Command-Line Interface

The amdinfer script provides an easy way to launch containers and build the server during development. The documentation for the options available in this Python script can be seen here or on the terminal using the --help flag.

AMD Inference Server helper script

usage: amdinfer [--dry-run] [-h] [-v] command ...

Commands

command

Possible choices: attach, benchmark, build, clean, dockerize, get, install, list, make, run, start, test, up

Options

--dry-run

print the actual commands that would be run without running them

Default: False

-v, --version

show program’s version number and exit

Sub-commands

attach

Attach to a running container

amdinfer attach [-h] [name]

Options

name

name of the container to attach to. Defaults to latest running container

benchmark

Run the benchmarks

amdinfer benchmark

build

Build the amdinfer project

amdinfer build [--coverage | --debug | --release] [-a] [-c] [-d DIR] [-l] [-r] [-t THREADS] [-h]

Named Arguments

--coverage

build the coverage configuration

--debug

build the debug configuration

--release

build the release configuration

Options

-a, --all

Build additional optional targets

Default: False

-c, --clean

clean prior to building

Default: False

-d, --dir

root path to place the build tree. Defaults to ./build

Default: “/workspace/amdinfer/build/Release/docs/build”

-l, --lint

Enable build-time linting in CMake. Defaults to false

Default: False

-r, --regen

Delete the CMakeCache and regenerate

Default: False

-t, --threads

number of threads for Make (defaults to number of processors)

Default: 64

clean

Delete all generated files, restoring the repo to clean state

amdinfer clean [--artifacts] [-h]

Options

--artifacts

Delete downloaded artifacts

Default: False

dockerize

Build a docker image

amdinfer dockerize [--dev-base DEV_BASE] [-h] [--vitis] [--production] [--push] [--registry REGISTRY] [--tfzendnn_path TFZENDNN_PATH] [--ptzendnn_path PTZENDNN_PATH] [--migraphx] [--suffix SUFFIX]

Options

--dev-base

Name of the image to use as dev image to build the production image.

Default: “”

--vitis

If provided, build with Vitis AI enabled

Default: False

--production

Build the production image. Defaults to false to build the dev image.

Default: False

--push

Push the image(s) to the registry after building.

Default: False

--registry

Docker image registry. Defaults to amdinfer-user

Default: “amdinfer-user”

--tfzendnn_path

TF+ZenDNN C++ package location. If provided, will build with TF+ZenDNN enabled

--ptzendnn_path

PT+ZenDNN C++ package location. If provided, will build with PT+ZenDNN enabled

--migraphx

If provided, will build with Migraphx enabled

Default: False

--suffix

String to append to the tag of the Docker image to use. Defaults to empty string

Default: “”

get

Get artifacts (images, videos and models) for examples and tests

amdinfer get [--all] [--vitis] [--ptzendnn] [--tfzendnn] [--migraphx] [--dry-run] [-h]

Options

--all

download all files

Default: False

--vitis

download vitis-related files

Default: False

--ptzendnn

download ptzendnn-related files

Default: False

--tfzendnn

download tfzendnn-related files

Default: False

--migraphx

download migraphx-related files

Default: False

--dry-run

print the actual commands that would be run without running them

Default: False

install

Install amdinfer. Any unknown arguments are passed to the build command. Use – –help to see options.

amdinfer install [-d DIR] [--get-manifest] [-h]

Options

-d, --dir

root path to the build tree. Defaults to /tmp/amdinfer/build

Default: “/tmp/amdinfer/build”

--get-manifest

Print the list of files last installed.

Default: False

list

List containers and images on the host.

amdinfer list [-h] [type]

Positional Arguments

type

List the active containers (default) or specify ‘images’ or ‘labels’

Default: False

make

This is a wrapper around make for the current build configuration.

amdinfer make [--dir DIR] [-h] [-j [JOBS]]

Options

--dir

root path to the build tree. Defaults to ./build

Default: “/workspace/amdinfer/build/Release/docs/build”

-j

Allow N jobs at once; infinite jobs with no arg. Defaults to number of cores.

Default: 64

run

This is a wrapper around ‘docker run’ with preset options. Any unparsed arguments are directly passed to ‘docker run’. Use ‘docker run –help’ to see the options.

amdinfer run [--dev | --autotest-dev] [--command COMMAND] [--devices] [-h] [--image IMAGE] [--interactive] [--net-host] [--rm] [--user-config] [--working-dir] [--xclbins]

Presets

Presets define a set of options together for convenience. If additional options are provided, they override those set in the preset.

--dev

Starts the dev container in interactive mode with the working directory mounted.

--autotest-dev

Starts the dev container with the working directory mounted and runs the coverage test.

Options

These options may default to a value if a preset is selected.

--command

Override the default CMD of the image. Defaults to None.

--devices, --no-devices

Pass devices from host to container, if they exist

--image

Image to run. Defaults to a value if a preset is used.

--interactive, --no-interactive

Keep STDIN open even if not attached and allocate a pseudo-TTY

--net-host, --no-net-host

Use host networking mode

--rm, --no-rm

Remove container after exiting

--user-config, --no-user-config

Pass the user’s git config and SSH keys to the container, if they exist

--working-dir, --no-working-dir

Mount the current working directory from host to /workspace/amdinfer on the container

--xclbins, --no-xclbins

Pass /opt/xilinx/overlaybins from host to container, if it exists

start

Start amdinfer-server with the current build configuration. Unknown arguments are passed to amdinfer-server.

amdinfer start [--dir DIR] [-h] [--gdb]

Options

--dir

root path to the build tree. Defaults to ./build

Default: “/workspace/amdinfer/build/Release/docs/build”

--gdb

Run server with GDB.

Default: False

test

Run the test suite with pytest. All unknown arguments are passed to pytest

amdinfer test [--mode {tests,examples,all}] [--hostname HOSTNAME] [--http-port HTTP_PORT] [--load-before] [--build BUILD] [--benchmark {only,all,skip}] [--perf {only,all,skip}]
              [--cpp {only,all,skip}] [-h]

Options

--mode

Possible choices: tests, examples, all

Type of tests to run. Defaults to tests

Default: “tests”

--hostname

hostname of the server. Defaults to 127.0.0.1

Default: “127.0.0.1”

--http-port

port to use for the HTTP server. Defaults to 8998

Default: 8998

--load-before

attempt to load xclbins to all unloaded FPGAs before running tests. Defaults to false

Default: False

--build

build to test. Defaults to Debug

Default: “Debug”

--benchmark

Possible choices: only, all, skip

Run benchmark tests. Defaults to skip. Ignored if -m is specified

Default: “skip”

--perf

Possible choices: only, all, skip

Run perf tests. Defaults to skip. Ignored if -m is specified

Default: “skip”

--cpp

Possible choices: only, all, skip

Run cpp tests. Defaults to all. Ignored if -m is specified

Default: “all”

up

Start docker-compose services

amdinfer up [-h] [-p PROFILE] [--registry REGISTRY] [--suffix SUFFIX] [-v VERSION] [--write-only]

Options

-p, --profile

profile to run (dev|prod|test|autotest-dev|autotest). Defaults to autotest-dev

Default: “autotest-dev”

--registry

Registry to use for the Docker images. Defaults to amdinfer-user

Default: “amdinfer-user”

--suffix

String to append to the tag of the Docker image to use. Defaults to empty string

Default: “”

-v, --version

Version of the Docker images to run. Defaults to latest

Default: “latest”

--write-only

Only generate docker-compose.yml without running it

Default: False