AMD Inference Server
0.3.0
About
Introduction
Features
Documentation overview
Support
Dependencies
Docker Image
Base Image
Ubuntu Focal Repositories
Ubuntu PPAs
PyPI
Github
Others
Xilinx
AMD
Included
Downloaded Files
Roadmap
2022 Q1
2022 Q2
2022 Q3
Future
Changelog
Unreleased
Added
Changed
Deprecated
Removed
Fixed
0.2.0 - 2022-08-05
Added
Changed
Fixed
0.1.0 - 2022-02-08
Added
Quickstart
Inference
Get the library
Running the examples
Using the library
Deployment
Prerequisites
Prepare the model repository
Get the deployment image
Start the image
Development
Prerequisites
Set up the host
Get the code
Build or get the Docker image
Compiling the AMD Inference Server
Get test artifacts
Run the AMD Inference Server
Next steps
Libraries and API
C++
Clients
gRPC
HTTP
Native
WebSocket
Core
DataType
Exceptions
Prediction
Servers
Python
Install the Python library
Build wheels
API
REST Endpoints
Command-Line Interface
Commands
Options
Sub-commands
attach
benchmark
build
clean
dockerize
get
install
list
make
run
start
test
up
Examples
Hello World - Python
Import the library
Create our client and server objects
Is AMD Inference Server already running?
Load a worker
Inference
Validate the response
Clean up
Next steps
Running ResNet50 - C++
Include the header
Start the server
Create the client object
Load a worker
Prepare images
Construct requests
Make an inference
Running ResNet50 - Python
Include the module
Start the server
Create the client object
Load a worker
Prepare images
Construct requests
Make an inference
Using the Server
Platforms
CPUs - ZenDNN
Build an image
Get assets and models
Freezing PyTorch models
Run Tests
Tune performance
GPUs - MIGraphX
Set up the host and GPUs
Build an image
Start an image
Get assets and models
FPGAs - Vitis AI
Set up the host and FPGAs
Build an image
Start an image
Get assets and models
Deploying with Docker
Build the deployment Docker image
Push to a registry
Prepare the image for Docker deployment
Start the container
Make a request
Deploying with KServe
Set up Kubernetes and KServe
Get or build the AMD Inference Server Image
Start an inference service
Serving Runtime
Custom container
Making Requests
Debugging
Performance Factors
Hardware
Compile the right version
Parallelism
REST threads
Sending requests
Duplicating workers
Developers
Contributing
Ways to Contribute
Contributing Code
Sign Your Work
Style Guide
Documentation
Architecture
Overview
Ingestion
API
HTTP/REST and WebSocket
C++ API
Batching
Workers
Organization and Lifecycle
Improving Performance
External Processing
XModel
Buffering
Manager
Observation
Logging
Metrics
Tracing
AKS
Introduction to AKS
Using AKS in AMD Inference Server
Logs
AMD Inference Server Logs
Drogon Logs
Benchmarking
XModel Benchmarking
Kernel Simulation
Metrics
Quickstart
Tracing
Quickstart
Code Documentation
Full API
Namespaces
Classes and Structs
Enums
Functions
AMD Inference Server
»
Code Documentation
View page source
Code Documentation
¶
Full API
¶
Namespaces
¶
Namespace @85
Namespace AKS
Namespace amdinfer
Namespaces
Classes
Enums
Functions
Typedefs
Variables
Namespace amdinfer::detail
Functions
Namespace amdinfer::grpc
Functions
Namespace amdinfer::http
Classes
Functions
Namespace amdinfer::pre_post
Namespaces
Classes
Enums
Functions
Variables
Namespace amdinfer::pre_post::detail
Functions
Namespace amdinfer::util
Classes
Functions
Typedefs
Variables
Namespace amdinfer::workers
Classes
Enums
Functions
Variables
Namespace google
Namespaces
Namespace google::protobuf
Classes
Namespace grpc
Namespace inference
Namespace Json
Namespace std
Classes
Namespace vart
Classes and Structs
¶
Struct AddDataToTensor
Struct Documentation
Struct InferenceRequestInputSizes
Struct Documentation
Struct LogOptions
Struct Documentation
Struct Observer
Struct Documentation
Template Struct Overloaded
Inheritance Relationships
Base Type
Struct Documentation
Template Struct ImagePreprocessOptions
Struct Documentation
Struct Server::ServerImpl
Nested Relationships
Struct Documentation
Struct ServerMetadata
Struct Documentation
Struct SetInputData
Struct Documentation
Struct SetOutputData
Struct Documentation
Struct UpdateCommand
Struct Documentation
Template Struct is_any
Inheritance Relationships
Base Type
Struct Documentation
Struct DetectResponse
Struct Documentation
Struct WriteData
Struct Documentation
Template Struct less< amdinfer::RequestParameters >
Struct Documentation
Class bad_status
Inheritance Relationships
Base Type
Derived Type
Class Documentation
Class Batch
Class Documentation
Class Batcher
Inheritance Relationships
Derived Types
Class Documentation
Class Buffer
Inheritance Relationships
Derived Types
Class Documentation
Template Class CallData
Inheritance Relationships
Base Type
Derived Types
Class Documentation
Class CallDataBase
Inheritance Relationships
Derived Types
Class Documentation
Class CallDataHasHardware
Inheritance Relationships
Base Type
Class Documentation
Class CallDataModelInfer
Inheritance Relationships
Base Type
Class Documentation
Class CallDataModelList
Inheritance Relationships
Base Type
Class Documentation
Class CallDataModelLoad
Inheritance Relationships
Base Type
Class Documentation
Class CallDataModelMetadata
Inheritance Relationships
Base Type
Class Documentation
Class CallDataModelReady
Inheritance Relationships
Base Type
Class Documentation
Class CallDataModelUnload
Inheritance Relationships
Base Type
Class Documentation
Class CallDataServerLive
Inheritance Relationships
Base Type
Class Documentation
Class CallDataServerMetadata
Inheritance Relationships
Base Type
Class Documentation
Class CallDataServerReady
Inheritance Relationships
Base Type
Class Documentation
Template Class CallDataServerStream
Inheritance Relationships
Base Type
Class Documentation
Template Class CallDataUnary
Inheritance Relationships
Base Type
Class Documentation
Class CallDataWorkerLoad
Inheritance Relationships
Base Type
Class Documentation
Class CallDataWorkerUnload
Inheritance Relationships
Base Type
Class Documentation
Class Client
Inheritance Relationships
Derived Types
Class Documentation
Class connection_error
Inheritance Relationships
Base Type
Class Documentation
Class CppNativeApi
Inheritance Relationships
Base Type
Class Documentation
Class DataType
Class Documentation
Class DrogonHttp
Inheritance Relationships
Base Type
Class Documentation
Class environment_not_set_error
Inheritance Relationships
Base Type
Class Documentation
Class external_error
Inheritance Relationships
Base Type
Class Documentation
Class file_not_found_error
Inheritance Relationships
Base Type
Class Documentation
Class file_read_error
Inheritance Relationships
Base Type
Class Documentation
Class GrpcApiUnary
Inheritance Relationships
Base Type
Class Documentation
Class GrpcClient
Nested Relationships
Nested Types
Inheritance Relationships
Base Type
Class Documentation
Class GrpcClient::GrpcClientImpl
Nested Relationships
Class Documentation
Class GrpcServer
Class Documentation
Class HardBatcher
Inheritance Relationships
Base Type
Class Documentation
Class DrogonWs
Inheritance Relationships
Base Type
Class Documentation
Class WebsocketServer
Inheritance Relationships
Base Type
Class Documentation
Class HttpClient
Nested Relationships
Nested Types
Inheritance Relationships
Base Type
Class Documentation
Class HttpClient::HttpClientImpl
Nested Relationships
Class Documentation
Class InferenceRequest
Class Documentation
Template Class InferenceRequestBuilder
Class Documentation
Template Class InferenceRequestBuilder< CallDataModelInfer * >
Class Documentation
Template Class InferenceRequestBuilder< InferenceRequest >
Class Documentation
Template Class InferenceRequestBuilder< std::shared_ptr< Json::Value > >
Class Documentation
Class InferenceRequestInput
Inheritance Relationships
Base Type
Class Documentation
Template Class InferenceRequestInputBuilder
Class Documentation
Template Class InferenceRequestInputBuilder< inference::ModelInferRequest_InferInputTensor >
Class Documentation
Template Class InferenceRequestInputBuilder< InferenceRequestInput >
Class Documentation
Template Class InferenceRequestInputBuilder< std::shared_ptr< Json::Value > >
Class Documentation
Class InferenceRequestOutput
Class Documentation
Template Class InferenceRequestOutputBuilder
Class Documentation
Template Class InferenceRequestOutputBuilder< std::shared_ptr< Json::Value > >
Class Documentation
Class InferenceResponse
Class Documentation
Class Interface
Inheritance Relationships
Derived Types
Class Documentation
Class invalid_argument
Inheritance Relationships
Base Type
Class Documentation
Class Logger
Class Documentation
Class Manager
Nested Relationships
Nested Types
Class Documentation
Class Manager::Endpoints
Nested Relationships
Class Documentation
Class ModelMetadata
Class Documentation
Class ModelMetadataTensor
Class Documentation
Class ModelRepository
Nested Relationships
Nested Types
Class Documentation
Class ModelRepository::ModelRepositoryImpl
Nested Relationships
Class Documentation
Class NativeClient
Inheritance Relationships
Base Type
Class Documentation
Class RequestParameters
Inheritance Relationships
Base Type
Class Documentation
Class runtime_error
Inheritance Relationships
Base Type
Derived Types
Class Documentation
Class Serializable
Inheritance Relationships
Derived Types
Class Documentation
Class Server
Nested Relationships
Nested Types
Class Documentation
Class SoftBatcher
Inheritance Relationships
Base Type
Class Documentation
Class UpdateListener
Inheritance Relationships
Base Type
Class Documentation
Class ThreadPool
Class Documentation
Class Timer
Class Documentation
Class VartTensorBuffer
Inheritance Relationships
Base Type
Class Documentation
Class VectorBuffer
Inheritance Relationships
Base Type
Class Documentation
Class WebSocketClient
Nested Relationships
Nested Types
Inheritance Relationships
Base Type
Class Documentation
Class WebSocketClient::WebSocketClientImpl
Nested Relationships
Class Documentation
Class WorkerInfo
Class Documentation
Class Aks
Inheritance Relationships
Base Type
Class Documentation
Class AksDetect
Inheritance Relationships
Base Type
Class Documentation
Class AksDetectStream
Inheritance Relationships
Base Type
Class Documentation
Class Echo
Inheritance Relationships
Base Type
Class Documentation
Class EchoMulti
Inheritance Relationships
Base Type
Class Documentation
Class InvertImage
Inheritance Relationships
Base Type
Class Documentation
Class InvertVideo
Inheritance Relationships
Base Type
Class Documentation
Class MIGraphXWorker
Inheritance Relationships
Base Type
Class Documentation
Class PtZendnn
Inheritance Relationships
Base Type
Class Documentation
Class ResNet50
Inheritance Relationships
Base Type
Class Documentation
Class ResNet50Stream
Inheritance Relationships
Base Type
Class Documentation
Class TfZendnn
Inheritance Relationships
Base Type
Class Documentation
Class Worker
Inheritance Relationships
Derived Types
Class Documentation
Class XModel
Inheritance Relationships
Base Type
Class Documentation
Template Class Map
Class Documentation
Enums
¶
Enum BatcherStatus
Enum Documentation
Enum InterfaceType
Enum Documentation
Enum Loggers
Enum Documentation
Enum LogLevel
Enum Documentation
Enum ImageOrder
Enum Documentation
Enum ResizeAlgorithm
Enum Documentation
Enum Status
Enum Documentation
Enum UpdateCommandType
Enum Documentation
Enum WorkerStatus
Enum Documentation
Functions
¶
Function amdinfer::addHeaders
Function Documentation
Function amdinfer::checkError
Function Documentation
Function amdinfer::constructMessage(const std::string&, const std::string&, const std::string&)
Function Documentation
Function amdinfer::constructMessage(const std::string&, const std::string&)
Function Documentation
Template Function amdinfer::copy
Function Documentation
Function amdinfer::createGetRequest
Function Documentation
Function amdinfer::createInferenceRequest
Function Documentation
Function amdinfer::createPostRequest
Function Documentation
Function amdinfer::detail::hash
Function Documentation
Function amdinfer::errorHttpResponse
Function Documentation
Template Function amdinfer::expandType
Function Documentation
Function amdinfer::findFunc
Function Documentation
Function amdinfer::getHardware
Function Documentation
Function amdinfer::getLevel
Function Documentation
Function amdinfer::getLogDirectory
Function Documentation
Template Function amdinfer::getTensorContents
Function Documentation
Function amdinfer::getWorker
Function Documentation
Function amdinfer::grpc::start
Function Documentation
Function amdinfer::grpc::stop
Function Documentation
Function amdinfer::grpcUnaryCallback
Function Documentation
Function amdinfer::hasHardware
Function Documentation
Function amdinfer::http::start
Function Documentation
Function amdinfer::http::stop
Function Documentation
Function amdinfer::inferAsyncOrdered
Function Documentation
Function amdinfer::inferAsyncOrderedBatched
Function Documentation
Function amdinfer::initialize
Function Documentation
Function amdinfer::initializeClientLogging
Function Documentation
Function amdinfer::initializeServerLogging
Function Documentation
Function amdinfer::initLogger
Function Documentation
Template Function amdinfer::jsonValueToType
Function Documentation
Function amdinfer::logTraceBuffer
Function Documentation
Function amdinfer::mapJsonToModelMetadata
Function Documentation
Function amdinfer::mapJsonToParameters
Function Documentation
Function amdinfer::mapJsonToResponse
Function Documentation
Function amdinfer::mapModelMetadataToProto
Function Documentation
Function amdinfer::mapParametersToJson
Function Documentation
Function amdinfer::mapParametersToProto
Function Documentation
Function amdinfer::mapProtoToModelMetadata
Function Documentation
Function amdinfer::mapProtoToParameters(const google::protobuf::Map<std::string, inference::InferParameter>&, RequestParameters *)
Function Documentation
Function amdinfer::mapProtoToParameters(const google::protobuf::Map<std::string, inference::InferParameter>&)
Function Documentation
Function amdinfer::mapProtoToParameters(const google::protobuf::Map<std::string, inference::InferParameter>&, RequestParameters&)
Function Documentation
Function amdinfer::mapProtoToParameters2
Function Documentation
Function amdinfer::mapProtoToResponse
Function Documentation
Function amdinfer::mapRequestToJson
Function Documentation
Function amdinfer::mapRequestToProto(const InferenceRequest&, inference::ModelInferRequest&, ] const Observer&)
Function Documentation
Function amdinfer::mapRequestToProto(const InferenceRequest&, inference::ModelInferRequest&, const Observer&)
Function Documentation
Function amdinfer::mapResponseToProto
Function Documentation
Function amdinfer::modelInfer
Function Documentation
Function amdinfer::modelList
Function Documentation
Function amdinfer::modelLoad
Function Documentation
Function amdinfer::modelMetadata
Function Documentation
Function amdinfer::modelMetadataTensorToJson
Function Documentation
Function amdinfer::modelMetadataToJson
Function Documentation
Function amdinfer::modelReady
Function Documentation
Function amdinfer::modelUnload
Function Documentation
Function amdinfer::operator<<
Function Documentation
Template Function amdinfer::Overloaded
Function Documentation
Function amdinfer::parseJson
Function Documentation
Function amdinfer::parseResponse
Function Documentation
Template Function amdinfer::pre_post::calcSoftmax
Function Documentation
Function amdinfer::pre_post::centerCrop
Function Documentation
Template Function amdinfer::pre_post::detail::nestedLoop
Function Documentation
Template Function amdinfer::pre_post::detail::normalize
Function Documentation
Function amdinfer::pre_post::getTopK
Function Documentation
Template Function amdinfer::pre_post::imagePreprocess
Function Documentation
Template Function amdinfer::pre_post::resnet50Postprocess
Function Documentation
Function amdinfer::runInference
Function Documentation
Function amdinfer::serverHasExtension
Function Documentation
Function amdinfer::serverMetadata
Function Documentation
Template Function amdinfer::switchOverTypes
Function Documentation
Function amdinfer::terminate
Function Documentation
Function amdinfer::util::addressToString
Function Documentation
Function amdinfer::util::autoExpandEnvironmentVariables
Function Documentation
Function amdinfer::util::base64Decode(std::string)
Function Documentation
Function amdinfer::util::base64Decode(const char *, size_t)
Function Documentation
Function amdinfer::util::base64Encode(std::string)
Function Documentation
Function amdinfer::util::base64Encode(const char *, size_t)
Function Documentation
Template Function amdinfer::util::containerProduct(Iter, Iter)
Function Documentation
Template Function amdinfer::util::containerProduct(const Container&)
Function Documentation
Template Function amdinfer::util::containerSum(Iter, Iter)
Function Documentation
Template Function amdinfer::util::containerSum(const Container&)
Function Documentation
Function amdinfer::util::endsWith
Function Documentation
Function amdinfer::util::exec
Function Documentation
Function amdinfer::util::getTime
Function Documentation
Function amdinfer::util::minDecodeLength
Function Documentation
Function amdinfer::util::minEncodeLength
Function Documentation
Function amdinfer::util::readNthLine
Function Documentation
Function amdinfer::util::setThreadName(const char *)
Function Documentation
Function amdinfer::util::setThreadName(const std::string&)
Function Documentation
Function amdinfer::util::split
Function Documentation
Function amdinfer::util::startsWith
Function Documentation
Function amdinfer::util::toLower(const std::string&)
Function Documentation
Function amdinfer::util::toLower(std::string *)
Function Documentation
Function amdinfer::util::zDecompress
Function Documentation
Function amdinfer::waitUntilModelReady
Function Documentation
Function amdinfer::waitUntilServerReady
Function Documentation
Function amdinfer::workerLoad
Function Documentation
Function amdinfer::workers::openLibrary
Function Documentation
Function amdinfer::workers::toDataType
Function Documentation
Function amdinfer::workerUnload
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function getWorker()
Function Documentation
Function main
Function Documentation
Function signalCallbackHandler
Function Documentation
Read the Docs
v: 0.3.0
Languages
en
Versions
0.1.0
0.2.0
0.3.0
main