AMD Inference Server
0.3.0
About
Introduction
Features
Documentation overview
Support
Dependencies
Docker Image
Base Image
Ubuntu Focal Repositories
Ubuntu PPAs
PyPI
Github
Others
Xilinx
AMD
Included
Downloaded Files
Roadmap
2022 Q1
2022 Q2
2022 Q3
Future
Changelog
Unreleased
Added
Changed
Deprecated
Removed
Fixed
0.2.0 - 2022-08-05
Added
Changed
Fixed
0.1.0 - 2022-02-08
Added
Quickstart
Inference
Get the library
Running the examples
Using the library
Deployment
Prerequisites
Prepare the model repository
Get the deployment image
Start the image
Development
Prerequisites
Set up the host
Get the code
Build or get the Docker image
Compiling the AMD Inference Server
Get test artifacts
Run the AMD Inference Server
Next steps
Libraries and API
C++
Clients
gRPC
HTTP
Native
WebSocket
Core
DataType
Exceptions
Prediction
Servers
Python
Install the Python library
Build wheels
API
REST Endpoints
Command-Line Interface
Commands
Options
Sub-commands
attach
benchmark
build
clean
dockerize
get
install
list
make
run
start
test
up
Examples
Hello World - Python
Import the library
Create our client and server objects
Is AMD Inference Server already running?
Load a worker
Inference
Validate the response
Clean up
Next steps
Running ResNet50 - C++
Include the header
Start the server
Create the client object
Load a worker
Prepare images
Construct requests
Make an inference
Running ResNet50 - Python
Include the module
Start the server
Create the client object
Load a worker
Prepare images
Construct requests
Make an inference
Using the Server
Platforms
CPUs - ZenDNN
Build an image
Get assets and models
Freezing PyTorch models
Run Tests
Tune performance
GPUs - MIGraphX
Set up the host and GPUs
Build an image
Start an image
Get assets and models
FPGAs - Vitis AI
Set up the host and FPGAs
Build an image
Start an image
Get assets and models
Deploying with Docker
Build the deployment Docker image
Push to a registry
Prepare the image for Docker deployment
Start the container
Make a request
Deploying with KServe
Set up Kubernetes and KServe
Get or build the AMD Inference Server Image
Start an inference service
Serving Runtime
Custom container
Making Requests
Debugging
Performance Factors
Hardware
Compile the right version
Parallelism
REST threads
Sending requests
Duplicating workers
Developers
Contributing
Ways to Contribute
Contributing Code
Sign Your Work
Style Guide
Documentation
Architecture
Overview
Ingestion
API
HTTP/REST and WebSocket
C++ API
Batching
Workers
Organization and Lifecycle
Improving Performance
External Processing
XModel
Buffering
Manager
Observation
Logging
Metrics
Tracing
AKS
Introduction to AKS
Using AKS in AMD Inference Server
Logs
AMD Inference Server Logs
Drogon Logs
Benchmarking
XModel Benchmarking
Kernel Simulation
Metrics
Quickstart
Tracing
Quickstart
Code Documentation
Full API
Namespaces
Classes and Structs
Enums
Functions
AMD Inference Server
»
Typedef amdinfer::BatchPtrQueue
View page source
Typedef amdinfer::BatchPtrQueue
¶
Defined in
File batcher.hpp
Typedef Documentation
¶
using
amdinfer
::
BatchPtrQueue
=
BlockingQueue
<
BatchPtr
>
¶
Read the Docs
v: 0.3.0
Languages
en
Versions
0.1.0
0.2.0
0.3.0
main