AMD Inference Server Logo
main

Getting started

  • Introduction
    • Features
    • Documentation overview
    • Support
  • Terminology
    • amdinfer
    • Types of Docker images
      • Development
      • Deployment
    • Types of users
      • Clients
      • Administrators
      • Developers
  • Quickstart
    • Prerequisites
    • Prepare the model repository
    • Get the deployment image
    • Start the image
    • Server deployment summary
    • Get the Python library
    • Running an example
    • Inference summary
    • Next steps

User guide

  • Backends
    • CPlusPlus
      • Model support
      • Hardware support
      • Host setup
      • Build an image
      • Start a container
      • Get test assets
      • Loading the backend
      • Troubleshooting
    • MIGraphX
      • Model support
      • Hardware support
      • Host setup
      • Build an image
      • Start a container
      • Get test assets
      • Loading the backend
      • Troubleshooting
    • PtZenDNN
      • Model support
      • Hardware support
      • Host setup
      • Build an image
      • Start a container
      • Get test assets
      • Loading the backend
      • Troubleshooting
    • TfZenDNN
      • Model support
      • Hardware support
      • Host setup
      • Build an image
      • Start a container
      • Get test assets
      • Loading the backend
      • Troubleshooting
    • Vitis AI
      • Model support
      • Hardware support
      • Host setup
      • Build an image
      • Start a container
      • Get test assets
      • Loading the backend
      • Troubleshooting
  • Model Repository
    • Single models
    • Ensembles
  • Ensembles
    • Defining ensembles
      • Model repository
      • API
  • Deployment
    • Deployment image
      • Get the image
      • Build the image
      • Push to a registry
      • Prepare the image
    • Start a container
    • KServe
    • Development image
  • Client API
    • Include the API
    • Create a client object
    • Server status
    • Loading a backend
    • Making an inference request
    • Parsing the response
    • Next steps
  • KServe
    • Set up Kubernetes and KServe
    • Get or build the AMD Inference Server Image
    • Start an inference service
      • Serving Runtime
      • Custom container
    • Making Requests
    • Debugging
  • Performance
    • MLCommons
      • SingleStream
      • Server
  • Performance Factors
    • Hardware
    • Compile the right version
    • Parallelism
      • REST threads
      • Sending requests
      • Duplicating workers
  • Troubleshooting
    • Use the latest version
    • Use server logs
    • Build errors

Examples

  • Hello World - Python
    • Import the library
    • Create our client and server objects
    • Is AMD Inference Server already running?
    • Load a worker
    • Inference
    • Validate the response
    • Clean up
    • Next steps
  • Running ResNet50 - C++
    • Include the header
    • Start the server
    • Create the client object
    • Load a worker
    • Prepare images
    • Construct requests
    • Make an inference
  • Running ResNet50 - Python
    • Include the module
    • Start the server
    • Create the client object
    • Load a worker
    • Prepare images
    • Construct requests
    • Make an inference

Developers

  • Developer Quickstart
    • Prerequisites
    • Set up the host
    • Get the code
    • amdinfer script
    • Build or get the Docker image
    • Compiling the AMD Inference Server
    • Get test artifacts
    • Run the AMD Inference Server
    • Next steps
  • Testing
    • Add a new test
      • Add assets
  • Architecture
    • Overview
    • Ingestion
      • API
      • HTTP/REST and WebSocket
      • gRPC
      • C++ API
    • Batching
    • Workers
      • Organization and Lifecycle
      • Improving Performance
      • External Processing
      • XModel
    • Shared State
    • Observation
      • Logging
      • Metrics
      • Tracing
  • AKS
    • Introduction to AKS
    • Using AKS in AMD Inference Server
  • Logs
    • AMD Inference Server Logs
    • Drogon Logs
  • Benchmarking
    • XModel Benchmarking
    • Kernel Simulation
  • Metrics
    • Quickstart
  • Tracing
    • Quickstart

About

  • Contributing
    • Ways to contribute
      • Idea generation
      • Raise issues
      • Triage
      • Raise pull requests
    • Style guide
      • Documentation
  • Dependencies
    • Docker Image
      • Base Image
      • Ubuntu Focal Repositories
      • Ubuntu PPAs
      • PyPI
      • Github
      • Others
      • Xilinx
      • AMD
    • Included
    • Downloaded Files
  • Changelog
    • Unreleased
      • Added
      • Changed
      • Deprecated
      • Removed
      • Fixed
      • Security
    • 0.4.0 - 2023-09-07
      • Added
      • Changed
      • Deprecated
      • Removed
      • Fixed
      • Security
    • 0.3.0 - 2023-02-01
      • Added
      • Changed
      • Deprecated
      • Removed
      • Fixed
    • 0.2.0 - 2022-08-05
      • Added
      • Changed
      • Fixed
    • 0.1.0 - 2022-02-08
      • Added
  • Roadmap
    • 2022
      • 2022 Q1
      • 2022 Q2
      • 2022 Q3
    • 2023
      • 2023 Q1
      • 2023 Q2
      • 2023 Q3
      • 2023 Q4
    • Future

Libraries and API

  • amdinfer Script
    • Commands
    • Options
    • Sub-commands
      • attach
      • benchmark
      • build
      • clean
      • dockerize
      • get
      • install
      • list
      • make
      • run
      • start
      • test
      • up
  • C++
    • Clients
      • gRPC
      • HTTP
      • Native
      • WebSocket
    • Core
      • DataType
      • Exceptions
      • Prediction
    • Servers
  • Python
    • Install the Python library
      • Build wheels
    • API
  • REST Endpoints
AMD Inference Server
  • »
  • Search

© Copyright 2022 Advanced Micro Devices, Inc.. Last updated on September 07, 2023.

Terms and Conditions | Privacy | Cookie Policy | Trademarks | Statement on Forced Labor | Fair and Open Competition | UK Tax Strategy | Inclusive Terminology | Cookies Settings


Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: main
Languages
en
Versions
0.1.0
0.2.0
0.3.0
0.4.0
main