AMD Inference Server
dev

Getting started

  • Introduction
    • Features
    • Documentation overview
    • Support
  • Terminology
    • amdinfer
    • Types of Docker images
      • Development
      • Deployment
    • Types of users
      • Clients
      • Administrators
      • Developers
    • Glossary
  • Quickstart
    • Prerequisites
    • Prepare the model repository
    • Get the deployment image
    • Start the image
    • Server deployment summary
    • Get the Python library
    • Running an example
    • Inference summary
    • Next steps

User guide

  • Backends
    • CPlusPlus
    • MIGraphX
      • Set up the host and GPUs
      • Get assets and models
      • Build an image
      • Start an image
    • Vitis AI
      • Set up the host and FPGAs
      • Get assets and models
      • Build an image
      • Start an image
    • ZenDNN
      • Get assets and models
      • Build an image
      • Freezing PyTorch models
      • Run Tests
      • Tune performance
  • Model Repository
    • Single models
    • Ensembles
  • Ensembles
    • Defining ensembles
      • Model repository
      • API
  • Deploying with Docker
    • Build the deployment Docker image
      • Push to a registry
    • Prepare the image for Docker deployment
    • Start the container
    • Make a request
  • Deploying with KServe
    • Set up Kubernetes and KServe
    • Get or build the AMD Inference Server Image
    • Start an inference service
      • Serving Runtime
      • Custom container
    • Making Requests
    • Debugging
  • Performance Factors
    • Hardware
    • Compile the right version
    • Parallelism
      • REST threads
      • Sending requests
      • Duplicating workers
  • Troubleshooting
    • Use the latest version
    • Use server logs

Examples

  • Hello World - Python
    • Import the library
    • Create our client and server objects
    • Is AMD Inference Server already running?
    • Load a worker
    • Inference
    • Validate the response
    • Clean up
    • Next steps
  • Running ResNet50 - C++
    • Include the header
    • Start the server
    • Create the client object
    • Load a worker
    • Prepare images
    • Construct requests
    • Make an inference
  • Running ResNet50 - Python
    • Include the module
    • Start the server
    • Create the client object
    • Load a worker
    • Prepare images
    • Construct requests
    • Make an inference

Developers

  • Developer Quickstart
    • Prerequisites
    • Set up the host
    • Get the code
    • amdinfer script
    • Build or get the Docker image
    • Compiling the AMD Inference Server
    • Get test artifacts
    • Run the AMD Inference Server
    • Next steps
  • Testing
    • Add a new test
      • Add assets
  • Architecture
    • Overview
    • Ingestion
      • API
      • HTTP/REST and WebSocket
      • gRPC
      • C++ API
    • Batching
    • Workers
      • Organization and Lifecycle
      • Improving Performance
      • External Processing
      • XModel
    • Shared State
    • Observation
      • Logging
      • Metrics
      • Tracing
  • AKS
    • Introduction to AKS
    • Using AKS in AMD Inference Server
  • Logs
    • AMD Inference Server Logs
    • Drogon Logs
  • Benchmarking
    • XModel Benchmarking
    • Kernel Simulation
  • Metrics
    • Quickstart
  • Tracing
    • Quickstart

About

  • Contributing
    • Ways to contribute
      • Idea generation
      • Raise issues
      • Triage
      • Raise pull requests
    • Style guide
      • Documentation
  • Dependencies
    • Docker Image
      • Base Image
      • Ubuntu Focal Repositories
      • Ubuntu PPAs
      • PyPI
      • Github
      • Others
      • Xilinx
      • AMD
    • Included
    • Downloaded Files
  • Changelog
    • Unreleased
      • Added
      • Changed
      • Deprecated
      • Removed
      • Fixed
      • Security
    • 0.3.0 - 2023-02-01
      • Added
      • Changed
      • Deprecated
      • Removed
      • Fixed
    • 0.2.0 - 2022-08-05
      • Added
      • Changed
      • Fixed
    • 0.1.0 - 2022-02-08
      • Added
  • Roadmap
    • 2022
      • 2022 Q1
      • 2022 Q2
      • 2022 Q3
    • 2023
      • 2023 Q1
      • 2023 Q2
      • 2023 Q3
      • 2023 Q4
    • Future

Libraries and API

  • amdinfer Script
    • Commands
    • Options
    • Sub-commands
      • attach
      • benchmark
      • build
      • clean
      • dockerize
      • get
      • install
      • list
      • make
      • run
      • start
      • test
      • up
  • C++
    • Clients
      • gRPC
      • HTTP
      • Native
      • WebSocket
    • Core
      • DataType
      • Exceptions
      • Prediction
    • Servers
  • Python
    • Install the Python library
      • Build wheels
    • API
      • BadStatus
      • Client
      • ConnectionError
      • DataType
      • EnvironmentNotSetError
      • ExternalError
      • FileNotFoundError
      • FileReadError
      • GrpcClient
      • HttpClient
      • ImageInferenceRequest()
      • InferenceRequest
      • InferenceRequestInput
      • InferenceRequestOutput
      • InferenceResponse
      • InferenceResponseOutput
      • InferenceTensor
      • InvalidArgumentError
      • ModelMetadata
      • NativeClient
      • ParameterMap
      • RuntimeError
      • Server
      • ServerMetadata
      • Tensor
      • WebSocketClient
      • inferAsyncOrdered()
      • inferAsyncOrderedBatched()
      • inference_request_to_dict()
      • loadEnsemble()
      • parallel_infer()
      • serverHasExtension()
      • start_http_client_server()
      • stringToArray()
      • unloadModels()
      • waitUntilModelNotReady()
      • waitUntilModelReady()
      • waitUntilServerReady()
  • REST Endpoints
AMD Inference Server
  • »
  • Search

© Copyright 2022 Advanced Micro Devices, Inc.. Last updated on April 18, 2023.

Terms and Conditions | Privacy | Cookie Policy | Trademarks | Statement on Forced Labor | Fair and Open Competition | UK Tax Strategy | Inclusive Terminology | Cookies Settings


Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: dev
Languages
en
Versions
0.1.0
0.2.0
0.3.0
main