Class Worker

• Defined in File worker.hpp

Inheritance Relationships

Derived Types

• public amdinfer::workers::Aks (Class Aks)

• public amdinfer::workers::AksDetect (Class AksDetect)

• public amdinfer::workers::AksDetectStream (Class AksDetectStream)

• public amdinfer::workers::Echo (Class Echo)

• public amdinfer::workers::EchoMulti (Class EchoMulti)

• public amdinfer::workers::InvertImage (Class InvertImage)

• public amdinfer::workers::InvertVideo (Class InvertVideo)

• public amdinfer::workers::MIGraphXWorker (Class MIGraphXWorker)

• public amdinfer::workers::PtZendnn (Class PtZendnn)

• public amdinfer::workers::ResNet50 (Class ResNet50)

• public amdinfer::workers::ResNet50Stream (Class ResNet50Stream)

• public amdinfer::workers::TfZendnn (Class TfZendnn)

• public amdinfer::workers::XModel (Class XModel)

Class Documentation

class Worker

All workers should extend the Worker class which defines the methods to create, start, and run workers over their lifecycle.

Subclassed by amdinfer::workers::Aks, amdinfer::workers::AksDetect, amdinfer::workers::AksDetectStream, amdinfer::workers::Echo, amdinfer::workers::EchoMulti, amdinfer::workers::InvertImage, amdinfer::workers::InvertVideo, amdinfer::workers::MIGraphXWorker, amdinfer::workers::PtZendnn, amdinfer::workers::ResNet50, amdinfer::workers::ResNet50Stream, amdinfer::workers::TfZendnn, amdinfer::workers::XModel

Public Functions

inline Worker(const std::string &name, const std::string &platform)

The constructor only initializes the private class members.

virtual ~Worker() = default

Destroy the Worker object.

virtual std::thread spawn(BatchPtrQueue *input_queue) = 0

Starts the worker’s run() method as a separate thread and returns it.

Parameters

input_queue – queue used to send requests to the started worker thread

Returns

std::thread

inline void init(RequestParameters *parameters)

Perform low-cost initialization of the worker.

inline size_t allocate(size_t num)

Allocate some buffers that are used to hold input and output data.

inline void acquire(RequestParameters *parameters)

Acquire any hardware resources or perform high-cost initialization.

inline void run(BatchPtrQueue *input_queue)

The main body of the worker executes the work.

Parameters

input_queue – queue that receives incoming requests

inline void release()

Release any hardware resources.

inline void deallocate()

Free the input and output buffers.

inline void destroy()

Perform any final operations before the worker thread is joined.

inline void returnBuffers(std::unique_ptr<std::vector<BufferPtrs>> input_buffers, std::unique_ptr<std::vector<BufferPtrs>> output_buffers)

Return buffers to the worker’s buffer pool after use. The buffers’ reset() method is called prior to returning.

Parameters

• input_buffers –

• output_buffers –

inline void setInputBuffers(BufferPtrsQueue *buffers)

inline void setOutputBuffers(BufferPtrsQueue *buffers)

inline uint32_t getMaxBufferNum() const

inline size_t getBatchSize() const

inline WorkerStatus getStatus() const

inline virtual std::vector<std::unique_ptr<Batcher>> makeBatcher(int num, RequestParameters *parameters)

template<typename T>
inline std::vector<std::unique_ptr<Batcher>> makeBatcher(int num, RequestParameters *parameters)

inline ModelMetadata getMetadata()

Protected Attributes

BufferPtrsQueue *input_buffers_

BufferPtrsQueue *output_buffers_

uint32_t max_buffer_num_

size_t batch_size_

ModelMetadata metadata_