Class WorkerInfo

Class Documentation

class WorkerInfo

Stores the metadata associated with a worker. Instances of this class are saved in the Manager.

Public Functions

WorkerInfo(const std::string &name, RequestParameters *parameters)

Construct a new WorkerInfo object.

~WorkerInfo()
WorkerInfo(WorkerInfo const&) = delete

Destroy a WorkerInfo object

Copy constructor

WorkerInfo &operator=(const WorkerInfo&) = delete

Copy assignment constructor.

WorkerInfo(WorkerInfo &&other) = delete

Move constructor

WorkerInfo &operator=(WorkerInfo &&other) = delete

Move assignment constructor.

Batcher *getBatcher()

Get the queue associated with this worker to send it requests.

Returns

InferenceRequestPtrQueue*

void join(std::thread::id id)

Blocks until the associated worker’s thread joins.

void joinAll()

Blocks until all workers in the group join.

void addAndStartWorker(const std::string &name, RequestParameters *parameters)

Start a new worker in the group with the given parameters. The name is used to uniquely identify a particular worker to dynamically load.

Parameters

  • name –

  • parameters – pointer to parameters. Should not be nullptr

void unload()

unload one worker from this worker group

void shutdown()

unload all workers from this worker group

BufferPtrs getInputBuffer() const

Get an input buffer from this worker.

Returns

BufferPtrs

BufferPtrs getOutputBuffer() const

Get an output buffer from this worker.

Returns

BufferPtrs

void putInputBuffer(BufferPtrs &&buffer) const

Return an input buffer to the worker.

Parameters

buffer – the buffer to return

void putOutputBuffer(BufferPtrs &&buffer) const

Return an output buffer to the worker.

Parameters

buffer – the buffer to return

bool inputSizeValid(size_t size) const

Checks if this worker group supports a particular number of input tensors.

Parameters

size –

Throws

invalid_argument – too many input tensors for this worker

Returns

true if the number of input tensors is acceptable

Returns

false if more buffers need to be allocated but the number is okay

inline auto getBufferNum() const

Get the number of buffers currently allocated by this worker.

Returns

auto

inline auto getMaxBufferNum() const

Get the maximum number of buffers that this worker may allocate. -1 indicates there is no limit.

Returns

auto

void allocate(size_t request_size)

Allocate enough buffers to fulfill a particular request size.

Parameters

request_size –

size_t getGroupSize() const

get the number of workers in the group

inline auto getBatchSize() const

get the batch size of the worker group