Class SoftBatcher¶

Defined in File soft.hpp

Inheritance Relationships¶

Base Type¶

public amdinfer::Batcher (Class Batcher)

Class Documentation¶

class SoftBatcher : public amdinfer::Batcher ¶

The SoftBatcher attempts to batch requests to the requested batch size but has a timeout that passes an incomplete batch onwards if the batch cannot be completed.

Public Functions

Batcher()¶

Construct a new Batcher object.

The C++ Interface class encapsulates incoming requests from the C++ API to the batcher.

explicit Batcher(RequestParameters *parameters)¶

Batcher(const Batcher &batcher)¶

Construct a new Batcher object.

Parameters: name – the endpoint corresponding to the batcher’s worker group copy constructor

Batcher(Batcher &&other) = delete¶: Move constructor.