Class NativeClient

• Defined in File native.hpp

Inheritance Relationships

Base Type

• public amdinfer::Client (Class Client)

Class Documentation

class NativeClient : public amdinfer::Client

The NativeClient class implements the Client using the native C++ API. This client can be used if the client and backend are in the same C++ executable.

Usage:

NativeClient client; if (client.serverLive()){ … }

Public Functions

virtual ServerMetadata serverMetadata() const override

Returns the server metadata as a ServerMetadata object.

Returns

ServerMetadata

virtual bool serverLive() const override

Checks if the server is live.

Returns

bool - true if server is live, false otherwise

virtual bool serverReady() const override

Checks if the server is ready.

Returns

bool - true if server is ready, false otherwise

virtual bool modelReady(const std::string &model) const override

Checks if a model/worker is ready.

Parameters

model – name of the model to check

Returns

bool - true if model is ready, false otherwise

virtual ModelMetadata modelMetadata(const std::string &model) const override

Returns the metadata associated with a ready model/worker.

Parameters

model – name of the model/worker to get metadata

Returns

ModelMetadata

virtual void modelLoad(const std::string &model, RequestParameters *parameters) const override

Loads a model with the given name and load-time parameters. This method assumes that a directory with this model name already exists in the model repository directory for the server containing the model and its metadata in the right format.

Parameters

• model – name of the model to load from the model repository directory

• parameters – load-time parameters for the worker supporting the model

virtual void modelUnload(const std::string &model) const override

Unloads a previously loaded model and shut it down. This is identical in functionality to workerUnload and is provided for symmetry.

Parameters

model – name of the model to unload

virtual InferenceResponse modelInfer(const std::string &model, const InferenceRequest &request) const override

Makes a synchronous inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for.

Parameters

• model – name of the model/worker to request inference to

• request – the request

Returns

InferenceResponse

virtual InferenceResponseFuture modelInferAsync(const std::string &model, const InferenceRequest &request) const override

Makes an asynchronous inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for. The user must save the Future object and use it to get the results of the inference later.

Parameters

• model – name of the model/worker to request inference to

• request – the request

Returns

InferenceResponseFuture

virtual std::vector<std::string> modelList() const override

Gets a list of active models on the server, returning their names.

Returns

std::vector<std::string>

virtual std::string workerLoad(const std::string &worker, RequestParameters *parameters) const override

Loads a worker with the given name and load-time parameters.

Parameters

• worker – name of the worker to load

• parameters – load-time parameters for the worker

Returns

std::string

virtual void workerUnload(const std::string &worker) const override

Unloads a previously loaded worker and shut it down. This is identical in functionality to modelUnload and is provided for symmetry.

Parameters

worker – name of the worker to unload

virtual bool hasHardware(const std::string &name, int num) const override

Checks if the server has the requested number of a specific hardware device.

Parameters

• name – name of the hardware device to check

• num – number of the device that should exist at minimum

Returns

bool - true if server has at least the requested number of the hardware device, false otherwise