Class GrpcClient

• Defined in File grpc.hpp

Nested Relationships

Nested Types

• Class GrpcClient::GrpcClientImpl

Inheritance Relationships

Base Type

• public amdinfer::Client (Class Client)

Class Documentation

class GrpcClient : public amdinfer::Client

The GrpcClient class implements the Client using gRPC.

Usage:

GrpcClient client{“127:0.0.1:50051”}; if (client.serverLive()){ … }

Public Functions

explicit GrpcClient(const std::string &address)

Constructs a new GrpcClient object.

Parameters

address – Address of the server to connect to

explicit GrpcClient(const std::shared_ptr<::grpc::Channel> &channel)

Constructs a new GrpcClient object.

Parameters

channel – an existing gRPC channel to reuse to connect to the server

GrpcClient(GrpcClient const&) = delete

Copy constructor.

GrpcClient &operator=(const GrpcClient&) = delete

Copy assignment constructor.

GrpcClient(GrpcClient &&other) = default

Move constructor.

GrpcClient &operator=(GrpcClient &&other) = default

Move assignment constructor.

~GrpcClient() override

Destructor. This is needed because GrpcClientImpl is an incomplete type. The destructor is defaulted in the implementation. But having a non- default destructor here forces the need to explicitly specify the other special member functions by the Rule of 5.

virtual ServerMetadata serverMetadata() const override

Returns the server metadata as a ServerMetadata object.

Returns

ServerMetadata

virtual bool serverLive() const override

Checks if the server is live.

Returns

bool - true if server is live, false otherwise

virtual bool serverReady() const override

Checks if the server is ready.

Returns

bool - true if server is ready, false otherwise

virtual bool modelReady(const std::string &model) const override

Checks if a model/worker is ready.

Parameters

model – name of the model to check

Returns

bool - true if model is ready, false otherwise

virtual ModelMetadata modelMetadata(const std::string &model) const override

Returns the metadata associated with a ready model/worker.

Parameters

model – name of the model/worker to get metadata

Returns

ModelMetadata

virtual void modelLoad(const std::string &model, RequestParameters *parameters) const override

Loads a model with the given name and load-time parameters. This method assumes that a directory with this model name already exists in the model repository directory for the server containing the model and its metadata in the right format.

Parameters

• model – name of the model to load from the model repository directory

• parameters – load-time parameters for the worker supporting the model

virtual void modelUnload(const std::string &model) const override

Unloads a previously loaded model and shut it down. This is identical in functionality to workerUnload and is provided for symmetry.

Parameters

model – name of the model to unload

virtual InferenceResponse modelInfer(const std::string &model, const InferenceRequest &request) const override

Makes a synchronous inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for.

Parameters

• model – name of the model/worker to request inference to

• request – the request

Returns

InferenceResponse

virtual InferenceResponseFuture modelInferAsync(const std::string &model, const InferenceRequest &request) const override

Makes an asynchronous inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for. The user must save the Future object and use it to get the results of the inference later.

Parameters

• model – name of the model/worker to request inference to

• request – the request

Returns

InferenceResponseFuture

virtual std::vector<std::string> modelList() const override

Gets a list of active models on the server, returning their names.

Returns

std::vector<std::string>

virtual std::string workerLoad(const std::string &worker, RequestParameters *parameters) const override

Loads a worker with the given name and load-time parameters.

Parameters

• worker – name of the worker to load

• parameters – load-time parameters for the worker

Returns

std::string

virtual void workerUnload(const std::string &worker) const override

Unloads a previously loaded worker and shut it down. This is identical in functionality to modelUnload and is provided for symmetry.

Parameters

worker – name of the worker to unload

virtual bool hasHardware(const std::string &name, int num) const override

Checks if the server has the requested number of a specific hardware device.

Parameters

• name – name of the hardware device to check

• num – number of the device that should exist at minimum

Returns

bool - true if server has at least the requested number of the hardware device, false otherwise

class GrpcClientImpl

Public Functions

inline explicit GrpcClientImpl(const std::shared_ptr<::grpc::Channel> &channel)

inline inference::GRPCInferenceService::Stub *getStub()