Class HttpClient

• Defined in File http.hpp

Nested Relationships

Nested Types

• Class HttpClient::HttpClientImpl

Inheritance Relationships

Base Type

• public amdinfer::Client (Class Client)

Class Documentation

class HttpClient : public amdinfer::Client

The HttpClient class implements the Client using HTTP REST.

Usage:

HttpClient client{“http://127:0.0.1:8998”}; if (client.serverLive()){ … }

Public Functions

explicit HttpClient(const std::string &address)

Construct a new HttpClient object.

Parameters

address – Address of the server to connect to

HttpClient(const std::string &address, const StringMap &headers, int parallelism)

Construct a new HttpClient object.

Parameters

• address – Address of the server to connect to

• headers – Key-value pairs that should be added to the HTTP headers for all requests

• parallelism – Max number of requests that can be sent in parallel

HttpClient(HttpClient const&) = delete

Copy constructor.

HttpClient &operator=(const HttpClient&) = delete

Copy assignment constructor.

HttpClient(HttpClient &&other) = default

Move constructor.

HttpClient &operator=(HttpClient &&other) = default

Move assignment constructor.

~HttpClient() override

Destructor. This is needed because HttpClientImpl is an incomplete type. The destructor is defaulted in the implementation. But having a non- default destructor here forces the need to explicitly specify the other special member functions by the Rule of 5.

virtual ServerMetadata serverMetadata() const override

Returns the server metadata as a ServerMetadata object.

Returns

ServerMetadata

virtual bool serverLive() const override

Checks if the server is live.

Returns

bool - true if server is live, false otherwise

virtual bool serverReady() const override

Checks if the server is ready.

Returns

bool - true if server is ready, false otherwise

virtual bool modelReady(const std::string &model) const override

Checks if a model/worker is ready.

Parameters

model – name of the model to check

Returns

bool - true if model is ready, false otherwise

virtual ModelMetadata modelMetadata(const std::string &model) const override

Returns the metadata associated with a ready model/worker.

Parameters

model – name of the model/worker to get metadata

Returns

ModelMetadata

virtual void modelLoad(const std::string &model, RequestParameters *parameters) const override

Loads a model with the given name and load-time parameters. This method assumes that a directory with this model name already exists in the model repository directory for the server containing the model and its metadata in the right format.

Parameters

• model – name of the model to load from the model repository directory

• parameters – load-time parameters for the worker supporting the model

virtual void modelUnload(const std::string &model) const override

Unloads a previously loaded model and shut it down. This is identical in functionality to workerUnload and is provided for symmetry.

Parameters

model – name of the model to unload

virtual InferenceResponse modelInfer(const std::string &model, const InferenceRequest &request) const override

Makes a synchronous inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for.

Parameters

• model – name of the model/worker to request inference to

• request – the request

Returns

InferenceResponse

virtual InferenceResponseFuture modelInferAsync(const std::string &model, const InferenceRequest &request) const override

Makes an asynchronous inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for. The user must save the Future object and use it to get the results of the inference later.

Parameters

• model – name of the model/worker to request inference to

• request – the request

Returns

InferenceResponseFuture

virtual std::vector<std::string> modelList() const override

Gets a list of active models on the server, returning their names.

Returns

std::vector<std::string>

virtual std::string workerLoad(const std::string &worker, RequestParameters *parameters) const override

Loads a worker with the given name and load-time parameters.

Parameters

• worker – name of the worker to load

• parameters – load-time parameters for the worker

Returns

std::string

virtual void workerUnload(const std::string &worker) const override

Unloads a previously loaded worker and shut it down. This is identical in functionality to modelUnload and is provided for symmetry.

Parameters

worker – name of the worker to unload

virtual bool hasHardware(const std::string &name, int num) const override

Checks if the server has the requested number of a specific hardware device.

Parameters

• name – name of the hardware device to check

• num – number of the device that should exist at minimum

Returns

bool - true if server has at least the requested number of the hardware device, false otherwise

class HttpClientImpl

Public Functions

inline explicit HttpClientImpl(const std::string &address, StringMap headers, int parallelism)

inline drogon::HttpClient *getClient()

inline const StringMap &getHeaders() const

inline auto getClientNum() const