Class WebSocketClient

Nested Relationships

Nested Types

Inheritance Relationships

Base Type

Class Documentation

class WebSocketClient : public amdinfer::Client

The WebSocketClient class implements the Client using websocket. It reuses the HttpClient for most transactions with the exception of some operations that actually use websocket.

Usage:

WebSocketClient client{“ws://127.0.0.1:8998”, “http://127.0.0.1:8998”}; if (client.serverLive()){ … }

Public Functions

WebSocketClient(const std::string &ws_address, const std::string &http_address)

Constructs a new WebSocketClient object.

Parameters

  • ws_address – address of the websocket server to connect to

  • http_address – address of the HTTP server to connect to

WebSocketClient(WebSocketClient const&) = delete

Copy constructor.

WebSocketClient &operator=(const WebSocketClient&) = delete

Copy assignment constructor.

WebSocketClient(WebSocketClient &&other) = default

Move constructor.

WebSocketClient &operator=(WebSocketClient &&other) = default

Move assignment constructor.

~WebSocketClient() override

Destructor. This is needed because WebSocketClientImpl is an incomplete type. The destructor is defaulted in the implementation. But having a non- default destructor here forces the need to explicitly specify the other special member functions by the Rule of 5.

virtual ServerMetadata serverMetadata() const override

Returns the server metadata as a ServerMetadata object.

Returns

ServerMetadata

virtual bool serverLive() const override

Checks if the server is live.

Returns

bool - true if server is live, false otherwise

virtual bool serverReady() const override

Checks if the server is ready.

Returns

bool - true if server is ready, false otherwise

virtual bool modelReady(const std::string &model) const override

Checks if a model/worker is ready.

Parameters

model – name of the model to check

Returns

bool - true if model is ready, false otherwise

virtual ModelMetadata modelMetadata(const std::string &model) const override

Returns the metadata associated with a ready model/worker.

Parameters

model – name of the model/worker to get metadata

Returns

ModelMetadata

virtual void modelLoad(const std::string &model, RequestParameters *parameters) const override

Loads a model with the given name and load-time parameters. This method assumes that a directory with this model name already exists in the model repository directory for the server containing the model and its metadata in the right format.

Parameters

  • model – name of the model to load from the model repository directory

  • parameters – load-time parameters for the worker supporting the model

virtual void modelUnload(const std::string &model) const override

Unloads a previously loaded model and shut it down. This is identical in functionality to workerUnload and is provided for symmetry.

Parameters

model – name of the model to unload

virtual InferenceResponse modelInfer(const std::string &model, const InferenceRequest &request) const override

Makes a synchronous inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for.

Parameters

  • model – name of the model/worker to request inference to

  • request – the request

Returns

InferenceResponse

virtual InferenceResponseFuture modelInferAsync(const std::string &model, const InferenceRequest &request) const override

Makes an asynchronous inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for. The user must save the Future object and use it to get the results of the inference later.

Parameters

  • model – name of the model/worker to request inference to

  • request – the request

Returns

InferenceResponseFuture

virtual std::vector<std::string> modelList() const override

Gets a list of active models on the server, returning their names.

Returns

std::vector<std::string>

virtual std::string workerLoad(const std::string &worker, RequestParameters *parameters) const override

Loads a worker with the given name and load-time parameters.

Parameters

  • worker – name of the worker to load

  • parameters – load-time parameters for the worker

Returns

std::string

virtual void workerUnload(const std::string &worker) const override

Unloads a previously loaded worker and shut it down. This is identical in functionality to modelUnload and is provided for symmetry.

Parameters

worker – name of the worker to unload

virtual bool hasHardware(const std::string &name, int num) const override

Checks if the server has the requested number of a specific hardware device.

Parameters

  • name – name of the hardware device to check

  • num – number of the device that should exist at minimum

Returns

bool - true if server has at least the requested number of the hardware device, false otherwise

void modelInferWs(const std::string &model, const InferenceRequest &request) const

Makes a websocket inference request to the given model/worker. The contents of the request depends on the model/worker that the request is for. This method differs from the standard inference in that it submits an actual Websocket message. The user should use modelRecv to get results and must disambiguate different responses on the client-side using the IDs of the responses.

Parameters

  • model –

  • request –

std::string modelRecv() const

Gets one message from the websocket server sent in response to a modelInferWs request. The user should know beforehand how many messages are expected and should call this method the same number of times.

Returns

std::string a JSON object encoded as a string

void close() const

Closes the websocket connection.

class WebSocketClientImpl

Public Functions

inline WebSocketClientImpl(const std::string &ws_address, const std::string &http_address)
inline ~WebSocketClientImpl()
WebSocketClientImpl(WebSocketClientImpl const&) = delete

Copy constructor.

WebSocketClientImpl &operator=(const WebSocketClientImpl&) = delete

Copy assignment constructor.

WebSocketClientImpl(WebSocketClientImpl &&other) = delete

Move constructor.

WebSocketClientImpl &operator=(WebSocketClientImpl &&other) = delete

Move assignment constructor.

inline void connect()
inline std::string recv()
inline drogon::WebSocketClient *getWsClient()
inline HttpClient *getHttpClient()