Class vart::RunnerExt

class RunnerExt : public Runner

Public Functions

virtual std::vector<vart::TensorBuffer*> get_inputs() = 0

Gets all input TensorBuffers of RunnerExt.

Sample code:

auto runner = vart::RunnerExt::create_runner(subgraph, attrs);
auto input_tensor_buffers = runner->get_inputs();
    for (auto input : input_tensor_buffers) {
        auto shape = input->get_tensor()->get_shape();
}

Returns:

All input TensorBuffers. A vector of raw pointer to the input TensorBuffer.

virtual std::vector<vart::TensorBuffer*> get_outputs() = 0

Gets all output TensorBuffers of RunnerExt.

Sample code:

auto runner = vart::RunnerExt::create_runner(subgraph, attrs);
auto output_tensor_buffers = runner->get_outputs();
    for (auto output : output_tensor_buffers) {
        auto shape = output->get_tensor()->get_shape();
}

Returns:

All output TensorBuffers. A vector of raw pointer to the output TensorBuffer.

virtual std::pair<uint32_t, int> execute_async(const std::vector<TensorBuffer*> &input, const std::vector<TensorBuffer*> &output) = 0

Executes the runner.

This is a blocking function.

Parameters:

• input – A vector of TensorBuffer create by all input tensors of runner.

• output – A vector of TensorBuffer create by all output tensors of runner.

Returns:

pair<jobid, status> status 0 for exit successfully, others for customized warnings or errors

virtual std::pair<std::uint32_t, int> execute_async(InputType input, OutputType output) = 0

execute_async

Parameters:

• input – inputs with a customized type

• output – outputs with a customized type

Returns:

pair<jobid, status> status 0 for exit successfully, others for customized warnings or errors

virtual int wait(int jobid, int timeout) = 0

Waits for the end of DPU processing.

modes: 1. Blocking wait for specific ID. 2. Non-blocking wait for specific ID. 3. Blocking wait for any ID. 4. Non-blocking wait for any ID

Parameters:

• jobid – job id, neg for any id, others for specific job id

• timeout – timeout, neg for block for ever, 0 for non-block, pos for block with a limitation(ms).

Returns:

status 0 for exit successfully, others for customized warnings or errors

virtual TensorFormat get_tensor_format()

Get the tensor format of runner.

Sample code:

auto format = runner->get_tensor_format();
switch (format) {
    case vart::Runner::TensorFormat::NCHW:
        // do something
        break;
    case vart::Runner::TensorFormat::NHWC:
        // do something
        break;
}

Returns:

TensorFormat : NHWC / HCHW

virtual std::vector<const xir::Tensor*> get_input_tensors() = 0

Get all input tensors of runner.

Sample code:

inputTensors = runner->get_input_tensors();
for (auto input : inputTensor) {
    input->get_name();
    input->get_shape();
    input->get_element_num();
}

Returns:

All input tensors. A vector of raw pointer to the input tensor.

virtual std::vector<const xir::Tensor*> get_output_tensors() = 0

Get all output tensors of runner.

Sample code:

outputTensors = runner->get_output_tensors();
  for (auto output : outputTensor) {
      output->get_name();
      output->get_shape();
      output->get_element_num();
}

Returns:

All output tensors. A vector of raw pointer to the output tensor.

Public Static Functions

static std::unique_ptr<RunnerExt> create_runner(const xir::Subgraph *subgraph, xir::Attrs *attrs)

Factory fucntion to create an instance of runner by subgraph and attrs.

Parameters:

• subgraph – XIR Subgraph

• attrs – XIR attrs object, this object is shared among all runners on the same graph.

Returns:

An instance of runner.

static std::unique_ptr<Runner> create_runner(const xir::Subgraph *subgraph, const std::string &mode = std::string(""))

Factory function to create an instance of DPU runner by subgraph.

Sample code:

// This API can be used like:
auto runner = vart::Runner::create_runner(subgraph, "run");

Parameters:

• subgraph – XIR Subgraph

• mode – 1 mode supported: ‘run’ - DPU runner.

Returns:

An instance of DPU runner.

static std::unique_ptr<Runner> create_runner_with_attrs(const xir::Subgraph *subgraph, xir::Attrs *attrs)

Factory function to create an instance of DPU runner by subgraph, and attrs.

Parameters:

• subgraph – XIR Subgraph

• attrs – XIR attrs object, this object is shared among all runners on the same graph.

• attrs["mode"], 1 – mode supported: ‘run’ - DPU runner.

Returns:

An instance of DPU runner.