Embedded Runtime Library¶

XRT Embedded Runtime definition

Header file ert.h defines data structures used by Emebdded Runtime (ERT) and XRT xclExecBuf() API.

struct ert_packet

Definition

struct ert_packet {
  union {unnamed_union};
#if defined(__linux__
  uint32_t data;
#else
  uint32_t data;
#endif
};

Members

{unnamed_union}: anonymous
data: count number of words representing packet payload
data: count number of words representing packet payload

struct ert_start_kernel_cmd

Definition

struct ert_start_kernel_cmd {
  union {unnamed_union};
  uint32_t cu_mask;
#if defined(__linux__
  uint32_t data;
#else
  uint32_t data;
#endif
};

Members

{unnamed_union}: anonymous
cu_mask: first mandatory CU mask
data: count-1 number of words representing interpreted payload
data: count-1 number of words representing interpreted payload

Description

The packet payload is comprised of reserved id field, a mandatory CU mask, and extra_cu_masks per header field, followed by a CU register map of size (count - (1 + extra_cu_masks)) uint32_t words.

struct ert_dpu_data: interpretation of data payload for ERT_START_DPU

Definition

struct ert_dpu_data {
  uint64_t instruction_buffer;
  uint32_t instruction_buffer_size;
  uint16_t uc_index;
  uint16_t chained;
};

Members

instruction_buffer: address of instruction buffer
instruction_buffer_size: size of instruction buffer in bytes
uc_index: microblaze controller index
chained: number of following ert_dpu_data elements

Description

The ert_dpu_data is prepended to data payload of ert_start_kernel_cmd after any extra cu masks. The payload count of the ert packet is incremented with the size (words) of ert_dpu_data elements preprended to the data payload.

The data payload for ERT_START_DPU is interpreted as fixed instruction buffer address along with instruction count, followed by regular kernel arguments.

struct ert_npu_data: interpretation of data payload for ERT_START_NPU

Definition

struct ert_npu_data {
  uint64_t instruction_buffer;
  uint32_t instruction_buffer_size;
  uint32_t instruction_prop_count;
};

Members

instruction_buffer: address of instruction buffer
instruction_buffer_size: size of instruction buffer in bytes
instruction_prop_count: WORD length of property name value pairs

Description

The ert_npu_data is prepended to data payload of ert_start_kernel_cmd after any extra cu masks. The payload count of the ert packet is incremented with the size (words) of ert_npu_data elements preprended to the data payload.

The data payload for ERT_START_NPU is interpreted as instruction buffer address, instruction count along with instruction property, followed by regular kernel arguments.

When instruction_prop_count is non-zero, it indicates the length (in 32 bits WORD) of the instruction buffer properties after this fields. This count is reserved for future extension. One example propertiy is the number of actual columns this instruction used.

struct ert_npu_preempt_data: interpretation of data payload for ERT_START_NPU_PREEMPT

Definition

struct ert_npu_preempt_data {
  uint64_t instruction_buffer;
  uint64_t save_buffer;
  uint64_t restore_buffer;
  uint32_t instruction_buffer_size;
  uint32_t save_buffer_size;
  uint32_t restore_buffer_size;
  uint32_t instruction_prop_count;
};

Members

instruction_buffer: address of instruction buffer
save_buffer: address of save instruction buffer
restore_buffer: address of restrore instruction buffer
instruction_buffer_size: size of instruction buffer in bytes
save_buffer_size: size of save instruction buffer in bytes
restore_buffer_size: size of restore instruction buffer in bytes
instruction_prop_count: number of property name value pairs

Description

The ert_npu_preempt_data is prepended to data payload of ert_start_kernel_cmd after any extra cu masks. The payload count of the ert packet is incremented with the size (words) of ert_npu_preempt_data elements preprended to the data payload.

The data payload for ERT_START_NPU_PREEMPT is interpreted as instruction buffer, save instruction buffer, restore instruction buffer and their size, along with instruction property, followed by regular kernel arguments.

When instruction_prop_count is non-zero, it indicates the length (in 32 bits WORD) of the instruction buffer properties after this fields. This count is reserved for future extension. One example propertiy is the number of actual columns this instruction used.

struct ert_cmd_chain_data: interpretation of data payload for ERT_CMD_CHAIN

Definition

struct ert_cmd_chain_data {
  uint32_t command_count;
  uint32_t submit_index;
  uint32_t error_index;
};

Members

command_count: number of commands in chain
submit_index: index of last successfully submitted command in chain
error_index: index of failing command if cmd status is not completed **data**[]: address of each command in chain

Description

This is the payload of an ert_packet when the opcode is ERT_CMD_CHAIN

struct ert_init_kernel_cmd

Definition

struct ert_init_kernel_cmd {
  union {unnamed_union};
  uint32_t cu_mask;
#if defined(__linux__
  uint32_t data;
#else
  uint32_t data;
#endif
};

Members

{unnamed_union}: anonymous
cu_mask: first mandatory CU mask
data: count-9 number of words representing interpreted payload
data: count-9 number of words representing interpreted payload

Description

this command initializes CUs by writing CU registers. CUs are represented by cu_mask and extra_cu_masks.

cu_run_timeout the configured CU timeout value in Microseconds: setting to 0 means CU should not timeout
cu_reset_timeout the configured CU reset timeout value in Microseconds: when CU timeout, CU will be reset. this indicates CU reset should be completed within the timeout value. if cu_run_timeout is set to 0, this field is undefined.

The packet payload is comprised of reserved id field, 8 reserved fields, a mandatory CU mask, and extra_cu_masks per header field, followed by a CU register map of size (count - (9 + extra_cu_masks)) uint32_t words.

struct ert_configure_cmd

Definition

struct ert_configure_cmd {
  union {unnamed_union};
  uint32_t slot_size;
  uint32_t num_cus;
  uint32_t cu_shift;
  uint32_t cu_base_addr;
  uint32_t ert:1;
  uint32_t polling:1;
  uint32_t cu_dma:1;
  uint32_t cu_isr:1;
  uint32_t cq_int:1;
  uint32_t cdma:1;
  uint32_t dsa52:1;
#if defined(__linux__
  uint32_t data;
#else
  uint32_t data;
#endif
};

Members

{unnamed_union}: anonymous
slot_size: command queue slot size
num_cus: number of compute units in program
cu_shift: shift value to convert CU idx to CU addr
cu_base_addr: base address to add to CU addr for actual physical address
ert: 1 enable embedded HW scheduler
polling: 1 poll for command completion
cu_dma: 1 enable CUDMA custom module for HW scheduler
cu_isr: 1 enable CUISR custom module for HW scheduler
cq_int: 1 enable interrupt from host to HW scheduler
cdma: 1 enable CDMA kernel
dsa52: 1 reserved for internal use
data: addresses of num_cus CUs
data: addresses of num_cus CUs

struct ert_configure_sk_cmd

Definition

struct ert_configure_sk_cmd {
  union {unnamed_union};
  uint32_t num_image;
};

Members

{unnamed_union}: anonymous
num_image: number of images

struct ert_unconfigure_sk_cmd

Definition

struct ert_unconfigure_sk_cmd {
  union {unnamed_union};
  uint32_t start_cuidx;
  uint32_t num_cus;
};

Members

{unnamed_union}: anonymous
start_cuidx: start index of compute units
num_cus: number of compute units in program

struct ert_abort_cmd

Definition

struct ert_abort_cmd {
  union {unnamed_union};
  uint64_t exec_bo_handle;
};

Members

{unnamed_union}: anonymous
exec_bo_handle: The bo handle of execbuf command to abort

struct ert_validate_cmd

Definition

struct ert_validate_cmd {
  union {unnamed_union};
};

Members

{unnamed_union}: anonymous

struct ert_access_valid_cmd

Definition

struct ert_access_valid_cmd {
  union {unnamed_union};
};

Members

{unnamed_union}: anonymous

struct ert_ctx_health_data

Definition

struct ert_ctx_health_data {
  uint32_t version;
  uint32_t txn_op_idx;
  uint32_t ctx_pc;
  uint32_t fatal_error_type;
  uint32_t fatal_error_exception_type;
  uint32_t fatal_error_exception_pc;
  uint32_t fatal_error_app_module;
  uint32_t app_health_report_size;
  uint32_t app_health_report;
};

Members

version: context health data version (current 0.0)
txn_op_idx: index of last TXN control code executed
ctx_pc: program counter for that context
fatal_error_type: the fatal error type if context crashes
fatal_error_exception_type: LX7 exception type
fatal_error_exception_pc: LX7 program counter at the time of the exception
fatal_error_app_module: module name where the exception occurred
app_health_report_size: size in bytes of the entire app health report
app_health_report: binary blob of the entire app health report (contains aie states)

Description

which has context health data

Field Default value Comment txn_op_idx: 0xFFFFFFFF there is no txn control code is running or the

last txn control code op idx is not captured

ctx_pc: 0 context .text program counter is not captured fatal_error_type: 0 no fatal error or fatal error is not captured fatal_error_exception_type: 0 fatal_error_exception_pc: 0 fatal_error_app_module: 0

app_health_report_size: 0 The entire app health report size

Once an ert packet completes with state ERT_CMD_STATE_TIMEOUT, the ert packet starting from payload will have the following information.

ERT_WORD_SIZE ()

Parameters

ERT_STATUS_REGISTER_ADDR ()

Parameters

Description

MicroBlaze write, host reads. MB(W) / HOST(COR)

ERT_CU_DMA_ENABLE_ADDR ()

Parameters

Description

on a specific CU. MB selects a free CU on which the command can run, then writes the 1<<CU back to the command slot CU mask and writes the slot index to the CU DMA REGISTER. HW is notified when the register is written and now does the DMA transfer of CU regmap map from command to CU, while MB continues its work. MB(W) / HW(R)

ERT_CQ_SLOT_SIZE_ADDR ()

Parameters

Description

configurable per xclbin. MB(W) / HW(R)

ERT_CU_OFFSET_ADDR ()

Parameters

Description

example a 64K regmap is 2^16 so 16 is written to the CU_OFFSET_ADDR. MB(W) / HW(R)

ERT_CQ_NUMBER_OF_SLOTS_ADDR ()

Parameters

Description

MB(W) / HW(R)

ERT_CU_BASE_ADDRESS_ADDR ()

Parameters

Description

CU_BASE_ADDRESS is the address of the first CU. MB(W) / HW(R)

ERT_CQ_BASE_ADDRESS_ADDR ()

Parameters

Description

MB(W) / HW(R)

ERT_CU_ISR_HANDLER_ENABLE_ADDR ()

Parameters

Description

CU interrupts. When a CU interrupts (when done), hardware handles the interrupt and writes the index of the CU that completed into the CU_STATUS_REGISTER (HW(W)/MB(COR)) as a bitmask

ERT_CQ_STATUS_ENABLE_ADDR ()

Parameters

Description

MB to indicate the presense of a new command in some slot. The slot index is written to the CQ_STATUS_REGISTER (HOST(W)/MB(R))

ERT_NUMBER_OF_CU_ADDR ()

Parameters

Description

xclbin. This is an optimization that allows HW to only check CU completion on actual CUs.

ERT_HOST_INTERRUPT_ENABLE_ADDR ()

Parameters

Description

When enabled writing to STATUS_REGISTER causes an interrupt in HOST. MB(W)

ERT_INTC_ADDR ()

Parameters

Description

This value is per hardware BSP (XPAR_INTC_SINGLE_BASEADDR)

ERT_CUISR_LUT_ADDR ()

Parameters

ERT_EXIT_CMD ()

Parameters

ERT_HLS_MODULE_IDLE ()

Parameters

ERT_INTC_IPR_ADDR ()

Parameters

Description

CU are enabled