ZOCL Driver Interfaces

A GEM style driver for Xilinx edge based accelerators

File zynq_ioctl.h defines ioctl command codes and associated structures for interacting with zocl driver for Xilinx FPGA platforms(Zynq/ZynqMP/Versal).

Accelerator memory allocation is modeled as buffer objects (bo). zocl supports both SMMU based shared virtual memory and CMA based shared physical memory between PS and PL. zocl also supports memory management of PL-DDRs and PL-BRAMs. PL-DDR is reserved by zocl driver via device tree. Both PS Linux and PL logic can access PL-DDRs

Execution model is asynchronous where execute commands are submitted using command buffers and POSIX poll is used to wait for finished commands. Commands for a compute unit can only be submitted after an explicit context has been opened by the client for that compute unit.

zocl driver functionality is described in the following table.

# Functionality ioctl request code data format
1 Allocate buffer on device DRM_IOCTL_ZOCL_CREATE_BO drm_zocl_create_bo
2 Allocate buffer on device with userptr DRM_IOCTL_ZOCL_USERPTR_BO drm_zocl_userptr_bo
3 Get the buffer handle of given physical address DRM_IOCTL_ZOCL_GET_HOST_BO drm_zocl_host_bo
4 Prepare bo for mapping into user’s address space DRM_IOCTL_ZOCL_MAP_BO drm_zocl_map_bo
5 Synchronize (DMA) buffer contents in requested direction DRM_IOCTL_ZOCL_SYNC_BO drm_zocl_sync_bo
6 Obtain information about buffer object DRM_IOCTL_ZOCL_INFO_BO drm_zocl_info_bo
7 Update bo backing storage with user’s data DRM_IOCTL_ZOCL_PWRITE_BO drm_zocl_pwrite_bo
8 Read back data in bo backing storage DRM_IOCTL_ZOCL_PREAD_BO drm_zocl_pread_bo
9 Update device view with a specific xclbin image DRM_IOCTL_ZOCL_PCAP_DOWNLOAD drm_zocl_pcap_download
10 Read the xclbin and map the compute units. DRM_IOCTL_ZOCL_READ_AXLF drm_zocl_axlf
11 Send an execute job to a compute unit DRM_IOCTL_ZOCL_EXECBUF drm_zocl_execbuf
12 Get the soft kernel command (experimental) DRM_IOCTL_ZOCL_SK_GETCMD drm_zocl_sk_getcmd
13 Create the soft kernel (experimental) DRM_IOCTL_ZOCL_SK_CREATE drm_zocl_sk_create
14 Report the soft kernel state (experimental) DRM_IOCTL_ZOCL_SK_REPORT drm_zocl_sk_report
15 Get Information about Compute Unit (experimental) DRM_IOCTL_ZOCL_INFO_CU drm_zocl_info_cu
struct drm_zocl_create_bo

Create buffer object used with DRM_IOCTL_ZOCL_CREATE_BO ioctl

Definition

struct drm_zocl_create_bo {
  uint64_t size;
  uint32_t handle;
  uint32_t flags;
};

Members

size
Requested size of the buffer object
handle
bo handle returned by the driver
flags
DRM_ZOCL_BO_XXX flags
struct drm_zocl_userptr_bo

Create buffer object with user’s pointer used with DRM_IOCTL_ZOCL_USERPTR_BO ioctl

Definition

struct drm_zocl_userptr_bo {
  uint64_t addr;
  uint64_t size;
  uint32_t handle;
  uint32_t flags;
};

Members

addr
Address of buffer allocated by user
size
Requested size of the buffer object
handle
bo handle returned by the driver
flags
DRM_XOCL_BO_XXX flags
struct drm_zocl_map_bo

Prepare a buffer object for mmap used with DRM_IOCTL_ZOCL_MAP_BO ioctl

Definition

struct drm_zocl_map_bo {
  uint32_t handle;
  uint32_t pad;
  uint64_t offset;
};

Members

handle
bo handle
pad
Unused
offset
‘Fake’ offset returned by the driver which can be used with POSIX mmap
struct drm_zocl_sync_bo

Synchronize the buffer in the requested direction via cache flush/invalidation. used with DRM_ZOCL_SYNC_BO ioctl.

Definition

struct drm_zocl_sync_bo {
  uint32_t handle;
  enum drm_zocl_sync_bo_dir dir;
  uint64_t offset;
  uint64_t size;
};

Members

handle
GEM object handle
dir
DRM_ZOCL_SYNC_DIR_XXX
offset
Offset into the object to write to
size
Length of data to write
struct drm_zocl_info_bo

Obtain information about buffer object used with DRM_IOCTL_ZOCL_INFO_BO ioctl

Definition

struct drm_zocl_info_bo {
  uint32_t handle;
  uint32_t flags;
  uint64_t size;
  uint64_t paddr;
};

Members

handle
GEM object handle
flags
User BO flags
size
Size of BO
paddr
physical address
struct drm_zocl_host_bo

Get the buffer handle of given physical address used with DRM_IOCTL_ZOCL_GET_HOST_BO ioctl

Definition

struct drm_zocl_host_bo {
  uint64_t paddr;
  size_t size;
  uint32_t handle;
};

Members

paddr
physical address
size
Size of BO
handle
GEM object handle
struct drm_zocl_pwrite_bo

Update bo with user’s data used with DRM_IOCTL_ZOCL_PWRITE_BO ioctl

Definition

struct drm_zocl_pwrite_bo {
  uint32_t handle;
  uint32_t pad;
  uint64_t offset;
  uint64_t size;
  uint64_t data_ptr;
};

Members

handle
GEM object handle
pad
Padding
offset
Offset into the object to write to
size
Length of data to write
data_ptr
Pointer to read the data from (pointers not 32/64 compatible)
struct drm_zocl_pread_bo

Read data from bo used with DRM_IOCTL_ZOCL_PREAD_BO ioctl

Definition

struct drm_zocl_pread_bo {
  uint32_t handle;
  uint32_t pad;
  uint64_t offset;
  uint64_t size;
  uint64_t data_ptr;
};

Members

handle
GEM object handle
pad
Padding
offset
Offset into the object to read from
size
Length of data to wrreadite
data_ptr
Pointer to write the data into (pointers not 32/64 compatible)
struct drm_zocl_info_cu

Get information about Compute Unit (experimental) used with DRM_IOCTL_ZOCL_INFO_CU ioctl

Definition

struct drm_zocl_info_cu {
  uint64_t paddr;
  int apt_idx;
  int cu_idx;
};

Members

paddr
Physical address
apt_idx
Aperture index
cu_idx
CU index
struct drm_zocl_execbuf

Submit a command buffer for execution on a compute unit (experimental) used with DRM_IOCTL_ZOCL_EXECBUF ioctl

Definition

struct drm_zocl_execbuf {
  uint32_t ctx_id;
  uint32_t exec_bo_handle;
};

Members

ctx_id
Pass 0
exec_bo_handle
BO handle of command buffer formatted as ERT command
struct argument_info

Kernel argument information

Definition

struct argument_info {
  char name;
  uint32_t offset;
  uint32_t size;
  uint32_t dir;
};

Members

name
argument name
offset
argument offset in CU
size
argument size in bytes
dir
input or output argument for a CU
struct kernel_info

Kernel information

Definition

struct kernel_info {
  char name;
  int anums;
  struct argument_info args;
};

Members

name
kernel name
anums
number of argument
args
argument array
struct drm_zocl_kds

KDS user configuration

Definition

struct drm_zocl_kds {
  uint32_t polling:1;
};

Members

polling
poll for command completion
struct drm_zocl_axlf

Read xclbin (AXLF) device image and map CUs (experimental) used with DRM_IOCTL_ZOCL_READ_AXLF ioctl

Definition

struct drm_zocl_axlf {
  struct axlf * za_xclbin_ptr;
  uint32_t za_flags;
  int za_ksize;
  char * za_kernels;
};

Members

za_xclbin_ptr
Pointer to xclbin (AXLF) object
za_flags
platform flags
za_ksize
size of kernels in bytes
za_kernels
pointer of argument array
struct drm_zocl_sk_getcmd

Get the soft kernel command (experimental) used with DRM_IOCTL_ZOCL_SK_GETCMD ioctl

Definition

struct drm_zocl_sk_getcmd {
  uint32_t opcode;
  uint32_t start_cuidx;
  uint32_t cu_nums;
  char name;
  uint32_t bohdl;
};

Members

opcode
opcode for the Soft Kernel Command Packet
start_cuidx
start index of compute units
cu_nums
number of compute units in program
name
symbol name of soft kernel
bohdl
BO to hold soft kernel image
struct drm_zocl_aie_cmd

Get the aie command used with DRM_IOCTL_ZOCL_AIE_GETCMD and DRM_IOCTL_ZOCL_AIE_PUTCMD ioctl

Definition

struct drm_zocl_aie_cmd {
  uint32_t opcode;
  uint32_t size;
  char info;
};

Members

opcode
opcode for the Aie Command Packet
size
size in bytes of info data
info
information to transfer
struct drm_zocl_sk_create

Create a soft kernel (experimental) used with DRM_IOCTL_ZOCL_SK_CREATE ioctl

Definition

struct drm_zocl_sk_create {
  uint32_t cu_idx;
  uint32_t handle;
};

Members

cu_idx
Compute unit index
handle
Buffer object handle
struct drm_zocl_sk_report

Report the Soft Kernel State (experimental) used with DRM_IOCTL_ZOCL_SK_REPORT ioctl

Definition

struct drm_zocl_sk_report {
  uint32_t cu_idx;
  enum drm_zocl_scu_state cu_state;
};

Members

cu_idx
Compute unit index
cu_state
State of the Soft Compute Unit