XRT Core Library

Xilinx Runtime (XRT) Library Interface Definitions

Header file xrt.h defines data structures and function signatures exported by Xilinx Runtime (XRT) Library. XRT is part of software stack which is integrated into Xilinx reference platform.

typedef xclDeviceHandle

opaque device handle

Description

A device handle of xclDeviceHandle kind is obtained by opening a device. Clients pass this device handle to refer to the opened device in all future interaction with XRT.

XRT Device Management APIs

---

XCL_DRIVER_DLLESPEC unsigned xclProbe()

Enumerate devices found in the system

Parameters

Return

count of devices found

XCL_DRIVER_DLLESPEC xclDeviceHandle xclOpen(unsigned deviceIndex, const char * logFileName, enum xclVerbosityLevel level)

Open a device and obtain its handle.

Parameters

unsigned deviceIndex

Slot number of device 0 for first device, 1 for the second device…

const char * logFileName

Log file to use for optional logging

enum xclVerbosityLevel level

Severity level of messages to log

Return

Device handle

XCL_DRIVER_DLLESPEC void xclClose(xclDeviceHandle handle)

Close an opened device

Parameters

xclDeviceHandle handle

Device handle

XCL_DRIVER_DLLESPEC int xclResetDevice(xclDeviceHandle handle, enum xclResetKind kind)

Reset a device or its CL

Parameters

xclDeviceHandle handle

Device handle

enum xclResetKind kind

Reset kind

Return

0 on success or appropriate error number

Reset the device. All running kernels will be killed and buffers in DDR will be purged. A device may be reset if a user’s application dies without waiting for running kernel(s) to finish.

NOTE

Only implemeted Reset kind through user pf is XCL_USER_RESET

XCL_DRIVER_DLLESPEC int xclGetDeviceInfo2(xclDeviceHandle handle, struct xclDeviceInfo2 * info)

Obtain various bits of information from the device

Parameters

xclDeviceHandle handle

Device handle

struct xclDeviceInfo2 * info

Information record

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclGetUsageInfo(xclDeviceHandle handle, struct xclDeviceUsage * info)

Obtain usage information from the device

Parameters

xclDeviceHandle handle

Device handle

struct xclDeviceUsage * info

Information record

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclGetErrorStatus(xclDeviceHandle handle, struct xclErrorStatus * info)

Obtain error information from the device

Parameters

xclDeviceHandle handle

Device handle

struct xclErrorStatus * info

Information record

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclLoadXclBin(xclDeviceHandle handle, const struct axlf * buffer)

Download FPGA image (xclbin) to the device

Parameters

xclDeviceHandle handle

Device handle

const struct axlf * buffer

Pointer to device image (xclbin) in memory

Return

0 on success or appropriate error number

Download FPGA image (AXLF) to the device. The PR bitstream is encapsulated inside xclbin as a section. xclbin may also contains other sections which are suitably handled by the driver.

XCL_DRIVER_DLLESPEC int xclGetSectionInfo(xclDeviceHandle handle, void * info, size_t * size, enum axlf_section_kind kind, int index)

Get Information from sysfs about the downloaded xclbin sections

Parameters

xclDeviceHandle handle

Device handle

void * info

Pointer to preallocated memory which will store the return value.

size_t * size

Pointer to preallocated memory which will store the return size.

enum axlf_section_kind kind

axlf_section_kind for which info is being queried

int index

The (sub)section index for the “kind” type.

Return

0 on success or appropriate error number

Get the section information from sysfs. The index corrresponds to the (section) entry of the axlf_section_kind data being queried. The info and the size contain the return binary value of the subsection and its size.

XCL_DRIVER_DLLESPEC int xclReClock2(xclDeviceHandle handle, unsigned short region, const unsigned short * targetFreqMHz)

Configure PR region frequncies

Parameters

xclDeviceHandle handle

Device handle

unsigned short region

PR region (always 0)

const unsigned short * targetFreqMHz

Array of target frequencies in order for the Clock Wizards driving the PR region

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclLockDevice(xclDeviceHandle handle)

Get exclusive ownership of the device

Parameters

xclDeviceHandle handle

Device handle

Return

0 on success or appropriate error number

The lock is necessary before performing buffer migration, register access or bitstream downloads.

XCL_DRIVER_DLLESPEC int xclUnlockDevice(xclDeviceHandle handle)

Release exclusive ownership of the device

Parameters

xclDeviceHandle handle

Device handle

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclOpenContext(xclDeviceHandle handle, uuid_t xclbinId, unsigned int ipIndex, bool shared)

Create shared/exclusive context on compute units

Parameters

xclDeviceHandle handle

Device handle

uuid_t xclbinId

UUID of the xclbin image running on the device

unsigned int ipIndex

IP/CU index in the IP LAYOUT array

bool shared

Shared access or exclusive access

Return

0 on success or appropriate error number

The context is necessary before submitting execution jobs using xclExecBuf(). Contexts may be exclusive or shared. Allocation of exclusive contexts on a compute unit would succeed only if another client has not already setup up a context on that compute unit. Shared contexts can be concurrently allocated by many processes on the same compute units.

XCL_DRIVER_DLLESPEC int xclCloseContext(xclDeviceHandle handle, uuid_t xclbinId, unsigned ipIndex)

Close previously opened context

Parameters

xclDeviceHandle handle

Device handle

uuid_t xclbinId

UUID of the xclbin image running on the device

unsigned ipIndex

IP/CU index in the IP LAYOUT array

Return

0 on success or appropriate error number

Close a previously allocated shared/exclusive context for a compute unit.

XCL_DRIVER_DLLESPEC int xclBootFPGA(xclDeviceHandle handle)

Boot the FPGA from PROM

Parameters

xclDeviceHandle handle

Device handle

Return

0 on success or appropriate error number

This should only be called when there are no other clients. It will cause PCIe bus re-enumeration

XCL_DRIVER_DLLESPEC int xclLogMsg(xclDeviceHandle handle, enum xrtLogMsgLevel level, const char * tag, const char * format, ...)

Send message to log file as per settings in ini file.

Parameters

xclDeviceHandle handle

Device handle

enum xrtLogMsgLevel level

Severity level of the msg

const char * tag

Tag supplied by the client, like “OCL”, “XMA”, etc.

const char * format

Format of Msg string to write to log file

...

All other arguments as per the format

Return

0 on success or appropriate error number

XRT Buffer Management APIs

---

Buffer management APIs are used for managing device memory and migrating buffers between host and device memory

XCL_DRIVER_DLLESPEC unsigned int xclAllocBO(xclDeviceHandle handle, size_t size, int unused, unsigned flags)

Allocate a BO of requested size with appropriate flags

Parameters

xclDeviceHandle handle

Device handle

size_t size

Size of buffer

int unused

This argument is ignored

unsigned flags

Specify bank information, etc

Return

BO handle

XCL_DRIVER_DLLESPEC unsigned int xclAllocUserPtrBO(xclDeviceHandle handle, void * userptr, size_t size, unsigned flags)

Allocate a BO using userptr provided by the user

Parameters

xclDeviceHandle handle

Device handle

void * userptr

Pointer to 4K aligned user memory

size_t size

Size of buffer

unsigned flags

Specify bank information, etc

Return

BO handle

XCL_DRIVER_DLLESPEC void xclFreeBO(xclDeviceHandle handle, unsigned int boHandle)

Free a previously allocated BO

Parameters

xclDeviceHandle handle

Device handle

unsigned int boHandle

BO handle

XCL_DRIVER_DLLESPEC size_t xclWriteBO(xclDeviceHandle handle, unsigned int boHandle, const void * src, size_t size, size_t seek)

Copy-in user data to host backing storage of BO

Parameters

xclDeviceHandle handle

Device handle

unsigned int boHandle

BO handle

const void * src

Source data pointer

size_t size

Size of data to copy

size_t seek

Offset within the BO

Return

0 on success or appropriate error number

Copy host buffer contents to previously allocated device memory. seek specifies how many bytes to skip at the beginning of the BO before copying-in size bytes of host buffer.

XCL_DRIVER_DLLESPEC size_t xclReadBO(xclDeviceHandle handle, unsigned int boHandle, void * dst, size_t size, size_t skip)

Copy-out user data from host backing storage of BO

Parameters

xclDeviceHandle handle

Device handle

unsigned int boHandle

BO handle

void * dst

Destination data pointer

size_t size

Size of data to copy

size_t skip

Offset within the BO

Return

0 on success or appropriate error number

Copy contents of previously allocated device memory to host buffer. skip specifies how many bytes to skip from the beginning of the BO before copying-out size bytes of device buffer.

XCL_DRIVER_DLLESPEC void * xclMapBO(xclDeviceHandle handle, unsigned int boHandle, bool write)

Memory map BO into user’s address space

Parameters

xclDeviceHandle handle

Device handle

unsigned int boHandle

BO handle

bool write

READ only or READ/WRITE mapping

Return

Memory mapped buffer

Map the contents of the buffer object into host memory To unmap the buffer call POSIX unmap() on mapped void * pointer returned from xclMapBO

XCL_DRIVER_DLLESPEC int xclSyncBO(xclDeviceHandle handle, unsigned int boHandle, enum xclBOSyncDirection dir, size_t size, size_t offset)

Synchronize buffer contents in requested direction

Parameters

xclDeviceHandle handle

Device handle

unsigned int boHandle

BO handle

enum xclBOSyncDirection dir

To device or from device

size_t size

Size of data to synchronize

size_t offset

Offset within the BO

Return

0 on success or standard errno

Synchronize the buffer contents between host and device. Depending on the memory model this may require DMA to/from device or CPU cache flushing/invalidation

XCL_DRIVER_DLLESPEC int xclCopyBO(xclDeviceHandle handle, unsigned int dstBoHandle, unsigned int srcBoHandle, size_t size, size_t dst_offset, size_t src_offset)

Copy device buffer contents to another buffer

Parameters

xclDeviceHandle handle

Device handle

unsigned int dstBoHandle

Destination BO handle

unsigned int srcBoHandle

Source BO handle

size_t size

Size of data to synchronize

size_t dst_offset

dst Offset within the BO

size_t src_offset

src Offset within the BO

Return

0 on success or standard errno

Copy from source buffer contents to destination buffer, can be device to device or device to host. Always perform WRITE to achieve better performance, destination buffer can be on device or host require DMA from device

XCL_DRIVER_DLLESPEC int xclExportBO(xclDeviceHandle handle, unsigned int boHandle)

Obtain DMA-BUF file descriptor for a BO

Parameters

xclDeviceHandle handle

Device handle

unsigned int boHandle

BO handle which needs to be exported

Return

File handle to the BO or standard errno

Export a BO for import into another device or Linux subsystem which accepts DMA-BUF fd This operation is backed by Linux DMA-BUF framework

XCL_DRIVER_DLLESPEC unsigned int xclImportBO(xclDeviceHandle handle, int fd, unsigned flags)

Obtain BO handle for a BO represented by DMA-BUF file descriptor

Parameters

xclDeviceHandle handle

Device handle

int fd

File handle to foreign BO owned by another device which needs to be imported

unsigned flags

Unused

Return

BO handle of the imported BO

Import a BO exported by another device. * This operation is backed by Linux DMA-BUF framework

XCL_DRIVER_DLLESPEC int xclGetBOProperties(xclDeviceHandle handle, unsigned int boHandle, struct xclBOProperties * properties)

Obtain xclBOProperties struct for a BO

Parameters

xclDeviceHandle handle

Device handle

unsigned int boHandle

BO handle

struct xclBOProperties * properties

BO properties struct pointer

Return

0 on success

This is the prefered method for obtaining BO property information.

XRT Unmanaged DMA APIs

---

Unmanaged DMA APIs are for exclusive use by the debuggers and tools. The APIs allow clinets to read/write from/to absolute device address. No checks are performed if a buffer was allocated before at the specified location or if the address is valid. Users who want to take over the full memory managemnt of the device may use this API to synchronize their buffers between host and device.

XCL_DRIVER_DLLESPEC ssize_t xclUnmgdPread(xclDeviceHandle handle, unsigned flags, void * buf, size_t size, uint64_t offset)

Perform unmanaged device memory read operation

Parameters

xclDeviceHandle handle

Device handle

unsigned flags

Unused

void * buf

Destination data pointer

size_t size

Size of data to copy

uint64_t offset

Absolute offset inside device

Return

0 on success or appropriate error number

This API may be used to perform DMA operation from absolute location specified. Users may use this if they want to perform their own device memory management – not using the buffer object (BO) framework defined before.

XCL_DRIVER_DLLESPEC ssize_t xclUnmgdPwrite(xclDeviceHandle handle, unsigned flags, const void * buf, size_t size, uint64_t offset)

Perform unmanaged device memory read operation

Parameters

xclDeviceHandle handle

Device handle

unsigned flags

Unused

const void * buf

Source data pointer

size_t size

Size of data to copy

uint64_t offset

Absolute offset inside device

Return

0 on success or appropriate error number

This API may be used to perform DMA operation to an absolute location specified. Users may use this if they want to perform their own device memory management – not using the buffer object (BO) framework defined before.

XCL_DRIVER_DLLESPEC size_t xclWrite(xclDeviceHandle handle, enum xclAddressSpace space, uint64_t offset, const void * hostBuf, size_t size)

Perform register write operation, deprecated

Parameters

xclDeviceHandle handle

Device handle

enum xclAddressSpace space

Address space

uint64_t offset

Offset in the address space

const void * hostBuf

Source data pointer

size_t size

Size of data to copy

Return

size of bytes written or appropriate error number

This API may be used to write to device registers exposed on PCIe BAR. Offset is relative to the the address space. A device may have many address spaces. This API is deprecated. Please use :c:func:`xclRegWrite()`, instead.

XCL_DRIVER_DLLESPEC size_t xclRead(xclDeviceHandle handle, enum xclAddressSpace space, uint64_t offset, void * hostbuf, size_t size)

Perform register read operation, deprecated

Parameters

xclDeviceHandle handle

Device handle

enum xclAddressSpace space

Address space

uint64_t offset

Offset in the address space

void * hostbuf

Destination data pointer

size_t size

Size of data to copy

Return

size of bytes written or appropriate error number

This API may be used to read from device registers exposed on PCIe BAR. Offset is relative to the the address space. A device may have many address spaces. This API is deprecated. Please use :c:func:`xclRegRead()`, instead.

XRT Compute Unit Execution Management APIs

These APIs are under development. These functions will be used to start compute units and wait for them to finish.

XCL_DRIVER_DLLESPEC int xclExecBuf(xclDeviceHandle handle, unsigned int cmdBO)

Submit an execution request to the embedded (or software) scheduler

Parameters

xclDeviceHandle handle

Device handle

unsigned int cmdBO

BO handle containing command packet

Return

0 or standard error number

Submit an exec buffer for execution. The exec buffer layout is defined by struct ert_packet which is defined in file ert.h. The BO should been allocated with DRM_XOCL_BO_EXECBUF flag.

XCL_DRIVER_DLLESPEC int xclExecBufWithWaitList(xclDeviceHandle handle, unsigned int cmdBO, size_t num_bo_in_wait_list, unsigned int * bo_wait_list)

Submit an execution request to the embedded (or software) scheduler

Parameters

xclDeviceHandle handle

Device handle

unsigned int cmdBO

BO handle containing command packet

size_t num_bo_in_wait_list

Number of BO handles in wait list

unsigned int * bo_wait_list

BO handles that must complete execution before cmdBO is started

Return

0 or standard error number

Submit an exec buffer for execution. The BO handles in the wait list must complete execution before cmdBO is started. The BO handles in the wait list must have beeen submitted prior to this call to xclExecBufWithWaitList.

XCL_DRIVER_DLLESPEC int xclExecWait(xclDeviceHandle handle, int timeoutMilliSec)

Wait for one or more execution events on the device

Parameters

xclDeviceHandle handle

Device handle

int timeoutMilliSec

How long to wait for

Return

Same code as poll system call

Wait for notification from the hardware. The function essentially calls “poll” system call on the driver file handle. The return value has same semantics as poll system call. If return value is > 0 caller should check the status of submitted exec buffers Note that if you perform wait for the same handle from multiple threads, you may lose wakeup for some of them. So, use different handle in different threads.

XCL_DRIVER_DLLESPEC int xclRegisterInterruptNotify(xclDeviceHandle handle, unsigned int userInterrupt, int fd)

register eventfd file handle for a MSIX interrupt

Parameters

xclDeviceHandle handle

Device handle

unsigned int userInterrupt

MSIX interrupt number

int fd

Eventfd handle

Return

0 on success or standard errno

Support for non managed interrupts (interrupts from custom IPs). fd should be obtained from eventfd system call. Caller should use standard poll/read eventfd framework in order to wait for interrupts. The handles are automatically unregistered on process exit.

XRT Stream Queue APIs

These functions are used for next generation DMA Engine, QDMA. QDMA provides not only memory mapped DMA which moves data between host memory and board memory, but also stream DMA which moves data between host memory and kernel directly. XDMA memory mapped DMA APIs are also supported on QDMA. New stream APIs are provided here for preview and may be revised in a future release. These can only be used with platforms with QDMA engine under the hood. The higher level OpenCL based streaming APIs offer more refined interfaces and compatibility between releases and each stream maps to a QDMA queue underneath.

XCL_DRIVER_DLLESPEC int xclCreateWriteQueue(xclDeviceHandle handle, struct xclQueueContext * q_ctx, uint64_t * q_hdl)

Create Write Queue

Parameters

xclDeviceHandle handle

Device handle

struct xclQueueContext * q_ctx

Queue Context

uint64_t * q_hdl

Queue handle

Return

0 or appropriate error number

Create write queue based on information provided in Queue context. Queue handle is generated if creation successes.

XCL_DRIVER_DLLESPEC int xclCreateReadQueue(xclDeviceHandle handle, struct xclQueueContext * q_ctx, uint64_t * q_hdl)

Create Read Queue

Parameters

xclDeviceHandle handle

Device handle

struct xclQueueContext * q_ctx

Queue Context

uint64_t * q_hdl

Queue handle

Return

0 or appropriate error number

Create read queue based on information provided in Queue context. Queue handle is generated if creation successes.

XCL_DRIVER_DLLESPEC int xclDestroyQueue(xclDeviceHandle handle, uint64_t q_hdl)

Destroy Queue

Parameters

xclDeviceHandle handle

Device handle

uint64_t q_hdl

Queue handle

Return

0 or appropriate error number

Destroy read or write queue and release all queue resources.

XCL_DRIVER_DLLESPEC void * xclAllocQDMABuf(xclDeviceHandle handle, size_t size, uint64_t * buf_hdl)

Allocate DMA buffer

Parameters

xclDeviceHandle handle

Device handle

size_t size

Buffer size

uint64_t * buf_hdl

Buffer handle

Return

0 or appropriate error number

Allocate DMA buffer which is used for queue read and write.

XCL_DRIVER_DLLESPEC int xclFreeQDMABuf(xclDeviceHandle handle, uint64_t buf_hdl)

Free DMA buffer

Parameters

xclDeviceHandle handle

Device handle

uint64_t buf_hdl

Buffer handle

Return

0 or appropriate error number

Free DMA buffer allocated by xclAllocQDMABuf.

XCL_DRIVER_DLLESPEC ssize_t xclWriteQueue(xclDeviceHandle handle, uint64_t q_hdl, struct xclQueueRequest * wr_req)

write data to queue

Parameters

xclDeviceHandle handle

Device handle

uint64_t q_hdl

Queue handle

struct xclQueueRequest * wr_req

Queue request

Return

Number of bytes been written or appropriate error number

Move data from host memory to board. The destination is determined by flow id and route id which are provided to xclCreateWriteQueue. it returns number of bytes been moved or error code. By default, this function returns only when the entire buf has been written, or error. If XCL_QUEUE_REQ_NONBLOCKING flag is used, it returns immediately and xclPollCompletion needs to be used to determine if the data transmission is completed. If XCL_QUEUE_REQ_EOT flag is used, end of transmit signal will be added at the end of this tranmission.

XCL_DRIVER_DLLESPEC ssize_t xclReadQueue(xclDeviceHandle handle, uint64_t q_hdl, struct xclQueueRequest * rd_req)

read data from queue

Parameters

xclDeviceHandle handle

Device handle

uint64_t q_hdl

Queue handle

struct xclQueueRequest * rd_req

read request

Return

Number of bytes been read or appropriate error number

Move data from board to host memory. The source is determined by flow id and route id which are provided to xclCreateReadQueue. It returns number of bytes been moved or error code. This function returns until all the requested bytes is read or error happens. If XCL_QUEUE_REQ_NONBLOCKING flag is used, it returns immediately and xclPollCompletion needs to be used to determine if the data trasmission is completed. If XCL_QUEUE_REQ_EOT flag is used, data transmission for the current read request completes immediatly once end of transmit signal is received.

XCL_DRIVER_DLLESPEC int xclPollCompletion(xclDeviceHandle handle, int min_compl, int max_compl, struct xclReqCompletion * comps, int * actual_compl, int timeout)

poll read/write queue completion

Parameters

xclDeviceHandle handle

undescribed

int min_compl

Unblock only when receiving min_compl completions

int max_compl

Max number of completion with one poll

struct xclReqCompletion * comps

Completed request array

int * actual_compl

Number of requests been completed

int timeout

Timeout

Return

Number of events or appropriate error number

Poll completion events of non-blocking read/write requests. Once this function returns, an array of completed requests is returned.