XRT Core Library

Xilinx Runtime (XRT) Library Interface Definitions

Header file xrt.h defines data structures and function signatures exported by Xilinx Runtime (XRT) Library. XRT is part of software stack which is integrated into Xilinx reference platform.

typedef xclDeviceHandle

opaque device handle

Description

A device handle of xclDeviceHandle kind is obtained by opening a device. Clients pass this device handle to refer to the opened device in all future interaction with XRT.

XRT Device Management APIs


XCL_DRIVER_DLLESPEC unsigned xclProbe()

Enumerate devices found in the system

Parameters

Return

count of devices found

XCL_DRIVER_DLLESPEC xclDeviceHandle xclOpen(unsigned deviceIndex, const char * logFileName, enum xclVerbosityLevel level)

Open a device and obtain its handle.

Parameters

unsigned deviceIndex
Slot number of device 0 for first device, 1 for the second device…
const char * logFileName
Log file to use for optional logging
enum xclVerbosityLevel level
Severity level of messages to log

Return

Device handle

XCL_DRIVER_DLLESPEC void xclClose(xclDeviceHandle handle)

Close an opened device

Parameters

xclDeviceHandle handle
Device handle
XCL_DRIVER_DLLESPEC int xclResetDevice(xclDeviceHandle handle, enum xclResetKind kind)

Reset a device or its CL

Parameters

xclDeviceHandle handle
Device handle
enum xclResetKind kind
Reset kind

Return

0 on success or appropriate error number

Reset the device. All running kernels will be killed and buffers in DDR will be purged. A device may be reset if a user’s application dies without waiting for running kernel(s) to finish.

NOTE

Only implemeted Reset kind through user pf is XCL_USER_RESET

XCL_DRIVER_DLLESPEC int xclGetDeviceInfo2(xclDeviceHandle handle, struct xclDeviceInfo2 * info)

Obtain various bits of information from the device

Parameters

xclDeviceHandle handle
Device handle
struct xclDeviceInfo2 * info
Information record

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclGetUsageInfo(xclDeviceHandle handle, struct xclDeviceUsage * info)

Obtain usage information from the device

Parameters

xclDeviceHandle handle
Device handle
struct xclDeviceUsage * info
Information record

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclGetErrorStatus(xclDeviceHandle handle, struct xclErrorStatus * info)

Obtain error information from the device

Parameters

xclDeviceHandle handle
Device handle
struct xclErrorStatus * info
Information record

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclLoadXclBin(xclDeviceHandle handle, const struct axlf * buffer)

Download FPGA image (xclbin) to the device

Parameters

xclDeviceHandle handle
Device handle
const struct axlf * buffer
Pointer to device image (xclbin) in memory

Return

0 on success or appropriate error number

Download FPGA image (AXLF) to the device. The PR bitstream is encapsulated inside xclbin as a section. xclbin may also contains other sections which are suitably handled by the driver.

XCL_DRIVER_DLLESPEC int xclGetSectionInfo(xclDeviceHandle handle, void * info, size_t * size, enum axlf_section_kind kind, int index)

Get Information from sysfs about the downloaded xclbin sections

Parameters

xclDeviceHandle handle
Device handle
void * info
Pointer to preallocated memory which will store the return value.
size_t * size
Pointer to preallocated memory which will store the return size.
enum axlf_section_kind kind
axlf_section_kind for which info is being queried
int index
The (sub)section index for the “kind” type.

Return

0 on success or appropriate error number

Get the section information from sysfs. The index corrresponds to the (section) entry of the axlf_section_kind data being queried. The info and the size contain the return binary value of the subsection and its size.

XCL_DRIVER_DLLESPEC int xclReClock2(xclDeviceHandle handle, unsigned short region, const unsigned short * targetFreqMHz)

Configure PR region frequncies

Parameters

xclDeviceHandle handle
Device handle
unsigned short region
PR region (always 0)
const unsigned short * targetFreqMHz
Array of target frequencies in order for the Clock Wizards driving the PR region

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclLockDevice(xclDeviceHandle handle)

Get exclusive ownership of the device

Parameters

xclDeviceHandle handle
Device handle

Return

0 on success or appropriate error number

The lock is necessary before performing buffer migration, register access or bitstream downloads.

XCL_DRIVER_DLLESPEC int xclUnlockDevice(xclDeviceHandle handle)

Release exclusive ownership of the device

Parameters

xclDeviceHandle handle
Device handle

Return

0 on success or appropriate error number

XCL_DRIVER_DLLESPEC int xclOpenContext(xclDeviceHandle handle, uuid_t xclbinId, unsigned int ipIndex, bool shared)

Create shared/exclusive context on compute units

Parameters

xclDeviceHandle handle
Device handle
uuid_t xclbinId
UUID of the xclbin image running on the device
unsigned int ipIndex
IP/CU index in the IP LAYOUT array
bool shared
Shared access or exclusive access

Return

0 on success or appropriate error number

The context is necessary before submitting execution jobs using xclExecBuf(). Contexts may be exclusive or shared. Allocation of exclusive contexts on a compute unit would succeed only if another client has not already setup up a context on that compute unit. Shared contexts can be concurrently allocated by many processes on the same compute units.

XCL_DRIVER_DLLESPEC int xclCloseContext(xclDeviceHandle handle, uuid_t xclbinId, unsigned ipIndex)

Close previously opened context

Parameters

xclDeviceHandle handle
Device handle
uuid_t xclbinId
UUID of the xclbin image running on the device
unsigned ipIndex
IP/CU index in the IP LAYOUT array

Return

0 on success or appropriate error number

Close a previously allocated shared/exclusive context for a compute unit.

XCL_DRIVER_DLLESPEC int xclBootFPGA(xclDeviceHandle handle)

Boot the FPGA from PROM

Parameters

xclDeviceHandle handle
Device handle

Return

0 on success or appropriate error number

This should only be called when there are no other clients. It will cause PCIe bus re-enumeration

XCL_DRIVER_DLLESPEC int xclLogMsg(xclDeviceHandle handle, enum xrtLogMsgLevel level, const char * tag, const char * format, ...)

Send message to log file as per settings in ini file.

Parameters

xclDeviceHandle handle
Device handle
enum xrtLogMsgLevel level
Severity level of the msg
const char * tag
Tag supplied by the client, like “OCL”, “XMA”, etc.
const char * format
Format of Msg string to write to log file
...
All other arguments as per the format

Return

0 on success or appropriate error number

XRT Buffer Management APIs


Buffer management APIs are used for managing device memory and migrating buffers between host and device memory

XCL_DRIVER_DLLESPEC unsigned int xclAllocBO(xclDeviceHandle handle, size_t size, int unused, unsigned flags)

Allocate a BO of requested size with appropriate flags

Parameters

xclDeviceHandle handle
Device handle
size_t size
Size of buffer
int unused
This argument is ignored
unsigned flags
Specify bank information, etc

Return

BO handle

XCL_DRIVER_DLLESPEC unsigned int xclAllocUserPtrBO(xclDeviceHandle handle, void * userptr, size_t size, unsigned flags)

Allocate a BO using userptr provided by the user

Parameters

xclDeviceHandle handle
Device handle
void * userptr
Pointer to 4K aligned user memory
size_t size
Size of buffer
unsigned flags
Specify bank information, etc

Return

BO handle

XCL_DRIVER_DLLESPEC void xclFreeBO(xclDeviceHandle handle, unsigned int boHandle)

Free a previously allocated BO

Parameters

xclDeviceHandle handle
Device handle
unsigned int boHandle
BO handle
XCL_DRIVER_DLLESPEC size_t xclWriteBO(xclDeviceHandle handle, unsigned int boHandle, const void * src, size_t size, size_t seek)

Copy-in user data to host backing storage of BO

Parameters

xclDeviceHandle handle
Device handle
unsigned int boHandle
BO handle
const void * src
Source data pointer
size_t size
Size of data to copy
size_t seek
Offset within the BO

Return

0 on success or appropriate error number

Copy host buffer contents to previously allocated device memory. seek specifies how many bytes to skip at the beginning of the BO before copying-in size bytes of host buffer.

XCL_DRIVER_DLLESPEC size_t xclReadBO(xclDeviceHandle handle, unsigned int boHandle, void * dst, size_t size, size_t skip)

Copy-out user data from host backing storage of BO

Parameters

xclDeviceHandle handle
Device handle
unsigned int boHandle
BO handle
void * dst
Destination data pointer
size_t size
Size of data to copy
size_t skip
Offset within the BO

Return

0 on success or appropriate error number

Copy contents of previously allocated device memory to host buffer. skip specifies how many bytes to skip from the beginning of the BO before copying-out size bytes of device buffer.

XCL_DRIVER_DLLESPEC void * xclMapBO(xclDeviceHandle handle, unsigned int boHandle, bool write)

Memory map BO into user’s address space

Parameters

xclDeviceHandle handle
Device handle
unsigned int boHandle
BO handle
bool write
READ only or READ/WRITE mapping

Return

Memory mapped buffer

Map the contents of the buffer object into host memory To unmap the buffer call POSIX unmap() on mapped void * pointer returned from xclMapBO

XCL_DRIVER_DLLESPEC int xclSyncBO(xclDeviceHandle handle, unsigned int boHandle, enum xclBOSyncDirection dir, size_t size, size_t offset)

Synchronize buffer contents in requested direction

Parameters

xclDeviceHandle handle
Device handle
unsigned int boHandle
BO handle
enum xclBOSyncDirection dir
To device or from device
size_t size
Size of data to synchronize
size_t offset
Offset within the BO

Return

0 on success or standard errno

Synchronize the buffer contents between host and device. Depending on the memory model this may require DMA to/from device or CPU cache flushing/invalidation

XCL_DRIVER_DLLESPEC int xclCopyBO(xclDeviceHandle handle, unsigned int dstBoHandle, unsigned int srcBoHandle, size_t size, size_t dst_offset, size_t src_offset)

Copy device buffer contents to another buffer

Parameters

xclDeviceHandle handle
Device handle
unsigned int dstBoHandle
Destination BO handle
unsigned int srcBoHandle
Source BO handle
size_t size
Size of data to synchronize
size_t dst_offset
dst Offset within the BO
size_t src_offset
src Offset within the BO

Return

0 on success or standard errno

Copy from source buffer contents to destination buffer, can be device to device or device to host. Always perform WRITE to achieve better performance, destination buffer can be on device or host require DMA from device

XCL_DRIVER_DLLESPEC int xclExportBO(xclDeviceHandle handle, unsigned int boHandle)

Obtain DMA-BUF file descriptor for a BO

Parameters

xclDeviceHandle handle
Device handle
unsigned int boHandle
BO handle which needs to be exported

Return

File handle to the BO or standard errno

Export a BO for import into another device or Linux subsystem which accepts DMA-BUF fd This operation is backed by Linux DMA-BUF framework

XCL_DRIVER_DLLESPEC unsigned int xclImportBO(xclDeviceHandle handle, int fd, unsigned flags)

Obtain BO handle for a BO represented by DMA-BUF file descriptor

Parameters

xclDeviceHandle handle
Device handle
int fd
File handle to foreign BO owned by another device which needs to be imported
unsigned flags
Unused

Return

BO handle of the imported BO

Import a BO exported by another device. * This operation is backed by Linux DMA-BUF framework

XCL_DRIVER_DLLESPEC int xclGetBOProperties(xclDeviceHandle handle, unsigned int boHandle, struct xclBOProperties * properties)

Obtain xclBOProperties struct for a BO

Parameters

xclDeviceHandle handle
Device handle
unsigned int boHandle
BO handle
struct xclBOProperties * properties
BO properties struct pointer

Return

0 on success

This is the prefered method for obtaining BO property information.

XRT Unmanaged DMA APIs


Unmanaged DMA APIs are for exclusive use by the debuggers and tools. The APIs allow clinets to read/write from/to absolute device address. No checks are performed if a buffer was allocated before at the specified location or if the address is valid. Users who want to take over the full memory managemnt of the device may use this API to synchronize their buffers between host and device.

XCL_DRIVER_DLLESPEC ssize_t xclUnmgdPread(xclDeviceHandle handle, unsigned flags, void * buf, size_t size, uint64_t offset)

Perform unmanaged device memory read operation

Parameters

xclDeviceHandle handle
Device handle
unsigned flags
Unused
void * buf
Destination data pointer
size_t size
Size of data to copy
uint64_t offset
Absolute offset inside device

Return

0 on success or appropriate error number

This API may be used to perform DMA operation from absolute location specified. Users may use this if they want to perform their own device memory management – not using the buffer object (BO) framework defined before.

XCL_DRIVER_DLLESPEC ssize_t xclUnmgdPwrite(xclDeviceHandle handle, unsigned flags, const void * buf, size_t size, uint64_t offset)

Perform unmanaged device memory read operation

Parameters

xclDeviceHandle handle
Device handle
unsigned flags
Unused
const void * buf
Source data pointer
size_t size
Size of data to copy
uint64_t offset
Absolute offset inside device

Return

0 on success or appropriate error number

This API may be used to perform DMA operation to an absolute location specified. Users may use this if they want to perform their own device memory management – not using the buffer object (BO) framework defined before.

XCL_DRIVER_DLLESPEC size_t xclWrite(xclDeviceHandle handle, enum xclAddressSpace space, uint64_t offset, const void * hostBuf, size_t size)

Perform register write operation, deprecated

Parameters

xclDeviceHandle handle
Device handle
enum xclAddressSpace space
Address space
uint64_t offset
Offset in the address space
const void * hostBuf
Source data pointer
size_t size
Size of data to copy

Return

size of bytes written or appropriate error number

This API may be used to write to device registers exposed on PCIe BAR. Offset is relative to the the address space. A device may have many address spaces. This API is deprecated. Please use :c:func:`xclRegWrite()`, instead.

XCL_DRIVER_DLLESPEC size_t xclRead(xclDeviceHandle handle, enum xclAddressSpace space, uint64_t offset, void * hostbuf, size_t size)

Perform register read operation, deprecated

Parameters

xclDeviceHandle handle
Device handle
enum xclAddressSpace space
Address space
uint64_t offset
Offset in the address space
void * hostbuf
Destination data pointer
size_t size
Size of data to copy

Return

size of bytes written or appropriate error number

This API may be used to read from device registers exposed on PCIe BAR. Offset is relative to the the address space. A device may have many address spaces. This API is deprecated. Please use :c:func:`xclRegRead()`, instead.

XRT Compute Unit Execution Management APIs

These APIs are under development. These functions will be used to start compute units and wait for them to finish.

XCL_DRIVER_DLLESPEC int xclExecBuf(xclDeviceHandle handle, unsigned int cmdBO)

Submit an execution request to the embedded (or software) scheduler

Parameters

xclDeviceHandle handle
Device handle
unsigned int cmdBO
BO handle containing command packet

Return

0 or standard error number

Submit an exec buffer for execution. The exec buffer layout is defined by struct ert_packet which is defined in file ert.h. The BO should been allocated with DRM_XOCL_BO_EXECBUF flag.

XCL_DRIVER_DLLESPEC int xclExecBufWithWaitList(xclDeviceHandle handle, unsigned int cmdBO, size_t num_bo_in_wait_list, unsigned int * bo_wait_list)

Submit an execution request to the embedded (or software) scheduler

Parameters

xclDeviceHandle handle
Device handle
unsigned int cmdBO
BO handle containing command packet
size_t num_bo_in_wait_list
Number of BO handles in wait list
unsigned int * bo_wait_list
BO handles that must complete execution before cmdBO is started

Return

0 or standard error number

Submit an exec buffer for execution. The BO handles in the wait list must complete execution before cmdBO is started. The BO handles in the wait list must have beeen submitted prior to this call to xclExecBufWithWaitList.

XCL_DRIVER_DLLESPEC int xclExecWait(xclDeviceHandle handle, int timeoutMilliSec)

Wait for one or more execution events on the device

Parameters

xclDeviceHandle handle
Device handle
int timeoutMilliSec
How long to wait for

Return

Same code as poll system call

Wait for notification from the hardware. The function essentially calls “poll” system call on the driver file handle. The return value has same semantics as poll system call. If return value is > 0 caller should check the status of submitted exec buffers Note that if you perform wait for the same handle from multiple threads, you may lose wakeup for some of them. So, use different handle in different threads.

XCL_DRIVER_DLLESPEC int xclRegisterInterruptNotify(xclDeviceHandle handle, unsigned int userInterrupt, int fd)

register eventfd file handle for a MSIX interrupt

Parameters

xclDeviceHandle handle
Device handle
unsigned int userInterrupt
MSIX interrupt number
int fd
Eventfd handle

Return

0 on success or standard errno

Support for non managed interrupts (interrupts from custom IPs). fd should be obtained from eventfd system call. Caller should use standard poll/read eventfd framework in order to wait for interrupts. The handles are automatically unregistered on process exit.

XRT Stream Queue APIs

These functions are used for next generation DMA Engine, QDMA. QDMA provides not only memory mapped DMA which moves data between host memory and board memory, but also stream DMA which moves data between host memory and kernel directly. XDMA memory mapped DMA APIs are also supported on QDMA. New stream APIs are provided here for preview and may be revised in a future release. These can only be used with platforms with QDMA engine under the hood. The higher level OpenCL based streaming APIs offer more refined interfaces and compatibility between releases and each stream maps to a QDMA queue underneath.

XCL_DRIVER_DLLESPEC int xclCreateWriteQueue(xclDeviceHandle handle, struct xclQueueContext * q_ctx, uint64_t * q_hdl)

Create Write Queue

Parameters

xclDeviceHandle handle
Device handle
struct xclQueueContext * q_ctx
Queue Context
uint64_t * q_hdl
Queue handle

Return

0 or appropriate error number

Create write queue based on information provided in Queue context. Queue handle is generated if creation successes.

XCL_DRIVER_DLLESPEC int xclCreateReadQueue(xclDeviceHandle handle, struct xclQueueContext * q_ctx, uint64_t * q_hdl)

Create Read Queue

Parameters

xclDeviceHandle handle
Device handle
struct xclQueueContext * q_ctx
Queue Context
uint64_t * q_hdl
Queue handle

Return

0 or appropriate error number

Create read queue based on information provided in Queue context. Queue handle is generated if creation successes.

XCL_DRIVER_DLLESPEC int xclDestroyQueue(xclDeviceHandle handle, uint64_t q_hdl)

Destroy Queue

Parameters

xclDeviceHandle handle
Device handle
uint64_t q_hdl
Queue handle

Return

0 or appropriate error number

Destroy read or write queue and release all queue resources.

XCL_DRIVER_DLLESPEC void * xclAllocQDMABuf(xclDeviceHandle handle, size_t size, uint64_t * buf_hdl)

Allocate DMA buffer

Parameters

xclDeviceHandle handle
Device handle
size_t size
Buffer size
uint64_t * buf_hdl
Buffer handle

Return

0 or appropriate error number

Allocate DMA buffer which is used for queue read and write.

XCL_DRIVER_DLLESPEC int xclFreeQDMABuf(xclDeviceHandle handle, uint64_t buf_hdl)

Free DMA buffer

Parameters

xclDeviceHandle handle
Device handle
uint64_t buf_hdl
Buffer handle

Return

0 or appropriate error number

Free DMA buffer allocated by xclAllocQDMABuf.

XCL_DRIVER_DLLESPEC ssize_t xclWriteQueue(xclDeviceHandle handle, uint64_t q_hdl, struct xclQueueRequest * wr_req)

write data to queue

Parameters

xclDeviceHandle handle
Device handle
uint64_t q_hdl
Queue handle
struct xclQueueRequest * wr_req
Queue request

Return

Number of bytes been written or appropriate error number

Move data from host memory to board. The destination is determined by flow id and route id which are provided to xclCreateWriteQueue. it returns number of bytes been moved or error code. By default, this function returns only when the entire buf has been written, or error. If XCL_QUEUE_REQ_NONBLOCKING flag is used, it returns immediately and xclPollCompletion needs to be used to determine if the data transmission is completed. If XCL_QUEUE_REQ_EOT flag is used, end of transmit signal will be added at the end of this tranmission.

XCL_DRIVER_DLLESPEC ssize_t xclReadQueue(xclDeviceHandle handle, uint64_t q_hdl, struct xclQueueRequest * rd_req)

read data from queue

Parameters

xclDeviceHandle handle
Device handle
uint64_t q_hdl
Queue handle
struct xclQueueRequest * rd_req
read request

Return

Number of bytes been read or appropriate error number

Move data from board to host memory. The source is determined by flow id and route id which are provided to xclCreateReadQueue. It returns number of bytes been moved or error code. This function returns until all the requested bytes is read or error happens. If XCL_QUEUE_REQ_NONBLOCKING flag is used, it returns immediately and xclPollCompletion needs to be used to determine if the data trasmission is completed. If XCL_QUEUE_REQ_EOT flag is used, data transmission for the current read request completes immediatly once end of transmit signal is received.

XCL_DRIVER_DLLESPEC int xclPollCompletion(xclDeviceHandle handle, int min_compl, int max_compl, struct xclReqCompletion * comps, int * actual_compl, int timeout)

poll read/write queue completion

Parameters

xclDeviceHandle handle
undescribed
int min_compl
Unblock only when receiving min_compl completions
int max_compl
Max number of completion with one poll
struct xclReqCompletion * comps
Completed request array
int * actual_compl
Number of requests been completed
int timeout
Timeout

Return

Number of events or appropriate error number

Poll completion events of non-blocking read/write requests. Once this function returns, an array of completed requests is returned.