Xilinx Runtime (XRT) Core Library¶
Xilinx Runtime (XRT) Library Interface Definitions
Header file xrt.h defines data structures and function signatures exported by Xilinx Runtime (XRT) Library. XRT is part of software stack which is integrated into Xilinx reference platform.
-
typedef
xclDeviceHandle
¶ opaque device handle
Description
A device handle of xclDeviceHandle kind is obtained by opening a device. Clients pass this device handle to refer to the opened device in all future interaction with XRT.
XRT Device Management APIs
-
XCL_DRIVER_DLLESPEC unsigned
xclProbe
()¶ Enumerate devices found in the system
Parameters
Return
count of devices found
-
XCL_DRIVER_DLLESPEC xclDeviceHandle
xclOpen
(unsigned deviceIndex, const char * logFileName, enum xclVerbosityLevel level)¶ Open a device and obtain its handle.
Parameters
unsigned deviceIndex
- Slot number of device 0 for first device, 1 for the second device…
const char * logFileName
- Log file to use for optional logging
enum xclVerbosityLevel level
- Severity level of messages to log
Return
Device handle
-
XCL_DRIVER_DLLESPEC void
xclClose
(xclDeviceHandle handle)¶ Close an opened device
Parameters
xclDeviceHandle handle
- Device handle
-
XCL_DRIVER_DLLESPEC int
xclResetDevice
(xclDeviceHandle handle, enum xclResetKind kind)¶ Reset a device or its CL
Parameters
xclDeviceHandle handle
- Device handle
enum xclResetKind kind
- Reset kind
Return
0 on success or appropriate error number
Reset the device. All running kernels will be killed and buffers in DDR will be purged. A device may be reset if a user’s application dies without waiting for running kernel(s) to finish.
-
XCL_DRIVER_DLLESPEC int
xclGetDeviceInfo2
(xclDeviceHandle handle, struct xclDeviceInfo2 * info)¶ Obtain various bits of information from the device
Parameters
xclDeviceHandle handle
- Device handle
struct xclDeviceInfo2 * info
- Information record
Return
0 on success or appropriate error number
-
XCL_DRIVER_DLLESPEC int
xclGetUsageInfo
(xclDeviceHandle handle, struct xclDeviceUsage * info)¶ Obtain usage information from the device
Parameters
xclDeviceHandle handle
- Device handle
struct xclDeviceUsage * info
- Information record
Return
0 on success or appropriate error number
-
XCL_DRIVER_DLLESPEC int
xclGetErrorStatus
(xclDeviceHandle handle, struct xclErrorStatus * info)¶ Obtain error information from the device
Parameters
xclDeviceHandle handle
- Device handle
struct xclErrorStatus * info
- Information record
Return
0 on success or appropriate error number
-
XCL_DRIVER_DLLESPEC int
xclLoadXclBin
(xclDeviceHandle handle, const struct axlf * buffer)¶ Download FPGA image (xclbin) to the device
Parameters
xclDeviceHandle handle
- Device handle
const struct axlf * buffer
- Pointer to device image (xclbin) in memory
Return
0 on success or appropriate error number
Download FPGA image (AXLF) to the device. The PR bitstream is encapsulated inside xclbin as a section. xclbin may also contains other sections which are suitably handled by the driver.
-
XCL_DRIVER_DLLESPEC int
xclLoadXclBinMgmt
(xclDeviceHandle handle, const struct axlf * buffer)¶ Download FPGA image (xclbin) to the device via mgmtpf
Parameters
xclDeviceHandle handle
- Device handle
const struct axlf * buffer
- Pointer to device image (xclbin) in memory
Return
0 on success or appropriate error number
Download FPGA image (AXLF) to the device. The PR bitstream is encapsulated inside xclbin as a section. xclbin may also contains other sections which are suitably handled by the driver.
-
XCL_DRIVER_DLLESPEC int
xclGetSectionInfo
(xclDeviceHandle handle, void * info, size_t * size, enum axlf_section_kind kind, int index)¶ Get Information from sysfs about the downloaded xclbin sections
Parameters
xclDeviceHandle handle
- Device handle
void * info
- Pointer to preallocated memory which will store the return value.
size_t * size
- Pointer to preallocated memory which will store the return size. kind: axlf_section_kind for which info is being queried index: The (sub)section index for the “kind” type.
enum axlf_section_kind kind
- undescribed
int index
- undescribed
Return
0 on success or appropriate error number
Get the section information from sysfs. The index corrresponds to the (section) entry of the axlf_section_kind data being queried. The info and the size contain the return binary value of the subsection and its size.
-
XCL_DRIVER_DLLESPEC int
xclReClock2
(xclDeviceHandle handle, unsigned short region, const unsigned short * targetFreqMHz)¶ Configure PR region frequncies
Parameters
xclDeviceHandle handle
- Device handle
unsigned short region
- PR region (always 0)
const unsigned short * targetFreqMHz
- Array of target frequencies in order for the Clock Wizards driving the PR region
Return
0 on success or appropriate error number
-
XCL_DRIVER_DLLESPEC int
xclReClockUser
(xclDeviceHandle handle, unsigned short region, const unsigned short * targetFreqMHz)¶ Configure PR region frequncies via userpf
Parameters
xclDeviceHandle handle
- Device handle
unsigned short region
- PR region (always 0)
const unsigned short * targetFreqMHz
- Array of target frequencies in order for the Clock Wizards driving the PR region
Return
0 on success or appropriate error number
-
XCL_DRIVER_DLLESPEC int
xclLockDevice
(xclDeviceHandle handle)¶ Get exclusive ownership of the device
Parameters
xclDeviceHandle handle
- Device handle
Return
0 on success or appropriate error number
The lock is necessary before performing buffer migration, register access or bitstream downloads.
-
XCL_DRIVER_DLLESPEC int
xclUnlockDevice
(xclDeviceHandle handle)¶ Release exclusive ownership of the device
Parameters
xclDeviceHandle handle
- Device handle
Return
0 on success or appropriate error number
-
XCL_DRIVER_DLLESPEC int
xclOpenContext
(xclDeviceHandle handle, uuid_t xclbinId, unsigned int ipIndex, bool shared)¶ Create shared/exclusive context on compute units
Parameters
xclDeviceHandle handle
- Device handle
uuid_t xclbinId
- UUID of the xclbin image running on the device
unsigned int ipIndex
- IP/CU index in the IP LAYOUT array
bool shared
- Shared access or exclusive access
Return
0 on success or appropriate error number
The context is necessary before submitting execution jobs using xclExecBuf()
. Contexts may be
exclusive or shared. Allocation of exclusive contexts on a compute unit would succeed
only if another client has not already setup up a context on that compute unit. Shared
contexts can be concurrently allocated by many processes on the same compute units.
-
XCL_DRIVER_DLLESPEC int
xclCloseContext
(xclDeviceHandle handle, uuid_t xclbinId, unsigned ipIndex)¶ Close previously opened context
Parameters
xclDeviceHandle handle
- Device handle
uuid_t xclbinId
- UUID of the xclbin image running on the device
unsigned ipIndex
- IP/CU index in the IP LAYOUT array
Return
0 on success or appropriate error number
Close a previously allocated shared/exclusive context for a compute unit.
-
XCL_DRIVER_DLLESPEC int
xclBootFPGA
(xclDeviceHandle handle)¶ Boot the FPGA from PROM
Parameters
xclDeviceHandle handle
- Device handle
Return
0 on success or appropriate error number
This should only be called when there are no other clients. It will cause PCIe bus re-enumeration
-
XCL_DRIVER_DLLESPEC int
xclLogMsg
(xclDeviceHandle handle, enum xrtLogMsgLevel level, const char * tag, const char * format, ...)¶ Send message to log file as per settings in ini file.
Parameters
xclDeviceHandle handle
- Device handle
enum xrtLogMsgLevel level
- Severity level of the msg
const char * tag
- Tag supplied by the client, like “OCL”, “XMA”, etc.
const char * format
- Format of Msg string to write to log file
...
- All other arguments as per the format
Return
0 on success or appropriate error number
XRT Buffer Management APIs
Buffer management APIs are used for managing device memory and migrating buffers between host and device memory
-
XCL_DRIVER_DLLESPEC unsigned int
xclAllocBO
(xclDeviceHandle handle, size_t size, enum xclBOKind domain, unsigned flags)¶ Allocate a BO of requested size with appropriate flags
Parameters
xclDeviceHandle handle
- Device handle
size_t size
- Size of buffer
enum xclBOKind domain
- Memory domain
unsigned flags
- Specify bank information, etc
Return
BO handle
-
XCL_DRIVER_DLLESPEC unsigned int
xclAllocUserPtrBO
(xclDeviceHandle handle, void * userptr, size_t size, unsigned flags)¶ Allocate a BO using userptr provided by the user
Parameters
xclDeviceHandle handle
- Device handle
void * userptr
- Pointer to 4K aligned user memory
size_t size
- Size of buffer
unsigned flags
- Specify bank information, etc
Return
BO handle
-
XCL_DRIVER_DLLESPEC void
xclFreeBO
(xclDeviceHandle handle, unsigned int boHandle)¶ Free a previously allocated BO
Parameters
xclDeviceHandle handle
- Device handle
unsigned int boHandle
- BO handle
-
XCL_DRIVER_DLLESPEC size_t
xclWriteBO
(xclDeviceHandle handle, unsigned int boHandle, const void * src, size_t size, size_t seek)¶ Copy-in user data to host backing storage of BO
Parameters
xclDeviceHandle handle
- Device handle
unsigned int boHandle
- BO handle
const void * src
- Source data pointer
size_t size
- Size of data to copy
size_t seek
- Offset within the BO
Return
0 on success or appropriate error number
Copy host buffer contents to previously allocated device memory. seek
specifies how many bytes
to skip at the beginning of the BO before copying-in size
bytes of host buffer.
-
XCL_DRIVER_DLLESPEC size_t
xclReadBO
(xclDeviceHandle handle, unsigned int boHandle, void * dst, size_t size, size_t skip)¶ Copy-out user data from host backing storage of BO
Parameters
xclDeviceHandle handle
- Device handle
unsigned int boHandle
- BO handle
void * dst
- Destination data pointer
size_t size
- Size of data to copy
size_t skip
- Offset within the BO
Return
0 on success or appropriate error number
Copy contents of previously allocated device memory to host buffer. skip
specifies how many bytes
to skip from the beginning of the BO before copying-out size
bytes of device buffer.
-
XCL_DRIVER_DLLESPEC void *
xclMapBO
(xclDeviceHandle handle, unsigned int boHandle, bool write)¶ Memory map BO into user’s address space
Parameters
xclDeviceHandle handle
- Device handle
unsigned int boHandle
- BO handle
bool write
- READ only or READ/WRITE mapping
Return
Memory mapped buffer
Map the contents of the buffer object into host memory
To unmap the buffer call POSIX unmap()
on mapped void * pointer returned from xclMapBO
-
XCL_DRIVER_DLLESPEC int
xclSyncBO
(xclDeviceHandle handle, unsigned int boHandle, enum xclBOSyncDirection dir, size_t size, size_t offset)¶ Synchronize buffer contents in requested direction
Parameters
xclDeviceHandle handle
- Device handle
unsigned int boHandle
- BO handle
enum xclBOSyncDirection dir
- To device or from device
size_t size
- Size of data to synchronize
size_t offset
- Offset within the BO
Return
0 on success or standard errno
Synchronize the buffer contents between host and device. Depending on the memory model this may require DMA to/from device or CPU cache flushing/invalidation
-
XCL_DRIVER_DLLESPEC int
xclCopyBO
(xclDeviceHandle handle, unsigned int dstBoHandle, unsigned int srcBoHandle, size_t size, size_t dst_offset, size_t src_offset)¶ Copy device buffer contents to another buffer
Parameters
xclDeviceHandle handle
- Device handle
unsigned int dstBoHandle
- Destination BO handle
unsigned int srcBoHandle
- Source BO handle
size_t size
- Size of data to synchronize
size_t dst_offset
- dst Offset within the BO
size_t src_offset
- src Offset within the BO
Return
0 on success or standard errno
Copy from source buffer contents to destination buffer, can be device to device or device to host. Always perform WRITE to achieve better performance, destination buffer can be on device or host require DMA from device
-
XCL_DRIVER_DLLESPEC int
xclExportBO
(xclDeviceHandle handle, unsigned int boHandle)¶ Obtain DMA-BUF file descriptor for a BO
Parameters
xclDeviceHandle handle
- Device handle
unsigned int boHandle
- BO handle which needs to be exported
Return
File handle to the BO or standard errno
Export a BO for import into another device or Linux subsystem which accepts DMA-BUF fd This operation is backed by Linux DMA-BUF framework
-
XCL_DRIVER_DLLESPEC unsigned int
xclImportBO
(xclDeviceHandle handle, int fd, unsigned flags)¶ Obtain BO handle for a BO represented by DMA-BUF file descriptor
Parameters
xclDeviceHandle handle
- Device handle
int fd
- File handle to foreign BO owned by another device which needs to be imported
unsigned flags
- Unused
Return
BO handle of the imported BO
Import a BO exported by another device. * This operation is backed by Linux DMA-BUF framework
-
XCL_DRIVER_DLLESPEC int
xclGetBOProperties
(xclDeviceHandle handle, unsigned int boHandle, struct xclBOProperties * properties)¶ Obtain xclBOProperties struct for a BO
Parameters
xclDeviceHandle handle
- Device handle
unsigned int boHandle
- BO handle
struct xclBOProperties * properties
- BO properties struct pointer
Return
0 on success
This is the prefered method for obtaining BO property information.
XRT Legacy Buffer Management APIs
Do not develop new features using the following 5 API’s. These are for backwards compatibility with classic XRT interface and will be deprecated in future. New clients should use BO based APIs defined above
-
XRT_DEPRECATED XCL_DRIVER_DLLESPEC uint64_t
xclAllocDeviceBuffer
(xclDeviceHandle handle, size_t size)¶ Allocate a buffer on the device
Parameters
xclDeviceHandle handle
- Device handle
size_t size
- Size of buffer
Return
Physical address of buffer on device or 0xFFFFFFFFFFFFFFFF in case of failure
Allocate a buffer on the device DDR and return its address. This API is deprecated and
will be removed in future release. Use xclAllocBO()
in all new code.
-
XRT_DEPRECATED XCL_DRIVER_DLLESPEC uint64_t
xclAllocDeviceBuffer2
(xclDeviceHandle handle, size_t size, enum xclMemoryDomains domain, unsigned flags)¶ Allocate a buffer on the device on a specific DDR
Parameters
xclDeviceHandle handle
- Device handle
size_t size
- Size of buffer
enum xclMemoryDomains domain
- Memory domain
unsigned flags
- Desired DDR bank as a bitmap.
Return
Physical address of buffer on device or 0xFFFFFFFFFFFFFFFF in case of failure
Allocate a buffer on a specific device DDR and return its address. This API is deprecated and
will be removed in future release. Use xclAllocBO()
in all new code.
-
XRT_DEPRECATED XCL_DRIVER_DLLESPEC void
xclFreeDeviceBuffer
(xclDeviceHandle handle, uint64_t buf)¶ Free a previously buffer on the device
Parameters
xclDeviceHandle handle
- Device handle
uint64_t buf
- Physical address of buffer
Description
The physical address should have been previously allocated by xclAllocDeviceBuffe()
or xclAllocDeviceBuffer2()
.
The address should point to the beginning of the buffer and not at an offset in the buffer. This API
is deprecated and will be removed in future release. Use xclFreeBO()
together with BO allocation APIs.
-
XRT_DEPRECATED XCL_DRIVER_DLLESPEC size_t
xclCopyBufferHost2Device
(xclDeviceHandle handle, uint64_t dest, const void * src, size_t size, size_t seek)¶ Write to device memory
Parameters
xclDeviceHandle handle
- Device handle
uint64_t dest
- Physical address in the device
const void * src
- Source buffer pointer
size_t size
- Size of data to synchronize
size_t seek
- Seek within the segment pointed to physical address
Return
Size of data moved or standard error number
Copy host buffer contents to previously allocated device memory. seek
specifies how many bytes to skip
at the beginning of the destination before copying size
bytes of host buffer. This API is deprecated
and will be removed in future release. Use xclSyncBO()
together with other BO APIs.
-
XRT_DEPRECATED XCL_DRIVER_DLLESPEC size_t
xclCopyBufferDevice2Host
(xclDeviceHandle handle, void * dest, uint64_t src, size_t size, size_t skip)¶ Read from device memory
Parameters
xclDeviceHandle handle
- Device handle
void * dest
- Destination buffer pointer
uint64_t src
- Physical address in the device
size_t size
- Size of data to synchronize
size_t skip
- Skip within the segment pointed to physical address
Return
Size of data moved or standard error number
Copy contents of previously allocated device memory to host buffer. skip
specifies how many bytes to skip
from the beginning of the source before copying size
bytes of device buffer. This API is deprecated
and will be removed in future release. Use xclSyncBO()
together with other BO APIs.
XRT Unmanaged DMA APIs
Unmanaged DMA APIs are for exclusive use by the debuggers and tools. The APIs allow clinets to read/write from/to absolute device address. No checks are performed if a buffer was allocated before at the specified location or if the address is valid. Users who want to take over the full memory managemnt of the device may use this API to synchronize their buffers between host and device.
-
XCL_DRIVER_DLLESPEC ssize_t
xclUnmgdPread
(xclDeviceHandle handle, unsigned flags, void * buf, size_t size, uint64_t offset)¶ Perform unmanaged device memory read operation
Parameters
xclDeviceHandle handle
- Device handle
unsigned flags
- Unused
void * buf
- Destination data pointer
size_t size
- Size of data to copy
uint64_t offset
- Absolute offset inside device
Return
size of bytes read or appropriate error number
This API may be used to perform DMA operation from absolute location specified. Users may use this if they want to perform their own device memory management – not using the buffer object (BO) framework defined before.
-
XCL_DRIVER_DLLESPEC ssize_t
xclUnmgdPwrite
(xclDeviceHandle handle, unsigned flags, const void * buf, size_t size, uint64_t offset)¶ Perform unmanaged device memory read operation
Parameters
xclDeviceHandle handle
- Device handle
unsigned flags
- Unused
const void * buf
- Source data pointer
size_t size
- Size of data to copy
uint64_t offset
- Absolute offset inside device
Return
size of bytes written or appropriate error number
This API may be used to perform DMA operation to an absolute location specified. Users may use this if they want to perform their own device memory management – not using the buffer object (BO) framework defined before.
-
XCL_DRIVER_DLLESPEC size_t
xclWrite
(xclDeviceHandle handle, enum xclAddressSpace space, uint64_t offset, const void * hostBuf, size_t size)¶ Perform register write operation
Parameters
xclDeviceHandle handle
- Device handle
enum xclAddressSpace space
- Address space
uint64_t offset
- Offset in the address space
const void * hostBuf
- Source data pointer
size_t size
- Size of data to copy
Return
size of bytes written or appropriate error number
This API may be used to write to device registers exposed on PCIe BAR. Offset is relative to the the address space. A device may have many address spaces. This API will be deprecated in future. Please use this API only for IP bringup/debugging. For execution management please use XRT Compute Unit Execution Management APIs defined below.
-
XCL_DRIVER_DLLESPEC size_t
xclRead
(xclDeviceHandle handle, enum xclAddressSpace space, uint64_t offset, void * hostbuf, size_t size)¶ Perform register read operation
Parameters
xclDeviceHandle handle
- Device handle
enum xclAddressSpace space
- Address space
uint64_t offset
- Offset in the address space
void * hostbuf
- Destination data pointer
size_t size
- Size of data to copy
Return
size of bytes written or appropriate error number
This API may be used to read from device registers exposed on PCIe BAR. Offset is relative to the the address space. A device may have many address spaces. This API will be deprecated in future. Please use this API only for IP bringup/debugging. For execution management please use XRT Compute Unit Execution Management APIs defined below
XRT Compute Unit Execution Management APIs
These APIs are under development. These functions will be used to start compute units and wait for them to finish.
-
XCL_DRIVER_DLLESPEC int
xclExecBuf
(xclDeviceHandle handle, unsigned int cmdBO)¶ Submit an execution request to the embedded (or software) scheduler
Parameters
xclDeviceHandle handle
- Device handle
unsigned int cmdBO
- BO handle containing command packet
Return
0 or standard error number
Submit an exec buffer for execution. The exec buffer layout is defined by struct ert_packet which is defined in file ert.h. The BO should been allocated with DRM_XOCL_BO_EXECBUF flag.
-
XCL_DRIVER_DLLESPEC int
xclExecBufWithWaitList
(xclDeviceHandle handle, unsigned int cmdBO, size_t num_bo_in_wait_list, unsigned int * bo_wait_list)¶ Submit an execution request to the embedded (or software) scheduler
Parameters
xclDeviceHandle handle
- Device handle
unsigned int cmdBO
- BO handle containing command packet
size_t num_bo_in_wait_list
- Number of BO handles in wait list
unsigned int * bo_wait_list
- BO handles that must complete execution before cmdBO is started
Return
0 or standard error number
Submit an exec buffer for execution. The BO handles in the wait list must complete execution before cmdBO is started. The BO handles in the wait list must have beeen submitted prior to this call to xclExecBufWithWaitList.
-
XCL_DRIVER_DLLESPEC int
xclExecWait
(xclDeviceHandle handle, int timeoutMilliSec)¶ Wait for one or more execution events on the device
Parameters
xclDeviceHandle handle
- Device handle
int timeoutMilliSec
- How long to wait for
Return
Same code as poll system call
Wait for notification from the hardware. The function essentially calls “poll” system call on the driver file handle. The return value has same semantics as poll system call. If return value is > 0 caller should check the status of submitted exec buffers Note that if you perform wait for the same handle from multiple threads, you may lose wakeup for some of them. So, use different handle in different threads.
-
XCL_DRIVER_DLLESPEC int
xclRegisterInterruptNotify
(xclDeviceHandle handle, unsigned int userInterrupt, int fd)¶ register eventfd file handle for a MSIX interrupt
Parameters
xclDeviceHandle handle
- Device handle
unsigned int userInterrupt
- MSIX interrupt number
int fd
- Eventfd handle
Return
0 on success or standard errno
Support for non managed interrupts (interrupts from custom IPs). fd should be obtained from eventfd system call. Caller should use standard poll/read eventfd framework in order to wait for interrupts. The handles are automatically unregistered on process exit.
XRT Stream Queue APIs
These functions are used for next generation DMA Engine, QDMA. QDMA provides not only memory mapped DMA which moves data between host memory and board memory, but also stream DMA which moves data between host memory and kernel directly. XDMA memory mapped DMA APIs are also supported on QDMA. New stream APIs are provided here for preview and may be revised in a future release. These can only be used with platforms with QDMA engine under the hood. The higher level OpenCL based streaming APIs offer more refined interfaces and compatibility between releases.
Performance Monitoring Operations
These functions are used to read and write to the performance monitoring infrastructure. OpenCL runtime will be using the BUFFER MANAGEMNT APIs described above to manage OpenCL buffers. It would use these functions to initialize and sample the performance monitoring on the card. Note that the offset is wrt the address space