XMA Upper Edge API Library¶

XMA Application Interface

The interface used by stand-alone XMA applications or plugins

int32_t xma_initialize(XmaXclbinParameter *devXclbins, int32_t num_parms)¶: Initialie XMA Library and devices

Parameters

XmaXclbinParameter * devXclbins: array of device index and full path of xclbin to program the device with.
int32_t num_parms: Number of elements in above array input

Description

This is the entry point routine for utilzing the XMA library and must be the first call within any application before calling any other XMA APIs. Each device specified will be programmed with provided xclbin(s).

Return

XMA_SUCCESS or XMa_ERROR

XMA upper edge APIs can be divided based on the following five types of kernel

Encoder

Decoder

Scaler

Filter

Kernel (generic type)

Encoder¶

Introduction

The Xilinx media encoder API is comprised of two distinct interfaces: one interface for an external framework such as FFmpeg or a proprietary multi-media framework and the plugin interface used by Xilinx accelerator developers. This section illustrates both interfaces starting with the external framework view and moving on to the plugin developers view.

The external interface to the Xilinx video encoder is comprised of the following functions:

xma_enc_session_create()

xma_enc_session_destroy()

xma_enc_session_send_frame()

xma_enc_session_recv_data()

A media framework (such as FFmpeg) is responsible for creating an encoder session. The encoder session contains state information used by the encoder plugin to manage the hardware associated with a Xilinx accelerator device. Prior to creating an encoder session the media framework is responsible for initializing the XMA using the function xma_initialize(). The initialize function should be called by the media framework early in the framework initialization to ensure that all resources have been configured. Ideally, the xma_initialize() function should be called from the main() function of the media framework in order to guarantee it is only called once.

enum XmaEncoderType: Value specifies the precise type of encoder kernel requested

Constants

XMA_H264_ENCODER_TYPE: undescribed
XMA_HEVC_ENCODER_TYPE: undescribed
XMA_VP9_ENCODER_TYPE: undescribed
XMA_AV1_ENCODER_TYPE: undescribed
XMA_COPY_ENCODER_TYPE: undescribed
XMA_JPG_ENCODER_TYPE: undescribed
XMA_MULTI_ENCODER_TYPE: undescribed

struct XmaEncoderProperties: Properities used to specify which encoder is requested and how the encoder should be initalized by the plugin driver

Definition

struct XmaEncoderProperties {
};

Members

XmaEncoderSession *xma_enc_session_create(XmaEncoderProperties *enc_props)¶: This function creates an encoder session and must be called prior to encoding a frame. A session reserves hardware resources for the duration of a video stream. The number of sessions allowed depends on a number of factors that include: resolution, frame rate, bit depth, and the capabilities of the hardware accelerator.

Parameters

XmaEncoderProperties * enc_props: Pointer to a XmaEncoderProperties structure that contains the key configuration properties needed for finding available hardware resource.

Return

Not NULL on success

NULL on failure

Note

session create & destroy are thread safe APIs

int32_t xma_enc_session_destroy(XmaEncoderSession *session)¶: This function destroys an encoder session that was previously created with the xm_enc_session_create function.

Parameters

XmaEncoderSession * session: Pointer to XmaEncoderSession created with xma_enc_session_create

Return

XMA_SUCCESS on success

XMA_ERROR on failure.

Note

session create & destroy are thread safe APIs

int32_t xma_enc_session_send_frame(XmaEncoderSession *session, XmaFrame *frame)¶: This function invokes plugin->send_frame fucntion assigned to this session which handles sending data to the hardware decoder.

Parameters

XmaEncoderSession * session: Pointer to session created by xm_enc_sesssion_create
XmaFrame * frame: Pointer to a frame to be encoded. If the encoder has buffered input, the input will need to be flushed. To do so, an XmaFrame with a null pointer to the first data buffer will need to be sent until XMA_EOS is received (XmaFrame.data[0].buffer = NULL).

Description

Plugin and media framework (like FFMPEG) may handle events like errors, no empty lookup buffer, no output data, end of input stream, etc using XMA_FLUSH_AGAIN, XMA_TRY_AGAIN, XMA_SEND_MORE_DATA, XMA_EOS, XMA_ERROR, etc return codes

Return

XMA_SUCCESS on success; indicates that sufficient data has been received to begin producing output

XMA_SEND_MORE_DATA if additional frames are needed before output can be received

XMA_ERROR on error

int32_t xma_enc_session_recv_data(XmaEncoderSession *session, XmaDataBuffer *data, int32_t *data_size)¶: This function invokes plugin->recv_data assigned to this session which handles obtaining output data from the hardware encoder. This function returns a data buffer along with the length of the buffer if one is available. This function is called after calling the function xm_enc_session_send_frame. If a data buffer is not ready to be returned, this function returns -1 and sets the length of the data buffer to 0. In addition, the data buffer pointer is set to NULL. If a data buffer is ready, the data_size will be set with a non-zero value and a pointer to the data buffer will be set to a non-NULL value.

Parameters

XmaEncoderSession * session: Pointer to session created by xm_enc_sesssion_create
XmaDataBuffer * data: Pointer to a data buffer containing encoded data
int32_t * data_size: Pointer to hold the size of the data buffer returned

Return

XMA_SUCCESS on success.

XMA_EOS when all data has been flushed from the encoder

XMA_ERROR on error.

Decoder¶

Introduction

The Xilinx media decoder API is comprised of two distinct interfaces: one interface for an external framework such as FFmpeg or a proprietary multi-media framework and the plugin interface used by Xilinx accelerator developers. This section illustrates both interfaces starting with the external framework view and moving on to the plugin developers view.

The external interface to the Xilinx video decoder is comprised of the following functions:

xma_dec_session_create()

xma_dec_session_destroy()

xma_dec_session_send_data()

xma_dec_session_get_properties()

xma_dec_session_recv_frame()

A media framework (such as FFmpeg) is responsible for creating a decoder session. The decoder session contains state information used by the decoder plugin to manage the hardware associated with a Xilinx accelerator device. Prior to creating an decoder session the media framework is responsible for initializing the XMA using the function xma_initialize(). The initialize function should be called by the media framework early in the framework initialization to ensure that all resources have been configured. Ideally, the xma_initialize() function should be called from the main() function of the media framework in order to guarantee it is only called once.

enum XmaDecoderType: A decoder from this list forms part of a request for a specific decoder when creating a decoder session via xma_dec_session_create.

Constants

XMA_H264_DECODER_TYPE: undescribed
XMA_HEVC_DECODER_TYPE: undescribed
XMA_VP9_DECODER_TYPE: undescribed
XMA_AV1_DECODER_TYPE: undescribed
XMA_JPG_DECODER_TYPE: undescribed
XMA_MULTI_DECODER_TYPE: undescribed

struct XmaDecoderProperties: Properities used to specify which decoder is requested and how the decoder should be initalized by the plugin driver

Definition

struct XmaDecoderProperties {
};

Members

XmaDecoderSession *xma_dec_session_create(XmaDecoderProperties *dec_props)¶: This function creates a decoder session and must be called prior to decoding data. A session reserves hardware resources for the duration of a video stream. The number of sessions allowed depends on a number of factors that include: resolution, frame rate, bit depth, and the capabilities of the hardware accelerator.

Parameters

XmaDecoderProperties * dec_props: Pointer to a XmaDecoderProperties structure that contains the key configuration properties, device & cu index, etc

Return

Not NULL on success

NULL on failure

Note

session create & destroy are thread safe APIs

int32_t xma_dec_session_destroy(XmaDecoderSession *session)¶: This function destroys a decoder session that was previously created with the xma_dec_session_create function.

Parameters

XmaDecoderSession * session: Pointer to XmaDecoderSession created with xma_dec_session_create

Return

XMA_SUCCESS on success

XMA_ERROR on failure.

Note

session create & destroy are thread safe APIs

int32_t xma_dec_session_send_data(XmaDecoderSession *session, XmaDataBuffer *data, int32_t *data_used)¶: This function invokes plugin->send_data fucntion assigned to this session which handles sending data to the hardware decoder.

Parameters

XmaDecoderSession * session: Pointer to session created by xma_dec_sesssion_create
XmaDataBuffer * data: Pointer to a data buffer to be decoded
int32_t * data_used: Pointer to an integer to receive the amount of data used

Description

Plugin and media framework (like FFMPEG) may handle events like errors, no empty lookup buffer, no output data, end of input stream, etc using XMA_FLUSH_AGAIN, XMA_TRY_AGAIN, XMA_SEND_MORE_DATA, XMA_EOS, XMA_ERROR, etc return codes

Return

XMA_SUCCESS on success.

XMA_ERROR on error.

int32_t xma_dec_session_get_properties(XmaDecoderSession *dec_session, XmaFrameProperties *fprops)¶: This function invokes plugin->get_properties assigned to this session which returns properties of the hardware decoder.

Parameters

XmaDecoderSession * dec_session: Pointer to session created by xma_dec_sesssion_create
XmaFrameProperties * fprops: Pointer to a frame properties structure to be filled in

Return

XMA_SUCCESS on success.

XMA_ERROR on error.

int32_t xma_dec_session_recv_frame(XmaDecoderSession *session, XmaFrame *frame)¶: This function invokes plugin->recv_frame assigned to this session which handles obtaining output frame from the hardware decoder. This function returns a frame if one is available.

Parameters

XmaDecoderSession * session: Pointer to session created by xma_dec_sesssion_create
XmaFrame * frame: Pointer to a frame containing decoded data

Description

This function is called after calling the function xma_dec_session_send_data. If a frame is not ready to be returned, plugin function should return -1. In addition, the

frame pointer should be set to NULL. If a frame is ready, a pointer to the frame will be set to a non-NULL value.

Return

XMA_SUCCESS on success.

XMA_ERROR on error.

Scaler¶

Introduction

The Xilinx media scaler API is comprised of two distinct interfaces: one interface for an external framework such as FFmpeg or a proprietary multi-media framework and the plugin interface used by Xilinx accelerator developers. This section illustrates both interfaces starting with the external framework view and moving on to the plugin developers view.

The external interface to the Xilinx video scaler is comprised of the following functions:

xma_scaler_session_create()

xma_scaler_session_destroy()

xma_scaler_session_send_frame()

xma_scaler_session_recv_frame_list()

A media framework (such as FFmpeg) is responsible for creating a scaler session. The scaler session contains state information used by the scaler plugin to manage the hardware associated with a Xilinx accelerator device. Prior to creating a scaler session the media framework is responsible for initializing the XMA using the function ref xma_initialize(). The initialize function should be called by the media framework early in the framework initialization to ensure that all resources have been configured. Ideally, the ref xma_initialize() function should be called from the main() function of the media framework in order to guarantee it is only called once.

enum XmaScalerType: Specific type of scaler to request

Constants

XMA_BICUBIC_SCALER_TYPE: undescribed
XMA_BILINEAR_SCALER_TYPE: undescribed
XMA_POLYPHASE_SCALER_TYPE: undescribed

struct XmaScalerInOutProperties: Properties which shall be used to specify configuration of kernel input or output as applicable

Definition

struct XmaScalerInOutProperties {
};

Members

struct XmaScalerFilterProperties: Filter coefficients to be used by kernel

Definition

struct XmaScalerFilterProperties {
};

Members

struct XmaScalerProperties: Properties structure used to request filter type and vendor as well as the manner in which it should be inialized by the plugin

Definition

struct XmaScalerProperties {
};

Members

void xma_scaler_default_filter_coeff_set(XmaScalerFilterProperties *props)¶: This helper function sets the default horizontal and vertical filter coefficients for a polyphase filter bank.

Parameters

XmaScalerFilterProperties * props: Pointer to a XmaScalerFilterProperties structure that will contain the filter coefficients.

XmaScalerSession *xma_scaler_session_create(XmaScalerProperties *props)¶: This function creates a scaler session and must be called prior to scaling a frame. A session reserves hardware resources for the duration of a video stream. The number of sessions allowed depends on a number of factors that include: resolution, frame rate, bit depth, and the capabilities of the hardware accelerator.

Parameters

XmaScalerProperties * props: undescribed

Description

props Pointer to a XmaScalerProperties structure that contains the key configuration properties needed for finding available hardware resource.

Return

Not NULL on success

NULL on failure

Note

session create & destroy are thread safe APIs

int32_t xma_scaler_session_destroy(XmaScalerSession *session)¶: This function destroys an scaler session that was previously created with the xma_scaler_session_create() function.

Parameters

XmaScalerSession * session: Pointer to XmaScalerSession created with xma_scaler_session_create

Return

XMA_SUCCESS on success

XMA_ERROR on failure.

Note

session create & destroy are thread safe APIs

int32_t xma_scaler_session_send_frame(XmaScalerSession *session, XmaFrame *frame)¶: This function invokes plugin->send_frame fucntion assigned to this session which handles sending data to the hardware scaler.

Parameters

XmaScalerSession * session: Pointer to session created by xma_scaler_sesssion_create
XmaFrame * frame: Pointer to a frame to be scaled. If the scaler is buffering input, then an XmaFrame with a NULL data buffer pointer to the first data buffer must be sent to flush the filter and to indicate that no more data will be sent: XmaFrame.data[0].buffer = NULL The application must then check for XMA_FLUSH_AGAIN for each such call when flushing the last few frames. When XMA_EOS is returned, no new data may be collected from the scaler.

Description

This function sends a frame to the hardware scaler. If a frame: buffer is not available, the plugin function should block.

Return

XMA_SUCCESS on success and the scaler is ready to produce output

XMA_SEND_MORE_DATA if the scaler is buffering input frames

XMA_FLUSH_AGAIN when flushing scaler with a null frame

XMA_EOS when the scaler has been flushed of all residual frames

XMA_ERROR on error

int32_t xma_scaler_session_recv_frame_list(XmaScalerSession *session, XmaFrame **frame_list)¶: This function invokes plugin->recv_frame_list assigned to this session which handles obtaining list of output frames from the hardware encoder. This function is called after calling the function xma_scaler_session_send_frame. If a data buffer is not ready to be returned, the plugin function should blocks.

Parameters

XmaScalerSession * session: Pointer to session created by xma_scaler_sesssion_create
XmaFrame ** frame_list: Pointer to a list of XmaFrame structures

Return

XMA_SUCCESS on success

XMA_ERROR on error

Filter¶

enum XmaFilterType: Identifier specifying precise type of video filter during session creation

Constants

XMA_2D_FILTER_TYPE: undescribed

struct XmaFilterPortProperties: Properties necessary for specifying how an input or output port should be configured by the plugin.

Definition

struct XmaFilterPortProperties {
};

Members

struct XmaFilterProperties: Properties necessary for specifying which filter kernel to select and how it should be configured by the plugin.

Definition

struct XmaFilterProperties {
};

Members

XmaFilterSession *xma_filter_session_create(XmaFilterProperties *props)¶: This function creates a filter session and must be called prior to filtering a frame. A session reserves hardware resources for the duration of a video stream. The number of sessions allowed depends on a number of factors that include: resolution, frame rate, bit depth, and the capabilities of the hardware accelerator.

Parameters

XmaFilterProperties * props: Pointer to a XmaFilterProperties structure that contains the key configuration properties needed for finding available hardware resource.

Return

Not NULL on success

NULL on failure

Note

session create & destroy are thread safe APIs

int32_t xma_filter_session_destroy(XmaFilterSession *session)¶: This function destroys an filter session that was previously created with the ref xma_filter_session_create function.

Parameters

XmaFilterSession * session: Pointer to XmaFilterSession created with xma_filter_session_create

Return

XMA_SUCCESS on success

XMA_ERROR on failure.

Note

session create & destroy are thread safe APIs

int32_t xma_filter_session_send_frame(XmaFilterSession *session, XmaFrame *frame)¶: This function invokes plugin->send_frame fucntion assigned to this session which handles sending frames to the hardware decoder.

Parameters

XmaFilterSession * session: Pointer to session created by xma_filter_session_create
XmaFrame * frame: Pointer to a frame to be filtered. If the filter is buffering input, then an XmaFrame with a NULL data buffer pointer to the first data buffer must be sent to flush the filter and to indicate that no more data will be sent: XmaFrame.data[0].buffer = NULL The application must then check for XMA_FLUSH_AGAIN for each such call when flushing the last few frames. When XMA_EOS is returned, no new data may be collected from the filter.

Return

XMA_SUCCESS on success and the filter is ready to produce output

XMA_SEND_MORE_DATA if the filter is buffering input frames

XMA_FLUSH_AGAIN when flushing filter with a null frame

XMA_EOS when the filter has been flushed of all residual frames

XMA_ERROR on error

int32_t xma_filter_session_recv_frame(XmaFilterSession *session, XmaFrame *frame)¶: This function invokes plugin->recv_frame assigned to this session which handles obtaining output frame with filtered data from the hardware filter. This function is called after calling the function xma_filter_session_send_frame. If a data buffer is not ready to be returned, the plugin function should block.

Parameters

XmaFilterSession * session: Pointer to session created by xma_filter_session_create
XmaFrame * frame: undescribed

Return

XMA_SUCCESS on success.

XMA_ERROR on error.

Kernel¶

Introduction

The kernel interface provides a generic method for controlling a kernel that does not need to send and receive video frame data. While it is possible to use the kernel plugin class for video, the class is intended for cases where only control information is required or the data is not readily classified as typical video frame data.

In most cases specific to video accelerators, the class of session typically fits into one of the followeing categories:

encoder

decoder

filter (with 1 input and 1 output)

ABR scaler (with 1 input and multiple output)

Each of the classes above expect frame data as input/output or both. As a result, these APIs provide a convenient way to send and/or receive frame data. In cases where frame data is not needed or more control over the type of data sent/recevied is needed, the kernel class of plugin may be a better fit. In this case, data is transferred between the host application and the plugin via private_session_data. private_session_data is managed by host application. As a result, it is up to the host application and plugin to decide the meaning/structure of the private_session_data.

enum XmaKernelType: Description of kernel represented by XmaKernel

Constants

XMA_KERNEL_TYPE: undescribed

struct XmaKernelProperties: XmaKernels represent unspecified or custom kernels that may not necessarily fit an existing video kernel type. As such, they may take custom parameters as properties. You should consult the documentation for the kernel plugin to get a list of XmaParameter parameters needed to specify how the kernel should be initalized.

Definition

struct XmaKernelProperties {
};

Members

XmaKernelSession *xma_kernel_session_create(XmaKernelProperties *props)¶: This function creates a kernel session and must be called prior to invoking other kernel session functions. A session reserves hardware resources until session destroy function is called.

Parameters

XmaKernelProperties * props: Pointer to a XmaKernelProperties structure that contains the key configuration properties needed for finding available hardware resource.

Return

Not NULL on success

NULL on failure

Note

session create & destroy are thread safe APIs

int32_t xma_kernel_session_destroy(XmaKernelSession *session)¶: This function destroys a kernel session that was previously created with the xma_kernel_session_create function.

Parameters

XmaKernelSession * session: Pointer to XmaKernelSession created with xma_kernel_session_create

Return

XMA_SUCCESS on success

XMA_ERROR on failure.

Note

session create & destroy are thread safe APIs

int32_t xma_kernel_session_write(XmaKernelSession *session, XmaParameter *param, int32_t param_cnt)¶: This function invokes plugin->write fucntion assigned to this session which handles sending data to the hardware kernel. The meaning of the data is managed by the caller and low-level XMA plugin. This means that the data provided could contain information about how kernel registers are programmed, how device DDR memory is set, or some combination of both.

Parameters

XmaKernelSession * session: Pointer to session created by xm_enc_sesssion_create
XmaParameter * param: Pointer to an XmaParameter list
int32_t param_cnt: Number of parameters provided in the list

Return

XMA_SUCCESS on success.

XMA_ERROR on error.

int32_t xma_kernel_session_read(XmaKernelSession *session, XmaParameter *param, int32_t *param_cnt)¶: This function invokes plugin->read assigned to this session which handles obtaining output data from the hardware kernel. The meaning of the data is managed by the caller and low-level XMA plugin.

Parameters

XmaKernelSession * session: Pointer to session created by xm_enc_sesssion_create
XmaParameter * param: Pointer to an XmaParameter
int32_t * param_cnt: Pointer number of parameters in the list

Return

XMA_SUCCESS on success.

XMA_ERROR on error.