XMA Examples for the Xilinx Video SDK

The examples in the examples/xma folder illustrate how C-based applications can natively interact with Xilinx devices using the Xilinx Video SDK plugins and the XMA (Xilinx Media Accelerator) APIs. The examples are for advanced use-cases where using the FFmpeg command-line interface is not appropriate.

Detailed documentation on the Xilinx Video SDK plugin interface and the XMA APIs can be found in the C API Programming Guide section of the documentation.

Build and Test Instructions

To build and test the sample XMA applications, the Xilinx Video SDK packages must be installed on the server, and the Xilinx devices must to be flashed with the shell provided in the release package.

To build and test all the XMA applications at once, run the following command from within the ./examples/xma directory:

source ./setup.sh

This script sets up the environment, builds all the applications and then runs a simple test to validate that each of the applications works correctly. The executables are placed in the build directory of the corresponding application subfolder.

To build and test each application individually, refer to the sections below.


XMA Decoder App

The examples/xma/decode_only folder contains a standalone decoder application using the C APIs (XMA) of the Xilinx Video SDK to interact with the Xilinx video accelerators.

Decoder Build Instructions

Export the XRT and XRM environment variables needed to build the decoder application:

source /opt/xilinx/xrt/setup.sh
source /opt/xilinx/xrm/setup.sh

In the exampes/xma/decode_only directory, do the following:

make clean
make

The resulting executable is placed in the build directory:

./build/u30_xma_decode

Decoder Test Instructions

The XMA decoder application supports most of the decode options supported by FFmpeg. The XMA application supports only NV12 semi-planar input and outputs NV12 encoded elementary stream. It ingests an H.264 or H.265 encoded file and utilizes hardware acceleration to get the decoded output.

Decoder Usage

Before running decoding application, source the xcdr environment:

source /opt/xilinx/xcdr/setup.sh

Running the decoder app with the --help option will print the complete list of options:

./build/u30_xma_decode --help
This is a standalone xma decoder app. It ingests an h264 or h265 
encoded file and utilizes hardware acceleration to get the decoded 
output.

Usage: 
	./u30_xma_decode [options] -i <input-file> -c:v <codec-type> 
	[codec_options] -o <output-file>

Arguments:
	--help                     Print this message and exit
	-log <level>               Specify the log level
	-d <device-id>             Specify a device on which to run.
	                           Default: 0

Input Arguments:

	-stream_loop <loop-count>  Number of times to loop the input
	                           file
	-i <input-file>            Input file to be used

Codec Arguments:

	-c:v <codec>               Specify H264 or H265 decoding. 
	                           (mpsoc_vcu_h264, mpsoc_vcu_hevc)
	-low_latency               Should low latency decoding be used
	-entropy_buf_cnt <count>   Specify number of internal entropy
	                           buffers. [2-10], default: 2
	-latency_logging           Log latency information to syslog
	-splitbuff_mode            Configure decoder in split/unsplit
	                           input buffer mode
	-frames <frame-count>      Number of frames to be processed.
	-o <file>                  File to which output is written.

Sample Decoder Command

Decode an h264 encoded file:

./build/u30_xma_decode  -i ~/CSGO_1920x1080_5000kbps.264 -c:v mpsoc_vcu_h264 -o ./CSGO_1920x1080.NV12

XMA Encoder App

The examples/xma/encode_only folder contains a standalone encoder application using the C APIs (XMA) of the Xilinx Video SDK to interact with the Xilinx video accelerators.

Encoder Build Instructions

Export the XRT and XRM environment variables needed to build the encoder application:

source /opt/xilinx/xrt/setup.sh
source /opt/xilinx/xrm/setup.sh

In the exampes/xma/encode_only directory, do the following:

make clean
make

The resulting executable is placed in the build directory:

./build/u30_xma_enc

Encoder Test Instructions

The encoder XMA application supports most of the encoder options supported by FFmpeg. The XMA application supports only YUV 4:2:0 semi-planar input and outputs H.264/HEVC encoded elementary stream.

Encoder Usage

Before running encoder application, source the xcdr environment:

source /opt/xilinx/xcdr/setup.sh

Running the encoder app with the -help option will print the complete list of options:

./u30_xma_enc -help
 XMA Encoder App Usage: 
	./program [input options] -i input-file -c:v <codec-option>  [encoder options] -o <output-file>

Arguments:

	--help                     Print this message and exit.
	-d <device-id>             Specify a device on which the encoder 
	                           to run. Default: 0
	-frames <frame-count>      Number of frames to be processed.

Input options:

	-stream_loop <loop-count>  Number of times to loop the input YUV 
	                           file.
	-w <width>                 Width of YUV input. 
	-h <height>                Height of YUV input. 
	-pix_fmt <pixel-format>    Pixel format of the input file (yuv420p / nv12). 
	                           Default input file format will be chosen as nv12. 
	-i <input-file>            Name and path of input YUV file 

Codec option:
	-c:v <codec>               Encoder codec to be used. Supported 
	                           are mpsoc_vcu_hevc, mpsoc_vcu_h264 
Encoder params: 
	-b:v <bitrate>             Bitrate can be given in Kbps or Mbps 
	                           or bits i.e., 5000000, 5000K, 5M. 
	                           Default is 200kbps 
	-fps <fps>                 Input frame rate. Default is 25. 
	-g <intraperiod>           Intra period. Default is 12. 
	-control-rate <mode>       Rate control mode. Supported are 0 
	                           to 3, default is 1.
	-max-bitrate <bitrate>     Maximum bit rate. Supported are 0 to 
	                           350000000, default is 5000 
	-slice-qp <qp>             Slice QP. Supported are -1 to 51, 
	                           default is -1 
	-min-qp <qp>               Minimum QP. Supported are 0 to 51, 
	                           default is 0. 
	-max-qp <qp>               Maximum QP. Supported values are 0 
	                           to 51, default is 51. 
	-bf <frames>               Number of B frames. Supported are 0 
	                           to 7, default is 2. 
	-periodicity-idr <value>   IDR picture frequency. Supported are 
	                           0 to UINT32_MAX, default is 
	                           UINT32_MAX. 
	-profile <value>           Encoder profile. 
	           For HEVC, supported are 0 or main and 1 or main-intra. 
	                           Default is 0/main. 
	                           ENC_HEVC_MAIN - 0 or main. 
	                           ENC_HEVC_MAIN_INTRA - 1 or main-intra. 
	           For H264, supported are 66 or baseline, 77 or main, 
	                           100 or high, default is 100/high. 
	                           ENC_H264_BASELINE - 66 or baseline. 
	                           ENC_H264_MAIN - 77 or main. 
	                           ENC_H264_HIGH - 100 or high. 
	-level <value>             Encoder level. 
	                           For HEVC, supported are 10 to 51, 
	                           default is 50. 
	                           For H264, supported are 10 to 52, 
	                           default is 50. 
	-slices <value>            Number of slices per frame. Supported 
	                           are 1 to 68, default is 1. 
	-qp-mode <mode>            QP mode. Supported are 0, 1, and 2, 
	                           default is 1. 
	-aspect-ratio <value>      Aspect ratio. Supported values are 0 
	                           to 3, default is 0. 
	-scaling-list <0/1>        Scaling list. Enable/Disable, 
	                           default enable. 
	-lookahead-depth <value>   Lookahead depth. Supported are 0 to 
	                           20, default is 0. 
	-temporal-aq <0/1>         Temporal AQ. Enable/Disable, 
	                           default disable. 
	-spatial-aq <0/1>          Spatial AQ. Enable/Disable, 
	                           default disable. 
	-spatial-aq-gain <value>   Spatial AQ gain. Supported are 0 to 
	                           100, default is 50. 
	-cores <value>             Number of cores to use, supported are 
	                           0 to 4, default is 0. 
	-tune-metrics <0/1>        Tunes MPSoC H.264/HEVC encoder's video 
	                           quality for objective metrics, default 
	                           disable. 
	-latency_logging <0/1>     Enable latency logging in syslog.
	-o <file>                  File to which output is written.

Sample Encoder Commands

H.264 encoding, best objective score, low latency:

./u30_xma_enc -w 1920 -h 1080  -i input_1080sp.yuv -c:v mpsoc_vcu_h264 -fps 60 -b:v 5000 -profile 2 -level 42 -g 120 -periodicity-idr 120 -qp-mode 0 -scaling-list 0 -bf 0 -o out1.264

H.264 encoding, best visual score, low latency:

./u30_xma_enc -w 1920 -h 1080  -i input_1080sp.yuv -c:v mpsoc_vcu_h264 -fps 60 -b:v 5000 -profile 2 -level 42 -g 120 -periodicity-idr 120 -qp-mode 1 -bf 0 -o out2.264

HEVC encoding, best visual score, low latency:

./u30_xma_enc -w 1920 -h 1080  -i input_1080sp.yuv -c:v mpsoc_vcu_hevc -fps 60 -b:v 4000 -g 120 -periodicity-idr 120 -qp-mode 1 -bf 0 -o out1.265

HEVC encoding, best visual score, normal latency:

./u30_xma_enc -w 1920 -h 1080  -i input_1080sp.yuv -c:v mpsoc_vcu_hevc -fps 60 -b:v 4000 -g 120 -periodicity-idr 120 -qp-mode 3 -bf 1 -lookahead-depth 20 -temporal-aq 1 -spatial-aq 1 -o out2.265

For more examples, refer to the encode_only/test/scripts/enc_app_test.sh script.

Limitations of the Encoder App

  • The encoder application assumes that the input YUV file is in NV12 format.

  • The application outputs only H264/HEVC elementary streams.

  • The application supports multi-process but not multi-channel.


XMA Scaler App

The examples/xma/scale_only folder contains a standalone scaler application using the C APIs (XMA) of the Xilinx Video SDK to interact with the Xilinx video accelerators.

Scaler Build Instructions

Export the XRT and XRM environment variables needed to build the scaler application:

source /opt/xilinx/xrt/setup.sh
source /opt/xilinx/xrm/setup.sh

In the exampes/xma/scale_only directory, do the following:

make clean
make

The resulting executable is placed in the build directory:

./build/u30_xma_scale

Scaler Test Instructions

The scaler XMA application supports most of the scaler options supported by FFmpeg. The XMA application supports only NV12 semi-planar input and outputs NV12 scaled elementary stream.

Scaler Usage

Before running scaler application, source the xcdr environment:

source /opt/xilinx/xcdr/setup.sh

Below is the list of all the options of the scaler app:

This program ingests an NV12 input file and utilizes hardware acceleration to scale to various resolutions. 

Usage:
	./u30_xma_scale [options] -w <input-width> -h <input-height> -i 
	<input-file> [scaler_options] -w <output-1-width> -h 
	<output-1-height> -o <output-1-file> -w ...

Arguments:
	--help                     Print this message and exit
	-log <level>               Specify the log level
	-d <device-id>             Specify a device on which to run.
	                           Default: 0

Input Arguments:

	-stream_loop <loop-count>  Number of times to loop the input
	                           file
	-w <width>                 Specify the input's width
	-h <height>                Specify the input's height
	-pix_fmt <fmt>             Specify the input's pixel format.
	                           nv12 is the only format supported.
	-fps <frame-rate>          Frame rate. Used for scaler load
	                           calculation.
	-i <input-file>            Input file to be used

Output Arguments:
	-coeff_load <load>         Specify the coefficient load. 0 Auto 
	                           (default), 1 static, 2 FilterCoef.txt.
	-enable-pipeline           Enable scaler pipeline
	-rate <half/full>          Set the rate to half. Half rate drops 
	                           frames to reduce resource usage. 
	                           Default: full.
	-enable-latency-logging    Enable latency logging
	-w <width>                 Specify the output's width
	-h <height>                Specify the output's height
	-frames <frame-count>      Number of frames to be processed.
	-o <file>                  File to which output is written.

Sample Scaler Command

Scale 1080p nv12 to 720p, 480p, 360p, and 240p nv12:

./u30_xma_scale -w 1920 -h 1080 -i ~/Kimono1_1920x1080_24.yuvNV12 -w 1280 -h \
      720 -o Kimono_1280x720.NV12 -w 852 -h 480 -o Kimono_852x480.NV12 -w \
      640 -h 360 -o Kimono_640x360.NV12 -w 480 -h 240 -o Kimono_480x240.NV12

Limitations of the Scaler App

  • The scaler supports nv12 format only

  • There may be a performance degradation relative to ffmpeg with a high number of outputs/processes.


XMA Transcoder App

The examples/xma/transcode folder contains a standalone transcoder application using the C APIs (XMA) of the Xilinx Video SDK to interact with the Xilinx video accelerators.

Transcoder Build Instructions

Export the XRT and XRM environment variables needed to build the transcoder application:

source /opt/xilinx/xrt/setup.sh
source /opt/xilinx/xrm/setup.sh

In the exampes/xma/transcode directory, do the following:

make clean
make

The resulting executable is placed in the build directory:

./build/u30_xma_transcode

Transcoder Test Instructions

The transcoder XMA application supports most of the options supported by ffmpeg. The XMA application supports only elementary H.264 and HEVC encoded stream input and outputs H.264/HEVC encoded elementary stream.

Transcoder Usage

Before running transcoder application, source the xcdr environment:

source /opt/xilinx/xcdr/setup.sh

Running the transcoder app with the --help option will print the complete list of options:

./u30_xma_transcode --help
 XMA Transcoder App Usage: 
	./program [generic options] -c:v <decoder codec> [decoder options]  -i input-file -multiscale_xma -outputs [num] [Scaler options]  -c:v <encoder codec> [encoder options] -o <output-file>  -c:v <encoder codec> [encoder options] -o <output-file>  -c:v <encoder codec> [encoder options] -o <output-file>..... 

Arguments:

	--help                     Print this message and exit.
	-d <device-id>             Specify a device on which the 
	                           transcoder to run. Default: 0
	-stream_loop <loop-count>  Number of times to loop the input file 
	-frames <frame-count>      Number of input frames to be processed 
	                           
Decoder options:

	-c:v <codec>               Decoder codec to be used. Supported 
	                           are mpsoc_vcu_hevc, mpsoc_vcu_h264 
	-low-latency <0/1>         Low latency for decoder. Default 
	                           disabled 
	-latency_logging <0/1>     Latency logging for decoder. Default 
	                           disabled 
	-i <input-file>            Name and path of input H.264/HEVC file 
	                           
Scaler options:

	-multiscale_xma            Name of the ABR scaler filter 
	-num-output <value>        Number of output files from scaler 
	-out_1_width <width>       Width of the scaler output channel 1 
	-out_1_height <height>     Height of the scaler output channel 1 
	-out_1_rate <full/half>    Full of Half rate for output channel 1 
	-out_2_width <width>       Width of the scaler output channel 2 
	-out_2_height <height>     Height of the scaler output channel 2 
	-out_2_rate <full/half>    Full of Half rate for output channel 2 
	-out_3_width <width>       Width of the scaler output channel 3 
	-out_3_height <height>     Height of the scaler output channel 3 
	-out_3_rate <full/half>    Full of Half rate for output channel 3 
	-out_4_width <width>       Width of the scaler output channel 4 
	-out_4_height <height>     Height of the scaler output channel 4 
	-out_4_rate <full/half>    Full of Half rate for output channel 4 
	-out_5_width <width>       Width of the scaler output channel 5 
	-out_5_height <height>     Height of the scaler output channel 5 
	-out_5_rate <full/half>    Full of Half rate for output channel 5 
	-out_6_width <width>       Width of the scaler output channel 6 
	-out_6_height <height>     Height of the scaler output channel 6 
	-out_6_rate <full/half>    Full of Half rate for output channel 6 
	-out_7_width <width>       Width of the scaler output channel 7 
	-out_7_height <height>     Height of the scaler output channel 7 
	-out_7_rate <full/half>    Full of Half rate for output channel 7 
	-out_8_width <width>       Width of the scaler output channel 8 
	-out_8_height <height>     Height of the scaler output channel 8 
	-out_8_rate <full/half>    Full of Half rate for output channel 8 
	-latency_logging <0/1>     Latency logging for scaler. Default 
	                           disabled 
Encoder options:

	-c:v <codec>               Encoder codec to be used. Supported 
	                           are mpsoc_vcu_hevc, mpsoc_vcu_h264 
	-b:v <bitrate>             Bitrate can be given in Kbps or Mbps 
	                           or bits i.e., 5000000, 5000K, 5M. 
	                           Default is 200kbps 
	-fps <fps>                 Input frame rate. Default is 25. 
	-g <intraperiod>           Intra period. Default is 12. 
	-control-rate <mode>       Rate control mode. Supported are 0 
	                           to 3, default is 1.
	-max-bitrate <bitrate>     Maximum bit rate. Supported are 0 to 
	                           350000000, default is 5000 
	-slice-qp <qp>             Slice QP. Supported are -1 to 51, 
	                           default is -1 
	-min-qp <qp>               Minimum QP. Supported are 0 to 51, 
	                           default is 0. 
	-max-qp <qp>               Maximum QP. Supported values are 0 
	                           to 51, default is 51. 
	-bf <frames>               Number of B frames. Supported are 0 
	                           to 7, default is 2. 
	-periodicity-idr <value>   IDR picture frequency. Supported are 
	                           0 to UINT32_MAX, default is 
	                           UINT32_MAX. 
	-profile <value>           Encoder profile. 
	           For HEVC, supported are 0 or main and 1 or main-intra. 
	                           Default is 0. 
	           For H264, supported are 66 or baseline, 77 or main 
	                           and 100 or high. Default is 100 
	-level <value>             Encoder level. 
	                           For HEVC, supported are 10 to 51, 
	                           default is 50. 
	                           For H264, supported are 10 to 52, 
	                           default is 50. 
	-slices <value>            Number of slices per frame. Supported 
	                           are 1 to 68, default is 1. 
	-qp-mode <mode>            QP mode. Supported are 0, 1, and 2, 
	                           default is 1. 
	-aspect-ratio <value>      Aspect ratio. Supported values are 0 
	                           to 3, default is 0. 
	-scaling-list <0/1>        Scaling list. Enable/Disable, 
	                           default enable. 
	-lookahead-depth <value>   Lookahead depth. Supported are 0 to 
	                           20, default is 0. 
	-temporal-aq <0/1>         Temporal AQ. Enable/Disable, 
	                           default disable. 
	-spatial-aq <0/1>          Spatial AQ. Enable/Disable, 
	                           default disable. 
	-spatial-aq-gain <value>   Spatial AQ gain. Supported are 0 to 
	                           100, default is 50. 
	-cores <value>             Number of cores to use, supported are 
	                           0 to 4, default is 0. 
	-tune-metrics <0/1>        Tunes MPSoC H.264/HEVC encoder's video 
	                           quality for objective metrics, default 
	                           disable. 
	-latency_logging <0/1>     Enable latency logging in syslog.
	-o <file>                  File to which output is written.

Sample Transcoder Commands

H.264 to HEVC ABR Transcoder:

./u30_xma_transcode -c:v mpsoc_vcu_h264 -i input_1080p.264 -multiscale_xma -num-output 4 -out_1_width 1280 -out_1_height 720 -out_2_width 848 -out_2_height 480 -out_3_width 640 -out_3_height 360 -out_4_width 288 -out_4_height 160 \
-c:v mpsoc_vcu_hevc -b:v 4000K  -o out1_test1.265 -c:v mpsoc_vcu_hevc -b:v 3000K -o out1_test2.265 -c:v mpsoc_vcu_hevc -b:v 2500K -o out1_test3.265 -c:v mpsoc_vcu_hevc -b:v 1250K -o out1_test4.265 -c:v mpsoc_vcu_hevc -b:v 625K -o out1_test5.265

HEVC ABR Transcoder with Look-Ahead:

./u30_xma_transcode -c:v mpsoc_vcu_hevc -i input_1080p.265 -multiscale_xma -num-output 4 -out_1_width 1280 -out_1_height 720 -out_2_width 848 -out_2_height 480 -out_3_width 640 -out_3_height 360 -out_4_width 288 -out_4_height 160 \
-c:v mpsoc_vcu_hevc -b:v 4000K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test1.265 -c:v mpsoc_vcu_hevc -b:v 3000K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test2.265 \
-c:v mpsoc_vcu_hevc -b:v 2500K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test3.265 -c:v mpsoc_vcu_hevc -b:v 1250K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test4.265 \
-c:v mpsoc_vcu_hevc -b:v 625K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test5.265

H.264 to HEVC ABR Transcoder with Scaler Pipeline:

./u30_xma_transcode -c:v mpsoc_vcu_h264 -i input_1080p.264 -multiscale_xma -num-output 4 -out_1_width 1280 -out_1_height 720 -out_2_width 848 -out_2_height 480 -out_3_width 640 -out_3_height 360 -out_4_width 288 -out_4_height 160 \
-c:v mpsoc_vcu_hevc -b:v 4000K  -o out3_sc_test1.265 -c:v mpsoc_vcu_hevc -b:v 3000K -o out3_sc_test2.265 -c:v mpsoc_vcu_hevc -b:v 2500K -o out3_sc_test3.265 -c:v mpsoc_vcu_hevc -b:v 1250K -o out3_sc_test4.265 -c:v mpsoc_vcu_hevc -b:v 625K -o out3_sc_test5.265

For more examples, refer to the transcode/test/scripts/transcoder_app_test.sh script.

Limitations of the Transcoder App

  • The transcoder application supports only elementary H264/HEVC encoded streams as input file. It cannot parse container formats like MP4, AVI, etc.

  • The transcoder outputs only H264/HEVC elementary streams.

  • The lookahead depth should be same for all the channels.