XMA Examples for Xilinx U30

The examples in the examples/xma folder illustrate how C-based applications can natively interact with U30 cards using the U30 plugins and the XMA (Xilinx Media Accelerator) APIs. The examples are for advanced use-cases where using the FFmpeg command-line interface is not appropriate.

Detailed documentation on the U30 plugin interface and the XMA APIs can be found in the C API Programming Guide section of the documentation.

Build and Test Instructions

To build and test the sample XMA applications, the U30 release packages must be installed on the server, and the U30 devices must to be flashed with the shell provided in the release package.

To build and test all the XMA applications at once, run the following command from within the ./examples/xma directory:

source ./setup.sh

This script sets up the environment, builds all the applications and then runs a simple test to validate that each of the applications works correctly. The executables are placed in the build directory of the corresponding application subfolder.

To build and test each application individually, refer to the sections below.

U30 XMA Decoder App

The examples/xma/decode_only folder contains a standalone XMA-based decoder application for U30.

Decoder Build Instructions

Export the XRT and XRM to build the decoder application:

source /opt/xilinx/xrt/setup.sh
source /opt/xilinx/xrm/setup.sh

In the exampes/xma/decode_only directory, do the following:

make clean
make

The resulting executable is placed in the build directory:

./build/u30_xma_decode

Decoder Test Instructions

The XMA decoder application supports most of the decode options supported by FFmpeg. The XMA application supports only NV12 semi-planar input and outputs NV12 encoded elementary stream. It ingests an H.264 or H.265 encoded file and utilizes hardware acceleration to get the decoded output.

Decoder Usage

Before running decoding application, source the xcdr environment:

source /opt/xilinx/xcdr/setup.sh

Running the decoder app with the --help option will print the complete list of options:

./build/u30_xma_decode --help
This is a standalone xma decoder app. It ingests an h264 or h265
encoded file and utilizes hardware acceleration to get the decoded
output.

Usage:
        ./u30_xma_decode [options] -i <input-file> -c:v <codec-type>
        [codec_options] -o <output-file>

Arguments:
        --help                     Print this message and exit
        -log <level>               Specify the log level
        -d <device-id>             Specify a device on which to run.
                                   Default: 0

Input Arguments:

        -stream_loop <loop-count>  Number of times to loop the input
                                   file
        -i <input-file>            Input file to be used

Codec Arguments:

        -c:v <codec>               Specify H264 or H265 decoding.
                                   (mpsoc_vcu_h264, mpsoc_vcu_hevc)
        -low_latency               Should low latency decoding be used
        -entropy_buf_cnt <count>   Specify number of internal entropy
                                   buffers. [2-10], default: 2
        -latency_logging           Log latency information to syslog
        -splitbuff_mode            Configure decoder in split/unsplit
                                   input buffer mode
        -frames <frame-count>      Number of frames to be processed.
        -o <file>                  File to which output is written.

Sample Decoder Command

Decode an h264 encoded file:

./build/u30_xma_decode  -i ~/CSGO_1920x1080_5000kbps.264 -c:v mpsoc_vcu_h264 -o ./CSGO_1920x1080.NV12

U30 XMA Encoder App

The examples/xma/encode_only folder contains a standalone XMA-based encoder application for U30.

Encoder Build Instructions

Export the XRT and XRM to build the encoder application:

source /opt/xilinx/xrt/setup.sh
source /opt/xilinx/xrm/setup.sh

In the exampes/xma/encode_only directory, do the following:

make clean
make

The resulting executable is placed in the build directory:

./build/u30_xma_enc

Encoder Test Instructions

The encoder XMA application supports most of the encoder options supported by FFmpeg. The XMA application supports only YUV 4:2:0 semi-planar input and outputs H.264/HEVC encoded elementary stream.

Encoder Usage

Before running encoder application, source the xcdr environment:

source /opt/xilinx/xcdr/setup.sh

Running the encoder app with the -help option will print the complete list of options:

./u30_xma_enc -help
XMA Encoder App Usage:
        ./program [input options] -i input-file -c:v <codec-option>  [encoder options] -o <output-file>

Arguments:

        --help                     Print this message and exit.
        -d <device-id>             Specify a device on which the encoder
                                   to run. Default: 0
        -frames <frame-count>      Number of frames to be processed.

Input options:

        -stream_loop <loop-count>  Number of times to loop the input YUV
                                   file.
        -w <width>                 Width of YUV input.
        -h <height>                Height of YUV input.
        -pix_fmt <pixel-format>    Pixel format of the input file (yuv420p / nv12).
                                   Default input file format will be chosen as nv12.
        -i <input-file>            Name and path of input YUV file

Codec option:
        -c:v <codec>               Encoder codec to be used. Supported
                                   are mpsoc_vcu_hevc, mpsoc_vcu_h264
Encoder params:
        -b:v <bitrate>             Bitrate can be given in Kbps or Mbps
                                   or bits i.e., 5000000, 5000K, 5M.
                                   Default is 200kbps
        -fps <fps>                 Input frame rate. Default is 25.
        -g <intraperiod>           Intra period. Default is 12.
        -control-rate <mode>       Rate control mode. Supported are 0
                                   to 3, default is 1.
        -max-bitrate <bitrate>     Maximum bit rate. Supported are 0 to
                                   350000000, default is 5000
        -slice-qp <qp>             Slice QP. Supported are -1 to 51,
                                   default is -1
        -min-qp <qp>               Minimum QP. Supported are 0 to 51,
                                   default is 0.
        -max-qp <qp>               Maximum QP. Supported values are 0
                                   to 51, default is 51.
        -bf <frames>               Number of B frames. Supported are 0
                                   to 7, default is 2.
        -periodicity-idr <value>   IDR picture frequency. Supported are
                                   0 to UINT32_MAX, default is
                                   UINT32_MAX.
        -profile <value>           Encoder profile.
                   For HEVC, supported are 0 or main and 1 or main-intra.
                                   Default is 0/main.
                                   ENC_HEVC_MAIN - 0 or main.
                                   ENC_HEVC_MAIN_INTRA - 1 or main-intra.
                   For H264, supported are 66 or baseline, 77 or main,
                                   100 or high, default is 100/high.
                                   ENC_H264_BASELINE - 66 or baseline.
                                   ENC_H264_MAIN - 77 or main.
                                   ENC_H264_HIGH - 100 or high.
        -level <value>             Encoder level.
                                   For HEVC, supported are 10 to 51,
                                   default is 50.
                                   For H264, supported are 10 to 52,
                                   default is 50.
        -slices <value>            Number of slices per frame. Supported
                                   are 1 to 68, default is 1.
        -qp-mode <mode>            QP mode. Supported are 0, 1, and 2,
                                   default is 1.
        -aspect-ratio <value>      Aspect ratio. Supported values are 0
                                   to 3, default is 0.
        -scaling-list <0/1>        Scaling list. Enable/Disable,
                                   default enable.
        -lookahead-depth <value>   Lookahead depth. Supported are 0 to
                                   20, default is 0.
        -temporal-aq <0/1>         Temporal AQ. Enable/Disable,
                                   default disable.
        -spatial-aq <0/1>          Spatial AQ. Enable/Disable,
                                   default disable.
        -spatial-aq-gain <value>   Spatial AQ gain. Supported are 0 to
                                   100, default is 50.
        -cores <value>             Number of cores to use, supported are
                                   0 to 4, default is 0.
        -tune-metrics <0/1>        Tunes MPSoC H.264/HEVC encoder's video
                                   quality for objective metrics, default
                                   disable.
        -latency_logging <0/1>     Enable latency logging in syslog.
        -o <file>                  File to which output is written.

Sample Encoder Commands

H.264 encoding, best objective score, low latency:

./u30_xma_enc -w 1920 -h 1080  -i input_1080sp.yuv -c:v mpsoc_vcu_h264 -fps 60 -b:v 5000 -profile 2 -level 42 -g 120 -periodicity-idr 120 -qp-mode 0 -scaling-list 0 -bf 0 -o out1.264

H.264 encoding, best visual score, low latency:

./u30_xma_enc -w 1920 -h 1080  -i input_1080sp.yuv -c:v mpsoc_vcu_h264 -fps 60 -b:v 5000 -profile 2 -level 42 -g 120 -periodicity-idr 120 -qp-mode 1 -bf 0 -o out2.264

HEVC encoding, best visual score, low latency:

./u30_xma_enc -w 1920 -h 1080  -i input_1080sp.yuv -c:v mpsoc_vcu_hevc -fps 60 -b:v 4000 -g 120 -periodicity-idr 120 -qp-mode 1 -bf 0 -o out1.265

HEVC encoding, best visual score, normal latency:

./u30_xma_enc -w 1920 -h 1080  -i input_1080sp.yuv -c:v mpsoc_vcu_hevc -fps 60 -b:v 4000 -g 120 -periodicity-idr 120 -qp-mode 3 -bf 1 -lookahead-depth 20 -temporal-aq 1 -spatial-aq 1 -o out2.265

For more examples, refer to the encode_only/test/scripts/enc_app_test.sh script.

Limitations of the Encoder App

  • The encoder application assumes that the input YUV file is in NV12 format.

  • The application outputs only H264/HEVC elementary streams.

  • The application supports multi-process but not multi-channel.

U30 XMA Scaler App

The examples/xma/scale_only folder contains a standalone XMA-based scaler application for U30.

Scaler Build Instructions

Export the XRT and XRM to build the scaler application:

source /opt/xilinx/xrt/setup.sh
source /opt/xilinx/xrm/setup.sh

In the exampes/xma/scale_only directory, do the following:

make clean
make

The resulting executable is placed in the build directory:

./build/u30_xma_scale

Scaler Test Instructions

The scaler XMA application supports most of the scaler options supported by FFmpeg. The XMA application supports only NV12 semi-planar input and outputs NV12 scaled elementary stream.

Scaler Usage

Before running scaler application, source the xcdr environment:

source /opt/xilinx/xcdr/setup.sh

Below is the list of all the options of the scaler app:

This program ingests an NV12 input file and utilizes hardware acceleration to scale to various resolutions.

Usage:
        ./u30_xma_scale [options] -w <input-width> -h <input-height> -i
        <input-file> [scaler_options] -w <output-1-width> -h
        <output-1-height> -o <output-1-file> -w ...

Arguments:
        --help                     Print this message and exit
        -log <level>               Specify the log level
        -d <device-id>             Specify a device on which to run.
                                   Default: 0

Input Arguments:

        -stream_loop <loop-count>  Number of times to loop the input
                                   file
        -w <width>                 Specify the input's width
        -h <height>                Specify the input's height
        -pix_fmt <fmt>             Specify the input's pixel format.
                                   nv12 is the only format supported.
        -fps <frame-rate>          Frame rate. Used for scaler load
                                   calculation.
        -i <input-file>            Input file to be used

Output Arguments:
        -coeff_load <load>         Specify the coefficient load. 0 Auto
                                   (default), 1 static, 2 FilterCoef.txt.
        -enable-pipeline           Enable scaler pipeline
        -rate <half/full>          Set the rate to half. Half rate drops
                                   frames to reduce resource usage.
                                   Default: full.
        -enable-latency-logging    Enable latency logging
        -w <width>                 Specify the output's width
        -h <height>                Specify the output's height
        -frames <frame-count>      Number of frames to be processed.
        -o <file>                  File to which output is written.

Sample Scaler Command

Scale 1080p nv12 to 720p, 480p, 360p, and 240p nv12:

./u30_xma_scale -w 1920 -h 1080 -i ~/Kimono1_1920x1080_24.yuvNV12 -w 1280 -h \
      720 -o Kimono_1280x720.NV12 -w 852 -h 480 -o Kimono_852x480.NV12 -w \
      640 -h 360 -o Kimono_640x360.NV12 -w 480 -h 240 -o Kimono_480x240.NV12

Limitations of the Scaler App

  • The scaler supports nv12 format only

  • There may be a performance degradation relative to ffmpeg with a high number of outputs/processes.

U30 XMA Transcoder App

The examples/xma/transcode folder contains a standalone XMA-based transcoder application for U30.

Transcoder Build Instructions

Export the XRT and XRM to build the transcoder application:

source /opt/xilinx/xrt/setup.sh
source /opt/xilinx/xrm/setup.sh

In the exampes/xma/transcode directory, do the following:

make clean
make

The resulting executable is placed in the build directory:

./build/u30_xma_transcode

Transcoder Test Instructions

The transcoder XMA application supports most of the options supported by ffmpeg. The XMA application supports only elementary H.264 and HEVC encoded stream input and outputs H.264/HEVC encoded elementary stream.

Transcoder Usage

Before running transcoder application, source the xcdr environment:

source /opt/xilinx/xcdr/setup.sh

Running the transcoder app with the --help option will print the complete list of options:

./u30_xma_transcode --help
XMA Transcoder App Usage:
      ./program [generic options] -c:v <decoder codec> [decoder options]  -i input-file -multiscale_xma -outputs [num] [Scaler options]  -c:v <encoder codec> [encoder options] -o <output-file>  -c:v <encoder codec> [encoder options] -o <output-file>  -c:v <encoder codec> [encoder options] -o <output-file>.....

Arguments:

        --help                     Print this message and exit.
        -d <device-id>             Specify a device on which the
                                   transcoder to run. Default: 0
        -stream_loop <loop-count>  Number of times to loop the input file
        -frames <frame-count>      Number of input frames to be processed

Decoder options:

        -c:v <codec>               Decoder codec to be used. Supported
                                   are mpsoc_vcu_hevc, mpsoc_vcu_h264
        -low-latency <0/1>         Low latency for decoder. Default
                                   disabled
        -latency_logging <0/1>     Latency logging for decoder. Default
                                   disabled
        -i <input-file>            Name and path of input H.264/HEVC file

Scaler options:

        -multiscale_xma            Name of the ABR scaler filter
        -num-output <value>        Number of output files from scaler
        -out_1_width <width>       Width of the scaler output channel 1
        -out_1_height <height>     Height of the scaler output channel 1
        -out_1_rate <full/half>    Full of Half rate for output channel 1
        -out_2_width <width>       Width of the scaler output channel 2
        -out_2_height <height>     Height of the scaler output channel 2
        -out_2_rate <full/half>    Full of Half rate for output channel 2
        -out_3_width <width>       Width of the scaler output channel 3
        -out_3_height <height>     Height of the scaler output channel 3
        -out_3_rate <full/half>    Full of Half rate for output channel 3
        -out_4_width <width>       Width of the scaler output channel 4
        -out_4_height <height>     Height of the scaler output channel 4
        -out_4_rate <full/half>    Full of Half rate for output channel 4
        -out_5_width <width>       Width of the scaler output channel 5
        -out_5_height <height>     Height of the scaler output channel 5
        -out_5_rate <full/half>    Full of Half rate for output channel 5
        -out_6_width <width>       Width of the scaler output channel 6
        -out_6_height <height>     Height of the scaler output channel 6
        -out_6_rate <full/half>    Full of Half rate for output channel 6
        -out_7_width <width>       Width of the scaler output channel 7
        -out_7_height <height>     Height of the scaler output channel 7
        -out_7_rate <full/half>    Full of Half rate for output channel 7
        -out_8_width <width>       Width of the scaler output channel 8
        -out_8_height <height>     Height of the scaler output channel 8
        -out_8_rate <full/half>    Full of Half rate for output channel 8
        -latency_logging <0/1>     Latency logging for scaler. Default
                                   disabled
Encoder options:

        -c:v <codec>               Encoder codec to be used. Supported
                                   are mpsoc_vcu_hevc, mpsoc_vcu_h264
        -b:v <bitrate>             Bitrate can be given in Kbps or Mbps
                                   or bits i.e., 5000000, 5000K, 5M.
                                   Default is 5000kbps
        -fps <fps>                 Input frame rate. Default is 25.
        -g <intraperiod>           Intra period. Default is 12.
        -control-rate <mode>       Rate control mode. Supported are 0
                                   to 3, default is 1.
        -max-bitrate <bitrate>     Maximum bit rate. Supported are 0 to
                                   350000000, default is 5000
        -slice-qp <qp>             Slice QP. Supported are -1 to 51,
                                   default is -1
        -min-qp <qp>               Minimum QP. Supported are 0 to 51,
                                   default is 0.
        -max-qp <qp>               Maximum QP. Supported values are 0
                                   to 51, default is 51.
        -bf <frames>               Number of B frames. Supported are 0
                                   to 7, default is 2.
        -periodicity-idr <value>   IDR picture frequency. Supported are
                                   0 to UINT32_MAX, default is
                                   UINT32_MAX.
        -profile <value>           Encoder profile.
                    For HEVC, supported are 0 or main and 1 or main-intra.
                                   Default is 0/main.
                                   ENC_HEVC_MAIN - 0 or main.
                                   ENC_HEVC_MAIN_INTRA - 1 or main-intra.
                    For H264, supported are 66 or baseline, 77 or main,
                                   and 100 or high.
                                   ENC_H264_BASELINE - 66 or baseline.
                                   ENC_H264_MAIN - 77 or main.
                                   ENC_H264_HIGH - 100 or high.
        -level <value>             Encoder level.
                                   For HEVC, supported are 10 to 51,
                                   default is 50.
                                   For H264, supported are 10 to 52,
                                   default is 50.
        -slices <value>            Number of slices per frame. Supported
                                   are 1 to 68, default is 1.
        -qp-mode <mode>            QP mode. Supported are 0, 1, and 2,
                                   default is 1.
        -aspect-ratio <value>      Aspect ratio. Supported values are 0
                                   to 3, default is 0.
        -scaling-list <0/1>        Scaling list. Enable/Disable,
                                   default enable.
        -lookahead-depth <value>   Lookahead depth. Supported are 0 to
                                   20, default is 0.
        -temporal-aq <0/1>         Temporal AQ. Enable/Disable,
                                   default disable.
        -spatial-aq <0/1>          Spatial AQ. Enable/Disable,
                                   default disable.
        -spatial-aq-gain <value>   Spatial AQ gain. Supported are 0 to
                                   100, default is 50.
        -cores <value>             Number of cores to use, supported are
                                   0 to 4, default is 0.
        -tune-metrics <0/1>        Tunes MPSoC H.264/HEVC encoder's video
                                   quality for objective metrics, default
                                   disable.
        -latency_logging <0/1>     Enable latency logging in syslog.
        -o <file>                  File to which output is written.

Sample Transcoder Commands

H.264 to HEVC ABR Transcoder:

./u30_xma_transcode -c:v mpsoc_vcu_h264 -i input_1080p.264 -multiscale_xma -num-output 4 -out_1_width 1280 -out_1_height 720 -out_2_width 848 -out_2_height 480 -out_3_width 640 -out_3_height 360 -out_4_width 288 -out_4_height 160 \
-c:v mpsoc_vcu_hevc -b:v 4000K  -o out1_test1.265 -c:v mpsoc_vcu_hevc -b:v 3000K -o out1_test2.265 -c:v mpsoc_vcu_hevc -b:v 2500K -o out1_test3.265 -c:v mpsoc_vcu_hevc -b:v 1250K -o out1_test4.265 -c:v mpsoc_vcu_hevc -b:v 625K -o out1_test5.265

HEVC ABR Transcoder with Look-Ahead:

./u30_xma_transcode -c:v mpsoc_vcu_hevc -i input_1080p.265 -multiscale_xma -num-output 4 -out_1_width 1280 -out_1_height 720 -out_2_width 848 -out_2_height 480 -out_3_width 640 -out_3_height 360 -out_4_width 288 -out_4_height 160 \
-c:v mpsoc_vcu_hevc -b:v 4000K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test1.265 -c:v mpsoc_vcu_hevc -b:v 3000K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test2.265 \
-c:v mpsoc_vcu_hevc -b:v 2500K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test3.265 -c:v mpsoc_vcu_hevc -b:v 1250K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test4.265 \
-c:v mpsoc_vcu_hevc -b:v 625K -qp-mode 3 -lookahead-depth 16 -temporal-aq 1 -spatial-aq 1 -spatial-aq-gain 75 -o out_la_test5.265

H.264 to HEVC ABR Transcoder with Scaler Pipeline:

./u30_xma_transcode -c:v mpsoc_vcu_h264 -i input_1080p.264 -multiscale_xma -num-output 4 -out_1_width 1280 -out_1_height 720 -out_2_width 848 -out_2_height 480 -out_3_width 640 -out_3_height 360 -out_4_width 288 -out_4_height 160 \
-c:v mpsoc_vcu_hevc -b:v 4000K  -o out3_sc_test1.265 -c:v mpsoc_vcu_hevc -b:v 3000K -o out3_sc_test2.265 -c:v mpsoc_vcu_hevc -b:v 2500K -o out3_sc_test3.265 -c:v mpsoc_vcu_hevc -b:v 1250K -o out3_sc_test4.265 -c:v mpsoc_vcu_hevc -b:v 625K -o out3_sc_test5.265

For more examples, refer to the transcode/test/scripts/transcoder_app_test.sh script.

Limitations of the Transcoder App

  • The transcoder application supports only elementary H264/HEVC encoded streams as input file. It cannot parse container formats like MP4, AVI, etc.

  • The transcoder outputs only H264/HEVC elementary streams.

  • The lookahead depth should be same for all the channels.