Using FFmpeg¶

This page documents how to use FFmpeg with the Xilinx Video SDK.

Introduction ¶

FFmpeg is an industry standard, open source, widely used utility for handling video. FFmpeg has many capabilities, including encoding and decoding all video compression formats, encoding and decoding audio, encapsulating, and extracting audio, and video from transport streams, and many more. The Xilinx Video SDK includes an enhanced version of FFmpeg that can communicate with the hardware accelerated transcode pipeline in Xilinx devices.

The FFmpeg executable included with version 3.0 of the Video SDK is a fork of the main FFmpeg GitHub (release 4.4, tag n4.4, commid ID dc91b913b6260e85e1304c74ff7bb3c22a8c9fb1) with a Xilinx patch applied to enable the Xilinx Video SDK plugins.

It is not within the scope of this document to provide an exhaustive guide on the usage of FFmpeg. Various resources can be found online, for example:

The following sections describe the options used with FFmpeg to configure the various hardware accelerators available on Xilinx devices.

Example Commands¶

A simple FFmpeg command for accelerated encoding with the Xilinx Video SDK will look similar to this one:

ffmpeg -c:v mpsoc_vcu_h264 -i infile.mp4 -c:v mpsoc_vcu_hevc -s:v 1920x1080 -b:v 1000K -r 60 -f mp4 -y transcoded.mp4

There are many other ways in which FFmpeg can be used to leverage the video transcoding features of Xilinx devices. Examples illustrating how to run FFmpeg for encoding, decoding, and transcoding with and without ABR scaling. To see examples of all these possibilities, refer to the FFmpeg tutorials included in this repository.

General FFmpeg Options¶

Options	Descriptions
-i¶	The input file.
-c:v¶	Specify the video codec. This option must be set for any video stream sent to a Xilinx device. Valid values are `mpsoc_vcu_hevc` (for HEVC) or `mpsoc_vcu_h264` (for H.264)
-s¶	The frame size (WxH). For example 1920x1080 or 3840x2160.
-f¶	The container format.
-r¶	The frame rate in fps (Hz).
-filter_complex¶	Used to specify ABR scaling options. Consult the section about Using the Xilinx Multiscale Filter for more details on how to use this option.
-xlnx_hwdev¶	Global option used to specify on which Xilinx device the FFmpeg job should run. Consult the Using Explicit Device IDs section for more details on how to use this option. Valid values are positive integers. Default is device 0.
-lxlnx_hwdev¶	Component-level option used to specify on which Xilinx device a specific component of the FFmpeg job should be run. When set, it overwrites the global `-xlnx_hwdev` option for that component. Consult the Using Explicit Device IDs section for more details on how to use this option. Valid values are positive integers. Default is device 0.
-latency_logging¶	Log latency information to syslog. Valid values: 0 (disabled, default) and 1 (enabled)
-loglevel¶	Configures the FFmpeg log level. Setting this option to `debug` displays comprehensive debug information about the job

Video Decoding ¶

For the complete list of features and capabilities of the Xilinx hardware decoder, refer to the Video Codec Unit section of the Specs and Features chapter of the documentation.

The Xilinx video decoder is leveraged in FFmpeg by setting the -c:v option to mpsoc_vcu_hevc for HEVC or to mpsoc_vcu_h264 for H.264.

The table below describes all the options for the Xilinx video decoder.

Options	Descriptions
-low_latency¶	Enable low-latency mode Valid values: 0 (default) and 1 Setting this flag to 1 reduces decoding latency when `splitbuff-mode` is also enabled. Streams containing B frames are not supported in low-latency mode.
-splitbuff_mode¶	Configure decoder in split/unsplit input buffer mode Valid values: 0 (default) and 1 The split buffer mode hands-off buffers to next pipeline stage earlier. Enabling both `splitbuff-mode` and `low-latency` reduces decoding latency.
-entropy_buffers_count¶	Number of internal entropy buffers Valid values: 2 (default) to 10 Can be used to improve the performance for input streams with a high bitrate (including 4k streams) or a high number of reference frames. 2 is enough for most cases. 5 is the practical limit.

Video Encoding ¶

For the complete list of features and capabilities of the Xilinx hardware encoder, refer to the Video Codec Unit section of the Specs and Features chapter of the documentation.

The Xilinx video encoder is leveraged in FFmpeg by setting the -c:v option to mpsoc_vcu_hevc for HEVC or to mpsoc_vcu_h264 for H.264.

The table below describes all the options for the Xilinx video encoder.

Options	Descriptions
-b:v¶	Specify the video bitrate You can specify this in Mb or Kb. For example -b:v 1M or -b:v 1000K. Can be specified in Mb or Kb. For example -b:v 1M or -b:v 1000K The bitrate can also be adjusted dynamically. Consult the Dynamic Encoder Parameters section for more details on how to change this option during runtime.
-max-bitrate¶	Maximum bitrate Valid values: 0 to 3.5e+10 (default 5000000) You may want to use this to limit encoding bitrate if you have not specified a `-b:v` bitrate
-g¶	GOP size Set the GOP size to 2x frame rate for a 2 second GOP The maximum supported GOP size is 1000.
-aspect-ratio¶	Aspect ratio Valid values: 0 to 3 (default is 0) (0) auto - 4:3 for SD video, 16:9 for HD video, unspecified for unknown format (1) 4:3 (2) 16:9 (3) none - Aspect ratio information is not present in the stream
-cores¶	Number of encoder cores in the Xilinx device to utilize Valid values: 0 to 4 (default is 0) (0) auto The FFmpeg encoder plugin automatically determines how many encoder cores are needed to sustain real-time performance (e.g. 1 for 1080p60, 4 for 4K60). The `-cores` option can be used to manually specify how many encoder cores are to be used for a given job. When encoding file-based clips with a resolution of 1080p60 or lower, leveraging additional cores may increase performance. This option will provide diminishing returns when multiple streams are processed on the same device. This option has no impact on live streaming use-cases as a video stream cannot be processed faster than it is received.
-slices¶	Number of slices to operate on at once within a core Valid values: 1 to 68 (default is 1) Slices are a fundamental part of the stream format. You can operate on slices in parallel to increase speed at which a stream is processed. However, operating on multiple “slices” of video at once will have a negative impact on video quality. This option must be used when encoding 4k streams to H.264 in order to sustain real-time performance. The maximum practical value for this option is 4 since there are 4 encoder cores in a device.
-level¶	Encoding level restriction If the user does not set this value, the encoder will automatically assign appropriate level based on resolution, frame rate and bitrate Valid values for H.264: 1, 1.1, 1.2, 1.3, 2, 2.1, 2.2, 3, 3.1, 3.2, 4, 4.1, 4.2, 5, 5.1, 5.2 Valid values for HEVC: 1, 2, 2.1, 3, 3.1, 4, 4.1, 5, 5.1, 5.2
-profile¶	Set the encoding profile Valid values for H.264: baseline (66), main (77), high (100, default), high-10 (110), high-10-intra (2158) Valid values for HEVC: main (0, default), main-intra (1), main-10 (2), main-10-intra (3)
-tier¶	Set the encoding tier (HEVC only) Valid values: 0, 1 (default is 0) (0) Main tier (1) High tier
-bf¶	Number of B frames Valid values: 0 to 4 (default is 2) For tuning use 1 or 2 to improve video quality at the cost of latency. Consult the B Frames section for more details on how to use this option.
-lookahead_depth¶	Number of frames to lookahead for qp maps Valid values: 0 to 20 (default is 0) The maximum lookahead depth allowed is 1/3 of the stream frame rate. When latency is tolerable, use the maximum lookahead depth to get optimum video quality to get optimum video quality. Consult the Lookahead section for more details on how to use this option.
-spatial-aq¶	Enable spatial AQ Valid values: 0, 1 (default is 1) (0) disable (1) enable - Default Consult the Adaptive Quantization section for more details on how to use this option.
-spatial-aq-gain¶	Percentage of spatial AQ gain. Valid values: 0 to 100 (default is 50) Consult the Adaptive Quantization section for more details on how to use this option.
-temporal-aq¶	Enable temporal AQ Valid values: 0, 1 (default is 1) (0) disable (1) enable - Default Consult the Adaptive Quantization section for more details on how to use this option.
-scaling-list¶	Determine if the quantization values are auto scaled Valid values: 0, 1 (default is 1) (0) flat - Flat scaling list mode, improves objective metrics (1) default - Default scaling list mode, improves video quality Consult the Scaling List section for more details on how to use this option.
-qp-mode¶	QP control mode Valid values: 0 to 2 (default is 1) (0) uniform (1) auto - default (2) relative_load For best objective scores use `uniform`. For best subjective quality use `auto` or `relative_load`. Consult the Adaptive Quantization section for more details on how to use this option.
-control-rate¶	Set the Rate Control mode Valid values: 0 to 3 (default is 1) (0) Constant QP (1) Constant Bitrate - default (2) Variable Bitrate (3) Low Latency
-min-qp¶	Minimum QP value allowed for rate control Valid values: 1 to 51 (default is 1) This option has no effect when `-control-rate` is set to Constant QP (0).
-max-qp¶	Maximum QP value allowed for rate control Valid values: 1 to 51 (default is 51) This option has no effect when `-control-rate` is set to Constant QP (0).
-slice-qp¶	Slice QP Valid values: -1 to 51 (default is -1) (-1) auto This sets the QP values when `-control-rate` is Constant QP (0). The specified QP value applies to all the slices. This parameter can also be used to provide QP for the first Intra frame when `-lookahead_depth` = 0. When set to -1, the QP for first Intra frame is internally calculated.
-periodicity-idr¶	IDR frame frequency Valid values: -1 to INT_MAX32 (default is -1) If this option is not specified, a closed GOP is generated: the IDR periodicity is aligned with the GOP size and IDR frame is inserted at the start of each GOP. To insert IDR frames less frequently, use a value which is a multiple of the GOP size.
-force_key_frames¶	Force insertion of IDR frames Valid values: time[,time…] or expr:expr Force an IDR frame to be inserted at the specified frame number(s). Consult the Dynamic IDR Frame Insertion section for more details on how to use this option.
-expert-options¶	Set advanced encoding options Valid values: dynamic-params=<options file> Consult the Dynamic Encoder Parameters section for more details on how to use this option.
-tune-metrics¶	Enable tuning video quality for objective metrics Valid value: 0, 1 (default is 0) (0) disable - Disable tune metrics (1) enable - Enable tune metrics Enabling `-tune-metrics` automatically forces `-qp-mode` = `uniform` and `-scaling-list` = `flat`, overwritting any explicit user settings of two values. This option improves objective quality by providing equal importance to all the blocks in the frame: the same quantization parameters and transform coefficients are used for all of them. This option should be disabled when measuring subjective quality or visually checking the video. This option should be enabled when measuring objective metrics such as PSNR/SSIM/VMAF.
-vsync¶	Add a vsync frame Valid values: 0, 1 Set this to 0 to prevent extra frames being added.
-disable-pipeline¶	Enable Ultra Low Latency (ULL) encoding Valid values: 0, 1 (default is 0) Set this to 1 to enable ultra low-latency encoding. Consult the Tuning Transcode Latency section for more details.
-avc-lowlat¶	AVC Ultra Low Latency (ULL) density tuning Valid values: 0, 1 (default is 0) This option only takes effect, when `-disable-pipeline` is enabled. Set this to 0 to allow for high density encoding, for resolutions up to 1080p60. Set this to 1 to allow for low density encoding, for resolutions up to 4kp60. Consult the Tuning Transcode Latency section for more details.
-dynamic-gop¶	Enable Dynamic GOP Valid values: 0, 1 (default is 0) Set this to 1 to enable dynamic insertion of B frames based on motion vectors. Requires a lookahead depth of at least 5. Consult the Dynamic GOP section for more details.

Video Scaling ¶

The Xilinx hardware scaler is leveraged in FFmpeg by using the multiscale_xma complex filter and the FFmpeg filter graph syntax. This section describes the options of the multiscale_xma complex filter.

For optimal visual quality in scaling operations, refer to ABR Scaling section of the Xilinx Video SDK user guide.

multiscale_xma¶: Filter implementing the Xilinx ABR multiscaler. Takes one input and up to 8 output streams. The complete list of options is described below.

Multiscale Filter Options¶
Options	Description
outputs¶	Specify the number of scaler outputs Valid values are integers between 1 and 8
out_{N}_width¶	Specify the width of each of the scaler outputs The output number {N} must be an integer value between 1 and 8, and must not exceed the number of outputs specified with `outputs` Valid values are integers between 3840 and 128, in multiples of 4
out_{N}_height¶	Specify the height of each of the scaler outputs The output number {N} must be an integer value between 1 and 8, and must not exceed the number of outputs specified with `outputs` Valid values are integers between 2160 and 128, in multiples of 4
out_{N}_rate¶	Specify the frame rate of each of the scaler outputs By default, the scaler uses the input stream frame rate for all outputs. While the encoder supports frame dropping with the -r option, there is also hardware support in the scaler for dropping frames. Dropping frames in the scaler is preferred since it saves scaler bandwidth, allowing the scaler and encoder to operate more efficiently. The output number {N} must be an integer value between 1 and 8, and must not exceed the number of outputs specified with `outputs` Valid values: `full` and `half` (default `full`). The first output has to be at full rate (`out_1_rate=full`).
enable_pipeline¶	Enable pipelining in multiscaler Pipelining provides additional performance at the cost of additional latency (2 frames). By default, pipelining is automatically controlled based on where the scaler input is coming from. If the input is coming from the host, pipelining is enabled. If the input is coming from the decoder then pipelining is disabled. Explicitly enabling pipelining has benefits in two situations: in a 2-device use cases where the output of the scaler is transfered to the host; in zero copy 4K ABR ladder use cases with multiple renditions. Valid values: -1 to 1 (default -1) auto (-1) disabled (0) enabled (1)

Using the Multiscale Filter¶

The filter graph specification for the multiscale_xma filter should be constructed in the following way:

Add the multiscale_xma filter to the graph
Set the number of scaler outputs
Set the width, height, and rate settings for each scaler output
Define the name each scaler output
If the outputs are not to encoded on the device, add xvbm_convert filters to the filter graph to copy the frames back to the host and convert them to AV frames.

The following example shows a complete command to decode, scale and encode to five different resolutions:

ffmpeg -c:v mpsoc_vcu_h264 -i input.mp4 \
  -filter_complex " \
    multiscale_xma=outputs=4: \
    out_1_width=1280: out_1_height=720: out_1_rate=full: \
    out_2_width=848:  out_2_height=480: out_2_rate=half: \
    out_3_width=640:  out_3_height=360: out_3_rate=half: \
    out_4_width=288:  out_4_height=160: out_4_rate=half  \
    [a][b][c][d]; [a]split[aa][ab]; [ab]fps=30[abb]" \
  -map "[aa]"  -b:v 4M    -c:v mpsoc_vcu_h264 -f mp4 -y ./scaled_720p60.mp4 \
  -map "[abb]" -b:v 3M    -c:v mpsoc_vcu_h264 -f mp4 -y ./scaled_720p30.mp4 \
  -map "[b]"   -b:v 2500K -c:v mpsoc_vcu_h264 -f mp4 -y ./scaled_480p30.mp4 \
  -map "[c]"   -b:v 1250K -c:v mpsoc_vcu_h264 -f mp4 -y ./scaled_360p30.mp4 \
  -map "[d]"   -b:v 625K  -c:v mpsoc_vcu_h264 -f mp4 -y ./scaled_288p30.mp4

This example can also be found in the FFmpeg introductory tutorials: Transcode With Multiple-Resolution Outputs.

Multiscale Usage Considerations¶

Managing Input Resolutions¶

The decoder supports resolutions which are multiples of 2, but the multiscaler supports resolutions which are multiples of 4. Therefore, pipelines leveraging both the decoder and the multiscaler will only work for streams with resolutions which are multiples of 4.

If the resolution out of the decoder is not a multiple of 4, the decoded output can be copied back to the host using the xvbm_convert filter and can be scaled to a suitable resolution using a software filter.

Performance Considerations¶

Encoded input streams with a high bitrate or with a high number of reference frames can degrade the performance of an ABR ladder. The -entropy_buffers_count decoder option can be used to help with this. A value of 2 is enough for most cases, 5 is the practical limit.

Encoding Scaler Outputs¶

The outputs of an ABR ladder can be encoded on the device using either the mpsoc_vcu_h264 or the mpsoc_vcu_hevc codec. All outputs must be encoded using the same codec.

Using Raw Scaler Outputs¶

To return raw video outputs from the ABR ladder, use the xvbm_convert filter to copy the frames from the device to the host and convert them to AV frames. The converted AV frames can then be used in FFmpeg software filters or directly saved to file as shown in this command:

ffmpeg -c:v mpsoc_vcu_h264 -i input.mp4 \
-filter_complex " \
  multiscale_xma=outputs=4: \
  out_1_width=1280: out_1_height=720:  out_1_rate=full: \
  out_2_width=848:  out_2_height=480:  out_2_rate=half: \
  out_3_width=640:  out_3_height=360:  out_3_rate=half: \
  out_4_width=288:  out_4_height=160:  out_4_rate=half  \
  [a][b][c][d]; [a]split[aa][ab]; [ab]fps=30[abb]; \
  [aa]xvbm_convert[aa1];[abb]xvbm_convert[abb1];[b]xvbm_convert[b1];[c]xvbm_convert[c1]; \
  [d]xvbm_convert[d1]" \
-map "[aa1]"  -pix_fmt yuv420p -f rawvideo ./scaled_720p60.yuv \
-map "[abb1]" -pix_fmt yuv420p -f rawvideo ./scaled_720p30.yuv \
-map "[b1]"   -pix_fmt yuv420p -f rawvideo ./scaled_480p30.yuv \
-map "[c1]"   -pix_fmt yuv420p -f rawvideo ./scaled_360p30.yuv \
-map "[d1]"   -pix_fmt yuv420p -f rawvideo ./scaled_288p30.yuv

This example can also be found in the FFmpeg introductory tutorials: Decode Only Into Multiple-Resolution Outputs.

Scaling and Encoding on Two Different Devices¶

The Xilinx Video SDK supports up to 32 scaled outputs streams per device, up to a maximum total equivalent bandwidth of 4kp60. For some use cases, such as 4K ladders or 1080p ladders many outputs, it may not be possible to scale or encode all streams on a single device. In this situation, it is possible to split the job across two devices and run part of job on one device and the other part on another device. This accomplished by using the -lxlnx_hwdev option which allows specifying the device on which a specific job component (decoder, scaler, encoder) should be run.

Consult the Using Explicit Device IDs section for more details on how to use the -lxlnx_hwdev option and work with multiple devices.

Moving Data through the Video Pipeline ¶

Automatic Data Movement¶

The Xilinx Video SDK takes care of moving data efficiently through the FFmpeg pipeline in these situations:

Individual operations:

Decoder input: encoded video is automatically sent from the host to the device
Scaler input: raw video is automatically sent from the host to the device
Encoder input: raw video is automatically sent from the host to the device
Encoder output: encoded video is automatically sent from the device to the host

Multistage pipelines

Pipelines with hardware accelerators only (such as transcoding with ABR ladder): the video frames remain on the device and are passed from one accelerator to the next, thereby avoiding unnecessary data movement between the host and the device
Pipelines with software filters: when using the ouput of the decoder or the scaler with a FFmpeg software filter, the video frames are automatically copied back to the host, as long as the software filter performs frame cloning. Examples of such filters include fps and split.

Explicit Data Movement¶

It is necessary to explicitly copy video frames from the device to the host in these situations:

Writing the output of the decoder or the scaler to file.
Using the output of the decoder or the scaler with a FFmpeg software filter which does not perform frame cloning.
Performing different operations on different devices, in which case the video frames must be copied from the first device to the host and then from the host to the second device.

This is done using the xvbm_convert filter.

IMPORTANT: Failing to use the xvbm_convert filter will result in garbage data being propagated throught the processing pipeline.

xvbm_convert¶: FFmpeg filter which converts and copies a XVBM frame on the device to an AV frame on the host. The pixel format of the frame on the host depends on the pixel format of the frame on the device. If the frame on the device is 8-bit, the frame will be stored as nv12 on the host. If the frame on the device is 10-bit, the frame will be stored as xv15 on the host. The xvbm_convert filter does not support other formats and does not support converting 8-bit frames to 10-bit frames.

Examples using the xvbm_convert filter can be found here:

FFmpeg tutorials Decode Only and Decode Only Into Multiple-Resolution Outputs
FFmpeg examples with Software Filters

Working with Multiple Devices ¶

By default (if no device identifier is specified) a job is submitted to device 0. When running large jobs or multiple jobs in parallel, device 0 is bound to run out of resources rapidly and additional jobs will error out due to insufficient resources.

By using the -lxlnx_hwdev and -xlnx_hwdev options, the different components (decoder, scaler, encoder) of a job can be individually submitted to a specific device. This makes it easy and straightforward to leverage the entire video acceleration capacity of your system, regardless of the number of cards and devices.

Consult the Using Explicit Device IDs section for more details on how to work with multiple devices.

Mapping Audio Streams ¶

When the FFmpeg job has a single input and a single output, the audio stream of the input is automatically mapped to the output video.

When the FFmpeg job has multiple outputs, FFmpeg must be explicitely told which audio stream to map to each of the output streams. The example below implements an transcoding pipeline with an ABR ladder. The input audio stream is split into 4 different channels using the asplit filter, one for each video output. Each audio channel is then uniquely mapped to one of the output video streams using the -map option.

ffmpeg -c:v mpsoc_vcu_h264 -i input.mp4
    -filter_complex "multiscale_xma=outputs=4:
    out_1_width=1280: out_1_height=720:  out_1_rate=full:
    out_2_width=848:  out_2_height=480:  out_2_rate=full:
    out_3_width=640:  out_3_height=360:  out_3_rate=full:
    out_4_width=288:  out_4_height=160:  out_4_rate=full
    [vid1][vid2][vid3][vid4]; [0:1]asplit=outputs=4[aud1][aud2][aud3][aud4]" \
    -map "[vid1]" -b:v 3M    -c:v mpsoc_vcu_h264 -map "[aud1]" -c:a aac -f mp4 -y output1.mp4
    -map "[vid2]" -b:v 2500K -c:v mpsoc_vcu_h264 -map "[aud2]" -c:a aac -f mp4 -y output2.mp4
    -map "[vid3]" -b:v 1250K -c:v mpsoc_vcu_h264 -map "[aud3]" -c:a aac -f mp4 -y output3.mp4
    -map "[vid4]" -b:v 625K  -c:v mpsoc_vcu_h264 -map "[aud4]" -c:a aac -f mp4 -y output4.mp4

Rebuilding FFmpeg ¶

There are two methods for rebuilding FFmpeg with the Xilinx Video SDK plugins enabled:

Using the complete source code
Using the git patch file

Using the Source Code¶

The sources/app-ffmpeg4-xma submodule contains the entire source code for the FFmpeg executable included with the Video SDK. This is a fork of the main FFmpeg GitHub (release 4.4, tag n4.4, commid ID dc91b913b6260e85e1304c74ff7bb3c22a8c9fb1) with a Xilinx patch applied to enable the Xilinx Video SDK plugins. Due to licensing restrictions, the FFmpeg executable included in the Video SDK is enabled with the Xilinx Video SDK plugins only.

You can rebuild the FFmpeg executable with optional plugins by following the instructions below. Additionally, comprehensive instructions for compiling FFmpeg can be found on the FFmpeg wiki page.

Make sure nasm and yasm are installed on your machine.
Navigate the top of the Xilinx Video SDK repository:
```
cd /path/to/video-sdk
```
Make sure the sources have been downloaded from the repository:
```
git submodule update --init --recursive
```
Navigate to the directory containing the FFmpeg sources:
```
cd sources/app-ffmpeg4-xma
```
Optionally install FFmpeg plugins you wish to enable (either from source or from your package manager like yum or apt). For example: libx264, or libx265.

Configure FFmpeg with --enable flags to enable the desired plugins. The -enable-libxma2api flag enables the Xilinx Video SDK plugins. The command below will configure the Makefile to install the custom FFmpeg in the /tmp/ffmpeg directory. To install in another location, modify the --prefix and --datadir options:

./configure --prefix=/tmp/ffmpeg --datadir=/tmp/ffmpeg/etc  --enable-x86asm --enable-libxma2api --disable-doc --enable-libxvbm --enable-libxrm --extra-cflags=-I/opt/xilinx/xrt/include/xma2 --extra-ldflags=-L/opt/xilinx/xrt/lib --extra-libs=-lxma2api --extra-libs=-lxrt_core --extra-libs=-lxrt_coreutil --extra-libs=-lpthread --extra-libs=-ldl --disable-static --enable-shared

Build and install the FFmpeg executable:
```
make -j && sudo make install
```
The /opt/xilinx/xcdr/setup.sh script puts the Xilinx-provided FFmpeg in the PATH environment variable. To use the newly built FFmpeg, update your PATH or provide the full path to the custom-built executable.

Using the Git Patch File¶

The sources/app-ffmpeg4-xma-patch folder contains a git patch file which can be applied to a FFmpeg fork to enable the Xilinx Video SDK plugins.

This patch is intended to be applied to FFmpeg n4.4. As such, applying this patch to earlier or later versions of FFmpeg may require edits to successfully merge these changes and represent untested configurations.

The patch makes edits to FFmpeg and adds new plugins to FFmpeg to initialize, configure and use Xilinx video accelerators.

The patch can be applied to a FFmpeg fork as follows:

Clone the n4.4 version of FFmpeg:

git clone https://github.com/FFmpeg/FFmpeg.git -b n4.4

After the git clone, you will have a directory named FFmpeg. Enter this directory:
```
cd FFmpeg
```

Copy the patch file into the FFmpeg directory:

cp /path/to/sources/app-ffmpeg4-xma-patch/0001-Updates-to-ffmpeg-n4.4-to-support-Alveo-U30-SDK-v3.patch .

Apply the patch:

git am 0001-Updates-to-ffmpeg-n4.4-to-support-Alveo-U30-SDK-v3.patch --ignore-whitespace --ignore-space-change

Optionally install FFmpeg plugins you wish to enable (either from source or from your package manager like yum or apt). For example: libx264, or libx265.

Configure FFmpeg with --enable flags to enable the desired plugins. The -enable-libxma2api flag enables the Xilinx Video SDK plugins. The command below will configure the Makefile to install the custom FFmpeg in the /tmp/ffmpeg directory. To install in another location, modify the --prefix and --datadir options:

./configure --prefix=/tmp/ffmpeg --datadir=/tmp/ffmpeg/etc  --enable-x86asm --enable-libxma2api --disable-doc --enable-libxvbm --enable-libxrm --extra-cflags=-I/opt/xilinx/xrt/include/xma2 --extra-ldflags=-L/opt/xilinx/xrt/lib --extra-libs=-lxma2api --extra-libs=-lxrt_core --extra-libs=-lxrt_coreutil --extra-libs=-lpthread --extra-libs=-ldl --disable-static --enable-shared

Build and install the FFmpeg executable:
```
make -j && sudo make install
```
The /opt/xilinx/xcdr/setup.sh script puts the Xilinx-provided FFmpeg in the PATH environment variable. To use the newly built FFmpeg, update your PATH or provide the full path to the custom-built executable.