vvas_xcompositor

vvas_xcompositor is a hardware accelerated N input, 1 output Gstreamer plugin that combines two or more video frames into a single frame. It can accept the below mentioned video formats. For each of the requested sink pads it will compare the incoming geometry and framerate to define the output parameters. Indeed, output video frames will have the geometry of the biggest incoming video stream and the framerate of the fastest incoming one.

In case input and output formats are different, then the color space conversion will be hardware accelerated by vvas_xcompositor. For implementation details, refer to vvas_xcompositor source code

Input and Output

This plug-in accepts buffers with the following color format standards:

  • RGBx

  • YUY2

  • r210

  • Y410

  • NV16

  • NV12

  • RGB

  • v308

  • I422_10LE

  • GRAY8

  • NV12_10LE32

  • BGRx

  • GRAY10_LE32

  • BGRx

  • UYVY

  • BGR

  • RGBA

  • BGRA

  • I420

  • GBR

Important

Make sure that the color formats needed for your application are supported by the image-processing hardware kernel.

As a reference, multi-scaler configuration for smart model select example design can be found in image-processing configuration

Individual parameters for each input stream can be configured on the GstCompositorPad:

  • “xpos”: The x-co-ordinate position of the top-left corner of the current frame in output buffer.

  • “ypos”: The y-co-ordinate position of the top-left corner of the current frame in output buffer.

  • “width”: The width of the current picture in the output buffer; If the input width and the width in output buffer are different, then hardware accelerated resize operation will be performed.

  • “height”: The height of the current picture in the output buffer; If the input height and the height in output buffer are different, then hardware accelerated resize operation will be performed.

  • “zorder”: The z-order position of the picture in the composition.

../../../_images/compositor.png

Plugin properties

Table 14: vvas_xcompositor Plug-in Properties

Property Name

Type

Range

Default

Description

avoid-output-copy

Boolean

true or false

false

Avoid output frames copy on all source pads even when downstream does not support GstVideoMeta metadata

best-fit

Boolean

true or false

false

downscale/upscale the input video to best-fit in each window

dev-idx

Integer

-1 to 31

-1

Device index Valid only for PCIe/Data Center platforms

enable-pipeline

Boolean

true or false

false

Enable buffer pipelining to improve performance in non zero-copy use cases

in-mem-bank

Unsigned Integer

0 - 65535

0

VVAS input memory bank to allocate memory

out-mem-bank

Unsigned Integer

0 - 65535

0

VVAS output memory bank to allocate memory

ppc

Integer

1, 2, 4

2

Pixel per clock supported by the image-processing kernel

scale-mode

Integer

0, 1, 2

0

Scale algorithm to use: 0:BILINEAR 1:BICUBIC 2:POLYPHASE

coef-load-type

Integer

0 => Fixed 1 => Auto

1

Type of filter Coefficients to be used: Fixed or Auto generated

num-taps

Integer

6=>6 taps 8=>8 taps 10=>10 taps 12=>12 taps

1

Number of filter taps to be used for scaling

kernel-name

String

NA

image_processing :{image_processing _1}

String defining the kernel name and instance as mentioned in xclbin

xclbin-location

String

NA

NULL

Location of the xclbin to program devices

vvas_xcompositor pad properties

Table 15: vvas_xcompositor Pad Properties

Property Name

Type

Range

Default

Description

xpos

Unsigned Integer

0 to 2147483647

0

The x-co-ordinate position of the top-left corner of the current farme in output buffer.

ypos

Unsigned Integer

0 to 2147483647

0

The y-co-ordinate position of the top-left corner of the current farme in output buffer.

height

Integer

-1 to 2147483647

-1

The height of the current picture in the output buffer; If the input height and the height in output buffer are different, then hardware accelerated resize operation will be performed. Setting default/-1 treats o/p height as input height

width

Integer

-1 to 2147483647

-1

The width of the current picture in the output buffer; If the input width and the width in output buffer are different, then hardware accelerated resize operation will be performed. Setting default/-1 treats o/p width as input width

zorder

Unsigned Integer

-1 to 16

-1

The z-order positon of the picture in the composition.

The example pipeline with vvas_xcompositor plug-in is as mentioned below.

#! /bin/bash

PAD_PROPERTIES="\
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1920 sink_1::ypos=0 \
sink_2::xpos=0 sink_2::ypos=1080 \
sink_3::xpos=1920 sink_3::ypos=1080 \
"

gst-launch-1.0 -v  filesrc location=$1 !\
h264parse !\
omxh264dec !\
comp.sink_0 \
vvas_xcompositor kernel-name=image_processing:{image_processing_1} \
xclbin-location=/run/media/mmcblk0p1/dpu.xclbin $PAD_PROPERTIES name=comp !\
video/x-raw , width=3840, height=2160 , format=NV12  !\
queue !\
fpsdisplaysink video-sink="kmssink  bus-id=a0130000.v_mix async=true" text-overlay=false sync=false \
filesrc location=$2 !\
h264parse !\
omxh264dec name=decoder_1 !\
queue !\
comp.sink_1 \
filesrc location=$3 !\
h264parse !\
omxh264dec !\
queue !\
comp.sink_2 \
filesrc location=$4 !\
h264parse !\
omxh264dec !\
queue !\
comp.sink_3