VVAS GStreamer Plug-ins for Embedded Platforms¶
The following section describes GStreamer plugins for Embedded platforms. The plug-ins source code is available in the vvas-gst-plugins repository/folder of the VVAS sources tree. The following table lists the GStreamer plug-ins.
Table 1: GStreamer Plug-ins
Plug-in Name |
Functionality |
---|---|
omxh264dec/omxh265dec |
GStreamer plug-in to perform hardware accelerated h264/h265 video decoding. |
omxh264enc/omxh265enc |
GStreamer plug-in to perform hardware accelerated h264/h265 video encoding. |
vvas_xroigen |
Plug-in to generate region of interest metadata. |
OMX Encoder/Decoder Plug-in¶
“omxh264decoder/omxh265decoder” are GStreamer Plugins for hardware accelerated H264/H265 Video decoding using VCU IP. Similarly, “omxh264encoder/omxh265encoder” are GStreamer Plugins for hardware accelerated H264/H265 Video encoding using VCU IP. For more details about these plugins, refer to the VCU PG252 Chapter 23: Encoder and Decoder Software Features.
Region of Interest Plug-in¶
The region of interest (ROI) GStreamer vvas_xroigen plug-in generates GstVideoRegionOfInterestMeta metadata information, which is expected by GStreamer OMX encoder plug-ins to encode raw frames with the desired quality parameters (QP) values/ level for specified ROIs. The vvas_xroigen plug-in prepares the GstVideoRegionOfInterestMeta metadata by parsing an VVAS supported list of metadata objects (GstInferenceMeta).
For implementation details, refer to vvas_xroigen source code.
Tip
This plug-in is only useful with embedded platforms because its main job is to generate metadata required for the GStreamer OMX encoder plug-in that exists on embedded platforms.
GStreamer ROI metadata information is attached with each outgoing buffer based on the following plug-in properties.
Input and Output¶
Accepts buffers with any of the GStreamer supported video formats on input and output GstPads.
Control Parameters and Plug-in Properties¶
The following table lists the GObject properties exposed by vvas_xroigen plug-in.
Table 2: vvas_xroigen Plug-in Properties
Property Name |
Type |
Range |
Default |
Description |
---|---|---|---|---|
class-filters |
GstValueArray |
None |
None |
Array of desired inference classes. vvas_xroigen prepares ROI metadata only for class-filters array of strings out of classes present in VVAS supported metadata objects. Default behavior: Plug-in allows all classes when this property is not set. e.g., class-filters =<”person”, “car”> |
insert-roi-sei |
Boolean |
true, false |
false |
When ‘true’, it sends custom events to OMX encoder to insert ROI information as SEI packet. |
resolution-range |
GstValueArray |
None |
None |
Array of integers that represent the minimum and maximum range of ROI resolutions. Plug-in uses this resolution range to filter out ROIs. Set this property: <min-width, min- height, max-width, max-height>) Default behavior: Plug-in allows all ROIs. e.g., resoluti on-range=<0 ,0,320,240> |
roi-max-num |
Unsigned integer |
0 to 4294967295 |
4294967295 |
Maximum number of ROI metadata to be attached per buffer |
roi-qp-delta |
Integer |
–32 to 31 |
0 |
QP delta to be used for ROI in encoder |
roi-qp-level |
Enum |
0, 1, 2, 3, 4 |
0 |
QP level to be used for ROI in encoder 0: high (delta QP of –5) 1: medium (delta QP of 0) 2: low (delta QP of +5) 3: don’t-care (maximum delta QP value) 4: intra (region all LCU encoded with intra prediction mode) |
roi-type |
Enum |
0, 1, 2 |
0 |
Type to be used to generate ROI metadata 0: default (ROI without encoder QP i nformation) 1: qp_level (ROI encoder QP level) 2: qp_delta (ROI encoder QP delta) |
Example Pipelines¶
The following pipeline takes input video in an MP4 container from the filesrc plug-in. The qtdemux
extracts the H.264
elementary stream from the MP4 container and pass it to the omxh264dec
plug-in for decoding. The output of the decoder goes to the vvas_xmultisrc
plug-in for resizing to a resolution of 640 x 360 and color-space conversion to BGR
format. If your design has other kernels, like multiscaler
, for re-size and colorspace conversion, then you may use it along with vvas_xabrscaler
plug-in. The output from vvas_xmultisrc goes to the vvas_xfilter plug-in for object detection using the densebox model. The inference operation generates the metadata and bounding box, for each detected object. The vvas_xroigen plug-in creates the ROI metadata from the incoming bounding box information and passes this metadata to omxh264enc for ROI based encoding. This is an example pipeline to demonstrate ROI feature. This pipeline uses omxh264enc hence your design must have VCU
encoder.
gst-launch-1.0 -v filesrc location=<filename>.mp4 \
! qtdemux \
! h264parse \
! omxh264dec internal-entropy-buffers=3 \
! queue \
! vvas_xmultisrc kconfig="/home/root/jsons/kernel_resize_bgr.json"
name=m2m_0 \
! video/x-raw, width=640, height=360, format=BGR \
! queue \
! vvas_xfilter kernels-config="/home/root/jsons/
kernel_densebox_640_360.json" \
! queue \
! vvas_xroigen roi-type=1 roi-qp-delta=-10 roi-max-num=10 resolution-
range="<0,0,500,300>" \
! queue \
! omxh264enc prefetch-buffer=true qp-mode=1
! queue
! fakesink