VVAS C API Samples¶

The VVAS release package contains two sample application code demonstrating the usage of the VVAS Core APIs.

Single Stream Car Classification Pipeline Sample Application
Multiple Streams Car Classification Pipeline Sample Application

Note

These sample applications are merely used to help you to understand the VVAS Core APIs functionalities and how they can be used in code. For that purpose, the implementations such as control flow and data structures used in these applications are just for the sake of API usage demonstration, and not intended to demonstrate an ideal optimized implementation. You, as an application developer, can use your desired control-flow/data structure implementation to use the VVAS Core APIs based on your application requirements.

Sample Video Preparation¶

To run the sample application you need a sample video of moving cars.

Download this free video from https://pixabay.com/videos/traffic-cars-road-street-56696/ in 1920x1080 format.
Convert the mp4 video file to h264 file using the below command

gst-launch-1.0 -v filesrc location=/workspace/traffic-56696.mp4 ! \
                  qtdemux ! h264parse ! video/x-h264,stream-format=byte-stream ! \
                  filesink location=/workspace/Test.h264

Single Stream Car Classification Pipeline Sample Application¶

Sample application code path:

The sample application code is present in the file test_cascade_yolov3_3xresnet.cpp at VVAS github repo.

Running the application

The application code is already compiled inside the docker environment. To run the application you can use the following command:

test_video_ml -i /workspace/Test.h264 \
              -j /opt/xilinx/vvas/share/cascade_yolov3_3xresnet18/cascade_yolov3_3xresnet18.json \
              -x /opt/xilinx/xclbin/v70.xclbin \
              -d 0 \
              -n "kernel_vdu_decoder:{kernel_vdu_decoder_0}"

Standalone compilation of the sample application

To compile this sample application standalone you can use the following Makefile (make app)

APP =  test_video_ml
APP_SOURCE_FILES  = test_cascade_yolov3_3xresnet.cpp

XRT_PKG_CFG=`pkg-config --cflags --libs xrt`
VVAS_UTILS_PKG_CFG=`pkg-config --cflags --libs vvas-utils`
VVAS_CORE_PKG_CFG=`pkg-config --cflags --libs vvas-core`
VVAS_CORE_MACROS=-DVVAS_GLIB_UTILS -DXLNX_PCIe_PLATFORM -DXLNX_V70_PLATFORM

${APP}: ${APP_SOURCE_FILES}
     g++ -Wall -g $(APP_SOURCE_FILES) $(XRT_PKG_CFG) $(VVAS_UTILS_PKG_CFG) $(VVAS_CORE_PKG_CFG) $(VVAS_CORE_MACROS) -o $(APP)

app:
     make ${APP}

clean:
     rm ${APP}

Multiple Streams Car Classification Pipeline Sample Application¶

Sample application code path:

The sample application code is present in the file test_cascade_yolov3_3xresnet_mt.cpp at VVAS github repo.

Running the application: The application code is already compiled inside the docker environment. To run the application you can use the following command:

Before running the application update the /opt/xilinx/vvas/share/cascade_yolov3_3xresnet18/cascade_yolov3_3xresnet18.json with the path of downloaded sample video files. We will take 16 input streams with the same video.

"input-streams":[
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264",
                 "/workspace/Test.h264"
              ]

Run the application using the following command:

test_video_ml_mt -j /opt/xilinx/vvas/share/cascade_yolov3_3xresnet18/cascade_yolov3_3xresnet18.json

Standalone compilation of the sample application

To compile this sample application standalone you can use the following Makefile (make app)

APP =  test_video_ml_mt
APP_SOURCE_FILES  = test_cascade_yolov3_3xresnet_mt.cpp

XRT_PKG_CFG=`pkg-config --cflags --libs xrt`
VVAS_UTILS_PKG_CFG=`pkg-config --cflags --libs vvas-utils`
VVAS_CORE_PKG_CFG=`pkg-config --cflags --libs vvas-core`
VVAS_CORE_MACROS=-DVVAS_GLIB_UTILS -DXLNX_PCIe_PLATFORM -DXLNX_V70_PLATFORM

${APP}: ${APP_SOURCE_FILES}
       g++ -Wall -g $(APP_SOURCE_FILES) $(XRT_PKG_CFG) $(VVAS_UTILS_PKG_CFG) $(VVAS_CORE_PKG_CFG) $(VVAS_CORE_MACROS) -o $(APP) -pthread -lvvascore_pool-1.0

app:
       make ${APP}

clean:
       rm ${APP}

Description of the sample application

This sample application constructs a car classification pipeline for 16 streams, where 4 streams are fed to each funnel. This application will show car classification data in Car Color, Make and Type order.

Application displays performance in frames per second (FPS) like the GStreamer pipeline.

Below is a sample FPS log.

00:34.009185754 [Sink_2.0] rendered: 162, current FPS: 16.79, average FPS: 16.79
00:34.009219157 [Sink_2.1] rendered: 144, current FPS: 16.32, average FPS: 16.32
00:34.009236560 [Sink_2.2] rendered: 158, current FPS: 16.37, average FPS: 16.37
00:34.009247114 [Sink_2.3] rendered: 152, current FPS: 15.89, average FPS: 15.89
00:34.021963566 [Sink_3.0] rendered: 158, current FPS: 16.49, average FPS: 16.49
00:34.022015281 [Sink_3.1] rendered: 166, current FPS: 17.21, average FPS: 17.21
00:34.022030713 [Sink_3.2] rendered: 147, current FPS: 15.55, average FPS: 15.55
00:34.022044064 [Sink_3.3] rendered: 146, current FPS: 15.44, average FPS: 15.44
00:34.000343033 [Sink_0.0] rendered: 154, current FPS: 16.22, average FPS: 16.22
00:34.000447190 [Sink_0.1] rendered: 144, current FPS: 16.08, average FPS: 16.08
00:34.000478190 [Sink_0.2] rendered: 165, current FPS: 17.13, average FPS: 17.13
00:34.000499755 [Sink_0.3] rendered: 162, current FPS: 16.82, average FPS: 16.82

Considering the last line of the above log,

0:00:34.000499755 : This number shows the relative time since the application is started
Sink_0.3: Shows the sink number. The first number (0) shows the funnel number and 2nd number (3) shows the sink in that funnel.

Unlike GStreamer pipeline, this application does not have limitation that all streams should have same caps. Users can have streams with different resolution, framerate, and different video codec format as well.

This sample application supports following video file name extensions

For H.264: .h264, .264, .avc
For H.265: .h265, .265, .hevc

The input argument cascade_yolov3_3xresnet18.json is the master json file, it has all the configuration needed to run this application. The JSON file content is as below

{
"yolov3-config-path":"/opt/xilinx/vvas/share/cascade_yolov3_3xresnet18/yolov3_voc_tf.json",
"resnet18-carmake-config-path":"/opt/xilinx/vvas/share/cascade_yolov3_3xresnet18/resnet18_vehicle_make.json",
"resnet18-cartype-config-path":"/opt/xilinx/vvas/share/cascade_yolov3_3xresnet18/resnet18_vehicle_type.json",
"resnet18-carcolor-config-path":"/opt/xilinx/vvas/share/cascade_yolov3_3xresnet18/resnet18_vehicle_color.json",
"metaconvert-config-path":"/opt/xilinx/vvas/share/cascade_yolov3_3xresnet18/metaconvert_config.json",
"xclbin-location":"/opt/xilinx/xclbin/v70.xclbin",
"dev-idx" : 0,
"additional-decoder-buffers": 14,
"batch-timeout": 400,
"fps-display-interval": 5,
"repeat-count":1,
"sink-type":0,
"log-level":0,
"input-streams":[
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264",
   "/workspace/Test.h264"
]
}

Parameters in the master JSON file passed to the sample application

yolov3-config-path: Yolov3 inference configuration file.
resnet18-carmake-config-path: Car Make inference configuration file.
resnet18-cartype-config-path: Car Type inference configuration file.
resnet18-carcolor-config-path: Car Color inference configuration file.
metaconvert-config-path: Meta convert configuration file.
xclbin-location: XCLBIN location.
dev-idx: Device index
additional-decoder-buffers: Additional decoder output buffers to allocate to run pipeline efficiently. Refer section Calculating Decoder output buffers for more details. The default value in the JSON is calculated for playing 4 streams in one funnel.
batch-timeout: Maximum time (in milliseconds) to wait for inference batch to be formed before submitting the batch for inference. The default value is calculated based on batch size of 14 and 30 FPS video.
repeat-count: Number of times each stream should be repeated.
fps-display-interval: FPS display interval in seconds (Minimum 1 sec), Default 5 seconds
sink-type: Sink type, 0 - Fake sink, 1 – File sink
log-level: Logging level 0 -3 (0 – Error, 1 – Warning, 2 – Info, 3 – Debug)
input-streams: An array of input streams. Update default strings in above configuration file with the available H264/H265 files.