Vitis Video Analytics SDK Release V3.0¶
Based on 2022.2 Tools
Based on Vitis AI 3.0 for Machine Learning
Based on GStreamer 1.18.5
- Embedded Platforms
Zynq UltraScale+ MPSoCas well as
- PCIe/Data Center Platform
What is new in this release?¶
- Added support for below mentioned models
Added software resize and colorspace conversion support in
Added support for xmodel where user can use any model supported by Vitis AI and
vvas_xinferplug-in will perform the ML operation and returns the output tensor. User can perform his/her post-processing on the output tensor data. For more details, refer to vvas_xinfer
In Embedded platform example designs Autologin has been disabled and user need to enter login and password for each boot. “root” is the default user name and users are required to set the new password at the time of fist login.
VVAS Tracker implementation uses Simd <https://github.com/ermig1979/Simd> library for high performance image processing operations. User has option to build VVAS with Simd library or not.
Other changes compared to previous release¶
vvas_xboundingboxaccel-sw-lib has been discontinued and removed from this release.
vvas_xoverlayplug-ins can be used to implement the features supported by
vvas_xboundingboxand many more new features.
vvas_xdpuinferaccel-sw-lib has been discontinued and removed from this release.
vvas_xinferplug-in shall be used for the inference requirements.
JSON file schema and fields have been updated. Refer to the relavant section in this document to know details about the modifications. In case you were already using previous version of VVAS release, then please adapt your applications as per changes in this release.
vvas_xpreprocessoraccel-sw-lib has been discontinued and removed from this release. Pre-processing functionality can be achieved through
vvas_xabrscalerplug-in. Preprocessing needed by Machine Learning can also be done by
vvas_xvcudecplug-in has been discontinued and removed from this release and instead
vvas_xvideodecplug-in has been introduced for video decoding on PCIe platform. This plug-in will not work on Alveo U30 platform.
vvas_xvcuencplug-in is not supported this release.
vvas_xlookaheadplug-in is not supported in this release.
v_multiscalerkernel has been renamed to
image_processingkernel to align with the features it supports.
Support for Alveo U30 card has been discontinued and removed from this release
Encoding and transcoding support has been removed in PCIe platform.
GstInference metadata structure has been re-organized and hence it is not backward compatible with previous VVAS releases. In case you were already using previous version of VVAS release and GstInference metadata, then please adapt your applications as per changes in this release.
Supports different input sources like Camera, RTP/RTSP streaming, file source etc.
Hardware accelerated H264/H265 Video Encoding/Decoding.
Vitis AI based inferencing for detection and classification
VVAS offers extensive support for popular AI model classes like Object Detection, Object Classification, Segmentation, Pose Detection etc. For complete list of supported classes of models, refer to model-class in infer-config section of vvas_xinfer
Hardware accelerated Resize and color space conversion
On-screen displaying bounding box around objects, text, circle. arrows, polygon etc.
HDMI Tx and Display Port interface for displaying the contents
Image_processing kernel require minimum cropping width and height to be 16. In case the cropping width or height is less than 16, then cropping is not performed.
Due to the above limitation, during cascaded ML inferencing, in case the first stage ML inference generates objects with region of interest having width and/height less than 16, then the object is not cropped and the ML inference for that object in next stages will not be correct.
Image Processingkernel is being used for resize, scaling or cropping, then output resolution width must be multiple of 8*PPC (Pixel Per Clock).
Image_processingkernel can be configured to process 1, 2 or 4 Pixels Per Clock. Higher the PPC value, higher will be throughput. The kernel will always write the data in chunks of 8*PPC. In case the output width is not multiple of 8*PPC, then there may be some garbage data in the right side of the image, depending on the difference between the actual width and the 8*PPC aligned width.
All inputs to the
vvas_xcompositorshall have same frame rate otherwise the processing will be controlled by the slowest frame rate input stream.
For platform/example specific known limitations, refer to the example/application design.
On zcu104 boards, Cascaded pipelines OR several ML instances running simultaneously OR few models need more processing power when running in performance mode are sending board into bad state and needs reboot to recover from it. The default value of IOUT_OC_FAULT_LIMIT on PMIC chip irps5401 is too low and that is causing the temperature fault limit getting crossed. Workaround is to increase this limit. But there is risk of board getting damaged if running for long time.