Kria™ KV260 Vision AI Starter Kit
Known Issues & Limitations
Please first review the FAQ for commonly encountered issues across Kria SOM applications.
The “-R” option of smartcam application for reporting FPS doesn’t print any info when the target “-t” is RTSP.
When using VLC to play the RTSP stream sent out by the application, sometimes it doesn’t work on some client machines, while the FFplay works well.
The KR260 carrier card does not power the AR1335 sensor module auto-focus feature. This is due to cross-sensor pin compatibility allowing the board to support other IAS Sensor modules. The result is that the AR1335 sensor module will not have a dynamic focus and depending on positioning of the objects to the lens the images may appear blurry.
The smartcam application is only supported in desktop_disabled mode (direct display). This is because desktop mode requires RGB16 format, while smartcam assumes optimized input format NV12. Desktop rendering would require SW based color conversion that severely reduces system performance. Thus, it is not supported.
If Logitech Brio camera is only producing 5fps and is enumerated only as a USB2.0 device, ensure to upgrade its firmware to 1.0.423 or later. See Logitech support website for firmware update utilities. Frame rate is reported by the application using the -R option.
When using the MIPI input to the application if the system presents a very low frame-rate (< 2 fps) please either reboot the platform and restart the application or stop the application and unload/load the kv260-smartcam firmware before restarting the application. Frame rate is reported by the application using the -R option.
Jupyter notebook only supports MIPI/USB input, and DP/RTSP output.
The latency of USB/MIPI camera to DP path varies depending on the source type and monitor type (DP monitor / HDMI monitor), because of different inherent latencies of the specific sources devices and display devices. Following is a table of our testing results. We measure the glass to glass latency, by putting a running stopwatch before the camera, and then take a screenshot containing both the original stopwatch and the image of the stopwatch in the monitor, so that we can calculate the latency with
Time_of_stopwatch_in_monitor - Time_physical_stopwatch
|No AI||With AI||No AI||With AI|
|Monitor A||DP connector||60ms||90ms||90ms||120ms|
|Monitor A||HDMI connector||60-70ms||90-100ms||140ms||160ms|
|Monitor B||HDMI connector||260ms-270ms||280ms-290ms||220ms||260ms|
With AI Run smartcam without option “-n” NO AI Run smartcam with option “-n”, turning off AI detection. Monitor A Dell PC monitor P2317H 1080p Monitor B Skyworth 4K TV 50H9D
After invoking the command
xmutil loadappto load firmware, it needs several seconds for the whole firmware to be ready. If you launch docker before the firmware is loaded, it will not be able to use DPU node properly and you may see the below error while launching application
WARNING: Logging before InitGoogleLogging() is written to STDERR F0830 10:01:34.758910 22 xrt_bin_stream.cpp:60] Check failed: fd_ > 0 (-1 vs. 0) , open(/usr/lib/dpu.xclbin) failed. Check failure stack trace: *** Aborted by signal Aborted...
To resolve this issue, exit the docker container and unload the firmware using xmutil. Then delete folder /etc/vart.conf/ using
rm -rf /etc/vart.conf/
root@xlnx-docker/# exit ubuntu@kria:~$ sudo xmutil unloadapp ubuntu@kria:~$ sudo rm -rf /etc/vart.conf/
Now you can start with the app deployment steps from loading the firmware and launching docker.
A known issue with I2S driver will cause a random kernel crash when loading and unloading the kv260-smartcam firmware with the following commands several times.
xmutil unloadapp xmutil loadapp kv260-smartcam
So the audio feature of the application is disabled by disabling the I2S subsystem in the dtsi file, which means the “-A”/”–audio” option with RTSP target will not stream the audio.
The model we use for SSD AI task through the option “–aitask ssd” is ssd_adas_pruned_0_95 from Vitis-AI-Library, which is not a general SSD model, but only for the ADAS scenario as showing by the BDD dataset.
The model we use for Refinedet AI task through the option “–aitask refinedet” is refinedet_pruned_0_96 from Vitis-AI-Library, which is not a general object detection model, but only for the detection of pedestrian such as the person object in COCO dataset.
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright© 2022 Xilinx