Kria™ KR260 Robotics Starter Kit

Machine Vision Camera Tutorial

Introduction

The Machine vision application is built on the KR260 Robotics Starter Kit and demonstrates use of SLVS-EC and GigE Vision standards for transferring of video frames. The design consists of four pipelines

  • Capture pipeline : Capture images from IMX547 Sensor, color or monochrome

  • Image Sensor Processing (ISP) pipeline: Processes and improves the captured mango images

  • Acceleration pipeline: Runs defect detect function on the processed mango images

  • Output pipelines

    • 10GigE : Streams video data received from the the capture pipeline to a host system over 10G SFP+ Transceiver

    • Display Port: Displays detection results and images of the mango at various stages.

The design has a platform and integrated accelerator functions. The platform consists of capture pipeline, output pipelines, and some video processing functions. This approach makes the design leaner and provides maximum programmable logic (PL) for the accelerator development.

The Application Processing Unit (APU) in the Processing System (PS) consists of four ARM Cortex-A53 cores and is configured to run in Symmetric Multi-Processing (SMP) Linux mode in the design. The application running on Linux is responsible for configuring and controlling the pipelines.

The APU application controls the following video data paths implemented in the PL:

  • Capture pipeline captures video frames from the IMX547 sensor through SLVS EC Interface, and converts data to Video AXI streams. The Video AXI streams are broadcasted to 10GigE pipeline and ISP Pipeline

  • ISP Pipeline receives streaming data from the capture pipeline, improves the video frame quality and writes the frame to DDR memory.

  • Acceleration Pipeline is implemented as a memory to memory pipeline. Each block in acceleration pipeline reads from DDR memory, processes the data, and writes back to memory

  • Output pipeline

    • Display pipeline reads video frames from memory, written by the acceleration blocks and sends the frame to Display Port sink.

    • 10GigE pipeline transfers the AXI stream data carrying raw video frames via 10GigE Vision protocol through SFP+ transceiver, to the host machine.

The application running on the host is responsible for interpreting the raw video stream.

Design Components

Hardware components

  • KR260 Robotics Starter Kit including:

    • K26 SOM with an AMD Zynq™ UltraScale+™ MPSoC

    • Carrier Card with SLVS-EC Rx connector, SFP+ cage and Display Port connector

  • Sony IMX547 Camera Kit Monochrome or Color*

  • 10G SFP+ Transceiver*

  • Fiber optic cable* connecting KR260 to the host machine*

  • 10G NIC card to be installed in the host machine*

  • 1080p Monitor to display defect detection results*

[*] Not included in the KR260 Starter Kit

Interfaces and IP

  • Video inputs

    • SLVS-EC Rx

  • Video outputs

    • Display Port

    • 10GigeE

  • Video processing

    • Vitis Vsion Accelerator functions

    • PL and PS based pre and post processing specific to a accelerator function

Software components

  • Operating system

    • APU: SMP Linux

  • Linux kernel subsystems

    • Video source: Video4 Linux (V4L2)

    • Display: Direct Rendering Manager (DRM)/Kernel Mode Setting (KMS)

  • Linux user space frameworks on KR260

    • GStreamer/VVAS

    • AMD Vitis™ AI

    • Xilinx run-time (XRT)

  • Linux user space frameworks on host

    • Frame Grabber SW stack

    • Sphinx GEV Viewer

Supported Resolution and Formats

  • Resolutions

    • 1080p on DP

    • 2472x2128 on Sencor

  • Sensor Pixel formats

    • MONO8

    • BAYRG8

Next Steps

Copyright © 2023–2024 Advanced Micro Devices, Inc.

Terms and Conditions