 Kria™ KR260 Robotics Starter Kit |
Machine Vision Camera Tutorial |
Hardware Architecture of the Platform¶
Introduction¶
This section describes the design implemented in the programmable logic (PL). The following figure shows the top-level hardware architecture of the reference design.
At a high-level, the design comprises of four pipelines:
Capture/Input pipeline: This comprises of pipelines through which video data is captured.
Capture images from a file source (PS)
SLVS-EC capture pipeline (PL)
Image Sensor Processing (ISP) pipeline: This comprises of functions to improve image quality
ISP for color stream (PL)
ISP for mono stream (PL)
Display/Output pipeline: This comprises of pipelines through which video data is output.
Display port as sink for output images (PS)
10G Ethernet pipeline (PL)
Accelerator pipeline: This comprises of overlay accelerator functions integrated into the platform using AMD Vitis™.
The Pre-Process block modifies the input data as required by the Defect Detection function (PL).
The Defect Detection block identifies defects in a mango (PL + PS).
Both Pre-Process and Defect Detection use Vitis Vision Library functions.
NOTE: The PS interconnects in the figure are conceptual.
Capture Pipeline¶
SLVS-EC Capture
A capture pipeline receives video frames from an external source and writes it into memory or streams it to another IP. The SLVS-EC receive capture pipeline is shown in the following figure.
This pipeline consists of four components: two that are controlled by the application processing unit (APU) via an AXI4-Lite based register interface; one is controlled by the APU via an Inter Integrated Circuit (I2C) register interface, and one is configured statically.
The Sony IMX547 CMOS active pixel image sensor, sends sensor data to the SLVS RX IP through a two-lane Gigabit Trasceiver (GT) interface. It is controlled and programmed by via an I2C interface using an AXI I2C controller in the PL. The Framos Sensor Module host the Sony IMX547 camera sensor, and connects to the KR260 carrier card via a flex cable and Framos Sensor Adaptor (FSA). For more information, refer to MV Camera Sensor
The Framos SLVS-EC RX IP Core is a receiver that handles the byte-to-pixel conversion of the incoming sensor data stream. The IP Core provided by Framos encompasses Xilinx GTs, and the logic to process the sensor image and provide it on Parallel Pixel Interface (PPI). For more information, refer to FRAMOS SLVS-EC RX IP CORE
The Framos PPI to Video IP Core converts the PPI data to a parallel video stream
The LogiCORE™ IP Video In to AXI4-Stream Video In to AXI4-Stream is designed to interface to video data (clocked parallel video data with synchronization signals - active video with either syncs, blanks or both) and convert it to the AXI4-Stream Video Protocol Interface. For more information, refer to Video In to AXI4-Stream IP Product Guide
The IPs in this pipeline are configured to transport 4 ppc at 237.6 MHz.
An AXI4-Stream Broadcaster IP is used in the PL to broadcast 64-bit AXIS Video data from the SLVS_EC pipeline to the ISP pipeline and 10GigE pipeline.
Image Sensor Processing (ISP) pipeline¶
The ISP pipeline converts the sensor data to Y8 format, resizes the image to 1920x1080p and writes it to memory. The acceleration pipeline reads these frames. ISP pipeline is shown in the following figure.
The AXI subset converter, is a statically-configured IP core that converts the 64-bit AXI4-Stream from the capture pipeline to a 32-bit AXI4-Stream output data by dropping the least significant bit (LSB) of each data word. The most significant bits carry the relevant information.
The ISP function from the AMD Vitis™ Vision libraries is a collection of functions that enhances the overall visual quality of the raw image from the sensor.
Functions implemented for color sensor
Gain Control
Demoaosaicing
Auto white balancing
Color correction
Gamma correction
Resizing to 1920x1080
For color the raw image is converted to BGR8 format, 24 bits per pixel, 96-bits for 4ppc the
Functions implemented for mono sensor
Gain Control
Gamma correction
Resizing to 1920x1080
For mono the raw image is converted to Y8 format, 8 bits per pixel, 32-bits for 4ppc the
The AXI subset converter, is a statically-configured IP core that converts the 32-bit AXI4-Stream input data to a 96-bit AXI4-Stream output data by adding zeros to the most significant bit (MSB) of each data word. At four ppc, the AXI4-Stream width is 96-bits. This is applicable to the mono pipeline only.
The video frame buffer takes AXI4-Stream (RGB8 for color and Y8 for mono) input data and converts it to AXI4-MM format, which is written to memory. The AXI-MM (Memory Mapped) interface is connected to the S_AXI port of the PS. For each video frame transfer, an interrupt is generated. A general purpose I/O (GPIO) is used to reset the IP. For more information, see the (Video Frame Buffer Read and Video Frame Buffer Write LogiCORE IP Product Guide).
All the IPs in this pipeline are configured to transport 4 ppc at 237.6 MHz.
Output pipeline - 10GigE¶
The 10GigE pipeline transfers the captured raw data to the host system via an SFP connection. The block diagram of the 10GigE Vision pipeline is shown below. The major blocks in the pipeline are provided by Sensor to Image (S2I)
The pipeline consists of following modules:
Framebuffer with Gen DC: The AXI4 stream output of the SLVS-EC IP pipeline is forwarded to Framebuffer IP. This IP buffers the image data in memory. The Framebuffer also prepares the data to be sent in GenICam GenDC format, which allows for sending different types of data, for example, raw image, preprocessed image, metadata etc in a data container. It sends ethernet packet size data chunks to the 10GigE packet composer(XGigE).
XGigE core: This module handles all the low-level networking features to the rest of the system. It forms the GigE Vision stream channel and provides networking interface for the CPU system.
10GigE MAC: This module implements the Media Access Controller (MAC) and sends data to the PHY over XGMII (Media-Independant-Interface).
The AMD 10G/25G High Speed Ethernet Subsystemis configured to support the physical coding sublayer/physical medium attachment (PCS/PMA) functions at 10G speeds. The IP implements the PHY layer of the 10GigE protocol and transmits the packetized sensor data on the Gigabit transceiver
Resource Utilization after Synthesis¶
| Blocks | LUT | FF | BRAM | URAM | DSP | GT |
|---|---|---|---|---|---|---|
| SLVS-EC | 1886 | 2948 | 22.5 | 0 | 0 | 2 |
| 10GigE IPs | 13889 | 18809 | 35 | 0 | 0 | 0 |
| Ethernet System | 2745 | 5010 | 0 | 0 | 0 | 1 |
| ISP-Color | 24530 | 24744 | 26 | 14 | 166 | 0 |
| ISP-Mono | 8939 | 9787 | 9 | 6 | 26 | 0 |
References¶
To get the 10GigE IP license, contact Sensor to Image S2I.
S2I Release note here.
Download the packages from S2I here.