 Kria™ KV260 Vision AI Starter Kit Defect Detection Tutorial |
Hardware Architecture of the Platform |
Hardware Architecture of the Platform¶
Introduction¶
This section describes the design implemented in the programmable logic (PL). The following figure shows the top-level hardware architecture of the reference design.
At a high-level, the design comprises of three pipelines:
Capture/Input pipeline: This comprises of pipelines through which video data is captured.
Capture images from a file source (PS)
MIPI CSI-2 capture pipeline (PL)
Display/Output pipeline: This comprises of pipelines through which video data is output.
DisplayPort pipeline (PL + PS)
Ethernet pipeline (PS)
Accelerator pipeline: This comprises of overlay accelerator functions integrated into the platform using AMD Vitis™.
The Pre-Process block modifies the input data as required by the Defect Detection function (PL).
The Defect Detection block identifies defects in a mango (PL + PS).
Both Pre-Process and Defect Detection use Vitis Vision Library functions.
NOTE: The PS interconnects in the figure are conceptual.
Capture¶
Single Sensor MIPI Capture¶
A capture pipeline receives video frames from an external source and writes it into memory. The single sensor MIPI CSI-2 receiver capture pipeline is shown in the following figure.
This pipeline consists of four components: two that are controlled by the application processing unti (APU) via an AXI4-Lite based register interface; one is controlled by the APU via an inter IC (I2C) register interface, and one is configured statically.
The On Semi AR0144, which is a CMOS active pixel image sensor, sends sensor data to the ISP through a two-lane MIPI interface. It is controlled and programmed by the ISP via a two-wire serial interface. The sensor is connected to the IAS 0 module connector on the ML carrier card. For more information, refer to the datasheet from https://www.onsemi.com/products/sensors/image-sensors-processors/image-sensors/ar0144cs.
The On Semi AP1302 ISP is an image signal processor digital image sensor supporting image resolutions up to 4224x 3156V. AP1302 interfaces to CMOS imaging sensors and performs all the necessary operations required to capture video streams. It performs functions such as Auto White Balancing (AWB), Auto Exposure (AE), Auto Focus (AF), etc. The ISP is controlled via an I2C interface using an AXI I2C controller in the PL. It is mounted on a ML carrier card and has a MIPI output interface that is connected to the MIPI CSI-2 RX subsystem inside the PL. A GPIO is used to reset the IP. For more information, refer to the datasheet from https://www.onsemi.com/products/sensors/image-sensors-processors/image-processors/ap1302.
The MIPI CSI-2 receiver subsystem (CSI RX) includes a MIPI D-PHY core that connects one data lane and one clock lane to the ISP. It implements a CSI-2 receive interface according to the MIPI CSI-2 standard v2.0 with underlying MIPI D-PHY standard v1.2. The subsystem captures images from the ISP in YUV 4:2:2 format and outputs AXI4-Stream video data. At two pixels per clock (ppc) and eight bits per pixel, the AXI4-Stream width is 32-bits. For more information, see the MIPI CSI-2 Receiver Subsystem Product Guide (PG232).
The AXI subset converter, see the AXI4-Stream Infrastructure IP Suite LogiCORE IP Product Guide (PG085), is a statically-configured IP core that converts the 32-bit AXI4-Stream input data to a 48-bit AXI4-Stream output data by adding zeros to the most significant bit (MSB) of each data word. At two ppc, the AXI4-Stream width is 48-bits. The format on the AXI4-Stream is as follows.
The video frame buffer takes Y8 AXI4-Stream input data and converts it to AXI4-MM format, which is written to memory. The Y stream is stored in memory as follows.
The AXI-MM interface is connected to the S_AXI port of the PS. For each video frame transfer, an interrupt is generated. A general purpose I/O (GPIO) is used to reset the IP between resolution changes. For more information, see the Video Frame Buffer Read and Video Frame Buffer Write LogiCORE IP Product Guide (PG278).
All the IPs in this pipeline are configured to transport 2 ppc at 275 MHz, enabling 1280x800 at 30 fps resolution set on the sensor.
Display¶
An output pipeline reads video frames from memory and sends the frames to a sink. In this case, the sink is a display port in the PS. The display pipeline is shown in the following figure.
This PL pipeline consists of four components, two of which are controlled by the APU via an AXI4-Lite base register interface and the other two are statically configured.
The Video Mixer IP core is configured to support blending of up to four overlay layers into one single output video stream. The four layers are configured to be memory-mapped AXI4 interfaces connected to the PS via interconnects. The primary AXI4-MM layer has the resolution set to match the display. The other layers, whatever their resolution, is blended with this layer. In this design, the overlay layers are configured to be Y8, while the AXI4-Stream output from the mixer is set to YUV 422 format. A GPIO is used to reset the subsystem between resolution changes. For more information, refer to the Video Mixer LogiCORE IP Product Guide (PG243).
The AXI4-Stream subset converter passes the least signficant bits (LSBs) to the Video Out converter and drops the MSB.
All video systems require management of video timing signals, which are used to synchronize processes. The Video Timing Controller (VTC) serves the function of both detecting and generating these timing signals. In this design, it generates timing control signals which are used by the AXI4-Stream to Video Out converter to produce native video output signals. For more information, refer to the Video Timing Controller Product Guide (PG016).
The AXI4-Stream to Video Out core provides a bridge between video processing cores with AXI4-Stream interfaces and native video output (parallel video data, video syncs, and blanks). The core works with the Video Timing Controller (VTC) core. The synchronizer section of the core synchronizes timing from the VTC to the video data from the AXI4-Stream bus. For more information, refer to the AXI4-Stream to Video Out LogiCORE IP Product Guide (PG044).
The output from the AXI4-Stream to Video Out Converter goes through SOM slices and concat blocks to drive the DisplayPort Live signals on the PS. The slices and concat are required to the adhere to the Live Video data format. For more information on DisplayPort Controller in Live mode, refer to the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Clocks, Resets, and Interrupts¶
Clocks¶
The following table identifies the main clocks of the PL design, their source, their clock frequency, and their function.
| Clock | Clock Source | Clock Frequency | Function |
|---|---|---|---|
| pl_clk0 | PS | 100 MHz | Clock source for clocking wizard (clk_wiz_0) generating clocks |
| clk_200M | Clocking wizard | 200 MHz | MIPI D-PHY core clock |
| clk_100M* | Clocking wizard | 100 MHz | AXI4-Lite clock to configure the different video IPs in the design. |
| clk_275M* | Clocking wizard | 275 MHz | AXI4 MM clock and AXI4-Stream clock used in the capture pipeline, display pipeline, and processing pipeline |
| clk_50M | Clocking wizard | 50 MHz | phase-locked loop (PLL) reference clock for the VCU IP |
| clk_550M* | Clocking wizard | 550 MHz | Generated to be used by the accelerator |
| clk_out1 | Clocking wizard | Variable | A clocking wizard with an AXI4 control interface is used in the display path. Based on the video resolution, the software can configure the required clock frequency through the AXI control interface. The output clock is used to drive the Video Timing Controller, AXI4-Stream to Native Video converter, and the DP Live clock port on the PS. |
*Clocks that are exposed as a Platform interface and can be used by an accelerator.
Resets¶
The following table summarizes the resets used in this design.
| Reset Source | Function |
|---|---|
| pl_resetn0 | PL reset for proc_sys_reset modules and the clock_wizard generating clocks for the video domain |
| rst_processor_100MHz | Synchronous resets for clk_out100M clock domain |
| rst_processor_275MHz | Synchronous resets for clk_out275M clock domain |
| rst_processor_550MHz | Synchronous resets for clk_out550M clock domain |
| pros_sys_reset_0 | Synchronous resets for clk_out1 clock domain |
| GPIOs | |
| emio_gpio_o[0] | Video Mixer Reset |
| emio_gpio_o[1] | Frame Buffer Write Reset |
| emio_gpio_o[2] | AP1302 ISP Reset |
| ____ |
Interrupts¶
The following table lists the PL-to-PS interrupts used in this design.
| Interrupt ID | Instance |
|---|---|
| pl_ps_irq1[0] | Video Mixer IP |
| pl_ps_irq1[1] | Video Timing Controller IP |
| pl_ps_irq1[2] | AXI I2C IP |
| pl_ps_irq1[3] | MIPI CSI RX Subsystem IP |
| pl_ps_irq1[5] | Frame Buffer Write IP |
| pl_ps_irq0 | Exposed as a Platform interface and can be used by an accelerator |
Resource Utilization¶
The following table reports the resource utilization numbers on this platform post-implementation.
| Resource | Utilization | Available | Utilization % |
|---|---|---|---|
| LUT | 20912 | 117120 | 17.86 |
| LUTRAM | 1734 | 57600 | 3.01 |
| FF | 30371 | 234240 | 12.97 |
| BRAM | 18 | 144 | 12.50 |
| URAM | 1 | 64 | 1.56 |
| DSP | 30 | 1248 | 2.40 |
| IO | 9 | 186 | 4.84 |
| BUFG | 10 | 352 | 2.84 |
| MMCM | 2 | 4 | 50.00 |
| PLL | 1 | 8 | 12.50 |
Next Steps¶
Copyright © 2021-2024 Advanced Micro Devices, Inc