Image Sensor Processing (ISP) Pipeline

ISP Pipeline example resides in L3/examples/isppipeline directory.

This benchmark tests the performance of isppipeline function. Image Sensor Processing (ISP) is a pipeline of functions that enhance the overall visual quality of the raw image from the sensor.

The tutorial provides a step-by-step guide that covers commands for building and running kernel.

Executable Usage

  • Work Directory(Step 1)

The steps for library download and environment setup can be found in README of L3 folder. For getting the design,

cd L3/examples/isppipeline

  • Build kernel(Step 2)

Run the following make command to build your XCLBIN and host binary targeting a specific device. Please be noticed that this process will take a long time, maybe couple of hours.

export OPENCV_INCLUDE=< path-to-opencv-include-folder >
export OPENCV_LIB=< path-to-opencv-lib-folder >
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:< path-to-opencv-lib-folder >
export DEVICE=< path-to-platform-directory >/< platform >.xpfm
make host xclbin TARGET=hw

  • Run kernel(Step 3)

To get the benchmark results, please run the following command.

make run TARGET=hw

  • Example output(Step 4)

-----------ISP Pipeline Design--------------------------------------------------------------------------------
Found Platform
Platform Name: Xilinx
XCLBIN File Name: krnl_ISPPipeline
INFO: Importing vision/L3/examples/isppipeline/build_dir.hw.xilinx_u200_xdma_201830_2/krnl_ISPPipeline.xclbin
Loading: 'vision/L3/examples/isppipeline/build_dir.hw.xilinx_u200_xdma_201830_2/krnl_ISPPipeline.xclbin'
The maximum depth reached by any of the 8 hls::stream() instances in the design is 196608
---------------------------------------------------------------------------------------------------------------

Profiling

The ISP Pipeline design is validated on Alveo U200 board at 300 MHz frequency. The hardware resource utilizations are listed in the following table.

Table 1 Hardware resources for ISP Pipeline

Dataset

LUT

BRAM

FF

DSP

Resolution

NPPC

other params

4K

1

Filter - 3x3

18987

24

17713

91

FHD

1

Filter - 3x3

19254

19

17534

88

The performance is shown below

Table 2 Performance numbers in terms of FPS (Frames Per Second) for ISP Pipeline

Dataset

FPS(CPU)

FPS(FPGA)

4k (3840x2160)

0.11

135

Full HD(1920x1080)

0.44

520