Image Sensor Processing (ISP) Pipeline¶

ISP Pipeline example resides in L3/examples/isppipeline directory.

This benchmark tests the performance of isppipeline function. Image Sensor Processing (ISP) is a pipeline of functions that enhance the overall visual quality of the raw image from the sensor.

The tutorial provides a step-by-step guide that covers commands for building and running kernel.

Executable Usage¶

Work Directory(Step 1)

The steps for library download and environment setup can be found in README of L3 folder. For getting the design,

cd L3/examples/isppipeline

Build kernel(Step 2)

Run the following make command to build your XCLBIN and host binary targeting a specific device. Please be noticed that this process will take a long time, maybe couple of hours.

export OPENCV_INCLUDE=< path-to-opencv-include-folder >
export OPENCV_LIB=< path-to-opencv-lib-folder >
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:< path-to-opencv-lib-folder >
export DEVICE=< path-to-platform-directory >/< platform >.xpfm
make host xclbin TARGET=hw

Run kernel(Step 3)

To get the benchmark results, please run the following command.

make run TARGET=hw

Example output(Step 4)

-----------ISP Pipeline Design--------------------------------------------------------------------------------
Found Platform
Platform Name: Xilinx
XCLBIN File Name: krnl_ISPPipeline
INFO: Importing vision/L3/examples/isppipeline/build_dir.hw.xilinx_u200_xdma_201830_2/krnl_ISPPipeline.xclbin
Loading: 'vision/L3/examples/isppipeline/build_dir.hw.xilinx_u200_xdma_201830_2/krnl_ISPPipeline.xclbin'
The maximum depth reached by any of the 8 hls::stream() instances in the design is 196608
---------------------------------------------------------------------------------------------------------------

Profiling¶

The ISP Pipeline design is validated on Alveo U200 board at 300 MHz frequency. The hardware resource utilizations are listed in the following table.

Table 1 Hardware resources for ISP Pipeline¶
Dataset			LUT	BRAM	FF	DSP
Resolution	NPPC	other params	LUT	BRAM	FF	DSP
4K	1	Filter - 3x3	18987	24	17713	91
FHD	1	Filter - 3x3	19254	19	17534	88

The performance is shown below

Table 2 Performance numbers in terms of FPS (Frames Per Second) for ISP Pipeline¶
Dataset	FPS(CPU)	FPS(FPGA)
4k (3840x2160)	0.11	135
Full HD(1920x1080)	0.44	520