“Donkey Car featuring the Ultra96 board, a Raspberry Pi, FPGA accelerated stereo vision, MIPI CSI-2 image acquisition, a LiDAR sensor and AI.
In this project, I will document the build of a Donkey Car enhanced with an advanced vision system using FPGA accelerated stereo vision and LiDAR.
The project consists of the following main components:
- Raspberry Pi 3 featuring 4 x Cortex-A53 cores - runs the AI that drives the car
- Avnet / Xilinx Ultra96 Board - used for video acquisition and processing. The high speed MIPI CSI-2 input of two OV5647 cameras are captured using 2 video pipelines implemented in the programmable logic (FPGA) part of the Xilinx Zynq UltraScale+ MPSoC. The captured images are processed by an OpenCV / xfOpenCV application. Using a hardware accelerated stereo matching algorithm (xfOpenCV StereoBM) is applied on the input images and the depth of the scene is computed in real-time
- Donkey Car - an RC car used for PoC
- 2 x OV5647 5 MegaPixel NoIR Camera
- TFMini LiDAR Sensor”