“We have looked at thermal imaging before however, we did that from scratch using Vivado and Software. In this project we are going to combine the FLIR Lepton with PYNQ, doing so allows me to create a PYNQ overlay that fellow lepton users can use to get started accelerating there applications without the need to worry about how to interface with the lepton sensor.
This is going to be a fun project as we are going to create the following
VHDL module and test bench to interface with the Lepton
PYNQ Overlay to control the lepton from Python
Jupyter note book to pull it all together
All of these will be available for download and use from my GitHub. It will be fun as well as it is not often I write a lot of VHDL for these projects,
FPGA Approach
The Lepton outputs it video in a very interesting manner, it uses SPI to create something called VoSPI or video over SPI. This uses the SCLK, SS and MISO lines on a traditional SPI interface.
Over this SPI link the video is transferred as in packets, there are eighty 14 bit pixels, along with a CRC and header in a packet. For the Lepton 2 there are 60 packets, one for each row.
To comply with export restrictions, the frame rate of the Lepton is 9 Hz this means it outputs some frames which are to be discarded. These are discard frames are identified in the packet header as non-valid, valid frames are indicated by a packet number between 1 and 60.
As the frame rate is so slow we can have a simple interface which captures and processes the 16 bit serial word on the MISO signal. Determines if it is discard frame, or if valid reads in the packet.
Valid packets will be written into a Dual Port RAM such that the RAM will contain a linear increment of all pixels in the image all 4800 in total.
We need to be careful here when we develop the VHDL module to ensure the code increments addresses on the word boundary and not byte. As such it will increment by 4 for each new address.
This Dual Port RAM will be able to be accessed from the PS using PYNQ and the image taken. We can then use PYNQ and Python to do more advanced processing or even recording on the output.
The IP core will be capable of being reset if video synchronization is lost however, it is designed such that discard packets will be detected and ignored.
As such we will create the two VHDL files a source file which will be used in the overlay and a test bench which is used as to test source file. This enables us to simulate the design and ensure we have the functionality as required.
To ensure the correct behavior the test bench will apply the following
Three Discard frames following the assertion of SS
Sixty packets of 80 pixels, with valid header and CRC
This frame will repeat twice, to ensure the IP core works correctly once the first frame has been received.”