“Explore augmented reality on Ultra96-V2 by adding computer generated graphics to the live USB camera video stream.
Introduction
Augmented Reality (AR) is a live view of a real-world environment which has been augmented with computer-generated graphics. There are two ways of implementing AR:
marker-based AR
marker-less AR
Marker-less AR is more complex and relies on the real environment to determine references points. This usually involves recognizing objects and or background cues such as floors and walls.
Marker-based AR is simpler to implement and relies on square markers located in the scene.
This project will describe how to implement marker-based AR on the Ultra96-V2 using OpenCV.
Let’s get started !
Inspiration
Before getting started, I wanted to share the inspiration and motivation for this project.
I was inspired by the following innovative products/tutorials:
Pantone, Color Matching Card, https://www.pantone.com/pantone-color-match-card
Adrian Rosebrock, Detecting ArUco markers with OpenCV and Python, PyImageSearch, https://www.pyimagesearch.com/2020/12/21/detecting-aruco-markers-with-opencv-and-python/ (accessed on 07 May, 2021)
Motivation
My motivation for implementing something similar was to be able to use the markers to automatically trigger some kind of calibration, such as:
white balance, using a white reference chart
stereo calibration, using a checkerboard reference chart
In order to achieve this, I created the following three charts (with Microsoft Word) to experiment with.
In this project, I will detect the presence of these charts, and perform additional processing, depending on the chart:
Chart 1 - Draw a green box around the checkboard pattern
Chart 2 - Measure the average values of B, G.R pixels in the area within the markers, and display bar graph with values on chart
Chart 2 - Measure the color histograms in the area within the markers, and display histograms on chart
For more information on generating these markers, please refer to the following excellent tutorials:
Adrian Rosebrock, Generating ArUco markers with OpenCV and Python, PyImageSearch, https://www.pyimagesearch.com/2020/12/14/generating-aruco-markers-with-opencv-and-python/ (accessed on 07 May, 2021)”