Main Content

Marker-Based Augmented Reality on Ultra96-V2

Explore augmented reality on Ultra96-V2 by adding computer generated graphics to the live USB camera video stream.

Augmented Reality (AR) is a live view of a real-world environment which has been augmented with computer-generated graphics. There are two ways of implementing AR:

marker-based AR
marker-less AR
Marker-less AR is more complex and relies on the real environment to determine references points. This usually involves recognizing objects and or background cues such as floors and walls.

Marker-based AR is simpler to implement and relies on square markers located in the scene.

This project will describe how to implement marker-based AR on the Ultra96-V2 using OpenCV.

Let’s get started !

Before getting started, I wanted to share the inspiration and motivation for this project.

I was inspired by the following innovative products/tutorials:

Pantone, Color Matching Card,
Adrian Rosebrock, Detecting ArUco markers with OpenCV and Python, PyImageSearch, (accessed on 07 May, 2021)

My motivation for implementing something similar was to be able to use the markers to automatically trigger some kind of calibration, such as:

white balance, using a white reference chart
stereo calibration, using a checkerboard reference chart
In order to achieve this, I created the following three charts (with Microsoft Word) to experiment with.

In this project, I will detect the presence of these charts, and perform additional processing, depending on the chart:

Chart 1 - Draw a green box around the checkboard pattern
Chart 2 - Measure the average values of B, G.R pixels in the area within the markers, and display bar graph with values on chart
Chart 2 - Measure the color histograms in the area within the markers, and display histograms on chart
For more information on generating these markers, please refer to the following excellent tutorials:

Adrian Rosebrock, Generating ArUco markers with OpenCV and Python, PyImageSearch, (accessed on 07 May, 2021)”

Link to article