“A magic wand that uses machine learning to learn and recognize wand gestures, powered by Arduino Nano 33 BLE, TensorFlow, and PlatformIO.
Let the magic fills this year’s Christmas and upcoming new year. For that, I built a magic wand :) And I use it to properly greet the holidays in a geeky possibly way.
What “magical” about this wand is that it uses Machine Learning (ML) to learn and recognise the wand gestures, only from the data collected from accelerometer sensor (that measures acceleration force). By using ML, the wand movement doesn’t have to be precise, it can be still recognised as gesture.
Arduino Nano 33 BLE Sense board as the main hardware with ARM Cortex M4 Microcontroller (MCU), is attached on a stick, that functions to collect acceleration data from on-board accelerometer sensor, then process and infer the prediction (hence, inference) of performed gesture, right on the Microcontroller (MCU) itself. ML software framework powered by TensorFlow Lite is needed to perform such inference, and optimised to run on low power MCU that only runs at 64 MHz and 256KB of RAM.
No magic included, only pure code :)
However, ability to run TensorFlow for ML inference on low power MCU, is kinda magical.
Notes:
If you search “Magic Wand” in hackster.io projects collection, you’ll find some similar projects, but so far none of them that:
Allows “correct” or convenient board orientation to do the gestures.
Leverages PlatformIO. I’ll explain why it’s important. Keep on reading.
Removes you from some hassles, explained below.
My project offers better experience - both for the wand users, and for developers or makers like yourself.
Original Work
Some says, there’s nothing new under the sun, and this project is not exception. It is based on the original Magic Wand sample code from TensorFlow Lite for Microcontroller. The TensorFlow authors were kind enough to provide a simple yet complete and fun example on how to infer physical movement from accelerometer, all runs on MCU with no OS, not Android, not Linux. Something that I’ve been searching for quite some time.
The original Magic Wand example can be run on Arduino Nano 33 BLE (or the more expensive one, Nano 33 BLE Sense) and SparkFun Edge board. It all works but I found it quite hassle to setup and run for the first time, especially for early learners. It involves to clone the whole TensorFlow repo and some “make” stuffs to sort things up and build the code. Using Arduino IDE is easier and only works for Arduino Nano 33 BLE board out of the box. However, as I intent to do heavier and further programming ahead with TensorFlow on MCU, Arduino IDE doesn’t really fit my need. That’s just me, though, I don’t feel the need to explain why.
Another thing that I found quite disturbing is the board’s orientation so it can infer the gestures correctly. You have to hold like this (Figure 1) when you perform the gesture. I find it hard to attach the board to a stick. I think it’s a easy fix.”