“In this project we take a look at making the Arduino Nano 33 BLE learn a few gestures and how to integrate it with a game!
First of all, I would like to give credit to Arduino Team for providing the base framework. If you have not had the chance to go through it, I highly suggest you go through it thoroughly before proceeding further,
Link: https://blog.arduino.cc/2019/10/15/get-started-with-machine-learning-on-arduino/
So first of all we will be starting by generating a machine learning model using google collab and then we shall load it into Arduino IDE to generate and upload the sketch. Note that the collab is a linux framework to generate the necessary header file (model.h) to be included with the code.
Generating the Data for Machine Learning Model
First we need data to train our model. It will be in the form of a csv file with all the points along the columns. You need N datasets for N outputs.
Now you will find a program to create dataset with the first link. It is IMU_Capture.ino file. Import this into your Arduino IDE, and upload to the device. I personally went ahead with three movements which meant three separate datasets.
1. Punching move: Move the device carefully straight ahead to get one set point. It will be visible in the serial monitor. Be careful not to jerk the device and get junk datapoint, this will ruin your model!!!
Keep it consistent for 10 to 12 datapoints. Keep on repeating the move, till you get 10 different sets or more if you like.
2. Defend move: Use your imagination but keep it consistent for ten to 12 datapoints. Keep on repeating the move.
3. Circular Motion
Note we are training three separate gestures which will be later used in our game. If you have troubles, you may use the dataset given with this project!
Training the Model in Google Collab
Please follow the steps given in the link below to start off making your model
https://colab.research.google.com/github/arduino/ArduinoTensorFlowLiteTutorials/blob/master/GestureToEmoji/arduino_tinyml_workshop.ipynb
We will go through it step by step:
First we execute these two commands:
!apt-get -qq install xxd
!pip install pandas numpy matplotlib
This sets up the linux engine and loads the necessary packages.
Then we have to train the neural network but before that we need to edit the data and reformat it in order for TensorFlow to understand.”