Main Content

Mouse Fatigue Estimation by GSR and EMG Values W/ TensorFlow

Collate forearm muscle soreness data on the SD card, build and train a neural network model, and run the model directly on Wio Terminal.

1 x Wio Terminal
1 x Grove - GSR Sensor
1 x Grove - EMG Detector
1 x Creality CR-6 SE 3D Printer
1 x Grove - 4 pin Male Jumper Conversion Cable
1 x Grove - Universal 4 Pin Cable
1 x SD Card
1 x Xiaomi 20000 mAh 3 Pro Type-C Power Bank

Like many of us, I spent most of my waking hours on the computer while working or studying. However, recently, I have started to feel a precipitous pain on my forearm after utilizing the mouse longer than six hours. Therefore, I decided to create this project to get prescient warnings regarding forearm muscle soreness levels so as to prevent any injury risks.

Since utilizing the mouse involves monotonous movements of the same small muscle groups for protracted periods of time, overusing the mouse by positioning, traveling, scrolling, and clicking can make these muscle groups tired and overworked. Hence, after repetitive overexposure to monotonous mouse movements, muscle stress (soreness) increases precipitously and can engender[1]:

Pain (ache, soreness) on the top of the hand
Pain (ache, soreness) around the wrist
Pain (ache, soreness) along the forearm and the elbow
Numbness and tingling in the thumb and the index finger
Burning, stiffness, restricted range of motion
Also, it can cause soreness and fatigue by putting extra strain on the muscles in the upper back (trapezius muscle) and shoulder (deltoid muscle).

Generally, the aforementioned symptoms are temporary and get better on their own after reducing mouse use. Nevertheless, overexposure to monotonous mouse movements can cause more noxious and enervating disorders such as Repetitive strain injury (RSI) and Carpal tunnel syndrome (CTS).

Repetitive strain injury (RSI) results from forceful, awkward, and/or repetitive use of your limbs, producing damaged muscles, tendons, and nerves. Although RSI is a broad term that encompasses several disorders, general symptoms include tingling or loss of sensation in fingers, inability to grasp objects between thumb and fingers, decrease in the size of hand muscles, and pain in the wrist, elbow, shoulder, or neck. The severity of RSI cases varies widely. Tendonitis is the most common example of RSI, while Carpal tunnel syndrome is a more rare and serious disorder.

Carpal tunnel syndrome (CTS) and Thoracic outlet syndrome (TOS) are two of the most disabling repetitive strain injuries. These conditions are disorders of the tendons, nerves, arteries, or veins, occurring at the wrist and upper arm, respectively. In CTS, repeated bending or use of the wrist and fingers results in the compression of the median nerve (runs along the palm side of the wrist), causing intermittent numbness, tingling, and pain in the side of the hand, including the thumb through the inside of the ring finger[2].

After scrutinizing recent research papers on mouse fatigue, I decided to utilize GSR (galvanic skin response) and EMG (Electromyography) measurements denoting forearm muscle soreness and create a budget-friendly device to forecast muscle soreness levels in the hope of averting permanent mouse-related injuries.

Since muscle soreness levels depending on GSR and EMG measurements fluctuates for different muscle groups, it is not possible to extrapolate and interpret forearm muscle soreness levels by merely employing limited data without applying complex algorithms. Therefore, I decided to build and train an artificial neural network model by utilizing the empirically assigned soreness classes to predict forearm muscle soreness levels based on GSR and EMG measurements.

I decided to employ Wio Terminal in this project since it can easily collect muscle soreness data (GSR and EMG measurements) and run my neural network model after being trained to forecast forearm muscle soreness levels. To obtain the required data to train my model, I connected a GSR sensor (Grove) and an EMG detector (Grove) to Wio Terminal. Also, Wio Terminal can display the collected muscle soreness data on its integrated TFT LCD screen.

Since Wio Terminal supports reading and writing information from/to files on an SD card, I stored the collected data in a CSV file on the SD card to create a data set. In this regard, although Wio Terminal provides Wi-Fi connectivity, I was able to save data packets via Wio Terminal without requiring any additional procedures.

After completing my data set, I built my artificial neural network model (ANN) with TensorFlow to make predictions on forearm muscle soreness levels (classes) based on GSR and EMG measurements. As labels, I employed the empirically assigned muscle soreness classes for each data record while collecting data:

After training and testing my neural network model, I converted it from a TensorFlow Keras H5 model to a C array (.h file) to execute the model on Wio Terminal. Therefore, the device is capable of detecting precise forearm muscle soreness levels (classes) by running the model independently.

Lastly, to make the device as compact and robust as possible while operating, I designed a Pikachu-inspired case (3D printable).

So, this is my project in a nutshell

In the following steps, you can find more detailed information on coding, logging data on the SD card, building an artificial neural network model with TensorFlow, and running it on Wio Terminal.”

Link to article