“A wearable Sign Language Translator powered by the VIAM and Raspberry Pi.
Story
In a world where communication barriers persist for individuals unable to speak, sign language stands as a vital means of expression. However, the necessity of a third-party interpreter often impedes spontaneous and natural communication, posing challenges for seamless interaction. To address this prevalent issue, we’ve developed an innovative electronic device equipped with AI technology capable of detecting hand symbols and translating them into audible speech in real-time. Powered by the VIAM app and the Raspberry Pi Zero, our aim is to empower individuals who are deaf or hard of hearing to communicate more freely and independently, eliminating the constant reliance on interpretation assistance.
The electronic device we’ve created takes the form of wearable spectacles, providing a practical and discreet solution for communication barriers. Designed to be worn by individuals with disabilities, the spectacles seamlessly integrate into everyday life, allowing users to communicate effortlessly with those who do not know sign language. This wearable design not only enhances accessibility but also promotes a sense of independence and autonomy, empowering users to express themselves confidently in various social and professional settings.
In addition to promoting inclusivity and independence, our device represents a significant advancement in assistive technology. By leveraging AI-driven sign language detection and translation capabilities, we’re not only breaking down communication barriers but also fostering greater understanding and empathy among individuals of diverse abilities. Ultimately, our goal is to create a more inclusive society where communication is seamless and accessible to all, regardless of linguistic or physical barriers.
Hardware
For our project, we needed a powerful single-board computer (SBC) to handle the complex tasks involved in sign language translation. While the Raspberry Pi 5 was initially considered, its size constraints led us to choose the Raspberry Pi Zero 2 W, which offers a good balance of performance and compactness.
The key feature of the Raspberry Pi Zero 2 W is its impressive performance capabilities. It features a quad-core 64-bit Arm Cortex-A53 CPU clocked at 1GHz, powered by the Raspberry Pi RP3A0 system-in-package (SiP). With 512MB of LPDDR2 SDRAM, this upgraded processor delivers a significant 40% increase in single-threaded performance and an impressive fivefold boost in multi-threaded performance compared to the original Raspberry Pi Zero.
To capture accurate hand gestures for our project, we opted for the Raspberry Pi Camera Module 3 (v3). This updated version of the camera module offers several improvements over its predecessors. It features a 16:9 aspect ratio and a fixed autofocus lens, along with a larger camera lens and a new 11.9-megapixel sensor. These enhancements ensure precise image capture, essential for reliable sign language translation.
To install the OS on the Raspberry Pi just follow this guide.
These are the main hardware used. Then we can discuss the software.
VIAM
VIAM is a software platform designed to streamline the development and management of smart machines. These machines are essentially robots or devices equipped with sensors and computing power, allowing them to interact with their environment. VIAM simplifies tasks for developers by providing a consistent interface for controlling these machines, collecting data, and even incorporating machine learning capabilities.
VIAM’s open-source and modular design gives developers flexibility. They can choose the functionalities they need and integrate VIAM with various hardware components. Additionally, VIAM offers cloud-based solutions for managing and scaling large numbers of smart machines.
These are the main terminologies used in the VIAM app.
- smart machine in Viam consists of at least one computer, typically a single-board computer, running viam-server and communicating with any hardware connected to it by signalling through a digital data pin.
- The term component describes a piece of hardware that a computer controls, like an arm or a motor.
- The term Services are built-in software packages that make it easier to add complex capabilities such as motion planning or object detection to your machine.”