Gesture Talk: A Low-Cost Wearable Sign Language Recognition System

Abstract

Sign language is a primary mode of communication for individuals with hearing and speech impairments. However, communicating with those unfamiliar with sign language often presents challenges, creating barriers in areas such as education, healthcare, workplaces, and public services. While various sign language recognition systems utilizing computer vision and deep learning have been proposed, many require high-performance computing hardware, specialized cameras, or controlled environments; these requirements make them expensive and difficult to deploy in everyday situations. This research introduces 'Gesture Talk,' a wearable system that translates sign language into text and speech in real-time. The system is developed using an ESP32 microcontroller, flex sensors, an OLED display, and an ISD1820 voice playback module. Unlike camera-based methods, this proposed system recognizes hand gestures by directly measuring finger flexion via five flex sensors mounted on a glove. Analog sensor values are continuously acquired by the ESP32 and compared against predefined gesture patterns stored in memory. Once a gesture is recognized, the corresponding message is displayed on the OLED screen, and a pre-recorded voice message is simultaneously played using the ISD1820 voice module. The entire system is powered by a rechargeable lithium battery, with an LM2596 buck converter ensuring stable voltage regulation for reliable operation. A functional prototype was developed to evaluate performance and was tested under various operating conditions. Experimental observations have demonstrated that reliable gesture recognition is achievable for predefined static gestures with an average accuracy of approximately 97.3%, while maintaining a response time of less than 250 milliseconds. The proposed device offers a compact, portable, and cost-effective solution that can assist individuals with hearing and speech impairments in communicating more effectively, without the need for internet connectivity or external computing resources. Its modular architecture also provides flexibility for future enhancements, such as wireless communication, mobile application integration, multilingual voice output, and machine learning-based gesture expansion.

Keywords