An Intelligent Sign Language Interpretation System for Patient Communication

Abstract

Effective communication is a cornerstone of high-quality healthcare; however, individuals who are deaf or hard of hearing frequently encounter significant barriers in clinical environments due to the absence of qualified interpreters and limited proficiency in sign language among healthcare providers. To address this challenge, this study proposes a real-time sign language interpretation system that translates hand gestures into textual outputs, thereby facilitating inclusive and equitable patient–provider communication. The system combines computer vision and deep learning techniques, utilizing OpenCV for image preprocessing and MediaPipe for extracting 21-point hand landmarks. The gesture recognition model was trained using Google’s Teachable Machine on a curated dataset of seven medically relevant gesture classes (e.g., Doctor, Cough, Injection), with data augmentation and normalization enhancing robustness. The final model, based on convolutional neural networks (CNNs), achieved an overall accuracy of 82%, with the Doctor gesture attaining a precision of 0.93 and Chest gesture showing 97.31% recognition accuracy in real-time tests. The modular architecture, ease of deployment, and strong performance across diverse environmental conditions make it a practical solution for medical settings. This research contributes to the advancement of assistive technologies in healthcare and underscores the role of AI-driven tools in promoting accessible, responsive, and high-quality patient care for the deaf and hard-of-hearing community.