Development of a Malaysian Sign Language interpreter using image recognition for the community to understand the deaf

Abstract

In Malaysia, Person with Disabilities (PWD) with hearing problems or commonly known as the deaf, struggle to have a conversation with the communities who do not know or how to do sign language efficiently. The consequence is hearing PWD received unequal treatment in jobs and learning opportunities. This project aims to develop a Malaysian Sign Language interpreter to convert hand signs to texts, in order to facilitate the conversation between hearing PWD and the communities. The system would implement a camera and vision system to capture images of hand signs. Four hand signs have been selected. Stacks of these images were processed digitally using deep learning method, and eventually the trained network could recognize the hand signs successfully. This feasible study suggests that the proposed setup could be further implemented to train more hand signs and enrich the hand signs-to-text vocabulary.