Sign ka lang (SKL): A Filipino sign language interpreter utilizing video processing and machine learning for learners with hearing impairment

ABSTRACT: STATEMENT OF THE PROBLEM: The research aims to develop a hardware-based system that can detect and translate Filipino Sign Language (FSL) through the use of microcontrollers, computer vision and machine learning techniques. The researchers specifically aim to accomplish the following objectives • To design and build an embedded system with an external display monitor • To develop a software that can process input a camera and display the translated FSL through a text output in a display monitor • To test the functionalities and features and evaluate the accuracy of the system • To see if there is a significant difference between utilizing the system and the current process of the special education center of the research local RESEARCH METHODOLOGY: To accomplish the research objectives, the researchers employed both qualitative and quantitative methods to gather data that would help in designing, developing, and evaluating the Sign Ka Lang Filipino Interpreter System. To achieve the research objectives, the researchers conducted both online and face to face interviews with SPED teachers at Gen. Maximino Hizon Elementary School to gain insights and recommendations regarding the development of the system. The system was tested by the primary level students with hearing impairments at Ge. Maximino Hizon Elementary School and along with other SPED teachers and parents. The data and responses from the users are then used to answer the research questions of the study. SUMMARY OF FINDINGS: Sign Ka Lang or SKL is a Filipino Sign Language interpreter system which can moderately translate 19 categories of different FSL words and signs. The testing procedures showed that the participants were satisfied with the created system, and found that it performs and provides the necessary features relating to their needs. Based on the average System Usability Scale score of the system which was 71.07%, the system’s intended users found the system’s usability to the above average. As for the ISO25010 Product Quality Evaluation: the system scored effectively on the system’s Functional Stability, Performance Efficiency, Usability, Reliability, and Portability. Moreover, the system has an overall accuracy of 69.33% when translating FSL, into text. Lastly, there was no measured significant differences between the test scores between the Experimental Group and Control Group. On the other hand, observations by the SPED teacher implied that the Experimental Group was more motivated and participative when compared to the Control Group. CONCLUSION: The purpose of this study is to design and build a system that is able to translate Filipino Sign Language (FSL) into text through developing an embedded system capable of processing video input from a camera. To achieve this, the researchers created the Sign Ka Lang (SKL) Filipino Sign Language Interpreter System, a system that is capable of recognizing and translating 258 FSL signs under 19 various categories. The system was mainly tested by the LWHIs of Gen. Maximino Hizon Elementary School along with professional and licensed sign language instructors. The final system has two main features: Sign & Translate which directly translates the signed word into text, and the other feature is Tell A Story which is an incorporation of stories wherein students can answer questions by signing in FSL. In conclusion, the researchers were able to meet the defined research objectives which were: to design and build an embedded system for a specific purpose, to develop a software capable of processing input from an external module, test the functionalities and features of the developed system, test the system’s accuracy, and determine whether is a significant difference between the system and the current teaching process of the 1st Grade of the research locale. RECOMMENDATIONS: The researchers highly emphasize that the system could be improved with more data for each word in each category. The inclusion of the variations for each FSL, sign or gesture could also be useful in upgrading the current system. Furthermore, the researchers recommend that those interested in the study should carefully select relevant key points within the body frame to extract information from the signer. It is also suggested to upgrade the camera used by the system, a camera with a higher frame rate and reduction would be ideal. Also, to help the user/signer minimize the factors that may affect the translation given by the system, they should add a lighting feature or device to compensate for environments or setting with poor lighting conditions. Finally, the researchers recommend to communicate with Deaf organizations, groups, and or individuals when working on the current system to better insights on how it could be improved.

Filipiniana