Talkeasy: An interactive web-based platform for hiragana Japanese sign language fingerspelling acquisition using K-nearest neighbors (KNN) algorithm

ABSTRACT: The researchers developed TalkEasy, an interactive web-based platform designed to address the lack of accessible and interactive tools for learning Japanese Sign Language (JSL) fingerspelling, particularly for Hiragana characters. Traditional learning methods often lack real-time translation, audio-textual support, and intelligent language assistance, making it difficult for learners to associate spoken and written Japanese with corresponding hand gestures. To bridge this gap, TalkEasy integrates the K-Nearest Neighbors (KNN) algorithm for real-time visual recognition and translation of JSL fingerspelling into text and speech. The platform features text-to-speech and speech-to-text functionalities, allowing users to convert JSL fingerspelling into spoken Japanese and written characters, improving accessibility and communication. Additionally, TalkEasy includes as NLP-powered chatbot using the GIPT API, which provides intelligent assistance for JSL fingerspelling acquisition, answers language-related queries, and offers bilingual support in Japanese and English. The platform also features a speech-to-video function that displays corresponding hand gestures, helping users better understand and practice JSL fingerspelling while providing real-time feedback on signing accuracy and pronunciation to create an engaging and effective learning experience. Moreover, TalkEasy offers a comprehensive library of Hiragana characters with associated hand gestures, pronunciation guides, and descriptions, integrating learning and natural language processing to provide an accessible and innovative solution for individuals seeking to improve their JSL fingerspelling proficiency, fostering better communication in various social, educational and professional settings.

Filipiniana