Development of a wearable device for controlling virtual mouse and keyboard for individuals with disarticulated elbow

ABSTRACT: Daily tasks related to work, academics, and other responsibilities involving technology, like using computers and laptops, are routine for most but challenging for individuals with disarticulated elbow (DE). Disarticulated elbow refers to an Upper Limb Amputation (ULA) at the elbow joint; however, for the purposes of this study, those who are born without a forearm, or technically speaking, individuals with Congenital Upper Limb Deficiency (CULD), were also included in the scope of the study. While there is a wide range of assistive technologies available -upper limb prostheses, eye-gaze typing, and wearable wireless mice – that could help individuals with DE in using a computer, they all present their own limitations. Thus, this focused on the development of a pair of Inertial Measurement Unit (IMU)-based wearable armbands that could translate right-arm movements into mouse cursor movement and left-arm movements into mouse actions. The developed system is composed of a left armband which detects left arm gestures through a developed KNN-based gesture recognition model, a right armband which utilizes the Bluetooth Low Energy Human Interface Device (BLE HID) to simulate mouse cursor movements, a control box that receives the data from the left armband via WI-FI, and a virtual keyboard that can be opened and closed using a specific left arm gesture. The study conducted a series of trials to determine the mouse cursor movement polling rate, mouse action response time and accuracy, virtual keyboard response time, and the overall effectiveness of the armband system. The results of the study indicate that the device’s performance was generally responsive and accurate, with an average polling rate of 352.4Hz, an overall mouse action response time of less than 500 milliseconds, and an overall mouse action accuracy of 95.5%. In addition to that, most keys on the virtual keyboard recorded a response time that was imperceptible to users, while a few showed noticeable delays. Lastly, the findings gathered through the ISO 9241-411 based user experience questionnaire demonstrate that the armband device is effective, intuitive, and user-friendly for individuals with elbow disarticulation.

Filipiniana