AI-powered intelligent mindfulness-based cognitive therapy companion for emotion recognition and context-aware therapy

ABSTRACT: Mindfulness-Based Cognitive Therapy (MBCT) is effective in managing depression and other mental health challenges but faces barriers such as accessibility and engagement. A key challenge is the lack of integration of advanced technologies in these practices. To address this, research aims to develop an adaptive mobile application that incorporates three key technologies: (1) speech and facial recognition to provide users with an accessible MBCT companion, (2) a context aware algorithm to tailor therapeutic practices based on the user’s location and time, and (3) LSTM-based emotional forecasting to enhance emotion management. A mobile application was made called “Felii” using Android Studio, Java Kotlin for the app back-end development as well as Java, Figma and Photoshop 2020 for the front-end development and phototyping. The speech and facial emotion model demonstrates high accuracy in detecting Joy (30/30) and strong performance in recognizing Sad (26/29), Angry (25/26), Fear (30/39), and Neural (26/26), with minimal misclassifications. Meanwhile, the LSTM-based emotion forecasting model shows promise but has some errors, such as Sad being misclassified as Angry (3 instances) and Fear occasionally misclassified as Neural (4 instances), indicating a need for further refinement. Additionally, the system provides therapy recommendations tailored to the time of the day and user location, enhancing its adaptability. These results confirm the effectiveness of the intelligent companion in supporting MBCT through accurate emotion and personalized therapy.

Filipiniana