| Summary: | People with certain speech impediments, such as aphasia, face challenges to keep independent and active lives. The bedroom scenario assumes a strong relevance for these individuals due to the different difficulties that can occur and require assistance while in bed. In this context, it is important to pursue assistive solutions for this population. In this dissertation, we aim at creating a sensor-based solution that recognizes dynamic arm movements while in bed, to support communication, providing ways to raise alarms to some hazard conditions, and also enable bidirectional and simple communication between aphasics and their caregivers. The solution developed uses a common smartwatch to collect movement data (using the built-in accelerometer, gyroscope, and magnetometer), which are forwarded to a bedside unit for processing. The bedside unit is responsible for receiving, classifying, and deciding if the movement performed at any time is one of the supported predefined dynamic arm movements. If positive, the bedside unit sends an alert to the caregiver using a specially developed mobile application that allows sending basic “yes” or “no” questions back to the aphasic, to which he may respond using the supported dynamic arm movements. The system, which may be subdivided into three modules, includes a smartwatch app responsible for acquiring the data and sending them to a bed-side unit, a pipeline implemented on the bed-side unit responsible for receiving and classifying the data using previously trained machine learning models, and sending the result to a mobile app in the possession of the caregiver, and a mobile app developed for the caregiver to receive the notifications and allow sending messages to the user being cared for. The bedside unit also implements a speech output service that provides audio near the bed, allowing the aphasic to hear feedback from the caregiver. The gesture recognition results are encouraging, both for subject-dependent and subject-independent scenarios (i.e. mean accuracy and F1-score above 99% and 91% respectively), showing that a model generalization may be attained, making both approaches feasible. Although the system was built for the use case of aphasics, it is not limited to those users and can be generalized for scenarios that require connecting people in bed to their caregivers, when the use of voice is not feasible or practical.
|
|---|