Summary:
Researchers at the International Islamic University Malaysia have developed a gesture recognition system for QTrobot using MediaPipe, aiming to enhance therapy for children with Autism Spectrum Disorder (ASD). By integrating MediaPipe’s posture recognition technology, QTrobot can now recognize children’s gestures and estimate engagement during sessions. Testing with 20 adult participants showed highly promising results, with an impressive 94.33% average gesture recognition accuracy and positive feedback on ease of use and interaction quality. Participants found the system enjoyable and effective, with half achieving 100% gesture recognition accuracy and the rest performing between 77% and 91%.
This study highlights the potential of combining intelligent gesture recognition with social robots like QTrobot to create more engaging and effective autism therapy sessions. While further improvements are needed to fully automate the system and adapt to real-world conditions, the findings demonstrate strong potential for wider use in special needs education and therapy. The project paves the way for making therapy more interactive, motivating, and accessible to children who benefit from structured and supportive learning environments.
Reference:
El-Muhammady, M. F., et al. “Utilization of Mediapipe Posture Recognition for the Usage in Estimating ASD Children Engagement Interacting with QTrobot.” Mekatronika: Journal of Intelligent Manufacturing and Mechatronics 6.2 (2024): 1-11.
https://journal.ump.edu.my/mekatronika/article/view/10754/3372