Interaction Design of the Mixed Reality Application for Deaf Children
Keywords: mixed reality, sign language, human-computer interaction, human-robot interaction
TL;DR: The use of mixed reality (MR) technology to teach Kazakh-Russian Sign Language (K-RSL) to deaf and hard-of-hearing (DHH) children in Kazakhstan, proposing a design framework for an MR system that engages children in vocabulary learning activities.
Abstract: In recent years, using mixed reality (MR) for teaching sign language has become more feasible. Existing research highlights the emerging importance of MR in teaching by providing a unique way of acquiring information through a visual modality in an immersive environment. This adds a perceived value to the application of MR in tackling the challenges of teaching and learning sign language to deaf and hard-of-hearing (DHH) children. In Kazakhstan, responding to the needs of the DHH children is often overlooked by educational authorities and stakeholders. Central to this research is the exploration of deaf-friendly ways of teaching Kazakh-Russian Sign Language (K-RSL) to DHH children using state-of-the-art MR technology. In this paper, we propose an iterative design framework of a holistic MR system that is tailored to respond to the needs of the DHH children through an engaging vocabulary learning activity co-designed with the participants. To this end, we conducted a pilot experiment with four deaf children aged 6-14 years old, who evaluated the MR application and four different signing avatars. Based on the preliminary observations, we discuss the limitations and possible directions for system improvement.
Submission Number: 12
Loading