Adaptive Multimodal Communication Aids for Deaf-Blind in ICT Contexts

Thesis Abstract

Adaptive Multimodal Communication Aids for Deaf-Blind in ICT Contexts seeks to address the communication barriers faced by deaf-blind individuals in increasingly digitized environments. The study responds to the persistent gap between ICT design practices and the accessibility needs of dual-sensory users, where existing assistive technologies often rely on a single modality or require extensive user training, resulting in limited real-world adoption. The aim is to design, evaluate, and iteratively refine a multimodal communication aid that dynamically adapts to user context, preferences, and environmental factors, thereby enhancing message comprehension, interaction fluency, and independence in ICT-mediated tasks. Specific objectives include (1) identifying user-centered multimodal configurations that optimize information transfer for deaf-blind users in common ICT scenarios (communication, navigation, and task management); (2) developing an adaptive framework that selects and synchronizes tactile, haptic, vibrotactile, braille-based, and audio-visual cues based on real-time sensor data and user profiles; (3) implementing a prototype within a controlled ICT environment and evaluating usability, performance, and cognitive load; (4) validating the framework across diverse additional impairments (e.g., residual vision, tactile sensitivity) and age groups; and (5) providing design guidelines for inclusive ICT systems grounded in empirical evidence and theoretical models. The methodology adopts a mixed-methods, iterative design-based research approach anchored in the Activity Theory and the Theory of Multimodal Interaction to explore how adaptive multimodal cues influence user performance and satisfaction. The study comprises three phases (a) formative qualitative inquiry with 30 deaf-blind participants and 15 caregivers/experts to elicit modality preferences, task demands, and contextual constraints; (b) development and pilot testing of a multimodal adaptive prototype comprising tactile bracelets, braille display, vibrotactile wristbands, and haptic auditory notifications integrated with mainstream ICT platforms; and (c) a controlled experiment with 60 deaf-blind participants randomized to adaptive versus non-adaptive conditions across four ICT tasks (email, calendar management, video conferencing, and emergency alert response). Data collection instruments include semi-structured interviews, usability scales (System Usability Scale, SUS), cognitive load measures (NASA-TLX), objective task performance metrics (completion time, error rates, assistance required), biometric and environmental sensors (temperature, noise levels, light exposure, skin conductance), and system logs. Validity and reliability are established through triangulation, pilot testing, and inter-rater reliability for qualitative coding. Data analysis employs a convergent mixed-methods strategy. Quantitative data will be analyzed using multivariate analysis of variance (MANOVA) to examine effects of adaptive versus non-adaptive conditions on task performance and cognitive load, followed by regression analysis to identify predictors of successful communication, and repeated-measures ANOVA to assess learning effects over sessions. Qualitative data will undergo thematic analysis to extract patterns in user preferences, perceived ease of use, and contextual challenges, complemented by discourse analysis of interview transcripts. The analysis will also explore interaction effects between modality configurations and environmental context using hierarchical linear modeling to account for nested data (participants within sessions). The study will report effect sizes (Cohen’s d) and credibility intervals, ensuring results are robust across subgroups defined by residual vision and tactile sensitivity. Expected findings include (i) evidence that adaptive multimodal cues significantly improve communication efficiency and user satisfaction compared with static modalities; (ii) a scalable decision framework that maps contexts to optimal modality combinations; (iii) reduced cognitive load and increased independence in ICT tasks; and (iv) nuanced insights into design requirements for wearables and tactile interfaces in assistive ICT ecosystems. The contribution to knowledge encompasses a theoretically grounded, practically implementable model for adaptive multimodal communication in ICT contexts, detailed design specifications for an integrative prototype, and empirical guidance for policy-makers and designers on inclusive ICT development. The study concludes with actionable recommendations for standardizing multimodal accessibility benchmarks, guidelines for future research in deaf-blind HCI, and implications for the deployment of adaptive assistive technologies in education, workplace, and everyday digital life.