Intelligent Tutoring System for Multimodal Coding Education Analytics

Thesis Abstract

The rapid evolution of coding education in online and blended learning settings has exposed persistent gaps in learners’ engagement, conceptual understanding, and practical proficiency, particularly among diverse student cohorts and under-represented groups. This study addresses the problem of limited adaptive guidance and multimodal feedback in traditional coding curricula, which hampers timely remediation of misconceptions and impedes sustained skill development. The aim is to develop and evaluate an Intelligent Tutoring System (ITS) for multimodal coding education analytics that integrates code execution traces, eye-tracking indicators, keystroke dynamics, and natural language interaction to deliver personalized tutoring and real-time analytic feedback. The specific objectives are (1) to design an ITS architecture that fuses multimodal data streams for fine-grained learner modeling in introductory and intermediate programming tasks; (2) to implement adaptive instructional interventions grounded in constructivist and cognitive apprenticeship theories, and informed by Self-Determination Theory to enhance motivation; (3) to evaluate the system’s effectiveness on code-writing accuracy, debugging efficiency, and conceptual understanding across novice and underrepresented student groups; (4) to examine the relationship between multimodal learner features and performance outcomes using predictive analytics; and (5) to assess learner perceptions and engagement through qualitative inquiry to refine feedback strategies. A mixed-methods design is employed. The quantitative strand uses a quasi-experimental framework with two intact classes (n = 240) drawn from first- and second-year computer science cohorts across two universities, assigned to ITS-supported instruction vs. conventional teaching. Data sources include programmatic metrics (task completion time, error rates, and test scores), multimodal traces (eye gaze heatmaps, mouse/keyboard dynamics, and code revision histories), and standardized programming concept inventories administered at baseline and post-intervention. Predictive modeling will be conducted through hierarchical linear modeling and mixed-effects regression to account for nested data and repeated measures, complemented by feature engineering on multimodal signals. The qualitative strand comprises semi-structured interviews and focus groups (n ? 40 students, 10 instructors) to elicit experiences with feedback modality, perceived transparency of the tutoring decisions, and suggestions for system improvement. Thematic analysis will be guided by Braun and Clarke’s approach, with triangulation against ITS log data to corroborate qualitative findings. The ITS architecture integrates (a) an offline learner model that updates probabilistic estimates of mastery in programming constructs (loops, conditionals, data structures) using Bayesian Knowledge Tracing augmented with item response theory-based refinement, (b) a real-time multimodal analytics engine that processes eye-tracking, keystroke dynamics, and program execution traces to infer cognitive load, confusion, and strategy use, (c) a tutoring module delivering scaffolded hints, code templates, and incremental challenges calibrated by mastery levels, and (d) a feedback generator offering diagnostic, feedback-on-demand, and metacognitive prompts aligned with cognitive apprenticeship principles. The theoretical framework combines constructivism, cognitive apprenticeship, and Self-Determination Theory to justify adaptive guidance and motivational supports. Data analysis will test hypotheses regarding (i) higher post-test programming proficiency in the ITS condition, (ii) reductions in debugging time and error rates, (iii) stronger predictive validity of multimodal features for learning gains, and (iv) positive shifts in learner motivation and perceived autonomy. Mediation analyses will explore whether changes in cognitive load and metacognition mediate the relationship between ITS interventions and performance. Expected findings include statistically significant improvements in code-writing accuracy (p < .05), debugging efficiency (mean reduction in debugging time by 25%), and concept mastery (effect size d > 0.5) in the ITS group relative to controls, with multimodal indicators (e.g., decreased fixation durations on incorrect code regions, smoother keystroke patterns during scaffolding) predicting learning gains. Qualitative insights are anticipated to reveal heightened perceived relevance, transparency of tutoring decisions, and greater learner autonomy. The study contributes to knowledge by advancing multimodal data fusion in ITS for programming education, detailing an empirically validated architecture that combines cognitive theories with motivation-enhancing design, and providing transferable implications for scalable coding curricula in higher education. Practical recommendations include guidelines for deploying multimodal ITS in university settings, considerations for privacy and ethical use of biometric data, and strategies to tailor feedback for diverse learner populations. The main conclusion posits that an ITS leveraging multimodal analytics can deliver effective, scalable, and motivating coding instruction, with demonstrable gains in performance and learner engagement, while outlining actionable avenues for iterative refinements in pedagogy and system design.