Assessing AI-Powered Tutoring for Science Concept Mastery in Schools

Thesis Abstract

This study investigates the efficacy of AI-powered tutoring systems in promoting science concept mastery among secondary school students, addressing persistent gaps in conceptual understanding, unequal access to quality instructional support, and limited alignment between classroom instruction and evidence-based learning paths. The aim is to determine whether AI-driven personalized tutoring improves science concept mastery relative to traditional instruction, and to identify the mechanisms by which it influences learning outcomes. Specific objectives are (1) to evaluate the impact of AI-powered tutoring on standardized measures of science concept mastery across key domains (biology, chemistry, physics) over a 12-week intervention; (2) to examine changes in metacognitive skill use and self-regulated learning behaviors; (3) to explore student and teacher experiences with the AI tutoring environment; (4) to identify contextual moderators such as prior achievement and socio-economic status; and (5) to propose a scalable integration model for AI tutoring within standard curriculum pacing. A quasi-experimental design will be employed in which two matched groups of 360 Year 9 students from six diverse urban secondary schools participate, with 180 assigned to an AI-powered tutoring condition and 180 to conventional instruction. The intervention spans 12 weeks, delivering adaptive micro-lessons, formative assessments, and immediate feedback via an AI tutor integrated with the existing learning management system. Data collection comprises pre- and post-tests assessing science concept mastery using a validated instrument with established reliability (Cronbach’s alpha > .85 and test-retest reliability r > .80); weekly learning analytics capturing interaction patterns, time-on-task, hint usage, and progression rates; standardized surveys measuring self-regulated learning (motivational constructs, planning, monitoring), and perceptions of AI tutoring (engagement, perceived usefulness, and usability); semi-structured interviews with 24 students and 8 science teachers; and classroom observations guided by a structured protocol to triangulate qualitative data. Instrument validity and reliability will be established through content validity indices with expert panels and pilot testing (n=60). Quantitative analyses will proceed with descriptive statistics and multivariate techniques. ANCOVA will compare post-test scores between groups controlling for baseline mastery, while hierarchical linear modeling will account for nested data (students within classes within schools). Mediation analysis, using PROCESS or structural equation modeling, will test whether changes in self-regulated learning mediate effects of AI tutoring on mastery. Moderation analyses will explore how prior achievement and socio-economic status influence outcomes. For qualitative data, thematic analysis will identify themes related to engagement, perceived cognitive load, and instructional alignment with curricula; triangulation across data sources will validate findings. A conceptual model integrating education technology acceptance theories (TAM) with constructivist learning theories and cognitive load theory will guide interpretation and synthesis. Expected findings include statistically significant improvements in science concept mastery for the AI-tutoring group (effect size d ? 0.40–0.60), with higher gains among students with moderate prior achievement and those exhibiting stronger self-regulated learning, mediated by increased use of metacognitive strategies and timely feedback. Qualitative insights are anticipated to reveal enhanced student agency, reduced cognitive load through adaptive scaffolding, and challenges related to AI tutor transparency and teacher role in monitoring. The study contributes to knowledge by providing rigorous causal evidence on AI-powered tutoring effectiveness for science learning in real-world school settings, clarifying the interplay between adaptive feedback, learner autonomy, and conceptual development, and delineating contextual factors that optimize impact. It offers a scalable implementation framework, including guidelines for alignment with curricula, professional development for teachers, data governance considerations, and a cost–benefit perspective. The main conclusion is that AI-powered tutoring can augment science concept mastery when integrated with coherent instructional design, robust teacher oversight, and attention to equity in access. Recommendations include adopting adaptive tutoring as a complementary tool within a blended learning model, investing in teacher training to interpret analytics, and pursuing longitudinal studies to assess long-term retention and transfer of conceptual understanding.