Adaptive AI Tutoring for Librarian Education and Skill Assessment

Thesis Abstract

This study investigates the efficacy of an adaptive AI tutoring system designed to enhance librarian education and skill assessment in higher education settings, addressing the persistent gap between theoretical instruction and practical information literacy competencies among aspiring librarians. The problem addressed is the limited scalability and personalization of existing librarian education programs to accommodate diverse learner profiles, rapid changes in information technologies, and evolving professional standards. The aim is to evaluate whether an adaptive AI tutor can personalize instruction, scaffold core competencies, and provide valid, continuous skill assessments aligned with professional standards such as the ALA Competencies for Librarians. The specific objectives are (1) to design an adaptive tutoring architecture integrating mastery-based progression, intelligible feedback, and real-time assessment; (2) to examine the system’s impact on knowledge acquisition, information literacy skills, and metadata literacy relative to traditional instruction; (3) to investigate learner engagement, motivation, and perceived usefulness; (4) to validate automated skill assessments against expert human judgments; and (5) to identify contextual factors moderating effectiveness across diverse library science curricula. The methodology adopts a mixed-methods, quasi-experimental design conducted over two academic semesters in three university library science programs. The population comprises 240 enrolled graduate students across two cohorts; a randomised assignment yields 120 students in the adaptive AI tutoring condition and 120 in standard instruction. The data collection instruments include a) a validated Information Literacy and Metadata Competency Scale administered at baseline, mid-point, and post-intervention; b) system-generated analytics capturing time-on-task, hint requests, mastery speed, and progression; c) a performance-based assessment with rubrics aligned to ALA competencies evaluated by two independent librarians to establish inter-rater reliability (Cohen’s kappa ? 0.80); d) a usability and engagement questionnaire using the Technology Acceptance Model (TAM) constructs; and e) semi-structured interviews with a purposive subsample of 24 participants for thematic insights. Validity and reliability are established through pilot testing (n=40), item? difficulties, and pilot-revised scoring rubrics with Cronbach’s alpha thresholds above 0.78 for all scales. Data analysis employs a convergent mixed-methods approach. Quantitative data are analyzed using multivariate analysis of covariance (MANCOVA) to compare post-test outcomes while controlling for baseline scores, with regression analyses to explore predictors of mastery and time-to-proficiency. Feature importance is examined via random forest modeling to identify which adaptive interventions (e.g., spaced repetition, adaptive hints, and performance dashboards) most strongly predict competency gains. The qualitative data from interviews are subjected to thematic analysis, triangulated with system logs to illuminate mechanisms of learning, motivation, and perceived legitimacy of automated assessments. A conceptual model grounded in constructivist learning theory and Vygotsky’s zone of proximal development is used to interpret results, augmented by the Technology Acceptance Model and Self-Determination Theory to explain engagement and intrinsic motivation. Named theories include Cognitive Load Theory to interpret learner interactions with adaptive scaffolds and Self-Regulated Learning Theory to assess progression strategies. Key expected findings include (i) significant improvement in composite competency scores for the adaptive AI group compared to traditional instruction (p < .05); (ii) higher engagement metrics and faster attainment of predefined mastery thresholds; (iii) strong convergent validity between AI-generated assessments and human expert ratings (ICC > 0.85); (iv) positive and stable usability and acceptance over time, moderated by prior digital literacy; (v) qualitative evidence that adaptive feedback and transparent progress dashboards foster deeper metacognitive reflection. The study contributes to knowledge by providing empirical evidence on the viability of adaptive AI tutors in professional librarian education, clarifying how personalized scaffolding translates into measurable competencies and how automated assessment aligns with expert judgment. It offers a scalable model for integrating AI-driven tutoring into library science curricula, with practical implications for curriculum designers, instructional technologists, and accreditation bodies. Limitations include potential generalizability constraints to non-U.S. contexts and reliance on institutional resources for implementation. Recommendations emphasize embedding adaptive tutors within competency frameworks, ensuring ongoing alignment with evolving standards, and pursuing longitudinal studies to assess retention and professional practice impacts.