Developing an AI-powered Chatbot for Enhancing User Reference Services in Academic Libraries

Thesis Abstract

Academic libraries increasingly seek innovative solutions to improve reference services amidst rising user demands and technological advancements. Despite the proliferation of digital resources and expressive user expectations for immediate assistance, traditional reference services often experience limitations in responsiveness, coverage, and 24/7 availability. In this context, the integration of artificial intelligence (AI) through chatbot technology presents a promising avenue to supplement human librarians, enhance user engagement, and provide timely, accurate reference support around the clock. This study aims to develop and evaluate an AI-powered chatbot designed to augment user reference services in academic library settings. The primary objectives include identifying user requirements for chatbot functionalities, designing a chatbot prototype rooted in natural language processing (NLP), and assessing its effectiveness and user satisfaction. The research employs a mixed-methods approach, combining qualitative and quantitative techniques. The quantitative component involves a quasi-experimental design where a sample of 300 registered students and faculty members from a large public university are divided into control and experimental groups. The control group receives traditional reference services, while the experimental group interacts with the chatbot. Data collection instruments include structured questionnaires to measure user satisfaction, perceived usefulness, and response accuracy, alongside system logs to assess chatbot performance metrics. The qualitative component comprises focus group discussions with 30 users and interviews with librarians to gather in-depth insights into user experiences and librarian perceptions of the chatbot’s integration. Data analysis utilizes descriptive statistics, t-tests, and ANOVA for quantitative data, while thematic analysis is employed to interpret qualitative responses. The development phase incorporates the application of the Cognitive Load Theory to optimize chatbot interface design, ensuring ease of use and effective information delivery. The chatbot’s architecture is based on recurrent neural networks (RNN) and transformer models, leveraging NLP frameworks like BERT (Bidirectional Encoder Representations from Transformers). An iterative development process incorporates user feedback to refine the chatbot’s functionalities, aiming for high accuracy in query understanding and response generation. The effectiveness of the chatbot is primarily evaluated through system performance metrics, including response time, accuracy, and user satisfaction ratings, supplemented by thematic insights into user acceptance and interaction patterns. Expected findings indicate that the AI-powered chatbot significantly enhances users’ access to reference information, reduces response times, and increases overall user satisfaction compared to traditional services. It is anticipated that user satisfaction scores will demonstrate statistically significant improvements (p<0.05) in the experimental group, with qualitative data revealing high acceptance levels and perceived utility. The research is also expected to identify key factors influencing chatbot adoption, such as interface simplicity, response accuracy, and contextual understanding. The study contributes to knowledge by providing empirical evidence on the effectiveness of AI-driven chatbots in academic library contexts, filling a gap in the existing literature concerning practical deployment strategies, user acceptance, and performance evaluation. It advances theoretical understanding by applying models like the Technology Acceptance Model (TAM) and Human-Computer Interaction (HCI) principles within the specific domain of reference services. The findings will inform best practices for integrating AI chatbots into library operations and establishing sustainable digital reference services. In conclusion, the research demonstrates that AI-powered chatbots can substantially improve the efficiency and accessibility of reference services in academic libraries, with positive implications for user engagement and institutional resource management. Based on the results, recommendations include adopting user-centered design principles, continuous system training with user feedback, and strategic staff training to ensure seamless integration. Future research should explore longitudinal impacts, scalability, and integration with other digital library components to develop comprehensive AI-driven reference ecosystems.