Design and Evaluation of AI-Enhanced Search Systems for Academic Libraries

Thesis Abstract

The rapid evolution of digital technologies has transformed the landscape of academic information retrieval, emphasizing the need for intelligent systems capable of enhancing search accuracy, relevance, and user satisfaction within academic libraries. Despite the proliferation of traditional keyword-based search engines, users frequently encounter challenges such as irrelevant results, information overload, and difficulties in locating precise scholarly materials. This study aims to design and evaluate an AI-enhanced search system tailored for academic library settings to improve retrieval performance through advanced natural language processing (NLP) and machine learning (ML) techniques. The primary objectives include analyzing user search behaviors, developing an AI-driven search prototype incorporating semantic understanding and personalized recommendations, and empirically assessing its effectiveness compared to conventional systems. The research employs a mixed-methods approach, integrating quantitative and qualitative techniques within a descriptive and experimental research design. The population comprises 500 active postgraduate students and academic staff members at a leading university, from which a stratified random sample of 200 participants will be selected to participate in usability testing and survey instruments. Data collection will utilize structured questionnaires measuring user satisfaction, search efficiency, and perceived relevance, alongside system logs to capture search patterns, click-through rates, and query reformulations. To ensure instrument validity and reliability, pilot testing will be conducted, and Cronbach’s alpha coefficients will be calculated for internal consistency. In addition, semi-structured interviews will be conducted with 20 participants to elicit detailed insights into their search experiences and perceptions of the AI-enhanced system. Data analysis will involve descriptive statistics, paired sample t-tests to compare system performance metrics, and regression analysis to identify factors influencing user satisfaction. Thematic analysis will be applied to qualitative interview data, while system evaluation metrics such as precision, recall, and mean reciprocal rank (MRR) will quantify retrieval effectiveness. The study is anchored in the Information Search Process (ISP) theory and the User Satisfaction Model, which underpin the development of an AI-based conceptual framework incorporating semantic search algorithms, user profiling, and adaptive learning. It is hypothesized that the AI-enhanced search system will significantly outperform traditional keyword-based systems in terms of relevance, accuracy, and user satisfaction. Furthermore, the research anticipates revealing key factors that influence acceptance and effective usage of intelligent search systems in academic contexts. Expected findings suggest that the AI-enabled system will demonstrate superior precision (expected increase of 20%) and recall (expected increase of 15%) over existing search tools, alongside higher user satisfaction scores. The analysis aims to establish a positive correlation between system usability features and user engagement. The study’s contribution to knowledge lies in providing empirical evidence on the practical application of AI technologies in library search environments, demonstrating how semantic and personalized search mechanisms enhance scholarly information retrieval processes. In conclusion, the findings will reinforce the importance of integrating AI into academic library systems to meet evolving user expectations and information needs. Recommendations will include adopting AI-driven search models, investing in training programs for librarians and users, and further research into scalable AI solutions tailored for diverse academic disciplines. This research paves the way for further investigations into the long-term impacts of intelligent search systems on academic research productivity and information literacy, fostering continuous innovation in library sciences and information management.