A Theory-Driven Framework for Evaluating Digital Repository Impact Metrics

Thesis Abstract

This study addresses the persistent challenge of accurately assessing the impact of digital repositories within academic ecosystems, where conventional metrics often fail to capture scholarly influence, user engagement, and societal reach. The aim is to develop a theory-driven framework that integrates constructs from information behavior, technology acceptance, and open science to produce robust, multidimensional impact metrics for digital repositories. Specific objectives are (1) to identify latent dimensions of repository impact beyond traditional counts of deposits and downloads; (2) to operationalize a formal model linking repository features, user interactions, and scholarly outcomes; (3) to validate the framework using empirical data from a representative sample of institutional repositories; (4) to compare the new framework against existing maturity and impact models; and (5) to propose practical indicators and measurement protocols for repository managers and policymakers. The research adopts a mixed-methods design anchored in a theory-driven model. The population comprises 40 institutional repositories across public and private universities in a defined geographic region, with a target sample of 12 repositories selected via stratified sampling to ensure diversity in size, discipline coverage, and repository infrastructure. Data collection employs multiple instruments (a) repository data extraction templates capturing deposit volume, filetypes, metadata quality, reuse indicators, and API-enabled interactions; (b) web analytics logs to measure user sessions, search behavior, and time-on-page; (c) surveys of 300 active depositors and 600 regular users to gauge perceived usefulness, ease of use, and intention to reuse; and (d) semi-structured interviews with 24 repository managers and 18 disciplinary researchers to illuminate contextual factors and governance. Validity and reliability are established through triangulation, pilot testing of instruments (n=30), and Cronbach’s alpha thresholds above 0.8 for multi-item scales. Data analysis combines quantitative and qualitative techniques structural equation modeling (SEM) to test the hypothesized relationships among constructs derived from information behavior theory, Technology Acceptance Model variants, and the Open Science dissemination framework; regression analyses to identify key predictors of impact across repositories; and thematic analysis of interview transcripts to extract contextual moderators and governance considerations. Theoretically, the study integrates Rogers’ Diffusion of Innovations, the Theory of Planned Behavior, and the Open Science framework to form a cohesive, testable model that explains how repository features influence scholarly visibility, engagement, and reuse. Expected findings include (i) a multidimensional impact construct comprising scholarly impact, reuse and diversification of access, user engagement, and social reach; (ii) demonstrable positive effects of rich metadata quality, persistent identifiers, and interoperable APIs on influence metrics; and (iii) contextual moderators such as disciplinary norms and governance policies that shape metric validity. The study contributes to knowledge by operationalizing a theory-driven, empirically validated framework for evaluating digital repository impact, enabling more accurate benchmarking, cross-repository comparisons, and informed decision-making for repository development and policy formulation. It offers a suite of practical indicators, measurement protocols, and a scoring rubric aligned with the theoretical model, facilitating ongoing monitoring and improvement. The main conclusion anticipates that an integrated, theory-backed framework yields superior explanatory power for repository impact than conventional usage-based metrics alone, with implications for optimizing repository design, promoting open science practices, and informing national and institutional research assessment strategies. The recommendations emphasize (a) adopting the framework as a standard evaluation protocol for repository performance; (b) enhancing metadata quality, interoperability, and transparent governance to bolster impact signals; and (c) sustaining longitudinal data collection to examine changes over time and across disciplines.