Designing, Implementing, and Evaluating AI-Driven Performance Appraisals in SMEs

Thesis Abstract

In the contemporary SME sector, performance appraisal practices remain largely manual and retrospective, constraining timely feedback, objective evaluation, and evidence-based HR interventions; AI-driven approaches offer potential to enhance accuracy, bias reduction, and decision support, yet empirical validation within small- and medium-sized enterprises is scarce. The study aims to design, implement, and evaluate an AI-assisted performance appraisal framework tailored for SMEs, with specific objectives to (i) develop a pragmatic AI-enabled appraisal system integrating objective task metrics, supervisor ratings, and employee self-assessments; (ii) implement the system in a cohort of 12 SMEs across manufacturing and services sectors and calibrate its algorithms using a 12-month pilot; (iii) assess impacts on appraisal reliability, perceived justice, employee engagement, and subsequent performance using mixed methods; and (iv) formulate guidelines for SME-level adoption and governance of AI-driven HR analytics. The theoretical foundation draws on expectancy theory (Vroom) to link appraisal outcomes with motivation, and technological acceptance model (TAM) to explain adoption drivers, augmented by bias and fairness considerations from fairness in machine learning literature. Methodologically, the study adopts a convergent parallel mixed-methods design. The population comprises HR professionals, line managers, and employees within 12 SMEs employing 50–250 staff. A purposive sample of 48 HR practitioners and 312 employees will be recruited for quantitative surveys, complemented by semi-structured interviews with 24 managers and 24 employees to enrich interpretive insights. Data collection instruments include (a) a validated Likert-scale survey measuring appraisal reliability (inter-rilm consistency), perceived organizational justice, engagement (Utrecht Work Engagement Scale), and job performance indicators; (b) system usage logs and AI-generated appraisal outputs; (c) interview protocols exploring experiences with AI assistance, perceived bias, and governance. Validity and reliability will be established through pilot testing, CFA confirmatory analysis, and triangulation of survey and interview data. Quantitative data will be analyzed using multiple regression to test the effects of AI-assisted appraisal quality on perceived justice and engagement, followed by hierarchical linear modeling to assess cross-level effects at the SME level. AI components will be evaluated for fairness and bias using disparate impact analysis, calibration plots, and SHAP (SHapley Additive exPlanations) values to interpret feature contributions. The qualitative data will undergo thematic analysis guided by Braun and Clarke, with coding frames developed iteratively to capture themes on usability, trust, and organizational impact. Integration of findings will occur at the interpretation stage to converge quantitative and qualitative results. Expected findings include (i) improved reliability of performance ratings and reduced rating variance compared with traditional methods, (ii) enhanced perceived procedural justice and higher employee engagement in SMEs adopting AI-assisted appraisal, (iii) nuanced understanding of bias risks and the effectiveness of governance mechanisms (e.g., overseen by HR analytics boards), and (iv) differential effects across sectors and firm sizes, moderated by management AI literacy. The study contributes to knowledge by providing empirical evidence on the feasibility, effectiveness, and governance of AI-driven performance appraisal in resource-constrained organizations, clarifying the interaction between AI tools and motivational processes, and offering a practical implementation blueprint for SMEs. The main conclusion is that a carefully designed AI-assisted appraisal framework, underpinned by transparent explainability, fairness controls, and robust governance, can improve appraisal quality and employee outcomes without compromising trust, provided organizational readiness and continuous monitoring are ensured. Recommendations include establishing SME-friendly data governance protocols, ongoing stakeholder training, periodic bias audits, and scalable deployment guides that align AI capabilities with SME strategic HR objectives.