Designing and Evaluating a Moral Reasoning Debugger for AI Systems

Thesis Abstract

The rapid integration of autonomous decision-making systems into critical sectors has intensified concerns regarding the moral quality of AI behavior and the detectability of underlying ethical misalignments. This study addresses the gap between theoretical normative models of AI ethics and practical, observable decision-making processes by designing and evaluating a Moral Reasoning Debugger (MRD) capable of diagnosing, annotating, and guiding the refinement of moral reasoning in AI systems. The aim is to develop an actionable debugging tool that (i) exposes the implicit ethical assumptions encoded in AI agents, (ii) assesses alignment with deontological and virtue-theoretic criteria in real-time decision traces, and (iii) supports iterative improvement through actionable remediation recommendations. Specific objectives include (1) conceptualizing a modular MRD architecture grounded in the theories of rule-utilitarian ethics and virtue ethics; (2) implementing a provenance-rich instrumentation layer to capture decision rationale, outcome metadata, and contextual factors from a suite of benchmark AI agents; (3) developing and validating a mixed-methods evaluation protocol that combines quantitative metrics of moral alignment with qualitative interpretability analyses; and (4) empirically evaluating MRD across three domains—healthcare robotics, autonomous driving, and resource allocation—using synthetic and real-world scenarios. The research adopts a design, implementation, and evaluation paradigm, integrating technical, philosophical, and empirical components. The population comprises twenty AI agents spanning rule-based, learning-based, and hybrid architectures deployed in controlled simulation environments calibrated to ethically salient tasks. A purposive sample of 60 decision episodes per agent, totaling 1,200 episodes, will be collected, including both scripted moral-dilemma scenarios and emergent, unanticipated cases. Data collection instruments encompass (a) the MRD instrumentation framework capturing decision traces, rationale tags, and contextual metadata; (b) a Moral Alignment rubric derived from a fusion of RAW (Rule-Utilitarian-Augmented) criteria and virtue-ethics narratives; and (c) evaluation interviews with domain experts to assess interpretability and remediation feasibility. Validity and reliability will be pursued through triangulation across trace data, expert judgments, and inter-annotator agreement (Cohen’s kappa target ? 0.75). Analytical approaches combine quantitative and qualitative methods. The MRD’s diagnostic performance will be assessed via regression analysis to identify predictors of misalignment scores, ANOVA to compare performance across agent types and domains, and ROC analysis to evaluate the debugger’s sensitivity and specificity in flagging ethically salient episodes. Thematic analysis will be conducted on expert interview transcripts to extract recurring interpretability challenges and remediation design patterns. A structural equation model will test hypothesized relations among representation clarity, agent transparency, and downstream remediation efficacy. A conceptual model will integrate the MRD’s diagnostic signals with normative ethics theories, illustrating pathways from detected misalignment to concrete guideline-based interventions. Expected findings include (i) higher misalignment detection precision for hybrid agents relative to purely rule-based systems due to richer internal rationale representations; (ii) demonstrable improvements in subsequent agent decisions following MRD-guided interventions, evidenced by reduced misalignment scores in post-intervention episodes; and (iii) robust interpretability gains for domain experts, enabling more actionable remediation plans without compromising system performance. The study anticipates that alignment with RAW criteria and virtue-ethics considerations will significantly contribute to the explainability of moral reasoning in AI systems, revealing actionable design patterns for embedding ethical reflection into autonomous decision pipelines. The research contributes to knowledge by operationalizing a replicable, theory-informed MRD that bridges normative ethics and computational practice, providing a systematic methodology for auditing moral reasoning and guiding iterative improvements in AI agents. It informs policy and governance by demonstrating how transparent debugging tools can support accountability in high-stakes AI deployments. The study concludes with practical recommendations for MRD integration into development lifecycles, guidelines for sector-specific moral calibration, and avenues for extending the framework to emergent AI paradigms, such as continual learning systems and multi-agent moral ecology.