AI-powered Fact-Checking for Local News Credibility Systems

Thesis Abstract

This study addresses the escalating challenge of misinformation in local news ecosystems and the consequent erosion of public trust, by developing and evaluating an AI-powered fact-checking system designed to enhance credibility assessment in local news workflows. The aim is to design, implement, and validate a scalable verification framework that integrates automated content analysis with human-in-the-loop processes to improve end-user trust and newsroom efficiency. Specific objectives include (1) to identify key credibility indicators in local news content through a mixed-methods pre-study of 40 local outlets; (2) to develop an AI-powered fact-checking pipeline that leverages natural language processing, knowledge graphs, and source-verification heuristics to generate credibility scores for local articles; (3) to evaluate the system’s performance against expert human fact-checkers using a labeled dataset of 2,000 local news items with ground-truth annotations; (4) to examine newsroom adoption barriers and enablers via a longitudinal field study in five local newsrooms over six months; and (5) to propose a governance and ethics framework for deploying AI-based credibility tools in local journalism. The research adopts a pragmatic mixed-methods design, combining quantitative analysis of verification accuracy with qualitative insights from newsroom practitioners. The population comprises local journalists, editors, fact-checkers, and readers in a mid-sized metropolitan region. A stratified random sample of 120 news articles, 20 editors, and 15 journalists will be surveyed, supplemented by 60 in-depth interviews and 12 focus groups with newsroom staff and readers. The data collection instruments include a) a labeled corpus of 2,000 local news items with fact-check annotations, b) a credibility rubric aligned with the CRAAP framework (Currency, Relevance, Authority, Accuracy, Purpose), c) a semi-structured interview guide exploring workflow integration, perceived usefulness, and trust, and d) a reader survey measuring perceived credibility and engagement. Validity and reliability will be ensured through inter-rater reliability checks (Cohen’s kappa > 0.8 for fact-check annotations), pilot testing of instruments, and triangulation across data sources. The AI component comprises a multi-module pipeline a) claim extraction and classification using transformer-based models (BERT-like encoders) fine-tuned on local news corpora, b) evidence retrieval from a dynamic knowledge graph of local institutions and official records, c) source credibility scoring using policy-aware heuristics, and d) a decision-support interface presenting credibility scores, flag explanations, and suggested actions for editors. Data analysis will employ a) supervised evaluation metrics (precision, recall, F1) for the fact-checking module against the expert-labeled dataset, b) ROC-AUC analysis to assess discrimination between credible and non-credible articles, c) thematic analysis of interview and focus group transcripts to identify adoption determinants, and d) regression analysis to examine relationships between system use, perceived credibility, and reader trust. A comparative analysis will test the system against baseline human-only fact-checking and a rule-based verifier. The study anticipates findings that demonstrate improved verification speed and comparable accuracy to expert fact-checkers, with higher perceived credibility among readers when the AI-assisted process is transparent and explainable. Expected contributions include (i) a validated, scalable AI-powered fact-checking architecture tailored to local news contexts; (ii) an evidence-based understanding of how automated credibility signals influence newsroom decision-making and audience trust; (iii) a governance framework outlining ethical considerations, transparency requirements, and risk mitigation for deployment in local journalism; and (iv) policy recommendations for integrating AI-assisted verification into newsroom workflows without compromising journalistic autonomy. The main conclusion is that AI-powered fact-checking, when integrated with human oversight and transparent explanations, can enhance the credibility of local news and restore reader trust, provided that governance, workflow alignment, and continuous evaluation mechanisms are embedded. Recommendations include iterative refinement of the knowledge graph with local source ontologies, ongoing proficiency training for editors in AI-assisted workflows, and stakeholder engagement strategies to maintain ethical standards and public accountability.