AI-driven restoration workflow for cultural heritage paintings using multispectral imaging

Thesis Abstract

This study addresses the challenge of preserving cultural heritage paintings by integrating artificial intelligence (AI) with multispectral imaging to detect, document, and guide restoration interventions with objectivity and repeatability. The problem centers on the limited interpretability of multispectral data and the variability of aging processes across pigments, varnishes, and ground layers, which often leads to inconsistent restoration decisions and potential loss of original material. The aim is to develop an AI-driven restoration workflow that leverages multispectral imaging to segment, classify, and quantify degradation patterns, map pigment and binder distributions, and propose restoration scenarios with confidence estimates. Specific objectives include (1) to create a standardized imaging protocol encompassing visible, near-infrared, short-wave infrared, and ultraviolet-induced fluorescence channels for a representative corpus of paintings; (2) to train and validate machine learning models for pigment recognition, varnish detection, craquelure segmentation, and degradation heat-map generation using a curated dataset of 120 authenticated paintings and 60 scientifically documented restorations; (3) to integrate radiometric calibration and image fusion techniques to produce a unified restoration map aligned with a digital conservation notebook; (4) to develop a decision-support framework that recommends conservation actions and quantifies associated uncertainty using Bayesian neural networks and Monte Carlo dropout; and (5) to evaluate the workflow against expert conservator judgments and documented restoration outcomes. The methodology adopts a mixed-methods research design combining quantitative image analytics with qualitative expert evaluation. The population comprises paintings from a university conservation laboratory collection and regional museum loans, with a purposive sample of 180 high-value works spanning oil on canvas and panel supports from the 15th to 19th centuries. Data collection instruments include a multispectral imaging rig (RGB, NIR, SWIR, UVF channels), calibrated reflectance standards, hyperspectral data cubes, spectroscopic pigment libraries, and a workflow diary capturing conservation decisions. Instrument validation involves cross-validation with 20 independently annotated paintings and inter-rater reliability assessment (Cohen’s kappa). Data analysis employs convolutional neural networks (CNNs) for material classification, semantic segmentation (U-Net variant) for craquelure and delamination mapping, and variational autoencoders for anomaly detection in degradation patterns. Model calibration uses Bayesian inference to estimate predictive uncertainties, while image fusion relies on wavelet-based and spectral unmixing techniques to produce an integrated restoration map. Statistical analyses include regression modeling to relate model outputs to expert assessments, along with repeatability tests using intra-class correlation. The study anticipates key findings high accuracy in pigment and varnish classification (F1 scores >0.88), robust craquelure segmentation (Dice coefficient >0.84), and reliable degradation heat maps with quantified uncertainty (mean absolute error <0.07 on standardized scales). The Bayesian framework is expected to provide actionable confidence intervals for recommended interventions, enabling traceable decision-making. The study contributes to knowledge by operationalizing a reproducible AI-assisted restoration workflow that rigorously fuses multispectral data with conservator expertise, advancing precision conservatorship and digital documentation standards. It also advances theoretical understanding of how spectral signatures relate to aging processes in complex art matrices and demonstrates the applicability of explainable AI techniques in cultural heritage contexts. The main conclusion anticipated is that an AI-driven workflow can augment, not replace, conservator judgment by delivering transparent, data-backed restoration maps and intervention options with quantified uncertainty, thereby reducing subjectivity and increasing reproducibility across institutions. Practical recommendations include adopting the workflow as a supplementary tool in conservation studios, expanding the pigment library to regional materials, and developing open data standards for multispectral conservation imaging. The study also outlines future research directions, such as extending the framework to three-dimensional surface texture analysis, integrating non-destructive chemical sensing modalities, and exploring transfer learning to adapt models to diverse painting schools.