A Probabilistic Framework for Nonparametric Evolutionary Model Selection Systems

Thesis Abstract

This study addresses the challenge of selecting increasingly complex models in nonparametric settings where traditional criteria misalign with evolving data structures and sparse observations. The objective is to develop a probabilistic framework that unifies nonparametric evolutionary model selection with rigorous uncertainty quantification, enabling robust model adaptation as data accrue. Specific objectives include (i) to formulate a hierarchical probabilistic model that accommodates nonparametric function classes and evolutionary shifts in data-generating processes; (ii) to derive model-agnostic selection criteria based on posterior predictive performance, information criteria adapted to nonparametric contexts, and a sequential decision process inspired by Bayesian experimental design; (iii) to implement an efficient algorithmic pipeline leveraging reversible-jump Markov chain Monte Carlo (RJ-MCMC) and evolving kernel methods to enable tractable exploration of model space; and (iv) to evaluate the framework on synthetic benchmarks and real-world datasets across domains requiring flexible, data-driven model adaptation. The methodology adopts a mixed-methods approach anchored in Bayesian inference and probabilistic machine learning. The population comprises simulated datasets that encode nonstationary and nonparametric dynamics, complemented by empirical time-series and spatial-temporal datasets from environmental monitoring, finance, and epidemiology. A sample of 50 synthetic scenarios with varying degrees of smoothness, drift, and regime-switching is generated to validate methodological properties, followed by application to three real-world datasets (a) climate proxy series with irregular sampling, (b) high-frequency financial tick data exhibiting volatility clustering, and (c) infection count data with under-reporting and reporting delays. Instrumentation includes synthetic data generators with controlled evolutionary parameters, publicly available time-series from meteorological stations, intraday price feeds, and anonymized health surveillance records. Validity and reliability are ensured through comprehensive simulation studies, including calibration of prior distributions and sensitivity analyses across prior misspecification and sample size. The data analysis combines several analytical strands (i) RJ-MMC-based model trans-dimensional inference to explore nonparametric function classes and their evolutionary transitions; (ii) Gaussian process (GP) priors and deep kernel learning to capture complex dependencies while controlling overfitting; (iii) posterior predictive checks and cross-validation tailored to nonstationary contexts; and (iv) decision-theoretic criteria, including expected utility and Bayesian false discovery rates, to guide model selection as data accumulate. The theoretical foundation integrates the Bayesian nonparametric framework of the Indian buffet process for latent feature allocation with evolutionary Bayesian modeling concepts and concepts from information geometry to quantify model complexity changes over time. The analysis further employs kernel-based density estimation, time-varying coefficient models, and nonparametric regression under a probabilistic paradigm, with computational algorithms implemented in Python and Stan. Expected findings include (i) a coherent probabilistic mechanism to prefer models that balance fit with evolutionary parsimony, (ii) robust detection of regime changes and smooth vs. abrupt functional shifts without overfitting, and (iii) improved predictive performance on out-of-sample data relative to fixed-structure nonparametric models. Anticipated results will reveal that integrated posterior predictive loss and sequential Bayes factors provide reliable signals for model evolution, while RJ-MCMC efficiency improves with adaptive proposal mechanisms and sparse GP approximations. The study contributes to knowledge by delivering a unified probabilistic framework for nonparametric evolutionary model selection that explicitly accounts for time-varying dynamics and uncertainty in a principled manner. It demonstrates how combining nonparametric priors, evolutionary mechanisms, and decision-theoretic criteria enhances model adaptability and predictive reliability in complex data environments. Practical implications include improved model surveillance in environmental monitoring, adaptive risk assessment in finance, and timely detection of epidemiological shifts. Recommendations for future research include extending the framework to incorporate multi-fidelity data sources, exploring scalable variational approximations for real-time deployment, and integrating causal inference perspectives to disentangle evolutionary effects from confounders.