Assessment of Bayesian Model Averaging in Small-Sample Economic Forecasting with Real-Time Data

Thesis Abstract

This study addresses the challenge of reliable economic forecasting in small-sample environments by evaluating Bayesian Model Averaging (BMA) as a robust tool for real-time data assimilation and model uncertainty quantification. The problem arises when conventional single-model approaches exhibit overfitting, biased forecast intervals, and degraded predictive performance due to limited observation counts and rapidly arriving data. The aim is to assess whether BMA improves forecast accuracy and interval coverage relative to competing methods under real-time data constraints, and to identify conditions under which BMA yields superior performance. Specific objectives include (1) comparing predictive accuracy and calibrated prediction intervals of BMA against traditional Bayesian regression, autoregressive models, and ensemble methods across multiple small-sample macroeconomic forecasting tasks; (2) evaluating the impact of prior specification, model priors, and hyperparameters on out-of-sample predictive performance; (3) examining real-time updating dynamics of posterior model probabilities as new data arrive; (4) investigating the robustness of BMA to model misspecification and structural breaks; and (5) deriving practical guidelines for employing BMA in policy-relevant short-horizon forecasts. A quantitative, empirical research design is adopted. The population comprises macroeconomic time series from a developed economy, including quarterly real GDP growth, inflation, and unemployment rates, observed over a 15-year window with 60–80 distinct forecast horizons. The sampling frame focuses on small-sample periods with 20–40 observations per rolling window, reflecting realistic data scarcity in policy evaluation. Data will be drawn from official statistics releases and high-frequency indicators aggregated to quarterly frequency. The instruments of data collection are standardized macroeconomic indicators and their contemporaneous auxiliary variables (e.g., survey expectations, commodity prices, financial conditions indices). The primary method of analysis is Bayesian Model Averaging over a candidate set of parsimonious models estimated via hierarchical Bayesian regression, incorporating weakly informative priors and model priors that penalize overcomplexity. Forecast performance will be evaluated using out-of-sample metrics including root mean squared forecast error (RMSFE), mean absolute error (MAE), and probability integral transform (PIT) coverage, with emphasis on predictive interval calibration. Rolling-window backtesting with 1000 simulations will be employed to ensure robust inference about performance across varying data vintages. Complementary methods such as cross-validated predictive likelihood and Bayesian model selection criteria (e.g., Bayes factors) will be used to contextualize results. The analysis will specify a dynamic BMA framework that allows model weights to evolve as new observations arrive, enabling an adaptive ensemble that captures parameter uncertainty and model misspecification. Model space will include a diverse set of linear and sparse nonlinear specifications, with considerations of lag structure and exogenous predictors. Sensitivity analyses will explore alternative prior specifications, hierarchical pooling of coefficients, and the influence of different prior inclusion probabilities. Theoretical grounding draws on the Bayesian decision-theoretic framework and Agena Bayesian model averaging literature, with reference to information criteria as complementary benchmarks. Ethical considerations will address data provenance and reproducibility, including transparent reporting of priors and code. Expected findings anticipate that BMA will yield superior predictive accuracy and well-calibrated forecast intervals in small-sample settings, particularly when incorporating diverse model specifications and real-time updates. It is expected that performance gains will be most pronounced in scenarios with moderate structural change and when priors are carefully tuned to reflect domain knowledge about macroeconomic dynamics. The study aims to contribute to knowledge by (a) providing empirical evidence on the practical benefits and limitations of BMA in real-time, small-sample forecasting, (b) clarifying how prior and model-prior choices shape predictive performance, and (c) offering implementable guidance for researchers and policymakers on deploying BMA to enhance decision-making under data scarcity. The main conclusion is anticipated to be that Bayesian Model Averaging, with appropriate dynamic weighting and prudent prior elicitation, offers a robust framework for small-sample economic forecasting under real-time data constraints, outperforming single-model and static ensemble approaches in both accuracy and coverage. Recommendations will include guidelines for prior specification, model space design, rolling-window sizes, and reporting standards to facilitate reproducibility and practical adoption in central-bank and statistical agencies.