Estimating Small-Sample Robustness in Time Series Forecasts with Bootstrap

Thesis Abstract

This study addresses the challenge of assessing forecast robustness in small-sample time series settings, where traditional asymptotic properties of bootstrap methods may be compromised by limited data and complex dependency structures. The aim is to develop and evaluate a robust bootstrap framework that accurately quantifies forecast uncertainty for short-horizon predictions. The specific objectives are (1) to compare bootstrap variants—residual, block, and sieve bootstrap—in terms of bias, variance, and coverage probabilities for point forecasts and prediction intervals; (2) to adapt bootstrap procedures to autoregressive and moving-average processes with structural breaks and regime shifts commonly observed in real-world data; (3) to establish diagnostic criteria for selecting appropriate bootstrap schemes given sample size, dependence, and nonstationarity; and (4) to provide practical guidelines for practitioners on implementing small-sample bootstrap in financial and macroeconomic forecasting contexts. The methodology adopts a quantitative empirical research design, drawing on time series data from two domains with distinct dynamics to enhance external validity. The population comprises endogenous time series generated by autoregressive integrated moving average (ARIMA) processes observed at daily and monthly frequencies. In Study 1, a financial returns series (S&P 500 index) spanning 3,000 daily observations is utilized; in Study 2, a macroeconomic indicator (U.S. quarterly unemployment rate) spanning 120 quarters is employed. For each series, a set of plausible data-generating processes is specified to introduce potential nonstationarities, structural breaks, and regime changes. The sample sizes mirror realistic constraints 3,000 observations for high-frequency data and 120 observations for low-frequency data. A combination of residual-based, block-based (moving block and stationary bootstrap), and sieve bootstrap methods is implemented, with adjustments to block lengths and model selection criteria governed by cross-validation and information criteria (AIC/BIC). The data collection instruments are solely archival time series extracts from established databases (e.g., Bloomberg Finance, Federal Reserve Economic Data). Validity and reliability are addressed through replication across bootstrap schemes, sensitivity analyses for block length and bandwidth choices, and out-of-sample backtesting to evaluate predictive performance. Analytical procedures include estimation of point forecasts using ARIMA and VAR models augmented with exogenous predictors where appropriate, followed by construction of forecast prediction intervals via bootstrap replicates. The primary analytic focus is on the coverage accuracy and interval width of bootstrap-based prediction intervals under small-sample constraints. Comparative metrics comprise empirical coverage rates, average interval lengths, and quantile-based forecast error measures (CRPS and PI-score). Hypothesis testing targets the null of nominal 95% coverage being statistically indistinguishable from empirical coverage under each bootstrap method, assessed with Wilson binomial confidence intervals and simulation-based p-values. Additional analyses examine the robustness of conclusions to model misspecification and nonstationarity. Theoretical grounding rests on the bootstrap theories of Efron and Tibshirani, augmented by recent advances in dependent data bootstrap (Lahiri) and small-sample corrections for time series (Künsch, Bühlmann). The study also contemplates the practical relevance of the bootstrap in the presence of structural breaks, leveraging literature on regime-switching and time-varying parameter models. Key expected findings include (i) residual bootstrap will exhibit biased coverage in small samples with pronounced serial correlation, (ii) block bootstrap will improve interval validity relative to residual methods but may require adaptive block lengths to balance bias and variance, (iii) sieve bootstrap will provide superior coverage under strong autoregressive dynamics but is sensitive to the chosen approximating model order, and (iv) Bayesian or hybrid approaches incorporating prior information may outperform classical bootstrap in certain nonstationary contexts. The study anticipates that an adaptive composite bootstrap scheme—combining block resampling with model-based residuals—will offer robust coverage across diverse data-generating processes. The study contributes to knowledge by providing a rigorous, empirically validated framework for estimating small-sample forecast uncertainty in time series through bootstrap, including practical guidelines for method selection, block-length optimization, and diagnostic checks tailored to short sequences. It advances understanding of how bootstrap performance interacts with nonstationarity and structural breaks, informing both methodological development and applied forecasting practice in finance and macroeconomics. The main conclusion is that no single bootstrap variant universally outperforms others in small samples; instead, a data-driven, context-aware ensemble approach yields the most reliable forecast intervals. Recommendations include adopting adaptive block-length selection procedures, incorporating model misspecification diagnostics, and disseminating practitioner-oriented decision rules for method choice based on sample size, dependence structure, and detected regime changes.