Walk-Forward Evaluation of Early-Warning Models for Large Bitcoin Movements Under Fixed and Train-Only Event Definitions
Open Access
Abstract
Evaluations of financial early-warning models can appear stronger when temporal separation is incomplete or when the event definition uses information unavailable at the forecast origin. This study examines one-day-ahead risk ranking of large absolute Bitcoin returns using 334 daily observations from 31 August 2024 to 30 July 2025, five expanding-window folds, lagged predictors, training-fold preprocessing, and train-only classification-threshold selection. The original q85 and q90 cutoffs, estimated from the full study-period return distribution, are retained as retrospective fixed-label benchmarks and are complemented by fold-specific train-only cutoffs. Under fixed q85 labels, class-weighted Logistic Regression with technical and market-activity variables achieved pooled ROC-AUC = 0.723 (moving-block 95% CI [0.556, 0.822]) and Average Precision = 0.338 ([0.103, 0.565]); Random Forest yielded 0.739 and 0.352. Pooled MCC was 0.207, but the primary model issued no positive warnings in three of the five fixed-q85 test folds. Under train-only q85 labels, ROC-AUC was 0.674 and Average Precision was 0.174, but the retained MCC threshold-selection rule produced no true positives. Conventional GARCH(1,1), EWMA, and 14-day historical-volatility scores did not exceed the primary model on both ranking metrics. Adding the complete six-variable sentiment and attention block lowered point estimates, although one-variable ablations and a matched-dimension control do not isolate sentiment content from dimensionality and small-sample overfitting. The results provide limited, sample-specific evidence for risk ranking, not evidence of deployment readiness, economic profitability, or cross-asset generalizability.Keywords
This publication has 44 references indexed in Scilit:
- A Novel Approach to Forecasting After Large Forecast ErrorsJournal of Forecasting, 2025
- Forecasting realized volatility of crude oil futures prices based on machine learningJournal of Forecasting, 2024
- Deep learning models for price forecasting of financial time series: A review of recent advancements: 2020–2022WIREs Data Mining and Knowledge Discovery, 2023
- Modeling uncertainty in financial tail risk: A forecast combination and weighted quantile approachJournal of Forecasting, 2023
- Reddit financial image post sentiment datasetData in Brief, 2022
- Forecasting directional bitcoin price returns using aspect-based sentiment analysis on online text dataMachine Learning, 2021
- Evaluating time series forecasting models: an empirical study on performance estimation methodsMachine Learning, 2020
- Early Warning Systems for Currency Crises with Real-Time DataOpen Economies Review, 2019
- On policymakers’ loss functions and the evaluation of early warning systemsEconomics Letters, 2013
- Generalized autoregressive conditional heteroskedasticityJournal of Econometrics, 1986