A Nonlinear Target-Factor Model with Attention Mechanism for Mixed-Frequency Data
Preprint
Abstract
We propose Mixed-Panels-Transformer Encoder (MPTE), a novel framework for estimating factor models in panel datasets with mixed frequencies and nonlinear signals. Traditional factor models rely on linear signal extraction and require homogeneous sampling frequencies, limiting their applicability to modern high-dimensional datasets where variables are observed at different temporal resolutions. Our approach leverages Transformer-style attention mechanisms to enable context-aware signal construction through flexible, data-dependent weighting schemes that replace fixed linear combinations with adaptive reweighting based on similarity and relevance. We extend classical principal component analysis (PCA) to accommodate general temporal and cross-sectional attention matrices, allowing the model to learn how to aggregate information across frequencies without manual alignment or pre-specified weights. For linear activation functions, we establish consistency and asymptotic normality of factor and loading estimators, showing that our framework nests Target PCA as a special case while providing efficiency gains through transfer learning across auxiliary datasets. The nonlinear extension uses a Transformer architecture to capture complex hierarchical interactions while preserving the theoretical foundations. In simulations, MPTE demonstrates superior performance in nonlinear environments, and in an empirical application to 13 macroeconomic forecasting targets using a selected set of 48 monthly and quarterly series from the FRED-MD and FRED-QD databases, our method achieves competitive performance against established benchmarks. We further analyze attention patterns and systematically ablate model components to assess variable importance and temporal dependence. The resulting patterns highlight which indicators and horizons are most influential for forecasting.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- A novel probabilistic carbon price prediction model: Integrating the transformer framework with mixed-frequency modeling at different quartilesApplied Energy, 2025
- Target PCA: Transfer learning large dimensional panel dataJournal of Econometrics, 2023
- A multi-task encoder-dual-decoder framework for mixed frequency data predictionInternational Journal of Forecasting, 2023
- Characteristics are covariances: A unified model of risk and returnJournal of Financial Economics, 2019
- Factor models for matrix-valued high-dimensional time seriesJournal of Econometrics, 2019
- The key role of forests in meeting climate targets requires science for credible mitigationNature Climate Change, 2017
- Forecasting with factor-augmented regression: A frequentist model averaging approachJournal of Econometrics, 2015
- Nowcasting: The real-time informational content of macroeconomic dataJournal of Monetary Economics, 2008
- THE GENERALIZED DYNAMIC FACTOR MODEL: REPRESENTATION THEORYEconometric Theory, 2001
- Neural networks and principal component analysis: Learning from examples without local minimaNeural Networks, 1989