The Unreasonable Effectiveness of Randomized Representations in Online Continual Graph Learning
Open Access
- 15 September 2026
- journal article
- research article
- Published by Springer Nature in Machine Learning
- Vol. 115 (9) , 220
Abstract
Catastrophic forgetting is one of the main obstacles for Online Continual Graph Learning (OCGL), where nodes arrive one by one, distribution drifts may occur at any time and offline training on task-specific subgraphs is not feasible. In this work, we explore a surprisingly simple yet highly effective approach for OCGL: we use a fixed, randomly initialized encoder to generate robust and expressive node embeddings by aggregating neighborhood information, training online only a lightweight classifier. By freezing the encoder, we eliminate drifts of the representation parameters, a key source of forgetting, obtaining embeddings that are both expressive and stable. When evaluated across several OCGL benchmarks, despite its simplicity and lack of memory buffer, this approach yields consistent gains over state-of-the-art methods, with surprising improvements of up to 30% and performance often approaching that of the joint offline-training upper bound. These results suggest that in OCGL, catastrophic forgetting can be minimized without complex replay or regularization by embracing architectural simplicity and stability.Keywords
All Related Versions
Funding Information
- Università degli Studi di Padova
This publication has 48 references indexed in Scilit:
- Continual learning approaches for anomaly detectionEvolving Systems, 2025
- Distance-based change point detection for novelty detection in concept-agnostic continual anomaly detectionJournal of Intelligent Information Systems, 2025
- E-GRACL: an IoT intrusion detection system based on graph neural networksThe Journal of Supercomputing, 2024
- An Untrained Neural Model for Fast and Accurate Graph Classification2023
- Online continual learning in image classification: An empirical surveyNeurocomputing, 2022
- Memory Aware Synapses: Learning What (not) to Forget2018
- Randomness in neural networks: an overviewWIREs Data Mining and Knowledge Discovery, 2017
- Extreme learning machine: Theory and applicationsNeurocomputing, 2006
- On the Surprising Behavior of Distance Metrics in High Dimensional Space2001
- Learning and generalization characteristics of the random vector functional-link netNeurocomputing, 1994