LEAF: Navigating Concept Drift in Cellular Networks
Preprint
Abstract
Operational networks commonly rely on machine learning models for many tasks, including detecting anomalies, inferring application performance, and forecasting demand. Yet, model accuracy can degrade due to concept drift, whereby the relationship between the features and the target to be predicted changes. Mitigating concept drift is an essential part of operationalizing machine learning models in general, but is of particular importance in networking's highly dynamic deployment environments. In this paper, we first characterize concept drift in a large cellular network for a major metropolitan area in the United States. We find that concept drift occurs across many important key performance indicators (KPIs), independently of the model, training set size, and time interval -- thus necessitating practical approaches to detect, explain, and mitigate it. We then show that frequent model retraining with newly available data is not sufficient to mitigate concept drift, and can even degrade model accuracy further. Finally, we develop a new methodology for concept drift mitigation, Local Error Approximation of Features (LEAF). LEAF works by detecting drift; explaining the features and time intervals that contribute the most to drift; and mitigates it using forgetting and over-sampling. We evaluate LEAF against industry-standard mitigation approaches (notably, periodic retraining) with more than four years of cellular KPI data. Our initial tests with a major cellular provider in the US show that LEAF consistently outperforms periodic and triggered retraining on complex, real-world data while reducing costly retraining operations.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Realtime mobile bandwidth prediction using LSTM neural network and Bayesian fusionComputer Networks, 2020
- Reactive Soft Prototype Computing for Concept Drift StreamsNeurocomputing, 2020
- Machine Learning for Cognitive Network ManagementIEEE Communications Magazine, 2018
- Ensemble learning for data stream analysis: A surveyInformation Fusion, 2017
- Dynamic extreme learning machine for data stream classificationNeurocomputing, 2017
- Correlation and variable importance in random forestsStatistics and Computing, 2016
- Recommender system application developments: A surveyDecision Support Systems, 2015
- Classification with correlated features: unreliability of feature ranking and solutionsBioinformatics, 2011
- A hybrid machine learning approach to network anomaly detectionInformation Sciences, 2007
- Applying lazy learning algorithms to tackle concept drift in spam filteringExpert Systems with Applications, 2007