Stability and Generalization for Distributed SGDA
- 24 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (8) , 9197-9209
Abstract
Minimax optimization is gaining increasing attention in modern machine learning applications. Driven by large-scale models and massive volumes of data collected from edge devices, as well as the concern to preserve client privacy, distributed minimax optimization algorithms become popular, such as Local Stochastic Gradient Descent Ascent (Local-SGDA), and Local Decentralized SGDA (Local-DSGDA). While most existing research on distributed minimax algorithms focuses on convergence rates and communication efficiency, their generalization performance remains largely unexplored, whereas generalization ability is a pivotal indicator for evaluating the holistic performance of a model when fed with unknown data. In this paper, we propose the stability-based generalization analytical framework for Distributed-SGDA, which unifies two popular distributed minimax algorithms including Local-SGDA and Local-DSGDA, and conduct a comprehensive analysis of stability error, generalization gap, and population risk across different metrics under various settings, e.g., (S)C-(S)C, PL-SC, and NC-NC cases. Our theoretical results reveal the trade-off between the generalization gap and optimization error and suggest hyperparameters choice to obtain the optimal population risk. Numerical experiments for Local-SGDA and Local-DSGDA validate the theoretical results.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62225113)
- Innovative Research Group Project of Hubei Province (2024AFA017)
- New Cornerstone Science Foundation through the XPLORER PRIZE
- National Natural Science Foundation of China (62576364)
- Basic and Applied Basic Research Foundation of Guangdong Province (2026B1515020071)
- Shenzhen Basic Research Project (Natural Science Foundation) Basic Research Key Project (JCYJ20241202124430041)
- CCF-DiDi GAIA Collaborative Research Funds (CCF-DiDi GAIA 202508)
This publication has 17 references indexed in Scilit:
- CDMA: A Practical Cross-Device Federated Learning Algorithm for General Minimax ProblemsProceedings of the AAAI Conference on Artificial Intelligence, 2023
- Prescribed Safety Performance Imitation Learning From a Single Expert DatasetIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Local AdaGrad-type algorithm for stochastic convex-concave optimizationMachine Learning, 2022
- Near-Optimal Decentralized Algorithms for Saddle Point Problems over Time-Varying Networks2021
- Towards Deep Learning Models Resistant to Adversarial AttacksarXiv, 2019
- Network Topology and Communication-Computation Tradeoffs in Decentralized OptimizationProceedings of the IEEE, 2018
- Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition2016
- Matrix Analysis2012
- STABILITY RESULTS IN LEARNING THEORYAnalysis and Applications, 2005
- Internet pricing with a game theoretical approach: concepts and examplesIEEE/ACM Transactions on Networking, 2002