Abstract
Minimax optimization is gaining increasing attention in modern machine learning applications. Driven by large-scale models and massive volumes of data collected from edge devices, as well as the concern to preserve client privacy, distributed minimax optimization algorithms become popular, such as Local Stochastic Gradient Descent Ascent (Local-SGDA), and Local Decentralized SGDA (Local-DSGDA). While most existing research on distributed minimax algorithms focuses on convergence rates and communication efficiency, their generalization performance remains largely unexplored, whereas generalization ability is a pivotal indicator for evaluating the holistic performance of a model when fed with unknown data. In this paper, we propose the stability-based generalization analytical framework for Distributed-SGDA, which unifies two popular distributed minimax algorithms including Local-SGDA and Local-DSGDA, and conduct a comprehensive analysis of stability error, generalization gap, and population risk across different metrics under various settings, e.g., (S)C-(S)C, PL-SC, and NC-NC cases. Our theoretical results reveal the trade-off between the generalization gap and optimization error and suggest hyperparameters choice to obtain the optimal population risk. Numerical experiments for Local-SGDA and Local-DSGDA validate the theoretical results.
Funding Information
  • National Natural Science Foundation of China (62225113)
  • Innovative Research Group Project of Hubei Province (2024AFA017)
  • New Cornerstone Science Foundation through the XPLORER PRIZE
  • National Natural Science Foundation of China (62576364)
  • Basic and Applied Basic Research Foundation of Guangdong Province (2026B1515020071)
  • Shenzhen Basic Research Project (Natural Science Foundation) Basic Research Key Project (JCYJ20241202124430041)
  • CCF-DiDi GAIA Collaborative Research Funds (CCF-DiDi GAIA 202508)

This publication has 17 references indexed in Scilit: