Yan Sun
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yan Sun has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Dacheng Tao
Nanyang technological university, Singapore, Singapore
5 shared publications - Li Shen
Shenzhen Campus of Sun Yat-sen University
5 shared publications - Liang Ding
University of Sydney
2 shared publications - Bo Du
School of Computer Science, Wuhan University, Wuhan, China
1 shared publication Bridget R. Scanlon
Bureau of Economic Geology, Jackson School of Geosciences, The University of Texas at Austin, Austin, USA
1 shared publication- Hao Sun
School of Computer Science, University of Science and Technology of China, Hefei, 230026, Anhui, China
1 shared publication
Latest Publications
- Open Access
- 9 July 2026
- Published by Copernicus GmbH in EGUsphere
- p. 1-27
A physically guided deep learning reconstruction of terrestrial water storage anomalies at 0.1° across China
Terrestrial water storage (TWS), comprising all surface and subsurface water components, is a key indicator of water availability. The Gravity Recovery and Climate Experiment (GRACE) satellite mission provides large-scale estimates of TWS anomalies (TWSA), but its coarse spatial resolution (3°, approximately 300 km) limits the analysis of hydrologic processes at sub-regional scales. Using a physically-guided deep learning framework, we downscale TWSA from the original 3° GRACE mascons to 0.1° (approximately 10 km) across China, generating a standard version (2002–2019) with comprehensive observations used for model constraints and independent evaluation and an extended version (2020–2023) to support more recent hydrologic analyses. The downscaled TWSA preserves large-scale GRACE signals at the 3° grid scale (median correlation coefficient (CC): 0.95; root-mean-square error (RMSE): 1.38 cm) and basin scale (median CC: 0.94; RMSE: 1.72 cm), with a low median uncertainty (0.88 cm) across China. Its reliability is supported by high consistency with physically informed TWSA spatial patterns at the 0.1° resolution (median CC: 0.91) and internally consistent water balance closure beyond the native GRACE resolution (median CC: 0.80; RMSE: 1.44 cm). Evaluation against independent observations demonstrates that the downscaled TWSA agrees well with groundwater variations in intensively irrigated regions (CC: 0.65 for irrigation intensity > 50 %) and annual glacier elevation change in cryospheric areas (CC: 0.97). The datasets improve fine-scale characterization of TWS variability and associated hydrologic processes in China, and can be used as a reference for evaluating performance of high-resolution hydrologic models. The two versions of the dataset are available at https://doi.org/10.5281/zenodo.19502906.
- 9 July 2026
- Published by Copernicus GmbH in EGUsphere
- p. 1-27
Open AccessA physically guided deep learning reconstruction of terrestrial water storage anomalies at 0.1° across China
Terrestrial water storage (TWS), comprising all surface and subsurface water components, is a key indicator of water availability. The Gravity Recovery and Climate Experiment (GRACE) satellite mission provides large-scale estimates of TWS anomalies (TWSA), but its coarse spatial resolution (3°, approximately 300 km) limits the analysis of hydrologic processes at sub-regional scales. Using a physically-guided deep learning framework, we downscale TWSA from the original 3° GRACE mascons to 0.1° (approximately 10 km) across China, generating a standard version (2002–2019) with comprehensive observations used for model constraints and independent evaluation and an extended version (2020–2023) to support more recent hydrologic analyses. The downscaled TWSA preserves large-scale GRACE signals at the 3° grid scale (median correlation coefficient (CC): 0.95; root-mean-square error (RMSE): 1.38 cm) and basin scale (median CC: 0.94; RMSE: 1.72 cm), with a low median uncertainty (0.88 cm) across China. Its reliability is supported by high consistency with physically informed TWSA spatial patterns at the 0.1° resolution (median CC: 0.91) and internally consistent water balance closure beyond the native GRACE resolution (median CC: 0.80; RMSE: 1.44 cm). Evaluation against independent observations demonstrates that the downscaled TWSA agrees well with groundwater variations in intensively irrigated regions (CC: 0.65 for irrigation intensity > 50 %) and annual glacier elevation change in cryospheric areas (CC: 0.97). The datasets improve fine-scale characterization of TWS variability and associated hydrologic processes in China, and can be used as a reference for evaluating performance of high-resolution hydrologic models. The two versions of the dataset are available at https://doi.org/10.5281/zenodo.19502906.
- 24 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (8) , 9197-9209
Stability and Generalization for Distributed SGDA
Minimax optimization is gaining increasing attention in modern machine learning applications. Driven by large-scale models and massive volumes of data collected from edge devices, as well as the concern to preserve client privacy, distributed minimax optimization algorithms become popular, such as Local Stochastic Gradient Descent Ascent (Local-SGDA), and Local Decentralized SGDA (Local-DSGDA). While most existing research on distributed minimax algorithms focuses on convergence rates and communication efficiency, their generalization performance remains largely unexplored, whereas generalization ability is a pivotal indicator for evaluating the holistic performance of a model when fed with unknown data. In this paper, we propose the stability-based generalization analytical framework for Distributed-SGDA, which unifies two popular distributed minimax algorithms including Local-SGDA and Local-DSGDA, and conduct a comprehensive analysis of stability error, generalization gap, and population risk across different metrics under various settings, e.g., (S)C-(S)C, PL-SC, and NC-NC cases. Our theoretical results reveal the trade-off between the generalization gap and optimization error and suggest hyperparameters choice to obtain the optimal population risk. Numerical experiments for Local-SGDA and Local-DSGDA validate the theoretical results.
- 24 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (8) , 9197-9209
Stability and Generalization for Distributed SGDA
Minimax optimization is gaining increasing attention in modern machine learning applications. Driven by large-scale models and massive volumes of data collected from edge devices, as well as the concern to preserve client privacy, distributed minimax optimization algorithms become popular, such as Local Stochastic Gradient Descent Ascent (Local-SGDA), and Local Decentralized SGDA (Local-DSGDA). While most existing research on distributed minimax algorithms focuses on convergence rates and communication efficiency, their generalization performance remains largely unexplored, whereas generalization ability is a pivotal indicator for evaluating the holistic performance of a model when fed with unknown data. In this paper, we propose the stability-based generalization analytical framework for Distributed-SGDA, which unifies two popular distributed minimax algorithms including Local-SGDA and Local-DSGDA, and conduct a comprehensive analysis of stability error, generalization gap, and population risk across different metrics under various settings, e.g., (S)C-(S)C, PL-SC, and NC-NC cases. Our theoretical results reveal the trade-off between the generalization gap and optimization error and suggest hyperparameters choice to obtain the optimal population risk. Numerical experiments for Local-SGDA and Local-DSGDA validate the theoretical results.
- 20 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Networking
- Vol. 34, 3782-3791
Decentralized Partial Model Personalization With Guaranteed Nonconvex Convergence
Compared with general Federated Learning (FL), Decentralized FL (DFL) has diminished central communication burdens and lower risks of disruption. To be compatible with real-world scenarios, some existing works produce several local personalized models rather than a universal model for all edge devices. However, they still suffer from inferior performance due to the full model aggregation in heterogeneous datasets. Therefore, we propose a DFL framework DFedMDC through partial model personalization, which can adapt to resource-heterogeneous environments (e.g., the Internet of Things (IoT)). Specifically, it personalizes the “right” components of local models and trains the shared model ui and personal model vi alternately in each client i. To further accelerate the convergence process, we propose DFedSMDC with properly directed noise perturbation into the gradient update. Theoretically, we provide convergence analysis of both algorithms in the general non-convex setting. It can shed light on how vital factors affect the convergence rate, such as these alternate updates of partial models ui, vi, data heterogeneity δ2 as well as various communication topologies (characterized by the spectral gap 1 − λ). Empirically, we confirm the state-of-the-art (SOTA) superiority of the proposed methods on several real-world datasets with various data distributions, relative to both SOTA personalized FL (PFL) and DFL baselines.
- 20 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Networking
- Vol. 34, 3782-3791
Decentralized Partial Model Personalization With Guaranteed Nonconvex Convergence
Compared with general Federated Learning (FL), Decentralized FL (DFL) has diminished central communication burdens and lower risks of disruption. To be compatible with real-world scenarios, some existing works produce several local personalized models rather than a universal model for all edge devices. However, they still suffer from inferior performance due to the full model aggregation in heterogeneous datasets. Therefore, we propose a DFL framework DFedMDC through partial model personalization, which can adapt to resource-heterogeneous environments (e.g., the Internet of Things (IoT)). Specifically, it personalizes the “right” components of local models and trains the shared model ui and personal model vi alternately in each client i. To further accelerate the convergence process, we propose DFedSMDC with properly directed noise perturbation into the gradient update. Theoretically, we provide convergence analysis of both algorithms in the general non-convex setting. It can shed light on how vital factors affect the convergence rate, such as these alternate updates of partial models ui, vi, data heterogeneity δ2 as well as various communication topologies (characterized by the spectral gap 1 − λ). Empirically, we confirm the state-of-the-art (SOTA) superiority of the proposed methods on several real-world datasets with various data distributions, relative to both SOTA personalized FL (PFL) and DFL baselines.
- 24 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (4) , 4744-4755
Toward Understanding Generalization and Stability Gaps Between Centralized and Decentralized Federated Learning
As two mainstream frameworks in federated learning (FL), both centralized and decentralized approaches have shown great application value in practical scenarios. However, existing studies do not provide sufficient evidence and clear guidance for analysis of which performs better in the FL community. Although decentralized methods have been proven to approach the comparable convergence of centralized with less communication, their test performance always falls short of expectations in empirical studies. To comprehensively and fairly compare their efficiency gaps in FL, in this paper, we explore their stability and generalization efficiency. Specifically, we prove that on the general smooth non-convex objectives, 1) centralized FL (CFL) always generalizes better than decentralized FL (DFL); 2) CFL achieves the best performance via adopting partial participation instead of full participation; and, 3) there is a necessary requirement for the topology in DFL to avoid performance collapse as the training scale increases. We also conduct extensive experiments on several common setups in FL to validate that our theoretical analysis is consistent with experimental phenomena and contextually valid in several general and practical scenarios.
- 24 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (4) , 4744-4755
Toward Understanding Generalization and Stability Gaps Between Centralized and Decentralized Federated Learning
As two mainstream frameworks in federated learning (FL), both centralized and decentralized approaches have shown great application value in practical scenarios. However, existing studies do not provide sufficient evidence and clear guidance for analysis of which performs better in the FL community. Although decentralized methods have been proven to approach the comparable convergence of centralized with less communication, their test performance always falls short of expectations in empirical studies. To comprehensively and fairly compare their efficiency gaps in FL, in this paper, we explore their stability and generalization efficiency. Specifically, we prove that on the general smooth non-convex objectives, 1) centralized FL (CFL) always generalizes better than decentralized FL (DFL); 2) CFL achieves the best performance via adopting partial participation instead of full participation; and, 3) there is a necessary requirement for the topology in DFL to avoid performance collapse as the training scale increases. We also conduct extensive experiments on several common setups in FL to validate that our theoretical analysis is consistent with experimental phenomena and contextually valid in several general and practical scenarios.
- Open Access
- 11 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 57 (3) , 1-36
On Efficient Training of Large-Scale Deep Learning Models
The field of deep learning has witnessed significant progress in recent times, particularly in areas such as computer vision (CV), natural language processing (NLP), and speech. The use of large-scale models trained on vast amounts of data holds immense promise for practical applications, enhancing industrial productivity and facilitating social development. However, it suffers extremely from the unstable training process and stringent requirements of computational resources. With the increasing demands on the adaption of computational capacity, though numerous studies have explored the efficient training field to a certain extent, a comprehensive summarization/guideline on those general acceleration techniques of training large-scale deep learning models is still much anticipated. In this survey, we present a detailed review of the general techniques for training acceleration. We consider the fundamental update formulation and split its basic components into five main perspectives: (1) “data-centric,” including dataset regularization, data sampling, and data-centric curriculum learning techniques, which can significantly reduce the computational complexity of the data samples; (2) “model-centric,” including acceleration of basic modules, compression training, model initialization, and model-centric curriculum learning techniques, which focus on accelerating the training via reducing the calculations on parameters and providing better initialization; (3) “optimization-centric,” including the selection of learning rate, the employment of large batch size, the designs of efficient objectives, and model average techniques, which pay attention to the training policy and improving the generality for the large-scale models; (4) “budgeted training,” including some distinctive acceleration methods on source-constrained situations, e.g., for limitation on the total iterations; and (5) “system-centric,” including some efficient distributed frameworks and open source libraries that provide adequate hardware support for the implementation of the above-mentioned acceleration algorithms. By presenting this comprehensive taxonomy, our survey presents a comprehensive review to understand the general mechanisms within each component and their joint interaction. Meanwhile, we further provide a detailed analysis and discussion of future works on the development of general acceleration techniques, which could inspire us to re-think and design novel efficient paradigms. Overall, we hope that this survey will serve as a valuable guideline for general efficient training.
- 11 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 57 (3) , 1-36
Open AccessOn Efficient Training of Large-Scale Deep Learning Models
The field of deep learning has witnessed significant progress in recent times, particularly in areas such as computer vision (CV), natural language processing (NLP), and speech. The use of large-scale models trained on vast amounts of data holds immense promise for practical applications, enhancing industrial productivity and facilitating social development. However, it suffers extremely from the unstable training process and stringent requirements of computational resources. With the increasing demands on the adaption of computational capacity, though numerous studies have explored the efficient training field to a certain extent, a comprehensive summarization/guideline on those general acceleration techniques of training large-scale deep learning models is still much anticipated. In this survey, we present a detailed review of the general techniques for training acceleration. We consider the fundamental update formulation and split its basic components into five main perspectives: (1) “data-centric,” including dataset regularization, data sampling, and data-centric curriculum learning techniques, which can significantly reduce the computational complexity of the data samples; (2) “model-centric,” including acceleration of basic modules, compression training, model initialization, and model-centric curriculum learning techniques, which focus on accelerating the training via reducing the calculations on parameters and providing better initialization; (3) “optimization-centric,” including the selection of learning rate, the employment of large batch size, the designs of efficient objectives, and model average techniques, which pay attention to the training policy and improving the generality for the large-scale models; (4) “budgeted training,” including some distinctive acceleration methods on source-constrained situations, e.g., for limitation on the total iterations; and (5) “system-centric,” including some efficient distributed frameworks and open source libraries that provide adequate hardware support for the implementation of the above-mentioned acceleration algorithms. By presenting this comprehensive taxonomy, our survey presents a comprehensive review to understand the general mechanisms within each component and their joint interaction. Meanwhile, we further provide a detailed analysis and discussion of future works on the development of general acceleration techniques, which could inspire us to re-think and design novel efficient paradigms. Overall, we hope that this survey will serve as a valuable guideline for general efficient training.
- 1 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (12) , 14453-14464
Efficient Federated Learning Via Local Adaptive Amended Optimizer With Linear Speedup
Adaptive optimization has achieved notable success for distributed learning while extending adaptive optimizer to federated Learning (FL) suffers from severe inefficiency, including (i) rugged convergence due to inaccurate gradient estimation in global adaptive optimizer; (ii) client drifts exacerbated by local over-fitting with the local adaptive optimizer. In this work, we propose a novel momentum-based algorithm via utilizing the global gradient descent and locally adaptive amended optimizer to tackle these difficulties. Specifically, we incorporate a locally amended technique to the adaptive optimizer, named Federated Local ADaptive Amended optimizer ( FedLADA ), which estimates the global average offset in the previous communication round and corrects the local offset through a momentum-like term to further improve the empirical training speed and mitigate the heterogeneous over-fitting. Theoretically, we establish the convergence rate of FedLADA with a linear speedup property on the non-convex case under the partial participation settings. Moreover, we conduct extensive experiments on the real-world dataset to demonstrate the efficacy of our proposed FedLADA , which could greatly reduce the communication rounds and achieves higher accuracy than several baselines.
- 1 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (12) , 14453-14464
Efficient Federated Learning Via Local Adaptive Amended Optimizer With Linear Speedup
Adaptive optimization has achieved notable success for distributed learning while extending adaptive optimizer to federated Learning (FL) suffers from severe inefficiency, including (i) rugged convergence due to inaccurate gradient estimation in global adaptive optimizer; (ii) client drifts exacerbated by local over-fitting with the local adaptive optimizer. In this work, we propose a novel momentum-based algorithm via utilizing the global gradient descent and locally adaptive amended optimizer to tackle these difficulties. Specifically, we incorporate a locally amended technique to the adaptive optimizer, named Federated Local ADaptive Amended optimizer ( FedLADA ), which estimates the global average offset in the previous communication round and corrects the local offset through a momentum-like term to further improve the empirical training speed and mitigate the heterogeneous over-fitting. Theoretically, we establish the convergence rate of FedLADA with a linear speedup property on the non-convex case under the partial participation settings. Moreover, we conduct extensive experiments on the real-world dataset to demonstrate the efficacy of our proposed FedLADA , which could greatly reduce the communication rounds and achieves higher accuracy than several baselines.