Exploring the Effect of Multi-Step Ascent in Sharpness-Aware Minimization
Open Access
- 22 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 82166-82179
Abstract
Recently, Sharpness-Aware Minimization (SAM) has shown state-of-the-art performance by seeking flat minima. To minimize the maximum loss within a neighborhood in the parameter space, SAM uses an ascent step, which perturbs the weights along the direction of gradient ascent with a given radius. While single-step or multi-step can be taken during ascent steps, previous studies have shown that multi-step ascent SAM rarely improves generalization performance. However, this phenomenon is particularly interesting because the multi-step ascent is expected to provide a better approximation of the maximum neighborhood loss. Therefore, in this paper, we analyze the effect of the number of ascent steps and investigate the difference between both single-step ascent SAM and multi-step ascent SAM. We identify the effect of the number of ascents on SAM optimization and reveal that single-step ascent SAM and multi-step ascent SAM exhibit distinct loss landscapes. Based on these observations, we finally suggest a simple modification that can mitigate the suboptimal generalization of multi-step ascent SAM.Keywords
All Related Versions
Funding Information
- Ministry of Science and ICT, South Korea (RS-2024-00338859)
This publication has 10 references indexed in Scilit:
- GCSAM: Gradient Centralized Sharpness Aware MinimizationIEEE Access, 2025
- CR-SAM: Curvature Regularized Sharpness-Aware MinimizationProceedings of the AAAI Conference on Artificial Intelligence, 2024
- SAMPa: Sharpness-aware Minimization Parallelized2024
- Understanding Catastrophic Overfitting in Single-step Adversarial TrainingProceedings of the AAAI Conference on Artificial Intelligence, 2021
- PyHessian: Neural Networks Through the Lens of the Hessian2020
- Deep Residual Learning for Image Recognition2016
- Wide Residual Networks2016
- ImageNet: A large-scale hierarchical image database2009
- Adaptive estimation of a quadratic functional by model selectionThe Annals of Statistics, 2000
- PAC-Bayesian model averaging1999