Sparse Regularization by Smooth Non-separable Non-convex Penalty Function Based on Ultra-discretization Formula
Preprint
Abstract
In sparse optimization, the $\ell_{1}$ norm is widely adopted for its convexity, yet it often yields solutions with smaller magnitudes than expected. To mitigate this drawback, various non-convex sparse penalties have been proposed. Some employ non-separability, with ordered weighting as an effective example, to retain large components while suppressing small ones. Motivated by these approaches, we propose ULPENS, a non-convex, non-separable sparsity-inducing penalty function that enables control over the suppression of elements. Derived from the ultra-discretization formula, ULPENS can continuously interpolate between the $\ell_{1}$ norm and a non-convex selective suppressing function by adjusting parameters inherent to the formula. With the formula, ULPENS is smooth, allowing the use of efficient gradient-based optimization algorithms. We establish key theoretical properties of ULPENS and demonstrate its practical effectiveness through numerical experiments.
Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Nonsmooth Nonconvex Stochastic Heavy BallJournal of Optimization Theory and Applications, 2024
- A comprehensive survey on regularization strategies in machine learningInformation Fusion, 2022
- Quasi-Newton methods for machine learning: forget the past, just sampleOptimization Methods and Software, 2021
- A general framework for ADMM accelerationNumerical Algorithms, 2019
- Continuous, Discrete and Ultradiscrete Painlevé Equations2017
- A Survey of Sparse Representation: Algorithms and ApplicationsIEEE Access, 2015
- A comparison of typical ℓ minimization algorithmsNeurocomputing, 2013
- Enhancing Sparsity by Reweighted ℓ 1 MinimizationJournal of Fourier Analysis and Applications, 2008
- Iterative Thresholding for Sparse ApproximationsJournal of Fourier Analysis and Applications, 2008
- Some methods of speeding up the convergence of iteration methodsUSSR Computational Mathematics and Mathematical Physics, 1964