Xiaojun Chen

This paper introduces a class of two-stage stochastic minimax problems where the first-stage objective function is nonconvex-concave while the second-stage objective function is strongly convex-concave. We establish properties of the second-stage minimax value function and solution functions, and characterize the existence and relationships among saddle points, minimax points, and KKT points. We apply the sample average approximation (SAA) to the class of two-stage stochastic minimax problems and prove the convergence of the KKT points as the sample size tends to infinity. An inexact parallel proximal gradient descent ascent algorithm is proposed to solve this class of problems with the SAA. Numerical experiments demonstrate the effectiveness of the proposed algorithm and validate the convergence properties of the SAA approach.

This paper gives a necessary and sufficient condition for a nonnegative matrix that has an orthogonal nonnegative matrix factorization (ONMF) via characterization of the null space. We propose an optimization model to minimize the Frobenius norm of the product of a given nonnegative matrix and a variable matrix subject to the constraints defined by the necessary and sufficient condition. Moreover, we present an augmented Lagrangian algorithm for solving this minimization model and prove the global convergence to a stationary point. Two factor matrices for the ONMF of the given matrix can be easily obtained by the outputs of the algorithm. Preliminary numerical results using synthetic and real-world data with applications in clustering show that our approach outperforms some existing ONMF methods regarding accuracy and robustness. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as recognition that the authors have followed reproducibility principles valued by SIMAX and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/Qilun-Luo/ONMF . [Formula: see text]

We propose a stochastic minimization model to find a robust solution of a system of stochastic vertical linear complementarity problems. This model aims to minimize a risk function under stochastic vertical linear complementarity constraints. We reformulate the model with a finite support set as a linearly constrained piecewise smooth minimization problem by a penalty method. We prove the existence of exact penalty parameters regarding global and local minimizers. We define a smoothing function of the piecewise smooth objective function and show that the smoothing function satisfies the Kurdyka–Łojasiewicz (KL) property. Moreover, we propose a smoothing block coordinate descent algorithm and prove that the sequence generated by the algorithm globally converges to an [ ]-Clarke stationary point of the penalty problem by the KL property for any [ ]. Finally, we apply our model and algorithm to portfolio selection problems with real data. Numerical results demonstrate the robustness of our model. Funding: X. Chen was supported in part by the Hong Kong Research Grant Council [Grants PolyU15300123, and PolyU15300322] and the CAS-Croucher Funding Scheme for the AMSS-PolyU Joint Laboratory. Z. Allen-Zhao was supported in part by the National Natural Science Foundation of China [Grant 12301405], the Shaanxi Fundamental Science Research Project for Mathematics and Physics [Grant 23JSZ010], and the Fundamental Research Fund for Central Universities of China [Grant ZYTS25201].

This paper introduces a class of two-stage stochastic minimax problems where the first-stage objective function is nonconvex-concave while the second-stage objective function is strongly convex-concave. We establish properties of the second-stage minimax value function and solution functions, and characterize the existence and relationships among saddle points, minimax points, and KKT points. We apply the sample average approximation (SAA) to the class of two-stage stochastic minimax problems and prove the convergence of the KKT points as the sample size tends to infinity. An inexact parallel proximal gradient descent ascent algorithm is proposed to solve this class of problems with the SAA. Numerical experiments demonstrate the effectiveness of the proposed algorithm and validate the convergence properties of the SAA approach.

This paper proposes a convex majorant approach for training sparse neural networks by bilevel optimization, where the upper level problem minimizes a smooth nonconvex function, and the lower level problem minimizes a smooth nonconvex function with a nonsmooth convex group sparse regularizer over a box set for fixed sparse regularization hyperparameters. The convex majorant function approximates the objective function of the lower level problem. We establish the relationship between the original bilevel optimization and the bilevel optimization with the convex majorant approach regarding global and local minimizers. Moreover, we use a smoothing function to approximate the convex majorant function and derive the convergence of global minimizers to those of the corresponding nonsmooth bilevel problems with smoothing parameter converging to zero. A smoothing implicit function method is proposed to solve the smooth approximate bilevel optimization problem. Some numerical experiments including the tests on the data from machine learning repository show that the convex majorant approach performs better than the widely used Grid Search method, Random Search method, and Bayesian optimization method.

We consider a class of nonconvex nonsmooth multicomposite optimization problems where the objective function consists of a Tikhonov regularizer and a composition of multiple nonconvex nonsmooth component functions. Such optimization problems arise from tangible applications in machine learning and beyond. To define and compute its first-order and second-order d(irectional)-stationary points effectively, we first derive the closed-form expression of the tangent cone for the feasible region of its constrained reformulation. Building on this, we establish its equivalence with the corresponding constrained and [Formula: see text]-penalty reformulations in terms of global optimality and d-stationarity. The equivalence offers indirect methods to attain the first-order and second-order d-stationary points of the original problem in certain cases. We apply our results to the training process of recurrent neural networks (RNNs).