Jianze Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Jianze Li has a total of 21 co-authors. Here are the top co-authors by number of shared publications:
- Pierre Comon
GIPSA-Lab Univ. Grenoble Alpes CNRS Grenoble INP France
7 shared publications - Konstantin Usevich
Research Center for Automatic Control, French National Center for Scientific Research (CNRS), Universit de Lorraine, Nancy, France
5 shared publications - Shuzhong Zhang
Department of Industrial and Systems Engineering, University of Minnesota, Minneapolis, MN 55455 USA
3 shared publications - Guang Gui Ding
Nankai University
2 shared publications Konstantin Usevich
Université de Lorraine, CNRS, CRAN, Nancy 54000, France
2 shared publications- Qin Ni
Nanjing University of Aeronautics and Astronautics
2 shared publications
Latest Publications
- 19 November 2024
- Published by Elsevier in Applied Mathematics and Computation
- Vol. 490
A sparse optimization approach for simultaneous orthogonal tensor diagonalization
- 19 November 2024
- Published by Elsevier in Applied Mathematics and Computation
- Vol. 490
A sparse optimization approach for simultaneous orthogonal tensor diagonalization
- 16 January 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Matrix Analysis and Applications
- Vol. 45 (1) , 167-202
Projectively and Weakly Simultaneously Diagonalizable Matrices and their Applications
Characterizing simultaneously diagonalizable (SD) matrices has been receiving considerable attention in recent decades due to its wide applications and its role in matrix analysis. However, the notion of SD matrices is arguably still restrictive for wider applications. In this paper, we consider two error measures related to the simultaneous diagonalization of matrices and propose several new variants of SD thereof; in particular, TWSD, TWSD-B, [Formula: see text]-SD (SDO), DWSD, and [Formula: see text]-SD (SDO). Those are all weaker forms of SD. We derive various sufficient and/or necessary conditions of them under different assumptions and show the relationships between these new notions. Finally, we discuss the applications of these new notions in, e.g., quadratically constrained quadratic programming and independent component analysis.
- 16 January 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Matrix Analysis and Applications
- Vol. 45 (1) , 167-202
Projectively and Weakly Simultaneously Diagonalizable Matrices and their Applications
Characterizing simultaneously diagonalizable (SD) matrices has been receiving considerable attention in recent decades due to its wide applications and its role in matrix analysis. However, the notion of SD matrices is arguably still restrictive for wider applications. In this paper, we consider two error measures related to the simultaneous diagonalization of matrices and propose several new variants of SD thereof; in particular, TWSD, TWSD-B, [Formula: see text]-SD (SDO), DWSD, and [Formula: see text]-SD (SDO). Those are all weaker forms of SD. We derive various sufficient and/or necessary conditions of them under different assumptions and show the relationships between these new notions. Finally, we discuss the applications of these new notions in, e.g., quadratically constrained quadratic programming and independent component analysis.
- 11 May 2023
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Matrix Analysis and Applications
- Vol. 44 (2) , 592-621
Convergence of Gradient-Based Block Coordinate Descent Algorithms for Nonorthogonal Joint Approximate Diagonalization of Matrices
We propose a gradient-based block coordinate descent (BCD-G) framework to solve the joint approximate diagonalization of matrices defined on the product of the complex Stiefel manifold and the special linear group. Instead of the cyclic fashion, we choose a block optimization based on the Riemannian gradient. To update the first block variable in the complex Stiefel manifold, we use the well-known line search descent method. To update the second block variable in the special linear group, based on four different kinds of elementary transformations, we construct three classes, GLU, GQU and GU, and then get three BCD-G algorithms, BCD-GLU, BCD-GQU and BCD-GU. We establish the global convergence and weak convergence of these three algorithms using the Łojasiewicz gradient inequality under the assumption that the iterates are bounded. We also propose a gradient-based Jacobi-type framework to solve the joint approximate diagonalization of matrices defined on the special linear group. As in the BCD-G case, using the GLU and GQU classes of elementary transformations, we focus on the Jacobi-GLU and Jacobi-GQU algorithms and establish their global convergence and weak convergence as well. All the algorithms and convergence results described in this paper also apply to the real case.
- 11 May 2023
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Matrix Analysis and Applications
- Vol. 44 (2) , 592-621
Convergence of Gradient-Based Block Coordinate Descent Algorithms for Nonorthogonal Joint Approximate Diagonalization of Matrices
We propose a gradient-based block coordinate descent (BCD-G) framework to solve the joint approximate diagonalization of matrices defined on the product of the complex Stiefel manifold and the special linear group. Instead of the cyclic fashion, we choose a block optimization based on the Riemannian gradient. To update the first block variable in the complex Stiefel manifold, we use the well-known line search descent method. To update the second block variable in the special linear group, based on four different kinds of elementary transformations, we construct three classes, GLU, GQU and GU, and then get three BCD-G algorithms, BCD-GLU, BCD-GQU and BCD-GU. We establish the global convergence and weak convergence of these three algorithms using the Łojasiewicz gradient inequality under the assumption that the iterates are bounded. We also propose a gradient-based Jacobi-type framework to solve the joint approximate diagonalization of matrices defined on the special linear group. As in the BCD-G case, using the GLU and GQU classes of elementary transformations, we focus on the Jacobi-GLU and Jacobi-GQU algorithms and establish their global convergence and weak convergence as well. All the algorithms and convergence results described in this paper also apply to the real case.
- Open Access
Jacobi-type algorithms for homogeneous polynomial optimization on Stiefel manifolds with applications to tensor approximations
This paper mainly studies the gradient-based Jacobi-type algorithms to maximize two classes of homogeneous polynomials with orthogonality constraints, and establish their convergence properties. For the first class of homogeneous polynomials subject to a constraint on a Stiefel manifold, we reformulate it as an optimization problem on a unitary group, which makes it possible to apply the gradient-based Jacobi-type (Jacobi-G) algorithm. Then, if the subproblem can always be represented as a quadratic form, we establish the global convergence of Jacobi-G under any one of three conditions. The convergence result for the first condition is an easy extension of the result in [Usevich et al. SIOPT 2020], while other two conditions are new ones. This algorithm and the convergence properties apply to the well-known joint approximate symmetric tensor diagonalization. For the second class of homogeneous polynomials subject to constraints on the product of Stiefel manifolds, we reformulate it as an optimization problem on the product of unitary groups, and then develop a new gradient-based multi-block Jacobi-type (Jacobi-MG) algorithm to solve it. We establish the global convergence of Jacobi-MG under any one of the above three conditions, if the subproblem can always be represented as a quadratic form. This algorithm and the convergence properties are suitable to the well-known joint approximate tensor diagonalization. As the proximal variants of Jacobi-G and Jacobi-MG, we also propose the Jacobi-GP and Jacobi-MGP algorithms, and establish their global convergence without any further condition. Some numerical results are provided indicating the efficiency of the proposed algorithms.
Open AccessJacobi-type algorithms for homogeneous polynomial optimization on Stiefel manifolds with applications to tensor approximations
This paper mainly studies the gradient-based Jacobi-type algorithms to maximize two classes of homogeneous polynomials with orthogonality constraints, and establish their convergence properties. For the first class of homogeneous polynomials subject to a constraint on a Stiefel manifold, we reformulate it as an optimization problem on a unitary group, which makes it possible to apply the gradient-based Jacobi-type (Jacobi-G) algorithm. Then, if the subproblem can always be represented as a quadratic form, we establish the global convergence of Jacobi-G under any one of three conditions. The convergence result for the first condition is an easy extension of the result in [Usevich et al. SIOPT 2020], while other two conditions are new ones. This algorithm and the convergence properties apply to the well-known joint approximate symmetric tensor diagonalization. For the second class of homogeneous polynomials subject to constraints on the product of Stiefel manifolds, we reformulate it as an optimization problem on the product of unitary groups, and then develop a new gradient-based multi-block Jacobi-type (Jacobi-MG) algorithm to solve it. We establish the global convergence of Jacobi-MG under any one of the above three conditions, if the subproblem can always be represented as a quadratic form. This algorithm and the convergence properties are suitable to the well-known joint approximate tensor diagonalization. As the proximal variants of Jacobi-G and Jacobi-MG, we also propose the Jacobi-GP and Jacobi-MGP algorithms, and establish their global convergence without any further condition. Some numerical results are provided indicating the efficiency of the proposed algorithms.
- Open Access
Convergence of gradient-based block coordinate descent algorithms for non-orthogonal joint approximate diagonalization of matrices
In this paper, we propose a gradient-based block coordinate descent (BCD-G) framework to solve the joint approximate diagonalization of matrices defined on the product of the complex Stiefel manifold and the special linear group. Instead of the cyclic fashion, we choose a block optimization based on the Riemannian gradient. To update the first block variable in the complex Stiefel manifold, we use the well-known line search descent method. To update the second block variable in the special linear group, based on four kinds of different elementary transformations, we construct three classes: GLU, GQU and GU, and then get three BCD-G algorithms: BCD-GLU, BCD-GQU and BCD-GU. We establish the global and weak convergence of these three algorithms using the \L{}ojasiewicz gradient inequality under the assumption that the iterates are bounded. We also propose a gradient-based Jacobi-type framework to solve the joint approximate diagonalization of matrices defined on the special linear group. As in the BCD-G case, using the GLU and GQU classes of elementary transformations, we focus on the Jacobi-GLU and Jacobi-GQU algorithms and establish their global and weak convergence. All the algorithms and convergence results described in this paper also apply to the real case.
Open AccessConvergence of gradient-based block coordinate descent algorithms for non-orthogonal joint approximate diagonalization of matrices
In this paper, we propose a gradient-based block coordinate descent (BCD-G) framework to solve the joint approximate diagonalization of matrices defined on the product of the complex Stiefel manifold and the special linear group. Instead of the cyclic fashion, we choose a block optimization based on the Riemannian gradient. To update the first block variable in the complex Stiefel manifold, we use the well-known line search descent method. To update the second block variable in the special linear group, based on four kinds of different elementary transformations, we construct three classes: GLU, GQU and GU, and then get three BCD-G algorithms: BCD-GLU, BCD-GQU and BCD-GU. We establish the global and weak convergence of these three algorithms using the \L{}ojasiewicz gradient inequality under the assumption that the iterates are bounded. We also propose a gradient-based Jacobi-type framework to solve the joint approximate diagonalization of matrices defined on the special linear group. As in the BCD-G case, using the GLU and GQU classes of elementary transformations, we focus on the Jacobi-GLU and Jacobi-GQU algorithms and establish their global and weak convergence. All the algorithms and convergence results described in this paper also apply to the real case.
- 5 April 2023
- Published by American Mathematical Society (AMS) in Mathematics of Computation
- Vol. 92 (343) , 2217-2245
Jacobi-type algorithms for homogeneous polynomial optimization on Stiefel manifolds with applications to tensor approximations
This paper mainly studies the gradient-based Jacobi-type algorithms to maximize two classes of homogeneous polynomials with orthogonality constraints, and establish their convergence properties. For the first class of homogeneous polynomials subject to a constraint on a Stiefel manifold, we reformulate it as an optimization problem on a unitary group, which makes it possible to apply the gradient-based Jacobi-type (Jacobi-G) algorithm. Then, if the subproblem can always be represented as a quadratic form, we establish the global convergence of Jacobi-G under any one of three conditions. The convergence result for the first condition is an easy extension of the result by Usevich, Li, and Comon [SIAM J. Optim. 30 (2020), pp. 2998–3028], while other two conditions are new ones. This algorithm and the convergence properties apply to the well-known joint approximate symmetric tensor diagonalization. For the second class of homogeneous polynomials subject to constraints on the product of Stiefel manifolds, we reformulate it as an optimization problem on the product of unitary groups, and then develop a new gradient-based multiblock Jacobi-type (Jacobi-MG) algorithm to solve it. We establish the global convergence of Jacobi-MG under any one of the above three conditions, if the subproblem can always be represented as a quadratic form. This algorithm and the convergence properties are suitable to the well-known joint approximate tensor diagonalization. As the proximal variants of Jacobi-G and Jacobi-MG, we also propose the Jacobi-GP and Jacobi-MGP algorithms, and establish their global convergence without any further condition. Some numerical results are provided indicating the efficiency of the proposed algorithms.
- 5 April 2023
- Published by American Mathematical Society (AMS) in Mathematics of Computation
- Vol. 92 (343) , 2217-2245
Jacobi-type algorithms for homogeneous polynomial optimization on Stiefel manifolds with applications to tensor approximations
This paper mainly studies the gradient-based Jacobi-type algorithms to maximize two classes of homogeneous polynomials with orthogonality constraints, and establish their convergence properties. For the first class of homogeneous polynomials subject to a constraint on a Stiefel manifold, we reformulate it as an optimization problem on a unitary group, which makes it possible to apply the gradient-based Jacobi-type (Jacobi-G) algorithm. Then, if the subproblem can always be represented as a quadratic form, we establish the global convergence of Jacobi-G under any one of three conditions. The convergence result for the first condition is an easy extension of the result by Usevich, Li, and Comon [SIAM J. Optim. 30 (2020), pp. 2998–3028], while other two conditions are new ones. This algorithm and the convergence properties apply to the well-known joint approximate symmetric tensor diagonalization. For the second class of homogeneous polynomials subject to constraints on the product of Stiefel manifolds, we reformulate it as an optimization problem on the product of unitary groups, and then develop a new gradient-based multiblock Jacobi-type (Jacobi-MG) algorithm to solve it. We establish the global convergence of Jacobi-MG under any one of the above three conditions, if the subproblem can always be represented as a quadratic form. This algorithm and the convergence properties are suitable to the well-known joint approximate tensor diagonalization. As the proximal variants of Jacobi-G and Jacobi-MG, we also propose the Jacobi-GP and Jacobi-MGP algorithms, and establish their global convergence without any further condition. Some numerical results are provided indicating the efficiency of the proposed algorithms.
- 25 March 2023
- Published by Springer Nature in Journal of the Operations Research Society of China
- Vol. 12 (4) , 874-920
Polar Decomposition-based Algorithms on the Product of Stiefel Manifolds with Applications in Tensor Approximation
- 25 March 2023
- Published by Springer Nature in Journal of the Operations Research Society of China
- Vol. 12 (4) , 874-920
Polar Decomposition-based Algorithms on the Product of Stiefel Manifolds with Applications in Tensor Approximation
- Open Access
TAOTF: A Two-stage Approximately Orthogonal Training Framework in Deep Neural Networks
The orthogonality constraints, including the hard and soft ones, have been used to normalize the weight matrices of Deep Neural Network (DNN) models, especially the Convolutional Neural Network (CNN) and Vision Transformer (ViT), to reduce model parameter redundancy and improve training stability. However, the robustness to noisy data of these models with constraints is not always satisfactory. In this work, we propose a novel two-stage approximately orthogonal training framework (TAOTF) to find a trade-off between the orthogonal solution space and the main task solution space to solve this problem in noisy data scenarios. In the first stage, we propose a novel algorithm called polar decomposition-based orthogonal initialization (PDOI) to find a good initialization for the orthogonal optimization. In the second stage, unlike other existing methods, we apply soft orthogonal constraints for all layers of DNN model. We evaluate the proposed model-agnostic framework both on the natural image and medical image datasets, which show that our method achieves stable and superior performances to existing methods.
Open AccessTAOTF: A Two-stage Approximately Orthogonal Training Framework in Deep Neural Networks
The orthogonality constraints, including the hard and soft ones, have been used to normalize the weight matrices of Deep Neural Network (DNN) models, especially the Convolutional Neural Network (CNN) and Vision Transformer (ViT), to reduce model parameter redundancy and improve training stability. However, the robustness to noisy data of these models with constraints is not always satisfactory. In this work, we propose a novel two-stage approximately orthogonal training framework (TAOTF) to find a trade-off between the orthogonal solution space and the main task solution space to solve this problem in noisy data scenarios. In the first stage, we propose a novel algorithm called polar decomposition-based orthogonal initialization (PDOI) to find a good initialization for the orthogonal optimization. In the second stage, unlike other existing methods, we apply soft orthogonal constraints for all layers of DNN model. We evaluate the proposed model-agnostic framework both on the natural image and medical image datasets, which show that our method achieves stable and superior performances to existing methods.
- 3 November 2022
- Published by Springer Nature
- p. 175-191
Data-Free Backdoor Removal Based on Channel Lipschitzness
- 3 November 2022
- Published by Springer Nature
- p. 175-191
Data-Free Backdoor Removal Based on Channel Lipschitzness
- Open Access
Jacobi-type algorithm for low rank orthogonal approximation of symmetric tensors and its convergence analysis
In this paper, we propose a Jacobi-type algorithm to solve the low rank orthogonal approximation problem of symmetric tensors. This algorithm includes as a special case the well-known Jacobi CoM2 algorithm for the approximate orthogonal diagonalization problem of symmetric tensors. We first prove the weak convergence of this algorithm, \textit{i.e.} any accumulation point is a stationary point. Then we study the global convergence of this algorithm under a gradient based ordering for a special case: the best rank-2 orthogonal approximation of 3rd order symmetric tensors, and prove that an accumulation point is the unique limit point under some conditions. Numerical experiments are presented to show the efficiency of this algorithm.
Open AccessJacobi-type algorithm for low rank orthogonal approximation of symmetric tensors and its convergence analysis
In this paper, we propose a Jacobi-type algorithm to solve the low rank orthogonal approximation problem of symmetric tensors. This algorithm includes as a special case the well-known Jacobi CoM2 algorithm for the approximate orthogonal diagonalization problem of symmetric tensors. We first prove the weak convergence of this algorithm, \textit{i.e.} any accumulation point is a stationary point. Then we study the global convergence of this algorithm under a gradient based ordering for a special case: the best rank-2 orthogonal approximation of 3rd order symmetric tensors, and prove that an accumulation point is the unique limit point under some conditions. Numerical experiments are presented to show the efficiency of this algorithm.