Zhening Li

The spectral p-norm and nuclear p-norm of matrices and tensors appear in various applications, albeit both are NP-hard to compute. The former sets a foundation of [ ]-sphere-constrained polynomial optimization problems, and the latter has been found in many rank minimization problems in machine learning. We study approximation algorithms of the tensor nuclear p-norm with an aim to establish the approximation bound matching the best one of its dual norm, the tensor spectral p-norm. Driven by the application of sphere covering to approximate both tensor spectral and nuclear norms ([ ]), we propose several types of hitting sets that approximately represent the [ ]-sphere with adjustable parameters for different levels of approximations and cardinalities, providing an independent toolbox for decision making on [ ]-spheres. Using the idea in robust optimization and second-order cone programming, we obtain the first polynomial-time algorithm with an [ ]-approximation bound for the computation of the matrix nuclear p-norm when [ ] is a rational, paving a way for applications in modeling with the matrix nuclear p-norm. These two new results enable us to propose various polynomial-time approximation algorithms for the computation of the tensor nuclear p-norm using tensor partitions, convex optimization, and duality theory, attaining the same approximation bound to the best one of the tensor spectral p-norms. Effective performance of the proposed algorithms for the tensor nuclear p-norm is shown by numerical implementations. We believe the ideas of [ ]-sphere covering with its applications in approximating nuclear p-norm would be useful for tackling optimization problems on other sets, such as the binary hypercube with its applications in graph theory and neural networks and the nonnegative sphere with its applications in copositive programming and nonnegative matrix factorization. Funding: This work was supported by the National Natural Science Foundation of China [Grants 71825003, 72394360, 72394364, 72171141, 72192830, and 72192832], and the program for Innovative Research Team of Shanghai University of Finance and Economics.

Feature extraction plays an indispensable role in image and video technology. However, it is difficult for traditional matrix based feature extraction methods to handle massive multi-dimensional data. This, alongside with the ubiquitous uncertainty (noise) in real-world data, resulted in many robust tensor based feature extraction models. However, these existing models did not consider the worst-case model performance (i.e., the largest fitting error among all samples), which is critically important from a robust optimization perspective. In this paper, we propose a novel robust feature extraction model via ℓ _∞ -norm based nonnegative Tucker decomposition. The model is to minimize the maximum sample fitting error so as to overcome the influence of data uncertainty. Although the new model is nonconvex and nonsmooth, we design an effective iterative optimization algorithm with theoretical guarantee on its convergence for it. The performance of the new model on five real-world benchmark object classification and face recognition datasets under various corruption scenarios are evaluated, and the experimental results show the excellence of the new model by comparing to many existing models.

The matrix spectral and nuclear norms appear in enormous applications. The generalizations of these norms to higher-order tensors is becoming increasingly important but unfortunately they are NP-hard to compute or even approximate. Although the two norms are dual to each other, the best known approximation bound achieved by polynomial-time algorithms for the tensor nuclear norm is worse than that for the tensor spectral norm. In this paper, we bridge this gap by proposing deterministic algorithms with the best bound for both tensor norms. Our methods not only improve the approximation bound for the nuclear norm, but are also data independent and easily implementable comparing to existing approximation methods for the tensor spectral norm. The main idea is to construct a selection of unit vectors that can approximately represent the unit sphere, in other words, a collection of spherical caps to cover the sphere. For this purpose, we explicitly construct several collections of spherical caps for sphere covering with adjustable parameters for different levels of approximations and cardinalities. These readily available constructions are of independent interest as they provide a powerful tool for various decision making problems on spheres and related problems. We believe the ideas of constructions and the applications to approximate tensor norms can be useful to tackle optimization problems over other sets such as the binary hypercube.

Unsupervised feature selection has attracted remarkable attention recently. With the development of data acquisition technology, multi-dimensional tensor data has been appeared in enormous real-world applications. However, most existing unsupervised feature selection methods are non-tensor-based which results the vectorization of tensor data as a preprocessing step. This seemingly ordinary operation has led to an unnecessary loss of the multi-dimensional structural information and eventually restricted the quality of the selected features. To overcome the limitation, in this paper, we propose a novel unsupervised feature selection model: Nonnegative tensor CP (CANDECOMP/PARAFAC) decomposition based unsupervised feature selection, CPUFS for short. In specific, we devise new tensor-oriented linear classifier and feature selection matrix for CPUFS. In addition, CPUFS simultaneously conducts graph regularized nonnegative CP decomposition and newly-designed tensor-oriented pseudo label regression and feature selection to fully preserve the multi-dimensional data structure. To solve the CPUFS model, we propose an efficient iterative optimization algorithm with theoretically guaranteed convergence, whose computational complexity scales linearly in the number of features. A variation of the CPUFS model by incorporating nonnegativity into the linear classifier, namely CPUFSnn, is also proposed and studied. Experimental results on ten real-world benchmark datasets demonstrate the effectiveness of both CPUFS and CPUFSnn over the state-of-the-arts.

Hermitian matrices have played an important role in matrix theory and complex quadratic optimization. The high-order generalization of Hermitian matrices, conjugate partial-symmetric (CPS) tensors, have shown growing interest recently in tensor theory and computation, particularly in application-driven complex polynomial optimization problems. In this paper, we study CPS tensors with a focus on ranks, computing rank-one decompositions and approximations, as well as their applications. We prove constructively that any CPS tensor can be decomposed into a sum of rank-one CPS tensors, which provides an explicit method to compute such rank-one decompositions. Three types of ranks for CPS tensors are defined and shown to be different in general. This leads to the invalidity of the conjugate version of Comon’s conjecture. We then study rank-one approximations and matricizations of CPS tensors. By carefully unfolding CPS tensors to Hermitian matrices, rank-one equivalence can be preserved. This enables us to develop new convex optimization models and algorithms to compute best rank-one approximations of CPS tensors. Numerical experiments from data sets in radar wave form design, elasticity tensor, and quantum entanglement are performed to justify the capability of our methods.

Conjugate partial-symmetric (CPS) tensors are the high-order generalization of Hermitian matrices. As the role played by Hermitian matrices in matrix theory and quadratic optimization, CPS tensors have shown growing interest recently in tensor theory and optimization, particularly in many application-driven complex polynomial optimization problems. In this paper, we study CPS tensors with a focus on ranks, rank-one decompositions and approximations, as well as their applications. The analysis is conducted along side with a more general class of complex tensors called partial-symmetric tensors. We prove constructively that any CPS tensor can be decomposed into a sum of rank-one CPS tensors, which provides an alternative definition of CPS tensors via linear combinations of rank-one CPS tensors. Three types of ranks for CPS tensors are defined and shown to be different in general. This leads to the invalidity of the conjugate version of Comon's conjecture. We then study rank-one approximations and matricizations of CPS tensors. By carefully unfolding CPS tensors to Hermitian matrices, rank-one equivalence can be preserved. This enables us to develop new convex optimization models and algorithms to compute best rank-one approximation of CPS tensors. Numerical experiments from various data are performed to justify the capability of our methods.