Er-Wei Bai

This article considers identification of sparse Volterra systems. A method based on the almost orthogonal matching pursuit (AOMP) is proposed. The AOMP algorithm allows one to estimate one nonzero coefficient at a time until all nonzero coefficients are found without losing the optimality and the sparsity, thus avoiding the curse of dimensionality often encountered in Volterra system identification.

To achieve a parsimonious model, it is necessary to rank the importance of input variables according to some measures. The problem is nontrivial in the setting of nonlinear and non- parametric system identification. Difficulties lie in the lack of structural information of the unknown system, unknown underlying probabilistic distributions and unknown nonlinear correlations of variables. In this paper, we present a way to rank variables according to Goodness of Fit (GoF). Asymptotic results are established and numerical algorithms are proposed. The problem is cast in a reproducing kernel Hilbert space (RKHS) that allows us to deal with non-parametric nature of the unknown system, to avoid making strong conditions on the unknown distributions, to link GoFs to computable conditional covariance operators on RKHS, and to develop computationally-friendly numerical algorithms. Numerical simulations support the theoretical developments.

Focusing on identification, this paper develops techniques to reconstruct zero and nonzero elements of a sparse parameter vector of a stochastic dynamic system under feedback control, for which the current input may depend on the past inputs and outputs, system noises as well as exogenous dithers. First, a sparse parameter identification algorithm is introduced based on L2 norm with L1 regularization, where the adaptive weights are adopted in the optimization variables of L1 term. Second, estimates generated by the algorithm are shown to have both set and parameter convergence. That is, sets of the zero and nonzero elements in the parameter can be correctly identified with probability one using a finite number of observations, and estimates of the nonzero elements converge to the true values almost surely. Third, it is shown that the results are applicable to a large number of applications, including variable selection, open-loop identification, and closed-loop control of stochastic systems. Finally, numerical examples are given to support the theoretical analysis.

A parsimonious model is always preferred in engineering applications not only because it has a better prediction ability but also it suffers less from the curse of dimensionality for a high dimensional system. A parsimonious model can be obtained by identifying contributing variables from the candidate variables and then by eliminating non-contributing or redundant variables. However identifying which variables contribute and which variables do not contribute is not an easy task for a high dimensional nonparametric nonlinear system. This paper considers variable selection problems for a nonlinear nonparametric system of high dimension. Two approaches, inverse regression and variable selection regression, are proposed along with their theoretical analysis and numerical algorithms. Neither approach suffers from the curse of dimensionality, which is usually a problem for traditional variable selection methods for a high dimensional system. Further, no elliptic symmetry, nor independent input variables are assumed so both algorithms enjoy wide applications. Numerical algorithms for both approaches are fairly straightforward and simple.