Paul G. Constantine

We introduce the Lipschitz matrix: a generalization of the scalar Lipschitz constant for functions with many inputs. Among the Lipschitz matrices compatible with a particular function, we choose th...

We introduce the Lipschitz matrix: a generalization of the scalar Lipschitz constant for functions with many inputs. Among the Lipschitz matrices compatible a particular function, we choose the smallest such matrix in the Frobenius norm to encode the structure of this function. The Lipschitz matrix then provides a function-dependent metric on the input space. Altering this metric to reflect a particular function improves the performance of many tasks in computational science. Compared to the Lipschitz constant, the Lipschitz matrix reduces the worst-case cost of approximation, integration, and optimization; if the Lipschitz matrix is low-rank, this cost no longer depends on the dimension of the input, but instead on the rank of the Lipschitz matrix defeating the curse of dimensionality. Both the Lipschitz constant and matrix define uncertainty away from point queries of the function and by using the Lipschitz matrix we can reduce uncertainty. If we build a minimax space-filling design of experiments in the Lipschitz matrix metric, we can further reduce this uncertainty. When the Lipschitz matrix is approximately low-rank, we can perform parameter reduction by constructing a ridge approximation whose active subspace is the span of the dominant eigenvectors of the Lipschitz matrix. In summary, the Lipschitz matrix provides a new tool for analyzing and performing parameter reduction in complex models arising in computational science.

Multivariate functions encountered in high-dimensional uncertainty quantification problems often vary most strongly along a few dominant directions in the input parameter space. We propose a gradie...

Multivariate functions encountered in high-dimensional uncertainty quantification problems often vary most strongly along a few dominant directions in the input parameter space. We propose a gradient-based method for detecting these directions and using them to construct ridge approximations of such functions, in the case where the functions are vector-valued (e.g., taking values in $\mathbb{R}^n$). The methodology consists of minimizing an upper bound on the approximation error, obtained by subspace Poincar\'e inequalities. We provide a thorough mathematical analysis in the case where the parameter space is equipped with a Gaussian probability measure. The resulting method generalizes the notion of active subspaces associated with scalar-valued functions. A numerical illustration shows that using gradients of the function yields effective dimension reduction. We also show how the choice of norm on the codomain of the function has an impact on the function's low-dimensional approximation.

Many of the input-parameter-to-output-quantity-of-interest maps that arise in computational science admit a surprising low-dimensional structure, where the outputs vary primarily along a handful of directions in the high-dimensional input space. This type of structure is well modeled by a ridge function, which is a composition of a low-dimensional linear transformation with a nonlinear function. If the goal is to compute statistics of the output (e.g., as in uncertainty quantification or robust design) then one should exploit this low-dimensional structure, when present, to accelerate computations. We develop Gaussian quadrature and the associated polynomial approximation for one-dimensional ridge functions. The key elements of our method are (i) approximating the univariate density of the given linear combination of inputs by repeated convolutions and (ii) a Lanczos-Stieltjes method for constructing orthogonal polynomials and Gaussian quadrature.

Parameter reduction can enable otherwise infeasible design and uncertainty studies with modern computational science models that contain several input parameters. In statistical regression, techniques for sufficient dimension reduction (SDR) use data to reduce the predictor dimension of a regression problem. A computational scientist hoping to use SDR for parameter reduction encounters a problem: a computer prediction is best represented by a deterministic function of the inputs, so data comprised of computer simulation queries fail to satisfy the SDR assumptions. To address this problem, we interpret SDR methods sliced inverse regression (SIR) and sliced average variance estimation (SAVE) as estimating the directions of a ridge function, which is a composition of a low-dimensional linear transformation with a nonlinear function. Within this interpretation, SIR and SAVE estimate matrices of integrals whose column spaces are contained in the ridge directions' span; we analyze and numerically verify convergence of these column spaces as the number of computer model queries increases. Moreover, we show example functions that are not ridge functions but whose inverse conditional moment matrices are low-rank. Consequently, the computational scientist should beware when using SIR and SAVE for parameter reduction, since SIR and SAVE may mistakenly suggest that truly important directions are unimportant.

We investigate the application of sufficient dimension reduction (SDR) to a noiseless data set derived from a deterministic function of several variables. In this context, SDR provides a framework for ridge recovery. In this second part, we explore the numerical subtleties associated with using two inverse regression methods---sliced inverse regression (SIR) and sliced average variance estimation (SAVE)---for ridge recovery. This includes a detailed numerical analysis of the eigenvalues of the resulting matrices and the subspaces spanned by their columns. After this analysis, we demonstrate the methods on several numerical test problems.