Communications-Inspired Projection Design with Application to Compressive Sensing

We consider the recovery of an underlying signal $\mathbf{x}\in\mathbb{C}^m$ based on projection measurements of the form $\mathbf{y}=\mathbf{M}\mathbf{x}+\mathbf{w}$, where $\mathbf{y}\in\mathbb{C}^\ell$ and $\mathbf{w}$ is measurement noise; we are interested in the case $\ell\ll m$. It is assumed that the signal model $p(\mathbf{x})$ is known and that $\mathbf{w}\sim\mathcal{CN}(\mathbf{w};\boldsymbol{0},\bf \Sigma_w)$ for known $\bf \Sigma_w$. The objective is to design a projection matrix $\mathbf{M}\in\mathbb{C}^{\ell\times m}$ to maximize key information-theoretic quantities with operational significance, including the mutual information between the signal and the projections $\mathcal{I}(\mathbf{x};\mathbf{y})$ or the Rényi entropy of the projections $\mbox{h}_\alpha \left( \mathbf{y} \right)$ (Shannon entropy is a special case). By capitalizing on explicit characterizations of the gradients of the information measures with respect to the projection matrix, where we also partially extend the well-known results of Palomar and Verdú from the mutual information to the Rényi entropy domain, we reveal the key operations carried out by the optimal projection designs: mode exposure and mode alignment. Experiments are considered for the case of compressive sensing (CS) applied to imagery. In this context, we provide a demonstration of the performance improvement possible through the application of the novel projection designs in relation to conventional ones, as well as justification for a fast online projection design method with which state-of-the-art adaptive CS signal recovery is achieved.