Sergio Cruces

Divergences are fundamental to the information criteria that underpin most signal processing algorithms. The alpha-beta family of divergences, designed for non-negative data, offers a versatile framework that parameterizes and continuously interpolates several separable divergences found in existing literature. This work extends the definition of alpha-beta divergences to accommodate complex data, specifically when the arguments of the divergence are complex vectors. This novel formulation is designed in such a way that, by setting the divergence hyperparameters to unity, it particularizes to the well-known Euclidean and Mahalanobis squared distances. Other choices of hyperparameters yield practical separable and non-separable extensions of several classical divergences. In the context of the problem of approximating a complex random vector, the centroid obtained by optimizing the alpha-beta mean distortion has a closed-form expression, whose interpretation sheds light on the distinct roles of the divergence hyperparameters. These contributions may have wide potential applicability, as there are many signal processing domains in which the underlying data are inherently complex.

This paper introduces a broad class of Mirror Descent (MD) and Generalized Exponentiated Gradient (GEG) algorithms derived from trace-form entropies defined via deformed logarithms. Leveraging these generalized entropies yields MD and GEG algorithms with improved convergence behavior, robustness against vanishing and exploding gradients, and inherent adaptability to non-Euclidean geometries through mirror maps. We establish deep connections between these methods and Amari’s natural gradient, revealing a unified geometric foundation for additive, multiplicative, and natural gradient updates. Focusing on the Tsallis, Kaniadakis, Sharma–Taneja–Mittal, and Kaniadakis–Lissia–Scarfone entropy families, we show that each entropy induces a distinct Riemannian metric on the parameter space, leading to GEG algorithms that preserve the natural statistical geometry. The tunable parameters of deformed logarithms enable adaptive geometric selection, providing enhanced robustness and convergence over classical Euclidean optimization. Overall, our framework unifies key first-order MD optimization methods under a single information-geometric perspective based on generalized Bregman divergences, where the choice of entropy determines the underlying metric and dual geometric structure.

Stochastic gradient descent (SGD) and exponentiated gradient (EG) update methods are widely used in signal processing and machine learning. This study introduces a novel family of generalized Exponentiated Gradient updates (EGAB) derived from the alpha-beta (AB) divergence regularization. The EGAB framework provides enhanced flexibility for processing data with varying distributions, thanks to the tunable hyperparameters of the AB divergence. We explore the applicability of these updates in online portfolio selection (OLPS) for financial markets with the goal of developing algorithms that achieve high risk-adjusted returns, even under relatively high transaction costs. The proposed EGAB algorithms are developed using constrained gradient optimization with regularization terms, demonstrating their versatility in OLPS by unifying the directional search of various algorithms and enabling interpolation between them. Our analysis and extensive computer simulations reveal that EGAB updates outperform existing OLPS algorithms, delivering good results on several performance metrics, such as cumulative return, average excess return, Sharpe ratio, and Calmar ratio, especially when transaction costs are significant. In conclusion, this study introduces a new family of exponentiated gradient updates and demonstrates their flexibility and effectiveness through extensive simulations across a wide range of real-world financial datasets.

Bounded component analysis refers to a set of unsupervised techniques for the identification and recovery of underlying components of observations whose support has clear boundaries. This contribution is focused on the statistical justification and practical implementation of the minimum perimeter criterion, a bounded component analysis technique that can be used for the extraction of complex signals. In particular, with the help of the isoperimetric inequality, we reveal the link of the robust optimization of a statistical risk function with this criterion. We also address the difficulties for its optimization since, at the separation solutions, the criterion is non-differentiable. A reformulation of its optimization is shown to lead to a practical implementation that, for sources with clear boundaries, outperforms the normalized MSE performance of the state-of-the-art blind separation methods, being even close to the optimal performance of supervised methods.

In this article, a sparse-Bayesian treatment is proposed to solve the crucial questions posed by power amplifier (PA) and digital predistorter (DPD) modeling. To learn a model, the advanced Bayesian framework includes a group of specific processes that maximize the likelihood of the measured data: regressor pursuit and identification, coefficient estimation, stopping criterion, and regressor deselection. The relevance vector machine (RVM) method is reformulated theoretically to be implemented in complex-valued linear regression. In essence, given an initial set of candidate regressors, the result of this sparse-Bayesian learning approach is the most likely model. Experimental results are provided for the linearization of class AB and class J PAs driven by a 30-MHz fifth-generation new radio signal for a fixed average power, where the evolution of the figures of merit versus the number of active coefficients is examined for the proposed sparse-Bayesian pursuit (SBP) algorithm in comparison to other greedy algorithms. The SBP presents a good performance in terms of linearization capabilities and computational cost. Furthermore, the proposed Bayesian framework enabled the design of a DPD model structure, deselect regressors, and readjust coefficients in a direct learning architecture, demonstrating the robustness to changes in the power level over a 10-dB range.

This article considers the decomposition of a nonnegative signal into a non-negative linear combination of the contributions of pre-specified atomic units, which are also nonnegative. This model, referred as compositional model, is evident in the time-frequency characterisations of audio signals, where the sound can be viewed as a blending of spectral patterns of the component sounds that are present simultaneously. The algorithm proposed in this article obtains the activation vector of the atoms through an Active-Set Newton algorithm that employ the Alpha-Beta-divergence between the observed signal and the decomposition. This divergence family has been proved to be more efficient than other more common divergences, such as the generic Kullback-Leibler divergence in various audio signal processing applications. We have evaluated the proposed algorithm in a signal separation application of polyphonic music.

The common spatial pattern (CSP) method is a dimensionality reduction technique widely used in brain-computer interface (BCI) systems. In the two-class CSP problem, training data are linearly projected onto directions maximizing or minimizing the variance ratio between the two classes. The present contribution proves that kurtosis maximization performs CSP in an unsupervised manner, i.e., with no need for labelled data, when the classes follow Gaussian or elliptically symmetric distributions. Numerical analyses on synthetic and real data validate these findings in various experimental conditions, and demonstrate the interest of the proposed unsupervised approach.

The birth of Information Theory, right after the pioneering work of Claude Shannon and his celebrated publication of the paper “A mathematical theory of Communication”

In brain-computer interfaces the typical models of the EEG observations usually lead to a poor estimation of the trial covariance matrices, given the high non-stationarity of the EEG sources. We propose the application of two techniques that significantly improve the accuracy of these estimations and can be combined with a wide range of motor imagery BCI methods. The first one scales the observations in such a way that implicitly normalizes the common temporal strength of the source activities. When the scaling applies independently to the trials of the observations the procedure justifies and improves the classical preprocessing for the EEG data. Additionally, when the scaling is instantaneous and independent for each sample, the procedure particularizes to Tyler’s method in statistics for obtaining a distribution-free estimate of scattering. In this case, the proposal provides an original interpretation of this existing method as a technique that pursuits an implicit instantaneous power-normalization of the underlying source processes. The second technique applies to the classifier and improves its performance through a convenient regularization of the features covariance matrix. Experimental tests reveal that a combination of the proposed techniques with state-of-the-art algorithms for motor-imagery classification provides a significant improvement in the classification results.