Monica F. Bugallo

This article describes a pilot undergraduate course designed to introduce the fundamentals of machine learning (ML) and generative artificial intelligence (AI) in an accessible and practical way to students from diverse disciplines. Emerging from cross-departmental collaboration, the course aims to demystify AI and equip students, particularly those from nontechnical fields, with essential ML and signal processing (SP) concepts, while helping them understand their applications and practice as end users. The article outlines the course content, student outcomes, and considerations for evaluating its effectiveness and potential areas for improvement.

Our interest in obtaining predictive models from vast volumes of data has never been greater. Often, we want these models to be as simple as possible but not simpler. Reducing the complexity or size of a model is accomplished by enforcing sparsity in the model’s parameters or structure. In the context of sparse regression, LASSO and its variants have become standard practice, but they rely on the choice of the penalty parameter for desired degrees of sparsity or predictability. In this work, we develop a novel online inference approach for transdimensional model learning with online feature selection, built on the fundamental principles of predictive least squares. The solution path follows the predictive error at each step, prioritizing more predictive features in the model. We provide a variety of examples to analyze the capabilities of the proposed method and evaluate its performance against both standard and recently proposed feature selection methods.

An important and often overlooked aspect of particle filtering methods is the estimation of unknown static parameters. A simple approach for addressing this problem is to augment the unknown static parameters as auxiliary states that are jointly estimated with the time-varying parameters of interest. This can be impractical, especially when the system of interest is high-dimensional. Multiple particle filtering (MPF) methods were introduced to try to overcome the curse of dimensionality by using a divide and conquer approach, where the vector of unknowns is partitioned into a set of subvectors, each estimated by a separate particle filter. Each particle filter weighs its own particles by using predictions and estimates communicated from the other filters. Currently, there is no principled way to implement MPF methods where the particle filters share unknown parameters or states. In this work, we propose a fusion strategy to allow for the sharing of unknown static parameters in the MPF setting. Specifically, we study the systems which are separable in states and observations. It is proved that optimal Bayesian fusion can be obtained for state-space models with non-interacting states and observations. Simulations are performed to show that MPF with fusion strategy can provide more accurate estimates within fewer time steps comparing to existing algorithms.

While systems analysis has been studied for decades in the context of control theory, it has only been recently used to improve the convergence of Denoising Diffusion Probabilistic Models. This work describes a novel improvement to Third- Order Langevin Dynamics (TOLD), a recent diffusion method that performs better than its predecessors. This improvement, abbreviated TOLD++, is carried out by critically damping the TOLD forward transition matrix similarly to Dockhorn's Critically-Damped Langevin Dynamics (CLD). Specifically, it exploits eigen-analysis of the forward transition matrix to derive the optimal set of dynamics under the original TOLD scheme. TOLD++ is theoretically guaranteed to converge faster than TOLD, and its faster convergence is verified on the Swiss Roll toy dataset and CIFAR-10 dataset according to the FID metric.