Roberto Cavoretto

Mini-batch Stochastic Gradient Descent (SGD) is a standard optimization method for training neural networks and other large-scale parametric models. We study three loss-based mini-batch pruning strategies—Perceptron-Inspired SGD (PISGD), Pruning SGD (PSGD), and No-Release PSGD (NRPSGD)—that reduce the number of per-sample gradient evaluations by deprioritizing low-loss samples during training. The methods are evaluated in Python on two problems: polynomial function approximation and MNIST handwritten-digit classification with a one-hidden-layer neural network. We examine the sensitivity to pruning hyperparameters and compare the three strategies with standard mini-batch SGD in terms of training time and optimization or predictive performance. In the reported MNIST experiments, the pruning methods show competitive or improved time–accuracy and time–loss behavior relative to standard mini-batch SGD, with NRPSGD obtaining the most favorable results in the longer runs. These findings provide proof-of-concept evidence restricted to the experimental settings considered. They indicate that loss-based mini-batch pruning can reduce gradient-computation cost without degrading solution quality in the tested settings. Because the pruning mechanism leaves the parameter-update rule unchanged, it may also be combined with adaptive stochastic optimizers such as RMSProp and Adam.

In this work, we introduce an adaptive radial basis function (RBF) algorithm for numerical cubature over scattered data in a spherical polygon. In particular, we focus on the families of radial-power and thin-plate spline basis functions and we perform interpolation with an exponent parameter near-optimal for integration and data-driven. A further key ingredient of the algorithm is a novel adaptive routine for computing integrals over these domains. We make numerical experiments over various spherical polygons, even intricate ones as approximations of the African and Australian continents, highlighting the qualities of our approach. All the MATLAB routines used for this task are available as open-source codes.

In this study, we address the non-trivial problem of determining the optimal shape parameter in radial basis function interpolation. We propose the use of a genetic algorithm, an optimization technique inspired by evolutionary selection, to identify this parameter. Numerical experiments are presented to evaluate the effectiveness of this approach, with direct comparisons to the leave-one-out cross-validation method, thereby highlighting its computational efficiency.