Seyedsaman Emami

Matrix Factorization (MF) remains a strong and interpretable method for explicit feedback recommendation, but its prediction errors may pre- serve systematic structure associated with particular users, items, and interaction regimes, especially in highly sparse datasets. This paper intro- duces Residual-Calibrated Matrix Factorization (RC-MF), a novel two-stage residual-learning framework for pointwise recommendation that learns structured correction terms from the systematic errors left by a fitted MF model, rather than optimizing a ranking objective. RC-MF models the unexplained component of MF predictions through jointly estimated global, user-specific, item-specific, and user-activity-item popularity residual effects, while shrink- age regularization, bounded corrections, and optional out-of-fold (OOF) resid- ual estimation control the reliability of the learned error structure. Experiments on real-world benchmark datasets show that RC-MF improves predictive accuracy over standard MF in most cases, with the clearest and most consistent gains in sparse recommendation settings. Ranking quality is assessed only as a secondary outcome because RC-MF is designed to reduce pointwise prediction error rather than optimize pairwise or listwise objectives. Ablation results identify user and item-specific residual effects as the main sources of improvement, with group-level effects providing an additional structured correction. Residual diagnostics further show that RC-MF reduces systematic group-level bias and error after calibration. These findings position RC MF as an interpretable and model-agnostic residual learning method for correcting persistent error structure left by MF models.

Machine learning methods are increasingly used in economics, yet most applications rely on single-task approaches that model outcomes, markets, or regions in isolation. In this study, we adopt a multi-task learning (MTL) framework for economic and tabular data to analyze interdependencies across related economic units. Using benchmark housing datasets representing different economic environments, we define each task as a regional or categorical subgroup. The MTL approach exploits shared representations while allowing for task-specific effects, leading to improved predictive performance and revealing latent relationships across subgroups. An empirical evaluation across three datasets demonstrates the suitability of MTL for structured economic data. Overall, the results suggest that MTL provides a transparent and flexible framework for jointly modeling related entities such as regions, industries, or firms.

Multi-Task Learning (MTL) aims to boost predictive performance by sharing information across related tasks, yet conventional methods often suffer from negative transfer when unrelated or noisy tasks are forced to share representations. We propose Robust Multi-Task Boosting using Clustering and Local Ensembling (RMB-CLE), a principled MTL framework that integrates error-based task clustering with local ensembling. Unlike prior work that assumes fixed clusters or hand-crafted similarity metrics, RMB-CLE derives inter-task similarity directly from cross-task errors, which admit a risk decomposition into functional mismatch and irreducible noise, providing a theoretically grounded mechanism to prevent negative transfer. Tasks are grouped adaptively via agglomerative clustering, and within each cluster, a local ensemble enables robust knowledge sharing while preserving task-specific patterns. Experiments show that RMB-CLE recovers ground-truth clusters in synthetic data and consistently outperforms multi-task, single-task, and pooling-based ensemble methods across diverse real-world and synthetic benchmarks. These results demonstrate that RMB-CLE is not merely a combination of clustering and boosting but a general and scalable framework that establishes a new basis for robust multi-task learning.

This paper presents a computationally efficient variant of Gradient Boosting (GB) for multi-class classification and multi-output regression tasks. Standard GB uses a 1-vs-all strategy for classification tasks with more than two classes. This strategy entails that one tree per class and iteration has to be trained. In this work, we propose the use of multi-output regressors as base models to handle the multi-class problem as a single task. In addition, the proposed modification allows the model to learn multi-output regression problems. An extensive comparison with other multi-output based Gradient Boosting methods is carried out in terms of generalization and computational efficiency. The proposed method showed the best trade-off between generalization ability and training and prediction speeds. Furthermore, an analysis of space and time complexity was undertaken.

This paper presents a novel methodology to address multi-output regression problems through the incorporation of deep-neural networks and gradient boosting. The proposed approach involves the use of dense layers as additive models within the Gradient Boosting framework using an auto transfer learning technique. At each boosting iteration, the deep model is cloned with the already trained layers frozen, and a new dense layer is concatenated to the frozen ones. Subsequently, only the weights of the newly added layer are trained in order to reduce the complexity of the learning task. Each layer is trained on the residuals of the squared loss function from previous iterations, resulting in the creation of a robust sequentially deep-trained neural network ensemble. Our experimental results demonstrate that the proposed approach leads to a significant improvement in the performance of the deep framework, resulting in more accurate predictions and improved model interpretability.

Deep learning has revolutionized computer vision and image classification domains. In this context Convolutional Neural Networks (CNNs) based architectures and Deep Neural Networks (DNNs) are the most widely applied models. In this article, we introduced two procedures for training CNNs and DNNs based on Gradient Boosting (GB), namely GB-CNN and GB-DNN. These models are trained to fit the gradient of the loss function or pseudo-residuals of previous models. At each iteration, the proposed method adds one dense layer to an exact copy of the previous deep NN model. The weights of the dense layers trained on previous iterations are frozen to prevent over-fitting, permitting the model to fit the new dense as well as to fine-tune the convolutional layers (for GB-CNN) while still utilizing the information already learned. Through extensive experimentation on different 2D-image classification and tabular datasets, the presented models show superior performance in terms of classification accuracy with respect to standard CNN and DNN with the same architectures.

This paper presents a novel technique based on gradient boosting to train the final layers of a neural network (NN). Gradient boosting is an additive expansion algorithm in which a series of models are trained sequentially to approximate a given function. A neural network can also be seen as an additive expansion where the scalar product of the responses of the last hidden layer and its weights provide the final output of the network. Instead of training the network as a whole, the proposed algorithm trains the network sequentially in T steps. First, the bias term of the network is initialized with a constant approximation that minimizes the average loss of the data. Then, at each step, a portion of the network, composed of J neurons, is trained to approximate the pseudo-residuals on the training data computed from the previous iterations. Finally, the T partial models and bias are integrated as a single NN with T × J neurons in the hidden layer. Extensive experiments in classification and regression tasks, as well as in combination with deep neural networks, are carried out showing a competitive generalization performance with respect to neural networks trained with different standard solvers, such as Adam, L-BFGS, SGD and deep models. Furthermore, we show that the proposed method design permits to switch off a number of hidden units during test (the units that were last trained) without a significant reduction of its generalization ability. This permits the adaptation of the model to different classification speed requirements on the fly.