Alberto Palomo-Alonso
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Alberto Palomo-Alonso has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
Antonio Portilla-Figueras
Escuela Politécnica Superior Universidad de Alcalá
3 shared publicationsSancho Salcedo-Sanz
Escuela Politécnica Superior Universidad de Alcalá
3 shared publicationsSilvia Jiménez-Fernández
Escuela Politécnica Superior Universidad de Alcalá
3 shared publications- Sancho Salcedo-Sanz
Department of Signal Processing and Communications, University of Alcalá, 28805 Madrid, Spain
2 shared publications - Antonio Portilla-Figueras
Universidad de Alcalà
1 shared publication - Carlos Eduardo Pedreira
Systems and Computing Department (PESC), COPPE, Federal University of Rio de Janeiro (UFRJ), Brazil
1 shared publication
Latest Publications
- Open Access
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
BEAST: BYTE-WISE EMBEDDING ARCHITECTURE FOR SEMANTIC TRANSFER
In this work, we introduce BEAST (Byte-wise Embedding Architecture for Semantic Transfer), a lightweight, tokenizer-free static embedding model that operates directly on byte-level input. BEAST learns 256-dimensional word representations by distilling semantic knowledge from GPT-3, without relying on tokenization, subword vocabularies, or linguistic preprocessing. Trained on over 1.2 million Spanish words, BEAST outperforms classic static embeddings such as FastText, Word2Vec, and GloVe, as well as several multilingual contextual models, on the Spanish translation of the SimLex-999 benchmark. Despite its compact size (51MB), BEAST captures rich lexical semantics and generalizes well to out-of-vocabulary words. We also demonstrate its practical utility in semantic clustering tasks, showing its potential as a robust embedding backbone for low-resource and multilingual NLP pipelines.
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessBEAST: BYTE-WISE EMBEDDING ARCHITECTURE FOR SEMANTIC TRANSFER
In this work, we introduce BEAST (Byte-wise Embedding Architecture for Semantic Transfer), a lightweight, tokenizer-free static embedding model that operates directly on byte-level input. BEAST learns 256-dimensional word representations by distilling semantic knowledge from GPT-3, without relying on tokenization, subword vocabularies, or linguistic preprocessing. Trained on over 1.2 million Spanish words, BEAST outperforms classic static embeddings such as FastText, Word2Vec, and GloVe, as well as several multilingual contextual models, on the Spanish translation of the SimLex-999 benchmark. Despite its compact size (51MB), BEAST captures rich lexical semantics and generalizes well to out-of-vocabulary words. We also demonstrate its practical utility in semantic clustering tasks, showing its potential as a robust embedding backbone for low-resource and multilingual NLP pipelines.
- Open Access
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
LEARNING DISCOURSE STRUCTURE WITHOUT SEMANTICS: A STUDY ON TOPIC SEGMENTATION OBJECTIVES
Topic segmentation is commonly framed as a semantic similarity problem, yet it remains unclear whether optimizing for segmentation induces transferable sentence representations. In this work, we propose a differentiable, similarity-based framework that models topic segmentation as boundary detection over sentence-level similarity matrices. Our approach integrates a fully vectorized and differentiable segmentation head into a Transformer architecture, enabling end-to-end supervised training without assuming a predefined number of segments. Unlike classical matrix segmentation algorithms, the proposed head is designed for seamless integration with neural encoders and gradient-based optimization. We train the model from scratch on a newly curated Spanish Wikipedia dataset for topic segmentation, where sentence embeddings are transformed into cosine similarity matrices and processed to predict topic boundaries. The formulation explicitly decouples sentence encoding from segmentation while preserving full differentiability. Empirically, the model substantially outperforms zero-shot Transformer baselines and heuristic methods, achieving a WindowDiff score of 0.07 on the test set. Beyond performance, we conduct an in-depth analysis of training dynamics and representation transfer. Results indicate that, although the model effectively captures discourse-level structural shifts, the learned embeddings exhibit very limited transferability to semantic similarity tasks. These findings suggest that topic segmentation primarily exploits structural and discourse cues rather than inducing semantically aligned representation spaces.
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessLEARNING DISCOURSE STRUCTURE WITHOUT SEMANTICS: A STUDY ON TOPIC SEGMENTATION OBJECTIVES
Topic segmentation is commonly framed as a semantic similarity problem, yet it remains unclear whether optimizing for segmentation induces transferable sentence representations. In this work, we propose a differentiable, similarity-based framework that models topic segmentation as boundary detection over sentence-level similarity matrices. Our approach integrates a fully vectorized and differentiable segmentation head into a Transformer architecture, enabling end-to-end supervised training without assuming a predefined number of segments. Unlike classical matrix segmentation algorithms, the proposed head is designed for seamless integration with neural encoders and gradient-based optimization. We train the model from scratch on a newly curated Spanish Wikipedia dataset for topic segmentation, where sentence embeddings are transformed into cosine similarity matrices and processed to predict topic boundaries. The formulation explicitly decouples sentence encoding from segmentation while preserving full differentiability. Empirically, the model substantially outperforms zero-shot Transformer baselines and heuristic methods, achieving a WindowDiff score of 0.07 on the test set. Beyond performance, we conduct an in-depth analysis of training dynamics and representation transfer. Results indicate that, although the model effectively captures discourse-level structural shifts, the learned embeddings exhibit very limited transferability to semantic similarity tasks. These findings suggest that topic segmentation primarily exploits structural and discourse cues rather than inducing semantically aligned representation spaces.
- Open Access
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
OVERFITTING AS A FINITE-SAMPLE EMPIRICAL DISTRIBUTION SHIFT
Overfitting is often described as an intrinsic property of machine learning models or learning algorithms, commonly associated with excessive capacity or insufficient regularization. In this work, we argue that this perspective obscures the role of data availability in shaping measurable generalization behavior. We show that observable overfitting, defined as the train-validation empirical gap, is an unavoidable finite-sample effect under standard i.i.d. assumptions, whose magnitude can be influenced by model capacity. We formalize this distinction by introducing two learning regimes: the Finite Data Regime (FDR) and the Infinite Data Regime (IDR). Under the IDR, we prove that for any learning algorithm whose outputs lie in a VC-finite hypothesis class, including highly expressive parametric models under standard VC assumptions, the training-validation gap converges to zero in probability as the number of samples grows, independently of optimization choices, regularization strategies, or architectural details, in the asymptotic sense considered here. Beyond this asymptotic result, we derive explicit finite-sample conditions characterizing when a learning system effectively approximates the IDR, yielding operational convergence criteria and closed-form sample complexity bounds. This perspective frames measurable overfitting as a manifestation of finite-sample distributional mismatch, shifting the focus from model-centric explanations toward a regime-aware understanding of generalization.
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessOVERFITTING AS A FINITE-SAMPLE EMPIRICAL DISTRIBUTION SHIFT
Overfitting is often described as an intrinsic property of machine learning models or learning algorithms, commonly associated with excessive capacity or insufficient regularization. In this work, we argue that this perspective obscures the role of data availability in shaping measurable generalization behavior. We show that observable overfitting, defined as the train-validation empirical gap, is an unavoidable finite-sample effect under standard i.i.d. assumptions, whose magnitude can be influenced by model capacity. We formalize this distinction by introducing two learning regimes: the Finite Data Regime (FDR) and the Infinite Data Regime (IDR). Under the IDR, we prove that for any learning algorithm whose outputs lie in a VC-finite hypothesis class, including highly expressive parametric models under standard VC assumptions, the training-validation gap converges to zero in probability as the number of samples grows, independently of optimization choices, regularization strategies, or architectural details, in the asymptotic sense considered here. Beyond this asymptotic result, we derive explicit finite-sample conditions characterizing when a learning system effectively approximates the IDR, yielding operational convergence criteria and closed-form sample complexity bounds. This perspective frames measurable overfitting as a manifestation of finite-sample distributional mismatch, shifting the focus from model-centric explanations toward a regime-aware understanding of generalization.
- Open Access
- 5 September 2024
- Published by Wiley in Expert Systems
- Vol. 41 (12)
TensorCRO: A TensorFlow‐based implementation of a multi‐method ensemble for optimization
This paper presents a novel implementation of the Coral Reef Optimization with Substrate Layers (CRO‐SL) algorithm. Our approach, which we call TensorCRO, takes advantage of the TensorFlow framework to represent CRO‐SL as a series of tensor operations, allowing it to run on GPU and search for solutions in a faster and more efficient way. We evaluate the performance of the proposed implementation across a wide range of benchmark functions commonly used in optimization research (such as the Rastrigin, Rosenbrock, Ackley, and Griewank functions), and we show that GPU execution leads to considerable speedups when compared to its CPU counterpart. Then, when comparing TensorCRO to other state‐of‐the‐art optimization algorithms (such as the Genetic Algorithm, Simulated Annealing, and Particle Swarm Optimization), the results show that TensorCRO can achieve better convergence rates and solutions than other algorithms within a fixed execution time, given that the fitness functions are also implemented on TensorFlow. Furthermore, we also evaluate the proposed approach in a real‐world problem of optimizing power production in wind farms by selecting the locations of turbines; in every evaluated scenario, TensorCRO outperformed the other meta‐heuristics and achieved solutions close to the best known in the literature. Overall, our implementation of the CRO‐SL algorithm in TensorFlow GPU provides a new, fast, and efficient approach to solving optimization problems, and we believe that the proposed implementation has significant potential to be applied in various domains, such as engineering, finance, and machine learning, where optimization is often used to solve complex problems. Furthermore, we propose that this implementation can be used to optimize models that cannot propagate an error gradient, which is an excellent choice for non‐gradient‐based optimizers.
- 5 September 2024
- Published by Wiley in Expert Systems
- Vol. 41 (12)
Open AccessTensorCRO: A TensorFlow‐based implementation of a multi‐method ensemble for optimization
This paper presents a novel implementation of the Coral Reef Optimization with Substrate Layers (CRO‐SL) algorithm. Our approach, which we call TensorCRO, takes advantage of the TensorFlow framework to represent CRO‐SL as a series of tensor operations, allowing it to run on GPU and search for solutions in a faster and more efficient way. We evaluate the performance of the proposed implementation across a wide range of benchmark functions commonly used in optimization research (such as the Rastrigin, Rosenbrock, Ackley, and Griewank functions), and we show that GPU execution leads to considerable speedups when compared to its CPU counterpart. Then, when comparing TensorCRO to other state‐of‐the‐art optimization algorithms (such as the Genetic Algorithm, Simulated Annealing, and Particle Swarm Optimization), the results show that TensorCRO can achieve better convergence rates and solutions than other algorithms within a fixed execution time, given that the fitness functions are also implemented on TensorFlow. Furthermore, we also evaluate the proposed approach in a real‐world problem of optimizing power production in wind farms by selecting the locations of turbines; in every evaluated scenario, TensorCRO outperformed the other meta‐heuristics and achieved solutions close to the best known in the literature. Overall, our implementation of the CRO‐SL algorithm in TensorFlow GPU provides a new, fast, and efficient approach to solving optimization problems, and we believe that the proposed implementation has significant potential to be applied in various domains, such as engineering, finance, and machine learning, where optimization is often used to solve complex problems. Furthermore, we propose that this implementation can be used to optimize models that cannot propagate an error gradient, which is an excellent choice for non‐gradient‐based optimizers.
- Open Access
- 11 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 3055-3069
A Flexible Architecture Using Temporal, Spatial and Semantic Correlation-Based Algorithms for Story Segmentation of Broadcast News
In this article, we propose a novel flexible architecture, with different algorithmic procedures, for effective story segmentation of broadcast news from subtitle files. The proposed system exploits spatial and temporal distance, as well as sentence similarity, to classify different stories in news broadcasts. The computational algorithms which form the architecture mainly focus on each sentence's features (temporal distance, spatial distance, and semantic similarity), and are combined to build an overall classifier. The first algorithm in the architecture focuses on the segmentation task, detecting boundaries between news. The second and third algorithms identify high semantic correlation between pieces of text, whether they are consecutive in space or not. Video Text Track (VTT) subtitle files are used to evaluate the performance of the proposed approach, although any file format that includes temporal information could also be considered. These VTT files may contain text errors and inaccuracies, and the proposed algorithms have been designed to deal with noisy content.
- 11 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 3055-3069
Open AccessA Flexible Architecture Using Temporal, Spatial and Semantic Correlation-Based Algorithms for Story Segmentation of Broadcast News
In this article, we propose a novel flexible architecture, with different algorithmic procedures, for effective story segmentation of broadcast news from subtitle files. The proposed system exploits spatial and temporal distance, as well as sentence similarity, to classify different stories in news broadcasts. The computational algorithms which form the architecture mainly focus on each sentence's features (temporal distance, spatial distance, and semantic similarity), and are combined to build an overall classifier. The first algorithm in the architecture focuses on the segmentation task, detecting boundaries between news. The second and third algorithms identify high semantic correlation between pieces of text, whether they are consecutive in space or not. Video Text Track (VTT) subtitle files are used to evaluate the performance of the proposed approach, although any file format that includes temporal information could also be considered. These VTT files may contain text errors and inaccuracies, and the proposed algorithms have been designed to deal with noisy content.