FuDoBa: Fusing Document and Knowledge Graph Based Representations with Bayesian Optimisation
Open Access
- 6 March 2026
- journal article
- research article
- Published by Springer Nature in Machine Learning
- Vol. 115 (3) , 61
Abstract
Building on the success of large language models (LLMs), LLM-based representations have dominated the document representation landscape, achieving strong performance on document embedding benchmarks. However, high-dimensional, computationally expensive LLM embeddings can be too generic or inefficient for domain-specific and resource-scarce applications. To address these limitations, we introduce FuDoBa—a Bayesian optimisation-based representation learning method that integrates LLM embeddings with domain-specific structured knowledge, sourced both locally and from external repositories such as WikiData. This fusion produces low-dimensional, task-relevant representations while reducing training complexity and yielding interpretable early-fusion weights for improved classification performance. We demonstrate the effectiveness of our approach on six datasets across two domains, showing that when paired with robust AutoML-based classifiers, our method performs on par with, or surpasses, proprietary LLM-only embedding baselines, while offering modality-wise interpretability and a smaller dimensional footprint.Keywords
All Related Versions
Funding Information
- The Slovenian Research and Innovation Agency (PR-12394)
- Ministero dell’Istruzione, dell’Università e della Ricerca (MIUR_PRIN 2020 2020ZSL9F9)
This publication has 31 references indexed in Scilit:
- Augmenting Multimodal LLMs with Self-Reflective Tokens for Knowledge-based Visual Question Answering2025
- AutoML-Guided Fusion of Entity and LLM-Based Representations for Document Classification2025
- Accurate predictions on small data with a tabular foundation modelNature, 2025
- Knowledge graph informed fake news classification via heterogeneous representation ensemblesNeurocomputing, 2022
- Compressibility of Distributed Document Representations2021
- Zero-Shot Text Classification via Knowledge Graph Embedding for Social Media DataIEEE Internet of Things Journal, 2021
- autoBOT: evolving neuro-symbolic representations for explainable low resource text classificationMachine Learning, 2021
- tax2vec: Constructing Interpretable Features from Taxonomies for Short Text ClassificationComputer Speech & Language, 2021
- Scaling tree-based automated machine learning to biomedical big data with a feature set selectorBioinformatics, 2019
- On the Surprising Behavior of Distance Metrics in High Dimensional Space2001