Gustav Šír
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Gustav Šír has a total of 27 co-authors. Here are the top co-authors by number of shared publications:
- Danial Hooshyar
Tallinn University
4 shared publications - Jakub Peleška
Czech Technical University in Prague Karlovo náměstí 13 121 35 Prague Czechia
4 shared publications - Tommi Kärkkäinen
University of Jyväskylä
4 shared publications - Yeongwook Yang
Department of Computer Science and Engineering, Gangneung-Wonju National University, Wonju-si, Gangwon-do, Republic of Korea
3 shared publications - Ekaterina Krivich
Tallinn University, Estonia
2 shared publications - Filip Železný
Czech Technical University
2 shared publications
Latest Publications
- Open Access
- 27 October 2026
- Published by MDPI AG in Electronics
- Vol. 15 (21) , 4975
Beyond AUC: Evaluating Deep Knowledge Tracing Through Temporal Coherence and Post Hoc Explanations
Deep knowledge tracing (KT) models estimate students’ evolving skill mastery from sequences of question–response events and predict whether the next response will be correct. They are often compared using AUC, a score of how well predictions distinguish correct from incorrect answers. We evaluate two complementary aspects beyond AUC: temporal prediction behavior for 11 KT models on four educational datasets, and attribution diagnostics on ASSISTments 2017 data. The relationship between AUC and response-direction consistency varies by dataset: models with higher AUC sometimes show more, and sometimes fewer, prediction drops after correct answers or prediction rises after incorrect answers on repeated skills. Reference baselines further show that stability must be interpreted together with responsiveness, because a model that never updates its prediction also appears perfectly stable. In an exploratory ASSISTments 2017 analysis of 50 histories from 33 learners per model, gradient-based selected question–response events produce modest, model-dependent signed probability changes (original minus post-replacement probability) relative to recent-history, same-skill, and random controls. For practitioners comparing KT models, the paper contributes two practical checks to report alongside AUC: whether predictions respond coherently after student answers, and how explanation-selected history events compare with simple recent-history or same-skill alternatives under a shared intervention.
- 27 October 2026
- Published by MDPI AG in Electronics
- Vol. 15 (21) , 4975
Open AccessBeyond AUC: Evaluating Deep Knowledge Tracing Through Temporal Coherence and Post Hoc Explanations
Deep knowledge tracing (KT) models estimate students’ evolving skill mastery from sequences of question–response events and predict whether the next response will be correct. They are often compared using AUC, a score of how well predictions distinguish correct from incorrect answers. We evaluate two complementary aspects beyond AUC: temporal prediction behavior for 11 KT models on four educational datasets, and attribution diagnostics on ASSISTments 2017 data. The relationship between AUC and response-direction consistency varies by dataset: models with higher AUC sometimes show more, and sometimes fewer, prediction drops after correct answers or prediction rises after incorrect answers on repeated skills. Reference baselines further show that stability must be interpreted together with responsiveness, because a model that never updates its prediction also appears perfectly stable. In an exploratory ASSISTments 2017 analysis of 50 histories from 33 learners per model, gradient-based selected question–response events produce modest, model-dependent signed probability changes (original minus post-replacement probability) relative to recent-history, same-skill, and random controls. For practitioners comparing KT models, the paper contributes two practical checks to report alongside AUC: whether predictions respond coherently after student answers, and how explanation-selected history events compare with simple recent-history or same-skill alternatives under a shared intervention.
- Open Access
- 28 September 2026
- Published by Wiley in Expert Systems
- Vol. 43 (11) , e70433
Mission‐Aligned Learning‐Informed Control of Autonomous Systems: Formulation and Foundations
Research, innovation and practical capital investment have been increasing rapidly toward the realization of autonomous physical agents. This includes industrial and service robots, unmanned aerial vehicles, embedded control devices, and a number of other realizations of cybernetic/mechatronic implementations of intelligent autonomous devices. In this paper, we consider a stylized version of robotic care, which would normally involve a two‐level Reinforcement Learning procedure that trains a policy for both lower level physical movement decisions as well as higher level conceptual tasks and their sub‐components. In order to deliver greater safety and reliability in the system, we present the general formulation of this as a two‐level expert defined optimization scheme which incorporates control with physics modelling of the system and environment at the lower level, and classical planning with complex task logic at the higher level, integrated with a capacity for learning. This synergistic integration of multiple methodologies—control, classical planning, and RL—presents an opportunity for greater insight for algorithm development, leading to more efficient and reliable performance. Here, the notion of reliability pertains to physical safety and interpretability into an otherwise black box operation of autonomous agents, concerning users and regulators. This work presents the principled integration of the two expert‐defined domains. The methodology is validated in a simulation of robotic care aid, with complex medication and meal preparation tasks at the upper level and locomotion at the lower level. The validation confirms the framework's adaptability to human preferences and symbolic to continuous feedback.
- 28 September 2026
- Published by Wiley in Expert Systems
- Vol. 43 (11) , e70433
Open AccessMission‐Aligned Learning‐Informed Control of Autonomous Systems: Formulation and Foundations
Research, innovation and practical capital investment have been increasing rapidly toward the realization of autonomous physical agents. This includes industrial and service robots, unmanned aerial vehicles, embedded control devices, and a number of other realizations of cybernetic/mechatronic implementations of intelligent autonomous devices. In this paper, we consider a stylized version of robotic care, which would normally involve a two‐level Reinforcement Learning procedure that trains a policy for both lower level physical movement decisions as well as higher level conceptual tasks and their sub‐components. In order to deliver greater safety and reliability in the system, we present the general formulation of this as a two‐level expert defined optimization scheme which incorporates control with physics modelling of the system and environment at the lower level, and classical planning with complex task logic at the higher level, integrated with a capacity for learning. This synergistic integration of multiple methodologies—control, classical planning, and RL—presents an opportunity for greater insight for algorithm development, leading to more efficient and reliable performance. Here, the notion of reliability pertains to physical safety and interpretability into an otherwise black box operation of autonomous agents, concerning users and regulators. This work presents the principled integration of the two expert‐defined domains. The methodology is validated in a simulation of robotic care aid, with complex medication and meal preparation tasks at the upper level and locomotion at the lower level. The validation confirms the framework's adaptability to human preferences and symbolic to continuous feedback.
- Open Access
- 24 August 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 17 (5) , 1-49
Deep Learning for Molecular Property Prediction with Logical Background Knowledge
Graph Neural Networks (GNNs) have emerged as powerful tools for molecular property prediction by leveraging graph-structured data representations of chemical compounds. However, standard GNN architectures are completely generic and do not allow for explicitly encoding domain-specific chemical knowledge, such as functional groups, molecular motifs, and atomic interactions, which can be critical for accurate predictions, especially in low-data regimes. In this work, we propose an enhanced learning framework that deeply integrates structured logical knowledge within GNN models. Particularly, by formulating the underlying message-passing GNN mechanisms as logic-driven transformations, we bridge the gap between symbolic domain knowledge and deep learning, improving predictive accuracy and explainability. Empirical results on multiple benchmark molecular datasets demonstrate that incorporating domain-specific relational knowledge leads to better generalization and robustness compared to standard GNNs.
- 24 August 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 17 (5) , 1-49
Open AccessDeep Learning for Molecular Property Prediction with Logical Background Knowledge
Graph Neural Networks (GNNs) have emerged as powerful tools for molecular property prediction by leveraging graph-structured data representations of chemical compounds. However, standard GNN architectures are completely generic and do not allow for explicitly encoding domain-specific chemical knowledge, such as functional groups, molecular motifs, and atomic interactions, which can be critical for accurate predictions, especially in low-data regimes. In this work, we propose an enhanced learning framework that deeply integrates structured logical knowledge within GNN models. Particularly, by formulating the underlying message-passing GNN mechanisms as logic-driven transformations, we bridge the gap between symbolic domain knowledge and deep learning, improving predictive accuracy and explainability. Empirical results on multiple benchmark molecular datasets demonstrate that incorporating domain-specific relational knowledge leads to better generalization and robustness compared to standard GNNs.
- Open Access
- 8 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 91267-91284
Why Large Language Models Alone Fall Short for Responsible Learner Modeling in K–12 Tutoring: A Case Study
The rapid rise of large language model (LLM)-based tutors in K–12 education has led to the misconception that generative models can replace traditional learner modelling and act as general-purpose engines for adaptive instruction. This is especially problematic in K–12 settings, which the EU AI Act classifies as a high-risk domain requiring responsible design. Motivated by concerns surrounding the role of learner modelling in responsible AI-powered tutoring, this study synthesises existing research evidence on key limitations of LLM-based tutoring systems and then presents an empirical case study investigating one critical aspect of these concerns: the accuracy, reliability, and temporal coherence of assessing learners’ evolving knowledge over time. To this end, we compare a deep knowledge tracing (DKT) model with a widely used LLM (with and without fine-tuning) that has demonstrated competitive performance in tutoring-related tasks, using a large-scale open-access dataset. Our findings show that DKT achieves the highest discrimination performance (AUC = 0.83) on next-step correctness prediction and consistently outperforms the LLM across evaluation settings. Although fine-tuning improves the LLM’s AUC by about 8% over the zero-shot baseline, it still remains 6% below DKT and produces higher early-sequence errors, precisely where incorrect predictions would be most harmful for adaptive learner support. Temporal-coherence analyses further reveal that while DKT maintains stable, directionally correct mastery updates, LLM variants display substantial temporal weaknesses, including smooth but wrong-direction updates, and, even after fine-tuning, remain inconsistent and unable to match DKT’s temporal stability. We also illustrate that these issues persist even though fine-tuned LLMs required nearly 198 hours of continuous high-compute training, far exceeding the computational demands of the lightweight DKT model. Our qualitative analysis of multi-skill mastery estimation further shows that, even after fine-tuning, the LLM produced unstable and inconsistent mastery trajectories, whereas DKT maintained smooth and coherent multi-skill updates. Collectively, these findings suggest that LLMs alone are unlikely to achieve the same positive effect sizes observed in decades long intelligent tutoring systems literature. Rather than replacing learner modelling, LLMs may be more appropriately deployed as pedagogical interfaces or content generators paired with dedicated learner modelling components to ensure responsible, accurate, reliable, and pedagogically sound support.
- 8 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 91267-91284
Open AccessWhy Large Language Models Alone Fall Short for Responsible Learner Modeling in K–12 Tutoring: A Case Study
The rapid rise of large language model (LLM)-based tutors in K–12 education has led to the misconception that generative models can replace traditional learner modelling and act as general-purpose engines for adaptive instruction. This is especially problematic in K–12 settings, which the EU AI Act classifies as a high-risk domain requiring responsible design. Motivated by concerns surrounding the role of learner modelling in responsible AI-powered tutoring, this study synthesises existing research evidence on key limitations of LLM-based tutoring systems and then presents an empirical case study investigating one critical aspect of these concerns: the accuracy, reliability, and temporal coherence of assessing learners’ evolving knowledge over time. To this end, we compare a deep knowledge tracing (DKT) model with a widely used LLM (with and without fine-tuning) that has demonstrated competitive performance in tutoring-related tasks, using a large-scale open-access dataset. Our findings show that DKT achieves the highest discrimination performance (AUC = 0.83) on next-step correctness prediction and consistently outperforms the LLM across evaluation settings. Although fine-tuning improves the LLM’s AUC by about 8% over the zero-shot baseline, it still remains 6% below DKT and produces higher early-sequence errors, precisely where incorrect predictions would be most harmful for adaptive learner support. Temporal-coherence analyses further reveal that while DKT maintains stable, directionally correct mastery updates, LLM variants display substantial temporal weaknesses, including smooth but wrong-direction updates, and, even after fine-tuning, remain inconsistent and unable to match DKT’s temporal stability. We also illustrate that these issues persist even though fine-tuned LLMs required nearly 198 hours of continuous high-compute training, far exceeding the computational demands of the lightweight DKT model. Our qualitative analysis of multi-skill mastery estimation further shows that, even after fine-tuning, the LLM produced unstable and inconsistent mastery trajectories, whereas DKT maintained smooth and coherent multi-skill updates. Collectively, these findings suggest that LLMs alone are unlikely to achieve the same positive effect sizes observed in decades long intelligent tutoring systems literature. Rather than replacing learner modelling, LLMs may be more appropriately deployed as pedagogical interfaces or content generators paired with dedicated learner modelling components to ensure responsible, accurate, reliable, and pedagogically sound support.
- Open Access
Problems With Large Language Models for Learner Modelling: Why LLMs Alone Fall Short for Responsible Tutoring in K--12 Education
The rapid rise of large language model (LLM)-based tutors in K--12 education has fostered a misconception that generative models can replace traditional learner modelling for adaptive instruction. This is especially problematic in K--12 settings, which the EU AI Act classifies as high-risk domain requiring responsible design. Motivated by these concerns, this study synthesises evidence on limitations of LLM-based tutors and empirically investigates one critical issue: the accuracy, reliability, and temporal coherence of assessing learners' evolving knowledge over time. We compare a deep knowledge tracing (DKT) model with a widely used LLM, evaluated zero-shot and fine-tuned, using a large open-access dataset. Results show that DKT achieves the highest discrimination performance (AUC = 0.83) on next-step correctness prediction and consistently outperforms the LLM across settings. Although fine-tuning improves the LLM's AUC by approximately 8\% over the zero-shot baseline, it remains 6\% below DKT and produces higher early-sequence errors, where incorrect predictions are most harmful for adaptive support. Temporal analyses further reveal that DKT maintains stable, directionally correct mastery updates, whereas LLM variants exhibit substantial temporal weaknesses, including inconsistent and wrong-direction updates. These limitations persist despite the fine-tuned LLM requiring nearly 198 hours of high-compute training, far exceeding the computational demands of DKT. Our qualitative analysis of multi-skill mastery estimation further shows that, even after fine-tuning, the LLM produced inconsistent mastery trajectories, while DKT maintained smooth and coherent updates. Overall, the findings suggest that LLMs alone are unlikely to match the effectiveness of established intelligent tutoring systems, and that responsible tutoring requires hybrid frameworks that incorporate learner modelling.
Open AccessProblems With Large Language Models for Learner Modelling: Why LLMs Alone Fall Short for Responsible Tutoring in K--12 Education
The rapid rise of large language model (LLM)-based tutors in K--12 education has fostered a misconception that generative models can replace traditional learner modelling for adaptive instruction. This is especially problematic in K--12 settings, which the EU AI Act classifies as high-risk domain requiring responsible design. Motivated by these concerns, this study synthesises evidence on limitations of LLM-based tutors and empirically investigates one critical issue: the accuracy, reliability, and temporal coherence of assessing learners' evolving knowledge over time. We compare a deep knowledge tracing (DKT) model with a widely used LLM, evaluated zero-shot and fine-tuned, using a large open-access dataset. Results show that DKT achieves the highest discrimination performance (AUC = 0.83) on next-step correctness prediction and consistently outperforms the LLM across settings. Although fine-tuning improves the LLM's AUC by approximately 8\% over the zero-shot baseline, it remains 6\% below DKT and produces higher early-sequence errors, where incorrect predictions are most harmful for adaptive support. Temporal analyses further reveal that DKT maintains stable, directionally correct mastery updates, whereas LLM variants exhibit substantial temporal weaknesses, including inconsistent and wrong-direction updates. These limitations persist despite the fine-tuned LLM requiring nearly 198 hours of high-compute training, far exceeding the computational demands of DKT. Our qualitative analysis of multi-skill mastery estimation further shows that, even after fine-tuning, the LLM produced inconsistent mastery trajectories, while DKT maintained smooth and coherent updates. Overall, the findings suggest that LLMs alone are unlikely to match the effectiveness of established intelligent tutoring systems, and that responsible tutoring requires hybrid frameworks that incorporate learner modelling.
- Open Access
- 23 October 2025
- Published by MDPI AG in Preprints.org
A Systematic Review of Deep Knowledge Tracing (2015-2025): Toward Responsible AI for Education
Background and Objectives: Tracking and adapting to learners’ evolving knowledge is essential for effective teaching. In digital learning, Deep Knowledge Tracing (DKT) employs deep neural networks to analyze sequential learner interactions, model their evolving knowledge, and predict skill mastery over time. While DKT is widely studied, its real-world adoption remains limited. This review examines DKT research from 2015–2025 through the lens of responsible AI principles, investigating modeling trends, evaluation practices, input features used for representing learner performance and context, strategies for mitigating data quality issues, assessment of sequential stability (consistency of knowledge estimates over time), and interpretability for educators. Methods: Following PRISMA guidelines, five major scholarly databases (Web of Science, Scopus, ScienceDirect, ACM Digital Library, IEEE Xplore) and Google Scholar were searched, yielding 1,047 peer-reviewed articles. After two rounds of screening and a quality appraisal focused on methodological rigor, 84 studies were included in the final synthesis. Results: Graph-based architectures were most common (26.2%), followed by Hybrid/Meta (23.8%) and Attentive models (17.9%). ASSIST datasets were used in 82.1% of studies, and 90.5% predominantly used Area Under the Curve (AUC) for evaluation. A wide variety of input features were used, ranging from basic question–answer pairs and knowledge concepts to time-based metrics, difficulty levels, behavioral indicators, and learning resource interactions. Approaches to address data quality challenges appeared in 44.0% of studies. Only 3.6% quantitatively assessed sequential stability of predictions. Interpretability techniques—designed to make predictions understandable to educators—were present in 11.9% of studies. Conclusions: Current DKT models often overlook responsible AI principles, including robust handling of data quality issues, assessment of sequential stability of predictions, and interpretability of predictions. As AI regulatory frameworks increasingly mandate trustworthy and interpretable AI in education, future research should prioritize these principles for practical and responsible deployment.
- 23 October 2025
- Published by MDPI AG in Preprints.org
Open AccessA Systematic Review of Deep Knowledge Tracing (2015-2025): Toward Responsible AI for Education
Background and Objectives: Tracking and adapting to learners’ evolving knowledge is essential for effective teaching. In digital learning, Deep Knowledge Tracing (DKT) employs deep neural networks to analyze sequential learner interactions, model their evolving knowledge, and predict skill mastery over time. While DKT is widely studied, its real-world adoption remains limited. This review examines DKT research from 2015–2025 through the lens of responsible AI principles, investigating modeling trends, evaluation practices, input features used for representing learner performance and context, strategies for mitigating data quality issues, assessment of sequential stability (consistency of knowledge estimates over time), and interpretability for educators. Methods: Following PRISMA guidelines, five major scholarly databases (Web of Science, Scopus, ScienceDirect, ACM Digital Library, IEEE Xplore) and Google Scholar were searched, yielding 1,047 peer-reviewed articles. After two rounds of screening and a quality appraisal focused on methodological rigor, 84 studies were included in the final synthesis. Results: Graph-based architectures were most common (26.2%), followed by Hybrid/Meta (23.8%) and Attentive models (17.9%). ASSIST datasets were used in 82.1% of studies, and 90.5% predominantly used Area Under the Curve (AUC) for evaluation. A wide variety of input features were used, ranging from basic question–answer pairs and knowledge concepts to time-based metrics, difficulty levels, behavioral indicators, and learning resource interactions. Approaches to address data quality challenges appeared in 44.0% of studies. Only 3.6% quantitatively assessed sequential stability of predictions. Interpretability techniques—designed to make predictions understandable to educators—were present in 11.9% of studies. Conclusions: Current DKT models often overlook responsible AI principles, including robust handling of data quality issues, assessment of sequential stability of predictions, and interpretability of predictions. As AI regulatory frameworks increasingly mandate trustworthy and interpretable AI in education, future research should prioritize these principles for practical and responsible deployment.
- Open Access
- 21 October 2025
- Published by IOS Press
- p. 2017-2024
Binarizing Physics-Inspired GNNs for Combinatorial Optimization
Physics-inspired graph neural networks (PI-GNNs) have been utilized as an efficient unsupervised framework for relaxing combinatorial optimization problems encoded through a specific graph structure and loss, reflecting dependencies between the problem’s variables. While the framework has yielded promising results in various combinatorial problems, we show that the performance of PI-GNNs systematically plummets with an increasing density of the combinatorial problem graphs. Our analysis reveals an interesting phase transition in the PI-GNNs’ training dynamics, associated with degenerate solutions for the denser problems, highlighting a discrepancy between the relaxed, real-valued model outputs and the binary-valued problem solutions. To address the discrepancy, we propose principled alternatives to the naive strategy used in PI-GNNs by building on insights from fuzzy logic and binarized neural networks. Our experiments demonstrate that the portfolio of proposed methods significantly improves the performance of PI-GNNs in increasingly dense settings.
- 21 October 2025
- Published by IOS Press
- p. 2017-2024
Open AccessBinarizing Physics-Inspired GNNs for Combinatorial Optimization
Physics-inspired graph neural networks (PI-GNNs) have been utilized as an efficient unsupervised framework for relaxing combinatorial optimization problems encoded through a specific graph structure and loss, reflecting dependencies between the problem’s variables. While the framework has yielded promising results in various combinatorial problems, we show that the performance of PI-GNNs systematically plummets with an increasing density of the combinatorial problem graphs. Our analysis reveals an interesting phase transition in the PI-GNNs’ training dynamics, associated with degenerate solutions for the denser problems, highlighting a discrepancy between the relaxed, real-valued model outputs and the binary-valued problem solutions. To address the discrepancy, we propose principled alternatives to the naive strategy used in PI-GNNs by building on insights from fuzzy logic and binarized neural networks. Our experiments demonstrate that the portfolio of proposed methods significantly improves the performance of PI-GNNs in increasingly dense settings.
- 23 September 2025
- Published by Springer Nature
- p. 438-456
ReDeLEx: A Framework for Relational Deep Learning Exploration
- 23 September 2025
- Published by Springer Nature
- p. 438-456
ReDeLEx: A Framework for Relational Deep Learning Exploration
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (12) , 20214-20228
Geometric Deep Learning for the Rubik’s Cube Group
The Rubik’s cube, a widely recognized combinatorial puzzle with an astronomically vast state space, has been the subject of various research experiments with neural networks used as heuristic estimators to navigate the state-space exploration. However, prior efforts have overlooked the intriguing symmetries inherent to this domain. Drawing on geometric deep learning principles, this article introduces a novel neural architecture that explicitly leverages these symmetries, grounded in a rigorous group-theoretical analysis. The design of the proposed symmetry-invariant model is then validated empirically through an innovative universal procedure for detecting model symmetry invariance. Finally, experimental results demonstrate that the symmetry-aware neural architecture exhibits enhanced generalization and problem-solving efficacy compared with the state of the art.
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (12) , 20214-20228
Geometric Deep Learning for the Rubik’s Cube Group
The Rubik’s cube, a widely recognized combinatorial puzzle with an astronomically vast state space, has been the subject of various research experiments with neural networks used as heuristic estimators to navigate the state-space exploration. However, prior efforts have overlooked the intriguing symmetries inherent to this domain. Drawing on geometric deep learning principles, this article introduces a novel neural architecture that explicitly leverages these symmetries, grounded in a rigorous group-theoretical analysis. The design of the proposed symmetry-invariant model is then validated empirically through an innovative universal procedure for detecting model symmetry invariance. Finally, experimental results demonstrate that the symmetry-aware neural architecture exhibits enhanced generalization and problem-solving efficacy compared with the state of the art.
- Open Access
- 22 July 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
Tabular Transformers Meet Relational Databases
Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts their extension to the more general case of relational databases . In this paper, we introduce a modular neural message-passing scheme that closely adheres to the formal relational model, enabling direct end-to-end learning of tabular transformers from database storage systems. We address the associated challenges of appropriate learning data representation and loading, which are critical in the database setting, and compare our approach against a number of representative models from various related fields across a significantly wide range of datasets. Our results then demonstrate superior performance of this newly proposed class of neural architectures.
- 22 July 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
Open AccessTabular Transformers Meet Relational Databases
Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts their extension to the more general case of relational databases . In this paper, we introduce a modular neural message-passing scheme that closely adheres to the formal relational model, enabling direct end-to-end learning of tabular transformers from database storage systems. We address the associated challenges of appropriate learning data representation and loading, which are critical in the database setting, and compare our approach against a number of representative models from various related fields across a significantly wide range of datasets. Our results then demonstrate superior performance of this newly proposed class of neural architectures.