Leonardo Ranaldi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Leonardo Ranaldi has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Fabio Massimo Zanzotto
University of Rome “Tor Vergata”, Rome, Italy
7 shared publications - Francesca Fallucchi
Guglielmo Marconi University
5 shared publications - Andrea Santilli
Sapienza University of Rome
1 shared publication - Dario Onorati
Department of Enterprise Engineering, University of Rome Tor Vergata, 00133 Roma, Italy
1 shared publication Elena Sofia Ruzzetti
Department of Enterprise Engineering ART Group University of Rome Tor Vergata Viale del Politecnico, 1 00133 Rome Italy
1 shared publication- Elena Sofia Ruzzetti
Human-centric ART, University of Rome Tor Vergata, Rome, Italy
1 shared publication
Latest Publications
- Open Access
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 9445-9448
Neural Networks meet Explicit Knowledge Representation: Towards Mechanistic Interpretability and Neuro-symbolic Modeling by-design
Transformer-based Large Language Models (Tb-LLMs) demonstrate extraordinary capabilities and, thus, change the approach of conducting research even in Knowledge Representation. Building on existing research that neural networks (NNs) on discrete symbols operate as holistic reasoners over ''classical'' symbolic approaches, this tutorial aims to present the dominant paradigm of Tb-LLMs according to the idea that these models can be understood with neuro-symbolic, ontology-grounded approaches and controlled through mechanistic interpretable or explainable KG reasoning methods. Offering a profound understanding of the inner workings of these models, the tutorial will help practitioners build novel, transparent architectures to model symbolic knowledge in NN.
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 9445-9448
Open AccessNeural Networks meet Explicit Knowledge Representation: Towards Mechanistic Interpretability and Neuro-symbolic Modeling by-design
Transformer-based Large Language Models (Tb-LLMs) demonstrate extraordinary capabilities and, thus, change the approach of conducting research even in Knowledge Representation. Building on existing research that neural networks (NNs) on discrete symbols operate as holistic reasoners over ''classical'' symbolic approaches, this tutorial aims to present the dominant paradigm of Tb-LLMs according to the idea that these models can be understood with neuro-symbolic, ontology-grounded approaches and controlled through mechanistic interpretable or explainable KG reasoning methods. Offering a profound understanding of the inner workings of these models, the tutorial will help practitioners build novel, transparent architectures to model symbolic knowledge in NN.
- Open Access
- 23 July 2025
- Published by MDPI AG in Big Data and Cognitive Computing
- Vol. 9 (8) , 193
Survey on the Role of Mechanistic Interpretability in Generative AI
The rapid advancement of artificial intelligence (AI) and machine learning has revolutionised how systems process information, make decisions, and adapt to dynamic environments. AI-driven approaches have significantly enhanced efficiency and problem-solving capabilities across various domains, from automated decision-making to knowledge representation and predictive modelling. These developments have led to the emergence of increasingly sophisticated models capable of learning patterns, reasoning over complex data structures, and generalising across tasks. As AI systems become more deeply integrated into networked infrastructures and the Internet of Things (IoT), their ability to process and interpret data in real-time is essential for optimising intelligent communication networks, distributed decision making, and autonomous IoT systems. However, despite these achievements, the internal mechanisms that drive LLMs’ reasoning and generalisation capabilities remain largely unexplored. This lack of transparency, compounded by challenges such as hallucinations, adversarial perturbations, and misaligned human expectations, raises concerns about their safe and beneficial deployment. Understanding the underlying principles governing AI models is crucial for their integration into intelligent network systems, automated decision-making processes, and secure digital infrastructures. This paper provides a comprehensive analysis of explainability approaches aimed at uncovering the fundamental mechanisms of LLMs. We investigate the strategic components contributing to their generalisation abilities, focusing on methods to quantify acquired knowledge and assess its representation within model parameters. Specifically, we examine mechanistic interpretability, probing techniques, and representation engineering as tools to decipher how knowledge is structured, encoded, and retrieved in AI systems. Furthermore, by adopting a mechanistic perspective, we analyse emergent phenomena within training dynamics, particularly memorisation and generalisation, which also play a crucial role in broader AI-driven systems, including adaptive network intelligence, edge computing, and real-time decision-making architectures. Understanding these principles is crucial for bridging the gap between black-box AI models and practical, explainable AI applications, thereby ensuring trust, robustness, and efficiency in language-based and general AI systems.
- 23 July 2025
- Published by MDPI AG in Big Data and Cognitive Computing
- Vol. 9 (8) , 193
Open AccessSurvey on the Role of Mechanistic Interpretability in Generative AI
The rapid advancement of artificial intelligence (AI) and machine learning has revolutionised how systems process information, make decisions, and adapt to dynamic environments. AI-driven approaches have significantly enhanced efficiency and problem-solving capabilities across various domains, from automated decision-making to knowledge representation and predictive modelling. These developments have led to the emergence of increasingly sophisticated models capable of learning patterns, reasoning over complex data structures, and generalising across tasks. As AI systems become more deeply integrated into networked infrastructures and the Internet of Things (IoT), their ability to process and interpret data in real-time is essential for optimising intelligent communication networks, distributed decision making, and autonomous IoT systems. However, despite these achievements, the internal mechanisms that drive LLMs’ reasoning and generalisation capabilities remain largely unexplored. This lack of transparency, compounded by challenges such as hallucinations, adversarial perturbations, and misaligned human expectations, raises concerns about their safe and beneficial deployment. Understanding the underlying principles governing AI models is crucial for their integration into intelligent network systems, automated decision-making processes, and secure digital infrastructures. This paper provides a comprehensive analysis of explainability approaches aimed at uncovering the fundamental mechanisms of LLMs. We investigate the strategic components contributing to their generalisation abilities, focusing on methods to quantify acquired knowledge and assess its representation within model parameters. Specifically, we examine mechanistic interpretability, probing techniques, and representation engineering as tools to decipher how knowledge is structured, encoded, and retrieved in AI systems. Furthermore, by adopting a mechanistic perspective, we analyse emergent phenomena within training dynamics, particularly memorisation and generalisation, which also play a crucial role in broader AI-driven systems, including adaptive network intelligence, edge computing, and real-time decision-making architectures. Understanding these principles is crucial for bridging the gap between black-box AI models and practical, explainable AI applications, thereby ensuring trust, robustness, and efficiency in language-based and general AI systems.
- Open Access
- 13 June 2025
- Published by MDPI AG in Information
- Vol. 16 (6) , 493
Animate, or Inanimate, That Is the Question for Large Language Models
The cognitive core of human beings is closely connected to the concept of animacy, which significantly influences their memory, vision, and complex language comprehension. While animacy is reflected in language through subtle constraints on verbs and adjectives, it is also acquired and honed through non-linguistic experiences. In the same vein, we suggest that the limited capacity of LLMs to grasp natural language, particularly in relation to animacy, stems from the fact that these models are trained solely on textual data. Hence, the question this paper aims to answer arises: Can LLMs, in their digital wisdom, process animacy in a similar way to what humans would do? We then propose a systematic analysis via prompting approaches. In particular, we probe different LLMs using controlled lexical contrasts (animate vs. inanimate nouns) and narrative contexts in which typically inanimate entities behave as animate. Results reveal that, although LLMs have been trained predominantly on textual data, they exhibit human-like behavior when faced with typical animate and inanimate entities in alignment with earlier studies, specifically on seven LLMs selected from three major families—OpenAI (GPT-3.5, GPT-4), Meta (Llama2 7B, 13B, 70B), and Mistral (Mistral-7B, Mixtral). GPT models generally achieve the most consistent and human-like performance, and in some tasks, such as sentence plausibility and acceptability judgments, even surpass human baselines. Moreover, although to a lesser degree, the other models also assume comparable results. Hence, LLMs can adapt to understand unconventional situations by recognising oddities as animated without needing to interface with unspoken cognitive triggers humans rely on to break down animations.
- 13 June 2025
- Published by MDPI AG in Information
- Vol. 16 (6) , 493
Open AccessAnimate, or Inanimate, That Is the Question for Large Language Models
The cognitive core of human beings is closely connected to the concept of animacy, which significantly influences their memory, vision, and complex language comprehension. While animacy is reflected in language through subtle constraints on verbs and adjectives, it is also acquired and honed through non-linguistic experiences. In the same vein, we suggest that the limited capacity of LLMs to grasp natural language, particularly in relation to animacy, stems from the fact that these models are trained solely on textual data. Hence, the question this paper aims to answer arises: Can LLMs, in their digital wisdom, process animacy in a similar way to what humans would do? We then propose a systematic analysis via prompting approaches. In particular, we probe different LLMs using controlled lexical contrasts (animate vs. inanimate nouns) and narrative contexts in which typically inanimate entities behave as animate. Results reveal that, although LLMs have been trained predominantly on textual data, they exhibit human-like behavior when faced with typical animate and inanimate entities in alignment with earlier studies, specifically on seven LLMs selected from three major families—OpenAI (GPT-3.5, GPT-4), Meta (Llama2 7B, 13B, 70B), and Mistral (Mistral-7B, Mixtral). GPT models generally achieve the most consistent and human-like performance, and in some tasks, such as sentence plausibility and acceptability judgments, even surpass human baselines. Moreover, although to a lesser degree, the other models also assume comparable results. Hence, LLMs can adapt to understand unconventional situations by recognising oddities as animated without needing to interface with unspoken cognitive triggers humans rely on to break down animations.
- Open Access
PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models
Pre-trained Language Models such as BERT are impressive machines with the ability to memorize, possibly generalized learning examples. We present here a small, focused contribution to the analysis of the interplay between memorization and performance of BERT in downstream tasks. We propose PreCog, a measure for evaluating memorization from pre-training, and we analyze its correlation with the BERT's performance. Our experiments show that highly memorized examples are better classified, suggesting memorization is an essential key to success for BERT.
Open AccessPreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models
Pre-trained Language Models such as BERT are impressive machines with the ability to memorize, possibly generalized learning examples. We present here a small, focused contribution to the analysis of the interplay between memorization and performance of BERT in downstream tasks. We propose PreCog, a measure for evaluating memorization from pre-training, and we analyze its correlation with the BERT's performance. Our experiments show that highly memorized examples are better classified, suggesting memorization is an essential key to success for BERT.
- Top Cited PapersOpen Access
- 4 January 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (2) , 677
Knowing Knowledge: Epistemological Study of Knowledge in Transformers
Statistical learners are leading towards auto-epistemic logic, but is it the right way to progress in artificial intelligence (AI)? Ways to discover AI fit the senses and the intellect. The structure of symbols–the operations by which the intellectual solution is realized–and the search for strategic reference points evoke essential issues in the analysis of AI. Studying how knowledge can be represented through methods of theoretical generalization and empirical observation is only the latest step in a long process of evolution. In this paper, we try to outline the origin of knowledge and how modern artificial minds have inherited it.
- 4 January 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (2) , 677
Top Cited PapersOpen AccessKnowing Knowledge: Epistemological Study of Knowledge in Transformers
Statistical learners are leading towards auto-epistemic logic, but is it the right way to progress in artificial intelligence (AI)? Ways to discover AI fit the senses and the intellect. The structure of symbols–the operations by which the intellectual solution is realized–and the search for strategic reference points evoke essential issues in the analysis of AI. Studying how knowledge can be represented through methods of theoretical generalization and empirical observation is only the latest step in a long process of evolution. In this paper, we try to outline the origin of knowledge and how modern artificial minds have inherited it.
- Open Access
- 2 November 2022
- Published by MDPI AG in Information
- Vol. 13 (11) , 524
CryptoNet: Using Auto-Regressive Multi-Layer Artificial Neural Networks to Predict Financial Time Series
When analyzing a financial asset, it is essential to study the trend of its time series. It is also necessary to examine its evolution and activity over time to statistically analyze its possible future behavior. Both retail and institutional investors base their trading strategies on these analyses. One of the most used techniques to study financial time series is to analyze its dynamic structure using auto-regressive models, simple moving average models (SMA), and mixed auto-regressive moving average models (ARMA). These techniques, unfortunately, do not always provide appreciable results both at a statistical level and as the Risk-Reward Ratio (RRR); above all, each system has its pros and cons. In this paper, we present CryptoNet; this system is based on the time series extraction exploiting the vast potential of artificial intelligence (AI) and machine learning (ML). Specifically, we focused on time series trends extraction by developing an artificial neural network, trained and tested on two famous crypto-currencies: Bitcoinand Ether. CryptoNet learning algorithm improved the classic linear regression model up to 31% of MAE (mean absolute error). Results from this work should encourage machine learning techniques in sectors classically reluctant to adopt non-standard approaches.
- 2 November 2022
- Published by MDPI AG in Information
- Vol. 13 (11) , 524
Open AccessCryptoNet: Using Auto-Regressive Multi-Layer Artificial Neural Networks to Predict Financial Time Series
When analyzing a financial asset, it is essential to study the trend of its time series. It is also necessary to examine its evolution and activity over time to statistically analyze its possible future behavior. Both retail and institutional investors base their trading strategies on these analyses. One of the most used techniques to study financial time series is to analyze its dynamic structure using auto-regressive models, simple moving average models (SMA), and mixed auto-regressive moving average models (ARMA). These techniques, unfortunately, do not always provide appreciable results both at a statistical level and as the Risk-Reward Ratio (RRR); above all, each system has its pros and cons. In this paper, we present CryptoNet; this system is based on the time series extraction exploiting the vast potential of artificial intelligence (AI) and machine learning (ML). Specifically, we focused on time series trends extraction by developing an artificial neural network, trained and tested on two famous crypto-currencies: Bitcoinand Ether. CryptoNet learning algorithm improved the classic linear regression model up to 31% of MAE (mean absolute error). Results from this work should encourage machine learning techniques in sectors classically reluctant to adopt non-standard approaches.
- Open Access
- 14 September 2022
- Published by MDPI AG in Information
- Vol. 13 (9) , 435
Shedding Light on the Dark Web: Authorship Attribution in Radical Forums
Online users tend to hide their real identities by adopting different names on the Internet. On Facebook or LinkedIn, for example, people usually appear with their real names. On other standard websites, such as forums, people often use nicknames to protect their real identities. Aliases are used when users are trying to protect their anonymity. This can be a challenge to law enforcement trying to identify users who often change nicknames. In unmonitored contexts, such as the dark web, users expect strong identity protection. Thus, without censorship, these users may create parallel social networks where they can engage in potentially malicious activities that could pose security threats. In this paper, we propose a solution to the need to recognize people who anonymize themselves behind nicknames—the authorship attribution (AA) task—in the challenging context of the dark web: specifically, an English-language Islamic forum dedicated to discussions of issues related to the Islamic world and Islam, in which members of radical Islamic groups are present. We provide extensive analysis by testing models based on transformers, styles, and syntactic features. Downstream of the experiments, we show how models that analyze syntax and style perform better than pre-trained universal language models.
- 14 September 2022
- Published by MDPI AG in Information
- Vol. 13 (9) , 435
Open AccessShedding Light on the Dark Web: Authorship Attribution in Radical Forums
Online users tend to hide their real identities by adopting different names on the Internet. On Facebook or LinkedIn, for example, people usually appear with their real names. On other standard websites, such as forums, people often use nicknames to protect their real identities. Aliases are used when users are trying to protect their anonymity. This can be a challenge to law enforcement trying to identify users who often change nicknames. In unmonitored contexts, such as the dark web, users expect strong identity protection. Thus, without censorship, these users may create parallel social networks where they can engage in potentially malicious activities that could pose security threats. In this paper, we propose a solution to the need to recognize people who anonymize themselves behind nicknames—the authorship attribution (AA) task—in the challenging context of the dark web: specifically, an English-language Islamic forum dedicated to discussions of issues related to the Islamic world and Islam, in which members of radical Islamic groups are present. We provide extensive analysis by testing models based on transformers, styles, and syntactic features. Downstream of the experiments, we show how models that analyze syntax and style perform better than pre-trained universal language models.
- 1 April 2022
- Published by Springer Nature
- p. 139-147
KERMITviz: Visualizing Neural Network Activations on Syntactic Trees
- 1 April 2022
- Published by Springer Nature
- p. 139-147
KERMITviz: Visualizing Neural Network Activations on Syntactic Trees
- Open Access
- 24 December 2021
- Published by MDPI AG in Future Internet
- Vol. 14 (1) , 10
Dis-Cover AI Minds to Preserve Human Knowledge
Modern AI technologies make use of statistical learners that lead to self-empiricist logic, which, unlike human minds, use learned non-symbolic representations. Nevertheless, it seems that it is not the right way to progress in AI. The structure of symbols—the operations by which the intellectual solution is realized—and the search for strategic reference points evoke important issues in the analysis of AI. Studying how knowledge can be represented through methods of theoretical generalization and empirical observation is only the latest step in a long process of evolution. For many years, humans, seeing language as innate, have carried out symbolic theories. Everything seems to have skipped ahead with the advent of Machine Learning. In this paper, after a long analysis of history, the rule-based and the learning-based vision, we would investigate the syntax as possible meeting point between the different learning theories. Finally, we propose a new vision of knowledge in AI models based on a combination of rules, learning, and human knowledge.
- 24 December 2021
- Published by MDPI AG in Future Internet
- Vol. 14 (1) , 10
Open AccessDis-Cover AI Minds to Preserve Human Knowledge
Modern AI technologies make use of statistical learners that lead to self-empiricist logic, which, unlike human minds, use learned non-symbolic representations. Nevertheless, it seems that it is not the right way to progress in AI. The structure of symbols—the operations by which the intellectual solution is realized—and the search for strategic reference points evoke important issues in the analysis of AI. Studying how knowledge can be represented through methods of theoretical generalization and empirical observation is only the latest step in a long process of evolution. For many years, humans, seeing language as innate, have carried out symbolic theories. Everything seems to have skipped ahead with the advent of Machine Learning. In this paper, after a long analysis of history, the rule-based and the learning-based vision, we would investigate the syntax as possible meeting point between the different learning theories. Finally, we propose a new vision of knowledge in AI models based on a combination of rules, learning, and human knowledge.
- Open Access
- 3 September 2021
- Published by MDPI AG in Preprints.org
Discover AI Knowledge to Preserve Cultural Heritage
Documenting cultural heritage by using artificial intelligence (AI) is crucial for preserving the memory of the past and a key point for future knowledge. However, modern AI technologies make use of statistical learners that lead to self-empiricist logic, which, unlike human minds, use learned non-symbolic representations. Nevertheless, it seems that it is not the right way to progress in AI. If we want to rely on AI for these tasks, it is essential to understand what lies behind these models. Among the ways to discover AI there are the senses and the intellect. We could consider AI as an intelligence. Intelligence has an essence, but we do not know whether it can be considered “something” or “someone”. Important issues in the analysis of AI concern the structure of symbols -operations with which the intellectual solution is carried out- and the search for strategic reference points, aspiring to create models with human-like intelligence. For many years, humans, seeing language as innate, have carried out symbolic theories. Everything seems to have skipped with the advent of Machine Learning. In this paper, after a long analysis of history, the rule-based and the learning-based vision, we propose KERMIT as a unit of investigation for a possible meeting point between the different learning theories. Finally, we propose a new vision of knowledge in AI models based on a combination of rules, learning and human knowledge.
- 3 September 2021
- Published by MDPI AG in Preprints.org
Open AccessDiscover AI Knowledge to Preserve Cultural Heritage
Documenting cultural heritage by using artificial intelligence (AI) is crucial for preserving the memory of the past and a key point for future knowledge. However, modern AI technologies make use of statistical learners that lead to self-empiricist logic, which, unlike human minds, use learned non-symbolic representations. Nevertheless, it seems that it is not the right way to progress in AI. If we want to rely on AI for these tasks, it is essential to understand what lies behind these models. Among the ways to discover AI there are the senses and the intellect. We could consider AI as an intelligence. Intelligence has an essence, but we do not know whether it can be considered “something” or “someone”. Important issues in the analysis of AI concern the structure of symbols -operations with which the intellectual solution is carried out- and the search for strategic reference points, aspiring to create models with human-like intelligence. For many years, humans, seeing language as innate, have carried out symbolic theories. Everything seems to have skipped with the advent of Machine Learning. In this paper, after a long analysis of history, the rule-based and the learning-based vision, we propose KERMIT as a unit of investigation for a possible meeting point between the different learning theories. Finally, we propose a new vision of knowledge in AI models based on a combination of rules, learning and human knowledge.