Mehrnoush Shamsfard
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Mehrnoush Shamsfard has a total of 60 co-authors. Here are the top co-authors by number of shared publications:
- Mostafa Masumi
Computer Science and Engineering Department Shahid Beheshti University
3 shared publications - Pegah Safari
Affiliation Faculty of Computer Science and Engineering, Shahid Beheshti University, Tehran, Iran
3 shared publications - Seyed Soroush Majd
Computer Science and Engineering Department Shahid Beheshti University
3 shared publications - Alireza Talebpour
Faculty of Computer Science and Engineering, Shahid Beheshti University
2 shared publications - Farhan Farsi
Amirkabir University of Technology
2 shared publications - Hamid Beigy
Department of Computer Engineering Sharif University of Technology Tehran Iran
2 shared publications
Latest Publications
- Open Access
- 11 May 2026
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 21 (5) , e0329934
Extracting user profile via large language models and ontologies
Extracting user-specific profiles that include general personal information, such as hobbies, occupation, or age, is a valuable asset for systems like recommendation engines or personalized chatbots. Currently, most approaches moved toward exploiting the capabilities of large language models (LLMs) on rich languages while less-resourced languages still present significant opportunities for exploration and improvement. In this research, we present a multi-step approach for profile extraction from dialogue systems in Persian as a use case. Through an extensive set of experiments and analyses on various models, we show that LLMs struggle with these languages due to limited language-specific resources and complex linguistic structures. To address this, we propose a hybrid method that combines techniques such as slot filling, in-context learning, and ontology-based inference. Our final results demonstrate a significant improvement over current state-of-the-art models, including LLMs with few-shot examples and even their fine-tuned version. Our method achieves an F-score of 90.46, outperforming GPT-4o and Llama-3-70B by an absolute difference of 17.08 and 24.51 respectively. Our system can also detect inconsistencies in presented information in which our performance substantially exceeds the best performance of GPT-4o and Llama-3-70B with an accuracy of 92%. It is 21% absolutely better than GPT-4o and 47% better than Llama.
- 11 May 2026
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 21 (5) , e0329934
Open AccessExtracting user profile via large language models and ontologies
Extracting user-specific profiles that include general personal information, such as hobbies, occupation, or age, is a valuable asset for systems like recommendation engines or personalized chatbots. Currently, most approaches moved toward exploiting the capabilities of large language models (LLMs) on rich languages while less-resourced languages still present significant opportunities for exploration and improvement. In this research, we present a multi-step approach for profile extraction from dialogue systems in Persian as a use case. Through an extensive set of experiments and analyses on various models, we show that LLMs struggle with these languages due to limited language-specific resources and complex linguistic structures. To address this, we propose a hybrid method that combines techniques such as slot filling, in-context learning, and ontology-based inference. Our final results demonstrate a significant improvement over current state-of-the-art models, including LLMs with few-shot examples and even their fine-tuned version. Our method achieves an F-score of 90.46, outperforming GPT-4o and Llama-3-70B by an absolute difference of 17.08 and 24.51 respectively. Our system can also detect inconsistencies in presented information in which our performance substantially exceeds the best performance of GPT-4o and Llama-3-70B with an accuracy of 92%. It is 21% absolutely better than GPT-4o and 47% better than Llama.
- 26 December 2025
- Published by Springer Nature in SN Computer Science
- Vol. 7 (1) , 37
HmBlogs: A Comprehensive Corpus and Benchmarking Study for Persian Word Embedding and Language Modeling
- 26 December 2025
- Published by Springer Nature in SN Computer Science
- Vol. 7 (1) , 37
HmBlogs: A Comprehensive Corpus and Benchmarking Study for Persian Word Embedding and Language Modeling
- Open Access
- 1 October 2025
- Published by TIB Open Publishing in Open Conference Proceedings
- Vol. 6
SBU-NLP at LLMs4OL 2025 Tasks A, B, and C: Stage-Wise Ontology Construction Through LLMs Without any Training Procedure
Automated ontology construction is a challenging task that traditionally requires extensive domain expertise, data preprocessing, and resource-intensive model training. While learning-based methods with fine-tuning are common, they often suffer from high computational costs and limited generalizability across domains. This paper explores a fully automated approach that leverages powerful large language models (LLMs) through prompt engineering, eliminating the need for training or fine-tuning. We participated in the LLMs4OL 2025 shared task, which includes four subtasks: extracting ontological terms and types (Text2Onto), assigning generalized types to terms (Term Typing), discovering taxonomic relations (Taxonomy Discovery), and extracting non-taxonomic semantic relations (Non-Taxonomic Relation Extraction). Our team focused on the first three tasks by using stratified random sampling, simple random sampling, and chunking-based strategies to include training sets in the prompts without limitations imposed by context window sizes. This simple yet general approach has proven effective across these tasks, enabling high-quality ontology construction without additional annotations or training. Additionally, we show that pretrained sentence embedding models ranging from 0.1B to 1.5B parameters perform comparably to a simple F1 token overlap baseline in taxonomy discovery, suggesting that embedding-based methods may not always offer significant advantages. Our findings highlight that prompt-based strategies with modern LLMs enable efficient, scalable, and domain-independent ontology construction, providing a promising alternative to traditional, resource-heavy methods.
- 1 October 2025
- Published by TIB Open Publishing in Open Conference Proceedings
- Vol. 6
Open AccessSBU-NLP at LLMs4OL 2025 Tasks A, B, and C: Stage-Wise Ontology Construction Through LLMs Without any Training Procedure
Automated ontology construction is a challenging task that traditionally requires extensive domain expertise, data preprocessing, and resource-intensive model training. While learning-based methods with fine-tuning are common, they often suffer from high computational costs and limited generalizability across domains. This paper explores a fully automated approach that leverages powerful large language models (LLMs) through prompt engineering, eliminating the need for training or fine-tuning. We participated in the LLMs4OL 2025 shared task, which includes four subtasks: extracting ontological terms and types (Text2Onto), assigning generalized types to terms (Term Typing), discovering taxonomic relations (Taxonomy Discovery), and extracting non-taxonomic semantic relations (Non-Taxonomic Relation Extraction). Our team focused on the first three tasks by using stratified random sampling, simple random sampling, and chunking-based strategies to include training sets in the prompts without limitations imposed by context window sizes. This simple yet general approach has proven effective across these tasks, enabling high-quality ontology construction without additional annotations or training. Additionally, we show that pretrained sentence embedding models ranging from 0.1B to 1.5B parameters perform comparably to a simple F1 token overlap baseline in taxonomy discovery, suggesting that embedding-based methods may not always offer significant advantages. Our findings highlight that prompt-based strategies with modern LLMs enable efficient, scalable, and domain-independent ontology construction, providing a promising alternative to traditional, resource-heavy methods.
- Open Access
FarsEval-PKBETS: A new diverse benchmark for evaluating Persian large language models
Research on evaluating and analyzing large language models (LLMs) has been extensive for resource-rich languages such as English, yet their performance in languages such as Persian has received considerably less attention. This paper introduces FarsEval-PKBETS benchmark, a subset of FarsEval project for evaluating large language models in Persian. This benchmark consists of 4000 questions and answers in various formats, including multiple choice, short answer and descriptive responses. It covers a wide range of domains and tasks,including medicine, law, religion, Persian language, encyclopedic knowledge, human preferences, social knowledge, ethics and bias, text generation, and respecting others' rights. This bechmark incorporates linguistics, cultural, and local considerations relevant to the Persian language and Iran. To ensure the questions are challenging for current LLMs, three models -- Llama3-70B, PersianMind, and Dorna -- were evaluated using this benchmark. Their average accuracy was below 50%, meaning they provided fully correct answers to fewer than half of the questions. These results indicate that current language models are still far from being able to solve this benchmark
Open AccessFarsEval-PKBETS: A new diverse benchmark for evaluating Persian large language models
Research on evaluating and analyzing large language models (LLMs) has been extensive for resource-rich languages such as English, yet their performance in languages such as Persian has received considerably less attention. This paper introduces FarsEval-PKBETS benchmark, a subset of FarsEval project for evaluating large language models in Persian. This benchmark consists of 4000 questions and answers in various formats, including multiple choice, short answer and descriptive responses. It covers a wide range of domains and tasks,including medicine, law, religion, Persian language, encyclopedic knowledge, human preferences, social knowledge, ethics and bias, text generation, and respecting others' rights. This bechmark incorporates linguistics, cultural, and local considerations relevant to the Persian language and Iran. To ensure the questions are challenging for current LLMs, three models -- Llama3-70B, PersianMind, and Dorna -- were evaluated using this benchmark. Their average accuracy was below 50%, meaning they provided fully correct answers to fewer than half of the questions. These results indicate that current language models are still far from being able to solve this benchmark
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 20053-20084
A Graph-Theoretical Framework for Analyzing the Behavior of Causal Language Models
Rashin Rahnamoun, Mehrnoush Shamsfard. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 20053-20084
Open AccessA Graph-Theoretical Framework for Analyzing the Behavior of Causal Language Models
Rashin Rahnamoun, Mehrnoush Shamsfard. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 85-96
FaBERT: Pre-training BERT on Persian Blogs
Mostafa Masumi, Seyed Soroush Majd, Mehrnoush Shamsfard, Hamid Beigy. Proceedings of the Tenth Workshop on Noisy and User-generated Text. 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 85-96
Open AccessFaBERT: Pre-training BERT on Persian Blogs
Mostafa Masumi, Seyed Soroush Majd, Mehrnoush Shamsfard, Hamid Beigy. Proceedings of the Tenth Workshop on Noisy and User-generated Text. 2025.
- Open Access
- 1 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 160741-160760
A Novel Open-Domain Question Answering System on Curated and Extracted Knowledge Bases With Consideration of Confidence Scores in Existing Triples
In this paper, we present a novel approach to improving Open-Domain Knowledge Base Question Answering by leveraging both curated (Freebase) and extracted (Reverb) knowledge bases. Our hybrid system combines Span Detection using BERT (SD-BERT) for precise entity and relation span detection with a Term Frequency–Inverse Document Frequency (TF-IDF) retrieval model enhanced by function scoring, achieving a balance between efficiency and accuracy. Additionally, we explore using Contextual Late Interaction over BERT (ColBERTv2), a scalable retrieval model optimized for token-level interaction, to handle more complex queries involving multiple entities and relations. Our system demonstrates significant improvements in handling large-scale, hybrid datasets such as ReverbSimpleQuestions and SimpleQuestions, providing both high accuracy and scalability. Through extensive evaluation, we achieved a Hit@1 of 67.87% using SD-BERT + TF-IDF with confidence scoring, outperforming several benchmark systems in fact-based query retrieval while maintaining real-time performance and the best performance, 83.63%, was achieved on the extracted knowledge base using the ColBERTv2 method.
- 1 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 160741-160760
Open AccessA Novel Open-Domain Question Answering System on Curated and Extracted Knowledge Bases With Consideration of Confidence Scores in Existing Triples
In this paper, we present a novel approach to improving Open-Domain Knowledge Base Question Answering by leveraging both curated (Freebase) and extracted (Reverb) knowledge bases. Our hybrid system combines Span Detection using BERT (SD-BERT) for precise entity and relation span detection with a Term Frequency–Inverse Document Frequency (TF-IDF) retrieval model enhanced by function scoring, achieving a balance between efficiency and accuracy. Additionally, we explore using Contextual Late Interaction over BERT (ColBERTv2), a scalable retrieval model optimized for token-level interaction, to handle more complex queries involving multiple entities and relations. Our system demonstrates significant improvements in handling large-scale, hybrid datasets such as ReverbSimpleQuestions and SimpleQuestions, providing both high accuracy and scalability. Through extensive evaluation, we achieved a Hit@1 of 67.87% using SD-BERT + TF-IDF with confidence scoring, outperforming several benchmark systems in fact-based query retrieval while maintaining real-time performance and the best performance, 83.63%, was achieved on the extracted knowledge base using the ColBERTv2 method.
- Open Access
- 2 October 2024
- Published by TIB Open Publishing in Open Conference Proceedings
- Vol. 4, 103-111
SKH-NLP at LLMs4OL 2024 Task B: Taxonomy Discovery in Ontologies Using BERT and LLaMA 3
Taxonomy discovery in ontologies refers to extracting the parent class from the child class. By modeling this task as a classification problem, we addressed it using two different approaches. The first approach involved fine-tuning the “BERT-Large” model with various prompts and using it in a classification system. In the second approach, we utilized the “LLaMA 3 70B” model, experimenting with different prompts and modifying them to achieve the best results. Additionally, we evaluated the correctness of the answers using substring and Levenshtein distance functions. The results indicate that, with appropriate fine-tuning, the BERT model can achieve performance levels comparable to those of more recent and significantly larger language models, such as LLaMA 3 70B. However, with appropriate prompts, LLaMA 3 70B performs slightly better than BERT, highlighting the importance of prompt quality. Ultimately, further experiments on different settings for fine-tuning BERT, few-shot learning, and using knowledge graphs for validating the model's answers for LLaMA are recommended to improve the results. Additionally, testing other models and examining the results of various encoder-based and decoder-based models can be employed.
- 2 October 2024
- Published by TIB Open Publishing in Open Conference Proceedings
- Vol. 4, 103-111
Open AccessSKH-NLP at LLMs4OL 2024 Task B: Taxonomy Discovery in Ontologies Using BERT and LLaMA 3
Taxonomy discovery in ontologies refers to extracting the parent class from the child class. By modeling this task as a classification problem, we addressed it using two different approaches. The first approach involved fine-tuning the “BERT-Large” model with various prompts and using it in a classification system. In the second approach, we utilized the “LLaMA 3 70B” model, experimenting with different prompts and modifying them to achieve the best results. Additionally, we evaluated the correctness of the answers using substring and Levenshtein distance functions. The results indicate that, with appropriate fine-tuning, the BERT model can achieve performance levels comparable to those of more recent and significantly larger language models, such as LLaMA 3 70B. However, with appropriate prompts, LLaMA 3 70B performs slightly better than BERT, highlighting the importance of prompt quality. Ultimately, further experiments on different settings for fine-tuning BERT, few-shot learning, and using knowledge graphs for validating the model's answers for LLaMA are recommended to improve the results. Additionally, testing other models and examining the results of various encoder-based and decoder-based models can be employed.
- Open Access
- 19 August 2024
- Published by Springer Nature in Research Square
Extracting explicit, implicit and contradictory information from chats in a Persian NLU engine
In this research, we present a Natural Language Understanding (NLU) component of an open-domain chatbot designed to extract explicit and implicit personal information from user dialogues, such as name, age, gender, education, occupation, marital status, residence, etc. The NLU component comprises two main units: explicit IE and implicit IE. The explicit IE unit detects the main topic (intent) of the utterance and extracts explicitly stated information to fill the values of predefined slots. It utilizes a custom joint training model leveraging the pretrained language model of XLM-RoBERTa and incorporating the previous utterance as an input feature. Training on a dataset of 6,652 utterances, it yields an intent accuracy of 91.22%, slot-F1 of 88.07% and exact match of 83.40%. The implicit IE unit is built upon an ontology containing common-sense knowledge, alongside an LLM-based inference engine. It infers the remaining slots based on explicit information or complementary data in the utterance. Our NLU component can even detect semantic conflicts in the dialogue and assessing the full pipeline from this aspect shows 92% accuracy which is 21% higher than GPT-4o.
- 19 August 2024
- Published by Springer Nature in Research Square
Open AccessExtracting explicit, implicit and contradictory information from chats in a Persian NLU engine
In this research, we present a Natural Language Understanding (NLU) component of an open-domain chatbot designed to extract explicit and implicit personal information from user dialogues, such as name, age, gender, education, occupation, marital status, residence, etc. The NLU component comprises two main units: explicit IE and implicit IE. The explicit IE unit detects the main topic (intent) of the utterance and extracts explicitly stated information to fill the values of predefined slots. It utilizes a custom joint training model leveraging the pretrained language model of XLM-RoBERTa and incorporating the previous utterance as an input feature. Training on a dataset of 6,652 utterances, it yields an intent accuracy of 91.22%, slot-F1 of 88.07% and exact match of 83.40%. The implicit IE unit is built upon an ontology containing common-sense knowledge, alongside an LLM-based inference engine. It infers the remaining slots based on explicit information or complementary data in the utterance. Our NLU component can even detect semantic conflicts in the dialogue and assessing the full pipeline from this aspect shows 92% accuracy which is 21% higher than GPT-4o.
- Open Access
RFBES at SemEval-2024 Task 8: Investigating Syntactic and Semantic Features for Distinguishing AI-Generated and Human-Written Texts
Nowadays, the usage of Large Language Models (LLMs) has increased, and LLMs have been used to generate texts in different languages and for different tasks. Additionally, due to the participation of remarkable companies such as Google and OpenAI, LLMs are now more accessible, and people can easily use them. However, an important issue is how we can detect AI-generated texts from human-written ones. In this article, we have investigated the problem of AI-generated text detection from two different aspects: semantics and syntax. Finally, we presented an AI model that can distinguish AI-generated texts from human-written ones with high accuracy on both multilingual and monolingual tasks using the M4 dataset. According to our results, using a semantic approach would be more helpful for detection. However, there is a lot of room for improvement in the syntactic approach, and it would be a good approach for future work.
Open AccessRFBES at SemEval-2024 Task 8: Investigating Syntactic and Semantic Features for Distinguishing AI-Generated and Human-Written Texts
Nowadays, the usage of Large Language Models (LLMs) has increased, and LLMs have been used to generate texts in different languages and for different tasks. Additionally, due to the participation of remarkable companies such as Google and OpenAI, LLMs are now more accessible, and people can easily use them. However, an important issue is how we can detect AI-generated texts from human-written ones. In this article, we have investigated the problem of AI-generated text detection from two different aspects: semantics and syntax. Finally, we presented an AI model that can distinguish AI-generated texts from human-written ones with high accuracy on both multilingual and monolingual tasks using the M4 dataset. According to our results, using a semantic approach would be more helpful for detection. However, there is a lot of room for improvement in the syntactic approach, and it would be a good approach for future work.