Sentential Cross-lingual Paraphrase Detection for English-Urdu Language Pair
Authors:
- 2 March 2025
- journal article
- Published by SAGE Publications in The European Journal on Artificial Intelligence
- Vol. 38 (3) , 309-329
Abstract
Due to vast digital data collections and paraphrasing tools, researchers have shown growing interest in Cross-lingual Paraphrase Detection (CLPD). Open-access data and tools make paraphrasing easier and detection more challenging. Translation tools further exacerbate the issue by enabling effortless text translation across languages, leading to increased cross-lingual paraphrasing. Most existing CLPD studies focus on European languages, particularly English, while the English-Urdu language pair remains underexplored due to limited standard approaches and benchmark corpora.This study addresses this gap by developing the CLPD Corpus for English-Urdu (CLPD-EU), a gold-standard benchmark corpus at the sentence level. The corpus includes 5,801 sentence pairs, comprising 3,900 paraphrased and 1,901 non-paraphrased instances. Additionally, the study implements classical machine learning methods based on bilingual dictionaries, cross-lingual word embeddings, and transfer learning using sentence transformers.The research further incorporates state-of-the-art Large Language Models (LLMs) such as Mistral and LLaMA, significantly improving detection accuracy. Our proposed Feature Fusion Approach, ‘Comb-ST+BD,’ demonstrates strong performance with an F1 score of 0.739 for the CLPD task. The CLPD-EU corpus will be publicly available to encourage further research in CLPD, especially for under-resourced languages like Urdu.Keywords
This publication has 47 references indexed in Scilit:
- Cross-Lingual Text Reuse Detection at sentence level for English–Urdu language pairComputer Speech & Language, 2022
- Cross-lingual Metaphor Paraphrase Detection – Experimental Corpus and Baselines2020
- Cross-Lingual Semantic Textual Similarity Modeling Using Neural Networks2019
- CLEU ‐ A Cross‐language english‐urdu corpus and benchmark for text reuse experimentsJournal of the Association for Information Science and Technology, 2018
- Cross-language plagiarism detection over continuous-space- and knowledge graph-based representations of languageKnowledge-Based Systems, 2016
- Cross Lingual Text Reuse Detection Based on Keyphrase Extraction and Similarity MeasuresLecture Notes in Computer Science, 2013
- PAN@FIRE: Overview of the Cross-Language !ndian Text Re-Use Detection Competition2013
- Cross-language plagiarism detectionLanguage Resources and Evaluation, 2010
- Multilingual Plagiarism DetectionLecture Notes in Computer Science, 2008
- The Problem of Cross-Language Information Retrieval1998