Vinay Setty
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Vinay Setty has a total of 63 co-authors. Here are the top co-authors by number of shared publications:
- Avishek Anand
Delft University of Technology
10 shared publications - V. Venktesh
Stockholm University
8 shared publications Roman Vitenberg
5 shared publications- Maarten van Steen
University of Twente, Enschede, Netherlands
4 shared publications Arunav Mishra
Max Planck Institute for Informatics, Saarbruecken, Germany
3 shared publicationsAvishek Anand
L3S Research Center, Hanover, Germany
3 shared publications
Latest Publications
- 21 September 2026
- Published by Springer Nature
- p. 336-356
Overview of CheckThat! 2026: Advancing Multilingual Fact-Checking
- 21 September 2026
- Published by Springer Nature
- p. 336-356
Overview of CheckThat! 2026: Advancing Multilingual Fact-Checking
- Open Access
- 24 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 381-391
Effectiveness of Sparse Retrieval for Automated Fact-Checking
As fact-checking pipelines grow increasingly complex, a critical question remains unanswered: does retrieval matter more than reasoning? We investigate this question on the AVeriTeC 2.0 Shared Task, a leading and highly competitive benchmark for automated fact-checking that requires systems to verify real-world claims against open-domain web evidence. On this benchmark, we compare dense retrieval (FAISS), learned sparse retrieval (SPLADE), late interaction (ColBERT), and contextual embeddings across two architecturally distinct systems. SPLADE consistently outperforms all alternatives, improving new AVeriTeC scores by up to 86% on the development set and up to 27% on the hidden test set, where it enables a lower-ranked system to claim the top leaderboard position. Its advantage stems from vocabulary-space representations that bridge lexical gaps, preserve fine-grained numerical and entity matches, and remain robust to noisy web evidence, capabilities that are difficult to achieve with dense retrieval without additional techniques such as query expansion, reranking, or entity-aware training. Strikingly, two systems of vastly different complexity, one consolidating all reasoning into a single LLM call and the other orchestrating five models across seven tasks, converge to virtually identical performance once equipped with SPLADE. This demonstrates that retrieval quality is a major bottleneck in these systems. Beyond accuracy, SPLADE reduces preprocessing time by an order of magnitude while allowing both pipelines to share a single index. Our findings establish learned sparse retrieval as a highly impactful design choice in automated fact-checking, with retrieval quality serving as a critical determinant of end-to-end performance in the studied systems.
- 24 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 381-391
Open AccessEffectiveness of Sparse Retrieval for Automated Fact-Checking
As fact-checking pipelines grow increasingly complex, a critical question remains unanswered: does retrieval matter more than reasoning? We investigate this question on the AVeriTeC 2.0 Shared Task, a leading and highly competitive benchmark for automated fact-checking that requires systems to verify real-world claims against open-domain web evidence. On this benchmark, we compare dense retrieval (FAISS), learned sparse retrieval (SPLADE), late interaction (ColBERT), and contextual embeddings across two architecturally distinct systems. SPLADE consistently outperforms all alternatives, improving new AVeriTeC scores by up to 86% on the development set and up to 27% on the hidden test set, where it enables a lower-ranked system to claim the top leaderboard position. Its advantage stems from vocabulary-space representations that bridge lexical gaps, preserve fine-grained numerical and entity matches, and remain robust to noisy web evidence, capabilities that are difficult to achieve with dense retrieval without additional techniques such as query expansion, reranking, or entity-aware training. Strikingly, two systems of vastly different complexity, one consolidating all reasoning into a single LLM call and the other orchestrating five models across seven tasks, converge to virtually identical performance once equipped with SPLADE. This demonstrates that retrieval quality is a major bottleneck in these systems. Beyond accuracy, SPLADE reduces preprocessing time by an order of magnitude while allowing both pipelines to share a single index. Our findings establish learned sparse retrieval as a highly impactful design choice in automated fact-checking, with retrieval quality serving as a critical determinant of end-to-end performance in the studied systems.
- 24 March 2026
- Published by Springer Nature
- p. 325-335
The CLEF-2026 CheckThat! Lab: Advancing Multilingual Fact-Checking
- 24 March 2026
- Published by Springer Nature
- p. 325-335
The CLEF-2026 CheckThat! Lab: Advancing Multilingual Fact-Checking
- 24 March 2026
- Published by Springer Nature
- p. 289-304
One LLM to Train Them All: Multi-task Learning Framework for Fact-Checking
- 24 March 2026
- Published by Springer Nature
- p. 289-304
One LLM to Train Them All: Multi-task Learning Framework for Fact-Checking
- Open Access
NumPert: Numerical Perturbations to Probe Language Models for Veracity Prediction
Large language models show strong performance on knowledge intensive tasks such as fact-checking and question answering, yet they often struggle with numerical reasoning. We present a systematic evaluation of state-of-the-art models for veracity prediction on numerical claims and evidence pairs using controlled perturbations, including label-flipping probes, to test robustness. Our results indicate that even leading proprietary systems experience accuracy drops of up to 62\% under certain perturbations. No model proves to be robust across all conditions. We further find that increasing context length generally reduces accuracy, but when extended context is enriched with perturbed demonstrations, most models substantially recover. These findings highlight critical limitations in numerical fact-checking and suggest that robustness remains an open challenge for current language models.
Open AccessNumPert: Numerical Perturbations to Probe Language Models for Veracity Prediction
Large language models show strong performance on knowledge intensive tasks such as fact-checking and question answering, yet they often struggle with numerical reasoning. We present a systematic evaluation of state-of-the-art models for veracity prediction on numerical claims and evidence pairs using controlled perturbations, including label-flipping probes, to test robustness. Our results indicate that even leading proprietary systems experience accuracy drops of up to 62\% under certain perturbations. No model proves to be robust across all conditions. We further find that increasing context length generally reduces accuracy, but when extended context is enriched with perturbed demonstrations, most models substantially recover. These findings highlight critical limitations in numerical fact-checking and suggest that robustness remains an open challenge for current language models.
- Open Access
Think Right, Not More: Test-Time Scaling for Numerical Claim Verification
Fact-checking real-world claims, particularly numerical claims, is inherently complex that require multistep reasoning and numerical reasoning for verifying diverse aspects of the claim. Although large language models (LLMs) including reasoning models have made tremendous advances, they still fall short on fact-checking real-world claims that require a combination of compositional and numerical reasoning. They are unable to understand nuance of numerical aspects, and are also susceptible to the reasoning drift issue, where the model is unable to contextualize diverse information resulting in misinterpretation and backtracking of reasoning process. In this work, we systematically explore scaling test-time compute (TTS) for LLMs on the task of fact-checking complex numerical claims, which entails eliciting multiple reasoning paths from an LLM. We train a verifier model (VERIFIERFC) to navigate this space of possible reasoning paths and select one that could lead to the correct verdict. We observe that TTS helps mitigate the reasoning drift issue, leading to significant performance gains for fact-checking numerical claims. To improve compute efficiency in TTS, we introduce an adaptive mechanism that performs TTS selectively based on the perceived complexity of the claim. This approach achieves 1.8x higher efficiency than standard TTS, while delivering a notable 18.8% performance improvement over single-shot claim verification methods. Our code and data can be found at https://github.com/VenkteshV/VerifierFC
Open AccessThink Right, Not More: Test-Time Scaling for Numerical Claim Verification
Fact-checking real-world claims, particularly numerical claims, is inherently complex that require multistep reasoning and numerical reasoning for verifying diverse aspects of the claim. Although large language models (LLMs) including reasoning models have made tremendous advances, they still fall short on fact-checking real-world claims that require a combination of compositional and numerical reasoning. They are unable to understand nuance of numerical aspects, and are also susceptible to the reasoning drift issue, where the model is unable to contextualize diverse information resulting in misinterpretation and backtracking of reasoning process. In this work, we systematically explore scaling test-time compute (TTS) for LLMs on the task of fact-checking complex numerical claims, which entails eliciting multiple reasoning paths from an LLM. We train a verifier model (VERIFIERFC) to navigate this space of possible reasoning paths and select one that could lead to the correct verdict. We observe that TTS helps mitigate the reasoning drift issue, leading to significant performance gains for fact-checking numerical claims. To improve compute efficiency in TTS, we introduce an adaptive mechanism that performs TTS selectively based on the perceived complexity of the claim. This approach achieves 1.8x higher efficiency than standard TTS, while delivering a notable 18.8% performance improvement over single-shot claim verification methods. Our code and data can be found at https://github.com/VenkteshV/VerifierFC
- Open Access
ShortCheck: Checkworthiness Detection of Multilingual Short-Form Videos
Short-form video platforms like TikTok present unique challenges for misinformation detection due to their multimodal, dynamic, and noisy content. We present ShortCheck, a modular, inference-only pipeline with a user-friendly interface that automatically identifies checkworthy short-form videos to help human fact-checkers. The system integrates speech transcription, OCR, object and deepfake detection, video-to-text summarization, and claim verification. ShortCheck is validated by evaluating it on two manually annotated datasets with TikTok videos in a multilingual setting. The pipeline achieves promising results with F1-weighted score over 70\%.
Open AccessShortCheck: Checkworthiness Detection of Multilingual Short-Form Videos
Short-form video platforms like TikTok present unique challenges for misinformation detection due to their multimodal, dynamic, and noisy content. We present ShortCheck, a modular, inference-only pipeline with a user-friendly interface that automatically identifies checkworthy short-form videos to help human fact-checkers. The system integrates speech transcription, OCR, object and deepfake detection, video-to-text summarization, and claim verification. ShortCheck is validated by evaluating it on two manually annotated datasets with TikTok videos in a multilingual setting. The pipeline achieves promising results with F1-weighted score over 70\%.
- Open Access
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 809-812
FactIR: A Real-World Zero-shot Open-Domain Retrieval Benchmark for Fact-Checking
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 809-812
Open AccessFactIR: A Real-World Zero-shot Open-Domain Retrieval Benchmark for Fact-Checking
- Open Access
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 789-792
Annotation Tool and Dataset for Fact-Checking Podcasts
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 789-792
Open AccessAnnotation Tool and Dataset for Fact-Checking Podcasts
- Open Access
- 10 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 1100-1101
Fact-checking Multilingual Podcasts
Long-form content, such as YouTube videos and podcasts, has become widely popular, particularly among younger audiences. However, the extended format of this content presents unique challenges for fact-checking them. This talk explores the feasibility of an end-to-end solution for transcribing and fact-checking long-form content in a multilingual context. I will conclude by presenting the selected technologies and models for end-to-end fact-checking in this domain.
- 10 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 1100-1101
Open AccessFact-checking Multilingual Podcasts
Long-form content, such as YouTube videos and podcasts, has become widely popular, particularly among younger audiences. However, the extended format of this content presents unique challenges for fact-checking them. This talk explores the feasibility of an end-to-end solution for transcribing and fact-checking long-form content in a multilingual context. I will conclude by presenting the selected technologies and models for end-to-end fact-checking in this domain.