Jeffrey Dalton
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Jeffrey Dalton has a total of 157 co-authors. Here are the top co-authors by number of shared publications:
- Shubham Chatterjee
Missouri University of Science and Technology, Rolla, MO, USA
11 shared publications - Mohammad Aliannejadi
University of Amsterdam, Amstedam, Netherlands
9 shared publications - Iain Mackie
University of Glasgow University of Glasgow University of Glasgow University of Glasgow
7 shared publications - Chuan Meng
The University of Edinburgh, Edinburgh, United Kingdom
5 shared publications - Laura Dietz
University of New Hampshire, Durham, NH, USA
5 shared publications - Zahra Abbasiantaeb
University of Amsterdam, Amsterdam, Netherlands
5 shared publications
Latest Publications
- Open Access
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 9429-9432
Agentic Search: Foundations, Optimization, and Long-Horizons
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 9429-9432
Open AccessAgentic Search: Foundations, Optimization, and Long-Horizons
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3190-3197
Sim.API: A Middleware to Simplify the Use of User Simulators for Shared Tasks in Conversational Search
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3190-3197
Open AccessSim.API: A Middleware to Simplify the Use of User Simulators for Shared Tasks in Conversational Search
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3092-3100
TREC iKAT 2025: A Test Collection for the Offline and Interactive Evaluation of Conversational Search
Conversational search agents, especially with the advent of large language models, have developed into useful tools to satisfy complex information needs of their users. Former research has shown that personalization (i.e., adaptation of agent responses to the preferences and traits of the user) can increase the relevance and perceived answer quality of these systems even further. However, developing accurate personalization methods typically requires rich datasets, both in terms of user profiles and complex conversations, for which only a few resources are publicly available. Over the past three years, the goal of the TREC Interactive Knowledge Assistance Track (iKAT) has been to bridge this gap. In organizing this shared task, we have developed a collection of complex information needs and associated conversations, made to challenge today's conversational agents and thus highlight aspects in need of further research. In this paper, we present the resources made for iKAT 2025, focusing on multi-session conversations (i.e., multiple dialogues per user), dynamically evolving user models, mixed-initiative dialogues, and large-scale human and automatic assessments. In addition to manually designed user profiles and conversations, the test collection for 2025 also contains dialogues between participating systems and our user simulators. All the resources are publicly available in our repository, including the system evaluation both as a result of the offline (i.e., test collection-based) and interactive tasks (i.e., user simulation-based), as well as their source code and model weights, to foster future research in this direction.
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3092-3100
Open AccessTREC iKAT 2025: A Test Collection for the Offline and Interactive Evaluation of Conversational Search
Conversational search agents, especially with the advent of large language models, have developed into useful tools to satisfy complex information needs of their users. Former research has shown that personalization (i.e., adaptation of agent responses to the preferences and traits of the user) can increase the relevance and perceived answer quality of these systems even further. However, developing accurate personalization methods typically requires rich datasets, both in terms of user profiles and complex conversations, for which only a few resources are publicly available. Over the past three years, the goal of the TREC Interactive Knowledge Assistance Track (iKAT) has been to bridge this gap. In organizing this shared task, we have developed a collection of complex information needs and associated conversations, made to challenge today's conversational agents and thus highlight aspects in need of further research. In this paper, we present the resources made for iKAT 2025, focusing on multi-session conversations (i.e., multiple dialogues per user), dynamically evolving user models, mixed-initiative dialogues, and large-scale human and automatic assessments. In addition to manually designed user profiles and conversations, the test collection for 2025 also contains dialogues between participating systems and our user simulators. All the resources are publicly available in our repository, including the system evaluation both as a result of the offline (i.e., test collection-based) and interactive tasks (i.e., user simulation-based), as well as their source code and model weights, to foster future research in this direction.
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3006-3016
Revisiting Text Ranking in Deep Research
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3006-3016
Open AccessRevisiting Text Ranking in Deep Research
- Open Access
- 28 May 2026
- Published by Association for Computing Machinery (ACM)
- p. 39-42
Conversational Search: From Fundamentals to Frontiers in the Age of Agents
- 28 May 2026
- Published by Association for Computing Machinery (ACM)
- p. 39-42
Open AccessConversational Search: From Fundamentals to Frontiers in the Age of Agents
- 24 March 2026
- Published by Springer Nature
- p. 11-18
Conversational Search: Foundations, Large Language Models, and Agents
- 24 March 2026
- Published by Springer Nature
- p. 11-18
Conversational Search: Foundations, Large Language Models, and Agents
- Open Access
Re-Rankers as Relevance Judges
Using large language models (LLMs) to predict relevance judgments has shown promising results. Most studies treat this task as a distinct research line, e.g., focusing on prompt design for predicting relevance labels given a query and passage. However, predicting relevance judgments is essentially a form of relevance prediction, a problem extensively studied in tasks such as re-ranking. Despite this potential overlap, little research has explored reusing or adapting established re-ranking methods to predict relevance judgments, leading to potential resource waste and redundant development. To bridge this gap, we reproduce re-rankers in a re-ranker-as-relevance-judge setup. We design two adaptation strategies: (i) using binary tokens (e.g., "true" and "false") generated by a re-ranker as direct judgments, and (ii) converting continuous re-ranking scores into binary labels via thresholding. We perform extensive experiments on TREC-DL 2019 to 2023 with 8 re-rankers from 3 families, ranging from 220M to 32B, and analyse the evaluation bias exhibited by re-ranker-based judges. Results show that re-ranker-based relevance judges, under both strategies, can outperform UMBRELA, a state-of-the-art LLM-based relevance judge, in around 40% to 50% of the cases; they also exhibit strong self-preference towards their own and same-family re-rankers, as well as cross-family bias.
Open AccessRe-Rankers as Relevance Judges
Using large language models (LLMs) to predict relevance judgments has shown promising results. Most studies treat this task as a distinct research line, e.g., focusing on prompt design for predicting relevance labels given a query and passage. However, predicting relevance judgments is essentially a form of relevance prediction, a problem extensively studied in tasks such as re-ranking. Despite this potential overlap, little research has explored reusing or adapting established re-ranking methods to predict relevance judgments, leading to potential resource waste and redundant development. To bridge this gap, we reproduce re-rankers in a re-ranker-as-relevance-judge setup. We design two adaptation strategies: (i) using binary tokens (e.g., "true" and "false") generated by a re-ranker as direct judgments, and (ii) converting continuous re-ranking scores into binary labels via thresholding. We perform extensive experiments on TREC-DL 2019 to 2023 with 8 re-rankers from 3 families, ranging from 220M to 32B, and analyse the evaluation bias exhibited by re-ranker-based judges. Results show that re-ranker-based relevance judges, under both strategies, can outperform UMBRELA, a state-of-the-art LLM-based relevance judge, in around 40% to 50% of the cases; they also exhibit strong self-preference towards their own and same-family re-rankers, as well as cross-family bias.
- Top Cited PapersOpen Access
- 18 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 218-229
Principles and Guidelines for the Use of LLM Judges
- 18 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 218-229
Top Cited PapersOpen AccessPrinciples and Guidelines for the Use of LLM Judges
- Open Access
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 2255-2265
QDER: Query-Specific Document and Entity Representations for Multi-Vector Document Re-Ranking
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 2255-2265
Open AccessQDER: Query-Specific Document and Entity Representations for Multi-Vector Document Re-Ranking
- Open Access
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 3455-3465
Conversational Gold: Evaluating Personalized Conversational Search System Using Gold Nuggets
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 3455-3465
Open AccessConversational Gold: Evaluating Personalized Conversational Search System Using Gold Nuggets