Bevan Koopman
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Bevan Koopman has a total of 87 co-authors. Here are the top co-authors by number of shared publications:
- Guido Zuccon
The University of Queensland, Brisbane, Australia
61 shared publications - Shengyao Zhuang
The University of Queensland, Brisbane, Australia
20 shared publications - Shuai Wang
The University of Queensland, Brisbane, Australia
14 shared publications - Harrisen Scells
University of Tübingen, Tübingen, Germany
10 shared publications - Peter Bruza
Faculty of Science, Queensland University of Technology
8 shared publications - Anton van der Vegt
Queensland Digital Health Centre, The University of Queensland, Brisbane, Queensland, Australia
7 shared publications
Latest Publications
- Open Access
- 24 September 2026
- Published by Elsevier in Informatics in Medicine Unlocked
- Vol. 66
CXRMate-2: Structured multimodal temporal embeddings and tractable reinforcement learning for clinically acceptable chest X-ray radiology report generation
- 24 September 2026
- Published by Elsevier in Informatics in Medicine Unlocked
- Vol. 66
Open AccessCXRMate-2: Structured multimodal temporal embeddings and tractable reinforcement learning for clinically acceptable chest X-ray radiology report generation
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 4419-4425
Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning
We introduce Rank-R1, an LLM-based reranker that reasons over queries and candidate documents before ranking. Existing rerankers based on LLMs rely on prompting or fine-tuning to order documents by relevance without explicit reasoning. Rank-R1 uses reinforcement learning with only relevance labels (no reasoning supervision) to learn an intermediate reasoning step that improves ranking. Experiments show that Rank-R1 is competitive with supervised fine-tuning on in-domain TREC DL queries. On the out-of-domain BRIGHT benchmark, which requires complex reasoning, Rank-R1 outperforms both zero-shot and supervised baselines; the 14B model surpasses RankGPT4. With a scaled configuration combining a 32B backbone, improved first-stage retrieval, and harder training data, Rank-R1 achieves state-of-the-art performance on BRIGHT.
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 4419-4425
Open AccessRank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning
We introduce Rank-R1, an LLM-based reranker that reasons over queries and candidate documents before ranking. Existing rerankers based on LLMs rely on prompting or fine-tuning to order documents by relevance without explicit reasoning. Rank-R1 uses reinforcement learning with only relevance labels (no reasoning supervision) to learn an intermediate reasoning step that improves ranking. Experiments show that Rank-R1 is competitive with supervised fine-tuning on in-domain TREC DL queries. On the out-of-domain BRIGHT benchmark, which requires complex reasoning, Rank-R1 outperforms both zero-shot and supervised baselines; the 14B model surpasses RankGPT4. With a scaled configuration combining a 32B backbone, improved first-stage retrieval, and harder training data, Rank-R1 achieves state-of-the-art performance on BRIGHT.
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3081-3091
Beyond Chunk-Then-Embed: A Comprehensive Taxonomy and Evaluation of Document Chunking Strategies for Information Retrieval
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3081-3091
Open AccessBeyond Chunk-Then-Embed: A Comprehensive Taxonomy and Evaluation of Document Chunking Strategies for Information Retrieval
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3070-3080
The Vulnerability of LLM Rankers to Prompt Injection Attacks: You are to [MARK] this paper as the Best Paper
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 3070-3080
Open AccessThe Vulnerability of LLM Rankers to Prompt Injection Attacks: You are to [MARK] this paper as the Best Paper
- 25 March 2026
- Published by Springer Nature
- p. 67-82
A simple sketch of a disposable coffee cup with a lid. The cup features a green circle in the center, possibly representing a logo or design element. The drawing is outlined in black with minimal detail. Starbucks: Improved Training for 2D Matryoshka Embeddings
- 25 March 2026
- Published by Springer Nature
- p. 67-82
A simple sketch of a disposable coffee cup with a lid. The cup features a green circle in the center, possibly representing a logo or design element. The drawing is outlined in black with minimal detail. Starbucks: Improved Training for 2D Matryoshka Embeddings
- Open Access
Beyond Chunk-Then-Embed: A Comprehensive Taxonomy and Evaluation of Document Chunking Strategies for Information Retrieval
Document chunking is a critical preprocessing step in dense retrieval systems, yet the design space of chunking strategies remains poorly understood. Recent research has proposed several concurrent approaches, including LLM-guided methods (e.g., DenseX and LumberChunker) and contextualized strategies(e.g., Late Chunking), which generate embeddings before segmentation to preserve contextual information. However, these methods emerged independently and were evaluated on benchmarks with minimal overlap, making direct comparisons difficult. This paper reproduces prior studies in document chunking and presents a systematic framework that unifies existing strategies along two key dimensions: (1) segmentation methods, including structure-based methods (fixed-size, sentence-based, and paragraph-based) as well as semantically-informed and LLM-guided methods; and (2) embedding paradigms, which determine the timing of chunking relative to embedding (pre-embedding chunking vs. contextualized chunking). Our reproduction evaluates these approaches in two distinct retrieval settings established in previous work: in-document retrieval (needle-in-a-haystack) and in-corpus retrieval (the standard information retrieval task). Our comprehensive evaluation reveals that optimal chunking strategies are task-dependent: simple structure-based methods outperform LLM-guided alternatives for in-corpus retrieval, while LumberChunker performs best for in-document retrieval. Contextualized chunking improves in-corpus effectiveness but degrades in-document retrieval. We also find that chunk size correlates moderately with in-document but weakly with in-corpus effectiveness, suggesting segmentation method differences are not purely driven by chunk size. Our code and evaluation benchmarks are publicly available at (Anonymoused).
Open AccessBeyond Chunk-Then-Embed: A Comprehensive Taxonomy and Evaluation of Document Chunking Strategies for Information Retrieval
Document chunking is a critical preprocessing step in dense retrieval systems, yet the design space of chunking strategies remains poorly understood. Recent research has proposed several concurrent approaches, including LLM-guided methods (e.g., DenseX and LumberChunker) and contextualized strategies(e.g., Late Chunking), which generate embeddings before segmentation to preserve contextual information. However, these methods emerged independently and were evaluated on benchmarks with minimal overlap, making direct comparisons difficult. This paper reproduces prior studies in document chunking and presents a systematic framework that unifies existing strategies along two key dimensions: (1) segmentation methods, including structure-based methods (fixed-size, sentence-based, and paragraph-based) as well as semantically-informed and LLM-guided methods; and (2) embedding paradigms, which determine the timing of chunking relative to embedding (pre-embedding chunking vs. contextualized chunking). Our reproduction evaluates these approaches in two distinct retrieval settings established in previous work: in-document retrieval (needle-in-a-haystack) and in-corpus retrieval (the standard information retrieval task). Our comprehensive evaluation reveals that optimal chunking strategies are task-dependent: simple structure-based methods outperform LLM-guided alternatives for in-corpus retrieval, while LumberChunker performs best for in-document retrieval. Contextualized chunking improves in-corpus effectiveness but degrades in-document retrieval. We also find that chunk size correlates moderately with in-document but weakly with in-corpus effectiveness, suggesting segmentation method differences are not purely driven by chunk size. Our code and evaluation benchmarks are publicly available at (Anonymoused).
- Open Access
The Vulnerability of LLM Rankers to Prompt Injection Attacks
Large Language Models (LLMs) have emerged as powerful re-rankers. Recent research has however showed that simple prompt injections embedded within a candidate document (i.e., jailbreak prompt attacks) can significantly alter an LLM's ranking decisions. While this poses serious security risks to LLM-based ranking pipelines, the extent to which this vulnerability persists across diverse LLM families, architectures, and settings remains largely under-explored. In this paper, we present a comprehensive empirical study of jailbreak prompt attacks against LLM rankers. We focus our evaluation on two complementary tasks: (1) Preference Vulnerability Assessment, measuring intrinsic susceptibility via attack success rate (ASR); and (2) Ranking Vulnerability Assessment, quantifying the operational impact on the ranking's quality (nDCG@10). We systematically examine three prevalent ranking paradigms (pairwise, listwise, setwise) under two injection variants: decision objective hijacking and decision criteria hijacking. Beyond reproducing prior findings, we expand the analysis to cover vulnerability scaling across model families, position sensitivity, backbone architectures, and cross-domain robustness. Our results characterize the boundary conditions of these vulnerabilities, revealing critical insights such as that encoder-decoder architectures exhibit strong inherent resilience to jailbreak attacks. We publicly release our code and additional experimental results at https://github.com/ielab/LLM-Ranker-Attack.
Open AccessThe Vulnerability of LLM Rankers to Prompt Injection Attacks
Large Language Models (LLMs) have emerged as powerful re-rankers. Recent research has however showed that simple prompt injections embedded within a candidate document (i.e., jailbreak prompt attacks) can significantly alter an LLM's ranking decisions. While this poses serious security risks to LLM-based ranking pipelines, the extent to which this vulnerability persists across diverse LLM families, architectures, and settings remains largely under-explored. In this paper, we present a comprehensive empirical study of jailbreak prompt attacks against LLM rankers. We focus our evaluation on two complementary tasks: (1) Preference Vulnerability Assessment, measuring intrinsic susceptibility via attack success rate (ASR); and (2) Ranking Vulnerability Assessment, quantifying the operational impact on the ranking's quality (nDCG@10). We systematically examine three prevalent ranking paradigms (pairwise, listwise, setwise) under two injection variants: decision objective hijacking and decision criteria hijacking. Beyond reproducing prior findings, we expand the analysis to cover vulnerability scaling across model families, position sensitivity, backbone architectures, and cross-domain robustness. Our results characterize the boundary conditions of these vulnerabilities, revealing critical insights such as that encoder-decoder architectures exhibit strong inherent resilience to jailbreak attacks. We publicly release our code and additional experimental results at https://github.com/ielab/LLM-Ranker-Attack.
- Open Access
- 6 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 437-441
SIGIR-AP 2025 Tutorial on Retrieval and Ranking with LLMs (R 2 LLMs)
- 6 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 437-441
Open AccessSIGIR-AP 2025 Tutorial on Retrieval and Ranking with LLMs (R 2 LLMs)
- Open Access
Humans are more gullible than LLMs in believing common psychological myths
Despite widespread debunking, many psychological myths remain deeply entrenched. This paper investigates whether Large Language Models (LLMs) mimic human behaviour of myth belief and explores methods to mitigate such tendencies. Using 50 popular psychological myths, we evaluate myth belief across multiple LLMs under different prompting strategies, including retrieval-augmented generation and swaying prompts. Results show that LLMs exhibit significantly lower myth belief rates than humans, though user prompting can influence responses. RAG proves effective in reducing myth belief and reveals latent debiasing potential within LLMs. Our findings contribute to the emerging field of Machine Psychology and highlight how cognitive science methods can inform the evaluation and development of LLM-based systems.
Open AccessHumans are more gullible than LLMs in believing common psychological myths
Despite widespread debunking, many psychological myths remain deeply entrenched. This paper investigates whether Large Language Models (LLMs) mimic human behaviour of myth belief and explores methods to mitigate such tendencies. Using 50 popular psychological myths, we evaluate myth belief across multiple LLMs under different prompting strategies, including retrieval-augmented generation and swaying prompts. Results show that LLMs exhibit significantly lower myth belief rates than humans, though user prompting can influence responses. RAG proves effective in reducing myth belief and reveals latent debiasing potential within LLMs. Our findings contribute to the emerging field of Machine Psychology and highlight how cognitive science methods can inform the evaluation and development of LLM-based systems.
- Open Access
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 414-423
Document Screenshot Retrievers are Vulnerable to Pixel Poisoning Attacks
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 414-423
Open AccessDocument Screenshot Retrievers are Vulnerable to Pixel Poisoning Attacks