Zhijie Nie

Dense retrieval usually fails in two ways: ranking non-relevant documents too high, or ranking relevant documents too low. We focus on the second case from the query side: documents that are genuinely relevant but receive low retrieval scores under certain query formulations, forming hard-positive query–document pairs. To study this failure mode systematically, we build on recent token-alignment work, which analyzes vocabulary logits obtained by projecting retriever representations through an architecture-specific token prediction head and shows that low overlap among top-ranked logit tokens can cause relevant documents to be scored low. This suggests a practical route to hard positives: generate relevant queries that rely on document phrases that receive low ranks under these logits. We therefore define model-specific Token and Phrase Representativeness Scores (TRS/PRS) to discover tokens and key phrases that appear in a document but are poorly expressed by its embedding. Using high-PRS phrases as anchors, we automatically construct challenging yet relevant queries, yielding hard-positive query–document pairs. Experiments across multiple retrievers and datasets show that PRS-anchored queries induce substantially larger drops for dense retrievers than other query construction strategies, revealing a dense-specific weakness. Moreover, when mixed with standard hard negatives during contrastive fine-tuning, these hard positives improve retrieval on our constructed hard-positive benchmark and can also improve standard in-domain retrieval benchmarks. Our code is publicly available at https://github.com/wzy2001wzy/HardPositive.

The Composed Image Retrieval (CIR) task aims to retrieve target images using a composed query consisting of a reference image and a modified text. Advanced methods often utilize contrastive learning as the optimization objective, which benefits from adequate positive and negative examples. However, the triplet for CIR incurs high manual annotation costs, resulting in limited positive examples. Furthermore, existing methods commonly use in-batch negative sampling, which reduces the negative number available for the model. To address the problem of lack of positives, we propose a data generation method by leveraging a multi-modal large language model to construct triplets for CIR. To introduce more negatives during fine-tuning, we design a two-stage fine-tuning framework for CIR, whose second stage introduces plenty of static representations of negatives to optimize the representation space rapidly. The above two improvements can be effectively stacked and designed to be plug-and-play, easily applied to existing CIR models without changing their original architectures. Extensive experiments and ablation analysis demonstrate that our method effectively scales positives and negatives and achieves state-of-the-art results on both FashionIQ and CIRR datasets. In addition, our method also performs well in zero-shot composed image retrieval, providing a new CIR solution for the low-resources scenario. Our code and data are released at https://github.com/BUAADreamer/SPN4CIR.

Cross-lingual Cross-modal Retrieval (CCR) is an essential task in web search, which aims to break the barriers between modality and language simultaneously and achieves image-text retrieval in the multi-lingual scenario with a single model. In recent years, excellent progress has been made based on cross-lingual cross-modal pre-training; particularly, the methods based on contrastive learning on large-scale data have significantly improved retrieval tasks. However, these methods directly follow the existing pre-training methods in the cross-lingual or cross-modal domain, leading to two problems of inconsistency in CCR: The methods with cross-lingual style suffer from the intra-modal error propagation, resulting in inconsistent recall performance across languages in the whole dataset. The methods with cross-modal style suffer from the inter-modal optimization direction bias, resulting in inconsistent rank across languages within each instance, which cannot be reflected by Recall@K. To solve these problems, we propose a simple but effective 1-to-K contrastive learning method, which treats each language equally and eliminates error propagation and optimization bias. In addition, we propose a new evaluation metric, Mean Rank Variance (MRV), to reflect the rank inconsistency across languages within each instance. Extensive experiments on four CCR datasets show that our method improves both recall rates and MRV with smaller-scale pre-trained data, achieving the new state-of-art.