LeCaRD: A Legal Case Retrieval Dataset for Chinese Law System
Authors:
Top Cited Papers
Open Access
- 11 July 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2342-2348
Abstract
Legal case retrieval is of vital importance for ensuring justice in different kinds of law systems and has recently received increasing attention in information retrieval (IR) research. However, the relevance judgment criteria of previous retrieval datasets are either not applicable to non-cited relationship cases or not instructive enough for future datasets to follow. Besides, most existing benchmark datasets do not focus on the selection of queries. In this paper, we construct the Chinese Legal Case Retrieval Dataset (LeCaRD), which contains 107 query cases and over 43,000 candidate cases. Queries and results are adopted from criminal cases published by the Supreme People's Court of China. In particular, to address the difficulty in relevance definition, we propose a series of relevance judgment criteria designed by our legal team and corresponding candidate case annotations are conducted by legal experts. Also, we develop a novel query sampling strategy that takes both query difficulty and diversity into consideration. For dataset evaluation, we implemented several existing retrieval models on LeCaRD as baselines. The dataset is now available to the public together with the complete data processing details.Keywords
Funding Information
- the national key Research and Development Program of China (2018YFC0831700)
- Beijing Academy of Artificial Intelligence (BAAI)
- Natural Science Foundation of China (61732008, 61532011, 62002194)
- Tsinghua University Guoqiang Research Institute
This publication has 9 references indexed in Scilit:
- The German Federal Courts Dataset 1950–2019: From Paper Archives to Linked Open DataJournal of Empirical Legal Studies, 2019
- Deeper Text Understanding for IR with Contextual Neural Language Modeling2019
- A Test Collection for Evaluating Legal Case Law Search2018
- Automatic Query Generation from Legal Texts for Case Law Retrieval2017
- Recognizing cited facts and principles in legal judgementsArtificial Intelligence and Law, 2017
- On the concept of relevance in legal information retrievalArtificial Intelligence and Law, 2017
- A history of AI and Law in 50 papers: 25 years of the international conference on AI and LawArtificial Intelligence and Law, 2012
- A language modeling approach to information retrieval1998
- The TREC-5 Filtering Track1996