Legal norm retrieval with variations of the bert model combined with TF-IDF vectorization
Authors:
Open Access
- 21 June 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 285-294
Abstract
In this work, we examine variations of the BERT model on the statute law retrieval task of the COLIEE competition. This includes approaches to leverage BERT's contextual word embeddings, fine-tuning the model, combining it with TF-IDF vectorization, adding external knowledge to the statutes and data augmentation. Our ensemble of Sentence-BERT with two different TF-IDF representations and document enrichment exhibits the best performance on this task regarding the F2 score. This is followed by a fine-tuned LEGAL-BERT with TF-IDF and data augmentation and our third approach with the BERTScore. As a result, we show that there are significant differences between the chosen BERT approaches and discuss several design decisions in the context of statute law retrieval.Keywords
Funding Information
- European Regional Development Fund (1704/00082)
This publication has 8 references indexed in Scilit:
- How to Fine-Tune BERT for Text Classification?2019
- Combining Similarity and Transformer Methods for Case Law Entailment2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks2019
- Textual Entailment in Legal Bar Exam Question Answering Using Deep Siamese Networks2018
- Universal Sentence Encoder for English2018
- Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books2015
- Research-paper recommender systems: a literature surveyInternational Journal on Digital Libraries, 2015