Can BERT Dig It? Named Entity Recognition for Information Retrieval in the Archaeology Domain
Authors:
Top Cited Papers
Open Access
- 16 September 2022
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Journal on Computing and Cultural Heritage
- Vol. 15 (3) , 1-18
Abstract
The amount of archaeological literature is growing rapidly. Until recently, these data were only accessible through metadata search. We implemented a text retrieval engine for a large archaeological text collection (~658 million words). In archaeological IR, domain-specific entities such as locations, time periods and artefacts play a central role. This motivated the development of a named entity recognition (NER) model to annotate the full collection with archaeological named entities. In this article, we present ArcheoBERTje, a BERT (Bidirectional Encoder Representations from Transformers) model pre-trained on Dutch archaeological texts. We compare the model’s quality and output on an NER task to a generic multilingual model and a generic Dutch model. We also investigate ensemble methods for combining multiple BERT models, and combining the best BERT model with a domain thesaurus using conditional random fields. We find that ArcheoBERTje outperforms both the multilingual and Dutch model significantly with a smaller standard deviation between runs, reaching an average F1 score of 0.735. The model also outperforms ensemble methods combining the three models. Combining ArcheoBERTje predictions and explicit domain knowledge from the thesaurus did not increase the F1 score. We quantitatively and qualitatively analyse the differences between the vocabulary and output of the BERT models on the full collection and provide some valuable insights in the effect of fine-tuning for specific domains. Our results indicate that for a highly specific text domain such as archaeology, further pre-training on domain-specific data increases the model’s quality on NER by a much larger margin than shown for other domains in the literature, and that domain-specific pre-training makes the addition of domain knowledge from a thesaurus unnecessary.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Named Entity Recognition using Word Embedding as a FeatureInternational Journal of Software Engineering and Its Applications, 2016
- Archaeological Discriminant Applications of the Lubischew Test2015
- Automatic Metadata Generation in an Archaeological Digital Library: Semantic Annotation of Grey Literature2013
- A Survey of Automatic Query Expansion in Information RetrievalACM Computing Surveys, 2012
- Efficient diversification of web search resultsProceedings of the VLDB Endowment, 2011
- The Archaeotools project: faceted classification and natural language processing in an archaeological contextPhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2009
- Faceted SearchSynthesis Lectures on Information Concepts, Retrieval, and Services, 2009
- A Chain of Text-mining to Extract Information in Archaeology2008
- Fuzzy Approach of Synonymy and Polysemy for Information Retrieval2007
- Extracting company names from text2002