Efficient Text Encoders for Labor Market Analysis
Authors:
- Jens-Joris DecorteJDJens-Joris Decorte
- Jeroen Van HautteJHJeroen Van Hautte
- Chris DevelderCDChris Develder
- Thomas DemeesterTDThomas Demeester
Preprint
Abstract
Labor market analysis relies on extracting insights from job advertisements, which provide valuable yet unstructured information on job titles and corresponding skill requirements. While state-of-the-art methods for skill extraction achieve strong performance, they depend on large language models (LLMs), which are computationally expensive and slow. In this paper, we propose \textbf{ConTeXT-match}, a novel contrastive learning approach with token-level attention that is well-suited for the extreme multi-label classification task of skill classification. \textbf{ConTeXT-match} significantly improves skill extraction efficiency and performance, achieving state-of-the-art results with a lightweight bi-encoder model. To support robust evaluation, we introduce \textbf{Skill-XL}, a new benchmark with exhaustive, sentence-level skill annotations that explicitly address the redundancy in the large label space. Finally, we present \textbf{JobBERT V2}, an improved job title normalization model that leverages extracted skills to produce high-quality job title representations. Experiments demonstrate that our models are efficient, accurate, and scalable, making them ideal for large-scale, real-time labor market analysis.Keywords
All Related Versions
This publication has 4 references indexed in Scilit:
- Data science for job market analysis: A survey on applications and techniquesExpert Systems with Applications, 2024
- Diversity Subsampling: Custom Subsamples from Large Data SetsINFORMS Journal on Data Science, 2023
- Automated occupation coding with hierarchical features: a data-centric approach to classification with pre-trained language modelsDiscover Artificial Intelligence, 2023
- A Survey on Skill Identification From Online Job AdsIEEE Access, 2021