Data Mining of Job Requirements in Online Job Advertisements Using Machine Learning and SDCA Logistic Regression
Open Access
- 3 October 2021
- journal article
- research article
- Published by MDPI AG in Mathematics
- Vol. 9 (19) , 2475
Abstract
There are currently many job portals offering job positions in the form of job advertisements. In this article, we are proposing an approach to mine data from job advertisements on job portals. Mainly, it would concern job requirements mining from individual job advertisements. Our proposed system consists of a data mining module, a machine learning module, and a postprocessing module. The machine learning module is based on the SDCA logistic regression. The postprocessing module includes several approaches to increase the success rate of the job requirements identification. The proposed system was verified on 20 most searched IT job positions from the selected job portal. In total, 9971 job advertisements were analyzed. Our system’s verification is finding all job requirements in 80% of analyzed advertisements. The detected job requirements were also compared with the Open Skills database. Based on this database and the extension of IT job positions with other typical job skills, we created a list of the most frequent job skills in selected IT job positions. The main contribution is the development of a universal system to detect job requirements in job advertisements. The proposed approach can be used not only for IT positions, but also for various job positions. The presented data mining module can also be used for various job portals.Keywords
Funding Information
- Czech Ministry of Education, Youth and Sports (SGS20/PřF-MF/2021)
This publication has 17 references indexed in Scilit:
- Classifying online Job Advertisements through Machine LearningFuture Generation Computer Systems, 2018
- A Machine Learning Analysis of the Non-academic Employment Opportunities for Ph.D. Graduates in AustraliaHigher Education Policy, 2018
- Skill Requirements in Big Data: A Content Analysis of Job AdvertisementsJournal of Computer Information Systems, 2017
- Scaling Up Stochastic Dual Coordinate Ascent2015
- Computing with competencies: Modelling organizational capacitiesExpert Systems with Applications, 2012
- Competency-based selection and assignment of human resources to construction projectsScientia Iranica, 2011
- DOING COMPETENCIES WELL: BEST PRACTICES IN COMPETENCY MODELINGPersonnel Psychology, 2011
- Mining for Computing JobsIEEE Software, 2009
- IT-based competency modeling and management: from theory to practice in enterprise engineering and operationsComputers in Industry, 2002
- THE PRACTICE OF COMPETENCY MODELINGPersonnel Psychology, 2000