Identifying Experts in Software Libraries and Frameworks Among GitHub Users
Authors:
- 1 May 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 276-287
Abstract
Software development increasingly depends on libraries and frameworks to increase productivity and reduce time-to-market. Despite this fact, we still lack techniques to assess developers expertise in widely popular libraries and frameworks. In this paper, we evaluate the performance of unsupervised (based on clustering) and supervised machine learning classifiers (Random Forest and SVM) to identify experts in three popular JavaScript libraries: facebook/react, mongodb/node-mongodb, and socketio/socket.io. First, we collect 13 features about developers activity on GitHub projects, including commits on source code files that depend on these libraries. We also build a ground truth including the expertise of 575 developers on the studied libraries, as self-reported by them in a survey. Based on our findings, we document the challenges of using machine learning classifiers to predict expertise in software libraries, using features extracted from GitHub. Then, we propose a method to identify library experts based on clustering feature data from GitHub; by triangulating the results of this method with information available on Linkedin profiles, we show that it is able to recommend dozens of GitHub users with evidences of being experts in the studied JavaScript libraries. We also provide a public dataset with the expertise of 575 developers on the studied libraries.Keywords
All Related Versions
This publication has 45 references indexed in Scilit:
- Degree-of-knowledgeACM Transactions on Software Engineering and Methodology, 2014
- Measuring and modeling programming experienceEmpirical Software Engineering, 2013
- A Large-Scale Empirical Study on Software Reuse in Mobile AppsIEEE Software, 2013
- Important characteristics of software development team members: an empirical investigation using Repertory GridInformation Systems Journal, 2007
- The Influence of Experience and Deliberate Practice on the Development of Superior Expert Performance2006
- Finding the Right Data for Software Cost ModelingIEEE Software, 2005
- Bayesian analysis of empirical software engineering cost modelsIEEE Transactions on Software Engineering, 1999
- Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments.Journal of Personality and Social Psychology, 1999
- Small sample size effects in statistical pattern recognition: recommendations for practitionersIEEE Transactions on Pattern Analysis and Machine Intelligence, 1991
- Silhouettes: A graphical aid to the interpretation and validation of cluster analysisJournal of Computational and Applied Mathematics, 1987