Amassing and indexing a large sample of version control systems: Towards the census of public source code history
Author:
- 1 May 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21601852,p. 11-20
Abstract
The source code and its history represent the output and process of software development activities and are an invaluable resource for study and improvement of software development practice. While individual projects and groups of projects have been extensively analyzed, some fundamental questions, such as the spread of innovation or genealogy of the source code, can be answered only by considering the entire universe of publicly available source code and its history. We describe methods we developed over the last six years to gather, index, and update an approximation of such a universal repository for publicly accessible version control systems and for the source code inside a large corporation. While challenging, the task is achievable with limited resources. The bottlenecks in network bandwidth, processing, and disk access can be dealt with using inherent parallelism of the tasks and suitable tradeoffs between the amount of storage and computations, but a completely automated discovery of public version control systems may require enticing participation of the sampled projects. Such universal repository would allow studies of global properties and origins of the source code that are not possible through other means.Keywords
This publication has 11 references indexed in Scilit:
- Merits of using repository metrics in defect prediction for open source projects2009
- Succession: Measuring transfer of code and developer productivity2009
- Evaluation of source code copy detection methods on freebsd2008
- Near-optimal hashing algorithms for approximate nearest neighbor in high dimensionsCommunications of the ACM, 2008
- Large-Scale Code Reuse in Open Source Software2007
- Constructing universal version history2006
- Discussion of a Large-Scale Open Source Data Collection Methodology2005
- An empirical study of software reuse vs. defect-density and stability2004
- Component rank: relative significance rank for software component search2003
- Analytical and empirical evaluation of software reuse metrics1996