PTMTorrent: A Dataset for Mining Open-source Pre-trained Model Packages
Authors:
- Wenxin JiangWJWenxin Jiang
- Nicholas SynovicNSNicholas Synovic
- Purvish JajalPJPurvish Jajal
- Taylor R. SchorlemmerTSTaylor R. Schorlemmer
- Arav TewariATArav Tewari
- Bhavesh PareekBPBhavesh Pareek
- George K. ThiruvathukalGTGeorge K. Thiruvathukal
- James C. DavisJDJames C. Davis
Preprint
Abstract
Due to the cost of developing and training deep learning models from scratch, machine learning engineers have begun to reuse pre-trained models (PTMs) and fine-tune them for downstream tasks. PTM registries known as "model hubs" support engineers in distributing and reusing deep learning models. PTM packages include pre-trained weights, documentation, model architectures, datasets, and metadata. Mining the information in PTM packages will enable the discovery of engineering phenomena and tools to support software engineers. However, accessing this information is difficult - there are many PTM registries, and both the registries and the individual packages may have rate limiting for accessing the data. We present an open-source dataset, PTMTorrent, to facilitate the evaluation and understanding of PTM packages. This paper describes the creation, structure, usage, and limitations of the dataset. The dataset includes a snapshot of 5 model hubs and a total of 15,913 PTM packages. These packages are represented in a uniform data schema for cross-hub mining. We describe prior uses of this data and suggest research opportunities for mining using our dataset. The PTMTorrent dataset (v1) is available at: https://app.globus.org/file-manager?origin_id=55e17a6e-9d8f-11ed-a2a2-8383522b48d9&origin_path=%2F~%2F. Our dataset generation tools are available on GitHub: https://doi.org/10.5281/zenodo.7570357.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- SoK: Taxonomy of Attacks on Open-Source Software Supply Chains2023
- An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry2023
- A Literature Review of Using Machine Learning in Software Development Life Cycle StagesIEEE Access, 2021
- World of code: enabling a research workflow for mining and analyzing the universe of open source VCS dataEmpirical Software Engineering, 2021
- TravisTorrent: Synthesizing Travis CI and GitHub for Full-Stack Research on Continuous Integration2017
- How Open Source Projects Use Static Code Analysis Tools in Continuous Integration Pipelines2017
- Oops, My Tests Broke the Build: An Explorative Analysis of Travis CI with GitHub2017
- Understanding user participation in online communities: A systematic literature review of empirical studiesComputers in Human Behavior, 2015
- An exploratory study of the pull-based software development model2014
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009