Collective knowledge: organizing research projects as a database of reusable components and portable workflows with common interfaces
- 29 March 2021
- journal article
- review article
- Published by The Royal Society in Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Vol. 379 (2197) , 20200211
Abstract
This article provides the motivation and overview of the Collective Knowledge Framework (CK or cKnowledge). The CK concept is to decompose research projects into reusable components that encapsulate research artifacts and provide unified application programming interfaces (APIs), command-line interfaces (CLIs), meta descriptions and common automation actions for related artifacts. The CK framework is used to organize and manage research projects as a database of such components. Inspired by the USB ‘plug and play’ approach for hardware, CK also helps to assemble portable workflows that can automatically plug in compatible components from different users and vendors (models, datasets, frameworks, compilers, tools). Such workflows can build and run algorithms on different platforms and environments in a unified way using the customizable CK program pipeline with software detection plugins and the automatic installation of missing packages. This article presents a number of industrial projects in which the modular CK approach was successfully validated in order to automate benchmarking, auto-tuning and co-design of efficient software and hardware for machine learning and artificial intelligence in terms of speed, accuracy, energy, size and various costs. The CK framework also helped to automate the artifact evaluation process at several computer science conferences as well as to make it easier to reproduce, compare and reuse research techniques from published papers, deploy them in production, and automatically adapt them to continuously changing datasets, models and systems. The long-term goal is to accelerate innovation by connecting researchers and practitioners to share and reuse all their knowledge, best practices, artifacts, workflows and experimental results in a common, portable and reproducible format at cKnowledge.io . This article is part of the theme issue ‘Reliability and reproducibility in computational science: implementing verification, validation and uncertainty quantification in silico ’.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learning2019
- Extreme scale multi-physics simulations of the tsunamigenic 2004 sumatra megathrust earthquake2017
- The Popper Convention: Making Reproducible Systems Evaluation Practical2017
- The FAIR Guiding Principles for scientific data management and stewardshipScientific Data, 2016
- The Spack package manager2015
- Community-driven reviewing and validation of publications2014
- The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloudNucleic Acids Research, 2013
- EasyBuild: Building Software with Ease2012
- Milepost GCC: Machine Learning Enabled Self-tuning CompilerInternational Journal of Parallel Programming, 2011
- Producing wrong data without doing anything obviously wrong!ACM SIGPLAN Notices, 2009