MG7: Configurable and scalable 16S metagenomics data analysis
Preprint
Abstract
As part of the Cambrian explosion of omics data, metagenomics brings to the table a specific, defining trait: its social essence. The meta prefix exerts its influence, with multitudes manifesting themselves everywhere; from samples to data analysis, from actors involved to (present and future) applications. Of these dimensions, data analysis is where needs lay further from what current tools provide. Key features are, among others, scalability, reproducibility, data provenance and distribution, process identity and versioning. These are the goals guiding our work in MG7, a 16S metagenomics data analysis system. The basic principle is a new approach to data analysis, where configuration, processes, or data locations are static, type-checked and subject to the standard evolution of a well-maintained software project. Cloud computing, in its Amazon Web Services incarnation, when coupled with these ideas, produces a robust, safely configurable, scalable tool. Processes, data, machine behaviors and their dependencies are expressed using a set of libraries which bring as much as possible checking and validation to the type level, without sacrificing expressiveness. Together they form a toolkit for defining scalable cloud-based workflows composed of stateless computations, with a static reproducible specification of dependencies, behavior and wiring of all steps. The modeling of taxonomy data is done using Bio4j, where the new paradigm of graph databases allows for both a simple expression of taxonomic assignment tasks and the calculation of taxa abundance values considering the hierarchic structure of the taxonomy tree. MG7 includes a new 16S reference database, 16S-DB7 , built with a flexible and sustainable update system, and the possibility of project-driven personalization.Keywords
This publication has 22 references indexed in Scilit:
- Marine metagenomics as a source for bioprospectingMarine Genomics, 2015
- Genome researchers raise alarm over big dataNature, 2015
- Metagenomics of extreme environmentsCurrent Opinion in Microbiology, 2015
- Metagenomics meets time series analysis: unraveling microbial community dynamicsCurrent Opinion in Microbiology, 2015
- Identifying personal microbiomes using metagenomic codesProceedings of the National Academy of Sciences, 2015
- Combining metagenomics, metatranscriptomics and viromics to explore novel microbial interactions: towards a systems-level understanding of human microbiomeComputational and Structural Biotechnology Journal, 2015
- Ribosomal Database Project: data and tools for high throughput rRNA analysisNucleic Acids Research, 2013
- Computational meta'omics for microbial community studiesMolecular Systems Biology, 2013
- FLASH: fast length adjustment of short reads to improve genome assembliesBioinformatics, 2011
- The 2010 Nucleic Acids Research Database Issue and online Database Collection: a community of data resourcesNucleic Acids Research, 2009