Fast probabilistic file fingerprinting for big data
Open Access
- 15 February 2013
- journal article
- Published by Springer Nature in BMC Genomics
- Vol. 14 (S2) , S8
Abstract
Biological data acquisition is raising new challenges, both in data analysis and handling. Not only is it proving hard to analyze the data at the rate it is generated today, but simply reading and transferring data files can be prohibitively slow due to their size. This primarily concerns logistics within and between data centers, but is also important for workstation users in the analysis phase. Common usage patterns, such as comparing and transferring files, are proving computationally expensive and are tying down shared resources.Keywords
This publication has 10 references indexed in Scilit:
- Fast probabilistic file fingerprinting for big dataBMC Genomics, 2013
- NCBI GEO: archive for functional genomics data sets--10 years onNucleic Acids Research, 2010
- Managing and Analyzing Next-Generation Sequence DataPLOS Computational Biology, 2009
- Big data: Welcome to the petacentreNature, 2008
- ArrayExpress--a public database of microarray experiments and gene expression profilesNucleic Acids Research, 2006
- Set reconciliation with nearly optimal communication complexityIEEE Transactions on Information Theory, 2003
- Fast PDA synchronization using characteristic polynomial interpolation2003
- Mersenne twisterACM Transactions on Modeling and Computer Simulation, 1998
- The MD5 Message-Digest Algorithm1992
- Universal classes of hash functions (Extended Abstract)1977