HarmonicIO: Scalable Data Stream Processing for Scientific Datasets
Authors:
- Preechakorn TorruangwatthanaPTPreechakorn Torruangwatthana
- Hakan WieslanderHWHakan Wieslander
- Ben BlameyBBBen Blamey
- Andreas HellanderAHAndreas Hellander
- Salman ToorSTSalman Toor
- 1 July 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 879-882
Abstract
Many streaming frameworks have been introduced to deal with the needs for online analysis of massive datasets. Scientific applications often require significant changes to make them compatible with these frameworks. Other issues include tight coupling with the underlying infrastructure, shared computing environment, static topology settings, and complex configuration. In this article we present HarmonicIO, a lightweight streaming framework specialized for scientific datasets. It boasts a smart dynamic architecture, is highly elastic, and enforces a clear separation between framework components and application execution environment using container technology.Keywords
This publication has 7 references indexed in Scilit:
- SNIC Science Cloud (SSC): A National-Scale Cloud Infrastructure for Swedish Academia2017
- Apache SparkCommunications of the ACM, 2016
- Optimization of Lipid Nanoparticle Formulations for mRNA Delivery in Vivo with Fractional Factorial and Definitive Screening DesignsNano Letters, 2015
- Twitter Heron2015
- Fluorescence live cell imaging2014
- Streamflex2007
- Data Stream Processing2007