Andrew Pawloski
Research Subjects
Andrew Pawloski has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Daniel Pilone
Element 84, Inc
2 shared publications - Amanda Tan
Science Institute, University of Washington, Seattle, WA 98195, USA
1 shared publication - Anthony Arendt
University of Washington
1 shared publication Dennis Fatland
Unversity of Washington
1 shared publication- Joseph Hamman
CarbonPlan San Francisco California USA
1 shared publication - Matthew Hanson
Development Seed
1 shared publication
Latest Publications
- Open Access
- 16 January 2020
- Published by Wiley in ESS Open Archive
Acquiring, processing, and analyzing open broadcast science data with extremely low latency at scale with AWS Groundstation and cloud processing.
Many high value data products made available from agencies such as NASA, NOAA, and USGS are broadcast in the clear directly from the satellite. Traditional access patterns involve downlinking the data to an agency ground segment, processing by a science team, and subsequent product distribution through a program such as the NOAA Big Data Initiative or a NASA Distributed Active Archive Center (DAAC). This talk demonstrates the ability to acquire, process, and analyze open broadcast science data entirely in the cloud. Leveraging the new AWS Elastic Ground Station service, we demonstrate selecting and scheduling satellite acquisitions for open science data then leveraging the scalable nature of the cloud to execute a full processing stack. This talk demonstrates the ability to acquire L0 data, leveraging open data processing algorithms from agencies like NASA and NOAA, and a highly scalable, cost effective, and secure processing architecture to produce near real-time data. By taking advantage of direct satellite acquisition and cloud scale processing, we demonstrate reduced latency from acquisition to data availability for scenarios like disaster response, tipping and cueing, and location monitoring. The deployed cloud stack leverages open source components and demonstrates ingest and archival of L0 data, processing to L1 products and above, horizontally scaling data processing algorithms, metadata generation, and search and discovery via APIs and a simple serverless UI.
- 16 January 2020
- Published by Wiley in ESS Open Archive
Open AccessAcquiring, processing, and analyzing open broadcast science data with extremely low latency at scale with AWS Groundstation and cloud processing.
Many high value data products made available from agencies such as NASA, NOAA, and USGS are broadcast in the clear directly from the satellite. Traditional access patterns involve downlinking the data to an agency ground segment, processing by a science team, and subsequent product distribution through a program such as the NOAA Big Data Initiative or a NASA Distributed Active Archive Center (DAAC). This talk demonstrates the ability to acquire, process, and analyze open broadcast science data entirely in the cloud. Leveraging the new AWS Elastic Ground Station service, we demonstrate selecting and scheduling satellite acquisitions for open science data then leveraging the scalable nature of the cloud to execute a full processing stack. This talk demonstrates the ability to acquire L0 data, leveraging open data processing algorithms from agencies like NASA and NOAA, and a highly scalable, cost effective, and secure processing architecture to produce near real-time data. By taking advantage of direct satellite acquisition and cloud scale processing, we demonstrate reduced latency from acquisition to data availability for scenarios like disaster response, tipping and cueing, and location monitoring. The deployed cloud stack leverages open source components and demonstrates ingest and archival of L0 data, processing to L1 products and above, horizontally scaling data processing algorithms, metadata generation, and search and discovery via APIs and a simple serverless UI.
- Open Access
- 14 January 2020
- Published by Wiley in ESS Open Archive
The Pangeo Platform: a community-driven open-source big data environment
In this presentation, we will describe the [Pangeo Project](http://pangeo.io), a coordinated community effort with support from NASA, NSF, AWS, Microsoft Azure and Google Cloud, to develop interactive and reproducible open source workflows for discovery, visualization, and quantitative analysis of large datasets used for research in the Earth Sciences. The Pangeo computational platform is based on JupyterHub and deployed wherever the data is stored. Python libraries such as Xarray, Rasterio, and Dask enable distributed parallel computations on HPC and Kubernetes clusters. We will discuss the design concepts central to the Pangeo platform and highlight specific applications using NASA satellite data archives on AWS. We will discuss recent progress in the integration of data discovery tools (e.g. STAC, CMR, Intake) with cloud-native storage formats for multidimensional data types (Cloud-Optimized Geotiff, Zarr, etc.) and highlight how they can be used to construct elegant, robust and reproducible scientific workflows. Finally, we will discuss performance, security, transferability across public cloud platforms, cost to operate, and approaches to encourage a cultural shift in scientific computation through educational events.
- 14 January 2020
- Published by Wiley in ESS Open Archive
Open AccessThe Pangeo Platform: a community-driven open-source big data environment
In this presentation, we will describe the [Pangeo Project](http://pangeo.io), a coordinated community effort with support from NASA, NSF, AWS, Microsoft Azure and Google Cloud, to develop interactive and reproducible open source workflows for discovery, visualization, and quantitative analysis of large datasets used for research in the Earth Sciences. The Pangeo computational platform is based on JupyterHub and deployed wherever the data is stored. Python libraries such as Xarray, Rasterio, and Dask enable distributed parallel computations on HPC and Kubernetes clusters. We will discuss the design concepts central to the Pangeo platform and highlight specific applications using NASA satellite data archives on AWS. We will discuss recent progress in the integration of data discovery tools (e.g. STAC, CMR, Intake) with cloud-native storage formats for multidimensional data types (Cloud-Optimized Geotiff, Zarr, etc.) and highlight how they can be used to construct elegant, robust and reproducible scientific workflows. Finally, we will discuss performance, security, transferability across public cloud platforms, cost to operate, and approaches to encourage a cultural shift in scientific computation through educational events.