High Energy Physics Data Popularity : ATLAS Datasets Popularity Case Study
Open Access
- 1 September 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 22-28
Abstract
The amount of scientific data generated by the LHC experiments has hit the exabyte scale. These data are transferred, processed and analyzed in hundreds of computing centers. The popularity of data among individual physicists and University groups has become one of the key factors of efficient data management and processing. It was actively used during LHC Run 1 and Run 2 by the experiments for the central data processing, and allowed the optimization of data placement policies and to spread the workload more evenly over the existing computing resources. Besides the central data processing, the LHC experiments provide storage and computing resources for physics analysis to thousands of users. Taking into account the significant increase of data volume and processing time after the collider upgrade for the High Luminosity Runs (2027– 2036) an intelligent data placement based on data access pattern becomes even more crucial than at the beginning of LHC. In this study we provide a detailed exploration of data popularity using ATLAS data samples. In addition, we analyze the geolocations of computing sites where the data were processed, and the locality of the home institutes of users carrying out physics analysis. Cartography visualization, based on this data, allows the correlation of existing data placement with physics needs, providing a better understanding of data utilization by different categories of user’s tasks.Keywords
Funding Information
- Russian Science Foundation
This publication has 7 references indexed in Scilit:
- CRIC: a unified information system for WLCG and beyondEPJ Web of Conferences, 2019
- Rucio: Scientific Data ManagementComputing and Software for Big Science, 2019
- PanDA for ATLAS distributed computing in the next decadeJournal of Physics: Conference Series, 2017
- C3PO - A Dynamic Data Placement Agent for ATLAS Distributed Data ManagementJournal of Physics: Conference Series, 2017
- Disk storage management for LHCb based on Data Popularity estimatorJournal of Physics: Conference Series, 2015
- PD2P: PanDA Dynamic Data Placement for ATLASJournal of Physics: Conference Series, 2012
- The ATLAS Experiment at the CERN Large Hadron ColliderJournal of Instrumentation, 2008