The ATLAS Distributed Data Management project: Past and Future
Open Access
- 13 December 2012
- conference paper
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 396 (3) , 032045
Abstract
ATLAS has recorded more than 8 petabyte(PB) of RAW data since the LHC started running at the end of 2009. Many more derived data products and complimentary simulation data have also been produced by the collaboration and, in total, 90PB are currently stored in the Worldwide LHC Computing Grid by ATLAS. All these data are managed by the ATLAS Distributed Data Management system, called Don Quijote 2 (DQ2). DQ2 has evolved rapidly to help ATLAS Computing operations manage these large quantities of data across the many grid sites at which ATLAS runs, and to help ATLAS physicists get access to these data. In this paper, we describe new and improved DQ2 services, and the experience of data management operation in ATLAS computing, showing how these services enable the management of PB scale computing operations. We also present the concepts of the new version of the ATLAS Distributed Data Management (DDM) system, Rucio.Keywords
This publication has 8 references indexed in Scilit:
- Popularity framework to process dataset traces and its application on dynamic replica reduction in the ATLAS experimentJournal of Physics: Conference Series, 2011
- Overview of ATLAS PanDA Workload ManagementJournal of Physics: Conference Series, 2011
- The consistency service of the ATLAS Distributed Data Management systemJournal of Physics: Conference Series, 2011
- ATLAS Operations: Experience and Evolution in the Data Taking EraJournal of Physics: Conference Series, 2011
- The ATLAS Tier-0: Overview and operational experienceJournal of Physics: Conference Series, 2010
- Dynamic and adaptive data-management in ATLASJournal of Physics: Conference Series, 2010
- Managing ATLAS data on a petabyte-scale with DQ2Journal of Physics: Conference Series, 2008
- The ATLAS computing modelJournal of Physics: Conference Series, 2008