Alberto Miranda
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Alberto Miranda has a total of 53 co-authors. Here are the top co-authors by number of shared publications:
- Ramon Nou
Barcelona Supercomputing Center
8 shared publications - Toni Cortes
Barcelona Supercomputing Center (BSC) Barcelona Spain
8 shared publications - André Brinkmann
Saarland University, Saarbrücken, Germany
6 shared publications - Marc-André Vef
Johannes Gutenberg University Mainz, Mainz, Germany
5 shared publications Ethan L. Miller
University of California, Santa Cruz, CA
2 shared publications- Jean-Baptiste Besnard
ParaTools SAS
2 shared publications
Latest Publications
- Open Access
- 29 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 35 (9) , 1551-1564
Malleability in Modern HPC Systems: Current Experiences, Challenges, and Future Opportunities
With the increase of complex scientific simulations driven by workflows and heterogeneous workload profiles, managing system resources effectively is essential for improving performance and system throughput, especially due to trends like heterogeneous HPC and deeply integrated systems with on-chip accelerators. For optimal resource utilization, dynamic resource allocation can improve productivity across all system and application levels, by adapting the applications' configurations to the system's resources. In this context, malleable jobs, which can change resources at runtime, can increase the system throughput and resource utilization while bringing various advantages for HPC users (e.g., shorter waiting time). Malleability has received much attention recently, even though it has been an active research area for almost two decades [1]. This paper presents the state-of-the-art of malleable implementations in HPC systems, targeting mainly malleability in compute and I/O resources. Based on our experiences, we state our current concerns and list future opportunities for research.
- 29 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 35 (9) , 1551-1564
Open AccessMalleability in Modern HPC Systems: Current Experiences, Challenges, and Future Opportunities
With the increase of complex scientific simulations driven by workflows and heterogeneous workload profiles, managing system resources effectively is essential for improving performance and system throughput, especially due to trends like heterogeneous HPC and deeply integrated systems with on-chip accelerators. For optimal resource utilization, dynamic resource allocation can improve productivity across all system and application levels, by adapting the applications' configurations to the system's resources. In this context, malleable jobs, which can change resources at runtime, can increase the system throughput and resource utilization while bringing various advantages for HPC users (e.g., shorter waiting time). Malleability has received much attention recently, even though it has been an active research area for almost two decades [1]. This paper presents the state-of-the-art of malleable implementations in HPC systems, targeting mainly malleability in compute and I/O resources. Based on our experiences, we state our current concerns and list future opportunities for research.
- 25 August 2023
- Published by Springer Nature
- p. 3-15
From Static to Malleable: Improving Flexibility and Compatibility in Burst Buffer File Systems
- 25 August 2023
- Published by Springer Nature
- p. 3-15
From Static to Malleable: Improving Flexibility and Compatibility in Burst Buffer File Systems
- Open Access
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 285-290
Adaptive multi-tier intelligent data manager for Exascale
The main objective of the ADMIRE project1 is the creation of an active I/O stack that dynamically adjusts computation and storage requirements through intelligent global coordination, the elasticity of computation and I/O, and the scheduling of storage resources along all levels of the storage hierarchy, while offering quality-of-service (QoS), energy efficiency, and resilience for accessing extremely large data sets in very heterogeneous computing and storage environments. We have developed a framework prototype that is able to dynamically adjust computation and storage requirements through intelligent global coordination, separated control, and data paths, the malleability of computation and I/O, the scheduling of storage resources along all levels of the storage hierarchy, and scalable monitoring techniques. The leading idea in ADMIRE is to co-design applications with ad-hoc storage systems that can be deployed with the application and adapt their computing and I/O behaviour on runtime, using malleability techniques, to increase the performance of applications and the throughput of the applications.
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 285-290
Open AccessAdaptive multi-tier intelligent data manager for Exascale
The main objective of the ADMIRE project1 is the creation of an active I/O stack that dynamically adjusts computation and storage requirements through intelligent global coordination, the elasticity of computation and I/O, and the scheduling of storage resources along all levels of the storage hierarchy, while offering quality-of-service (QoS), energy efficiency, and resilience for accessing extremely large data sets in very heterogeneous computing and storage environments. We have developed a framework prototype that is able to dynamically adjust computation and storage requirements through intelligent global coordination, separated control, and data paths, the malleability of computation and I/O, the scheduling of storage resources along all levels of the storage hierarchy, and scalable monitoring techniques. The leading idea in ADMIRE is to co-design applications with ad-hoc storage systems that can be deployed with the application and adapt their computing and I/O behaviour on runtime, using malleability techniques, to increase the performance of applications and the throughput of the applications.
- 17 January 2020
- Published by Springer Nature in Journal of Computer Science and Technology
- Vol. 35 (1) , 72-91
GekkoFS — A Temporary Burst Buffer File System for HPC Applications
- 17 January 2020
- Published by Springer Nature in Journal of Computer Science and Technology
- Vol. 35 (1) , 72-91
GekkoFS — A Temporary Burst Buffer File System for HPC Applications
- 1 September 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-12
NORNS: Extending Slurm to Support Data-Driven Workflows through Asynchronous Data Staging
As HPC systems move into the Exascale era, parallel file systems are struggling to keep up with the I/O requirements from data-intensive problems. While the inclusion of burst buffers has helped to alleviate this by improving I/O performance, it has also increased the complexity of the I/O hierarchy by adding additional storage layers each with its own semantics. This forces users to explicitly manage data movement between the different storage layers, which, coupled with the lack of interfaces to communicate data dependencies between jobs in a data-driven workflow, prevents resource schedulers from optimizing these transfers to benefit the cluster’s overall performance. This paper proposes several extensions to job schedulers, prototyped using the Slurm scheduling system, to enable users to appropriately express the data dependencies between the different phases in their processing workflows. It also introduces a new service for asynchronous data staging called NORNS that coordinates with the job scheduler to orchestrate data transfers to achieve better resource utilization. Our evaluation shows that a workflow-aware Slurm exploits node-local storage more effectively, reducing the filesystem I/O contention and improving job running times.
- 1 September 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-12
NORNS: Extending Slurm to Support Data-Driven Workflows through Asynchronous Data Staging
As HPC systems move into the Exascale era, parallel file systems are struggling to keep up with the I/O requirements from data-intensive problems. While the inclusion of burst buffers has helped to alleviate this by improving I/O performance, it has also increased the complexity of the I/O hierarchy by adding additional storage layers each with its own semantics. This forces users to explicitly manage data movement between the different storage layers, which, coupled with the lack of interfaces to communicate data dependencies between jobs in a data-driven workflow, prevents resource schedulers from optimizing these transfers to benefit the cluster’s overall performance. This paper proposes several extensions to job schedulers, prototyped using the Slurm scheduling system, to enable users to appropriately express the data dependencies between the different phases in their processing workflows. It also introduces a new service for asynchronous data staging called NORNS that coordinates with the job scheduler to orchestrate data transfers to achieve better resource utilization. Our evaluation shows that a workflow-aware Slurm exploits node-local storage more effectively, reducing the filesystem I/O contention and improving job running times.
- 1 September 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 225-228
ECHOFS: A Scheduler-Guided Temporary Filesystem to Leverage Node-Local NVMS
The growth in data-intensive scientific applications poses strong demands on the HPC storage subsystem, as data needs to be copied from compute nodes to I/O nodes and vice versa for jobs to run. The emerging trend of adding denser, NVM-based burst buffers to compute nodes, however, offers the possibility of using these resources to build temporary file systems with specific I/O optimizations for a batch job. In this work, we present echofs, a temporary filesystem that coordinates with the job scheduler to preload a job's input files into node-local burst buffers. We present the results measured with NVM emulation, and different FS backends with DAX/FUSE on a local node, to show the benefits of our proposal and such coordination.
- 1 September 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 225-228
ECHOFS: A Scheduler-Guided Temporary Filesystem to Leverage Node-Local NVMS
The growth in data-intensive scientific applications poses strong demands on the HPC storage subsystem, as data needs to be copied from compute nodes to I/O nodes and vice versa for jobs to run. The emerging trend of adding denser, NVM-based burst buffers to compute nodes, however, offers the possibility of using these resources to build temporary file systems with specific I/O optimizations for a batch job. In this work, we present echofs, a temporary filesystem that coordinates with the job scheduler to preload a job's input files into node-local burst buffers. We present the results measured with NVM emulation, and different FS backends with DAX/FUSE on a local node, to show the benefits of our proposal and such coordination.
- 1 September 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 319-324
GekkoFS - A Temporary Distributed File System for HPC Applications
We present GekkoFS, a temporary, highly-scalable burst buffer file system which has been specifically optimized for new access patterns of data-intensive High-Performance Computing (HPC) applications. The file system provides relaxed POSIX semantics, only offering features which are actually required by most (not all) applications. It is able to provide scalable I/O performance and reaches millions of metadata operations already for a small number of nodes, significantly outperforming the capabilities of general-purpose parallel file systems.
- 1 September 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 319-324
GekkoFS - A Temporary Distributed File System for HPC Applications
We present GekkoFS, a temporary, highly-scalable burst buffer file system which has been specifically optimized for new access patterns of data-intensive High-Performance Computing (HPC) applications. The file system provides relaxed POSIX semantics, only offering features which are actually required by most (not all) applications. It is able to provide scalable I/O performance and reaches millions of metadata operations already for a small number of nodes, significantly outperforming the capabilities of general-purpose parallel file systems.
- 1 November 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 63-70
Improving OpenStack Swift interaction with the I/O Stack to Enable Software Defined Storage
This paper analyses how OpenStack Swift, a distributed object storage service for a globally used middleware, interacts with the I/O subsystem through the Operating System. This interaction, which seems organised and clean on the middleware side, becomes disordered on the device side when using mechanical disk drives, due to the way threads are used internally to request data. We will show that only modifying the Swift threading model we achieve an 18% mean improvement in performance with objects larger than 512 KiB and obtain a similar performance with smaller objects. Compared to the original scenario, the performance obtained on both scenarios is obtained in a fair way: the bandwidth is shared equally between concurrently accessed objects. Moreover, this threading model allows us to apply techniques for Software Defined Storage (SDS). We show an implementation of a Bandwidth Differentiation technique that can control each data stream and that guarantees a high utilization of the device.
- 1 November 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 63-70
Improving OpenStack Swift interaction with the I/O Stack to Enable Software Defined Storage
This paper analyses how OpenStack Swift, a distributed object storage service for a globally used middleware, interacts with the I/O subsystem through the Operating System. This interaction, which seems organised and clean on the middleware side, becomes disordered on the device side when using mechanical disk drives, due to the way threads are used internally to request data. We will show that only modifying the Swift threading model we achieve an 18% mean improvement in performance with objects larger than 512 KiB and obtain a similar performance with smaller objects. Compared to the original scenario, the performance obtained on both scenarios is obtained in a fair way: the bandwidth is shared equally between concurrently accessed objects. Moreover, this threading model allows us to apply techniques for Software Defined Storage (SDS). We show an implementation of a Bandwidth Differentiation technique that can control each data stream and that guarantees a high utilization of the device.
- 20 October 2017
- Published by Springer Nature
- p. 691-700
HetFS: A Heterogeneous File System for Everyone
- 20 October 2017
- Published by Springer Nature
- p. 691-700
HetFS: A Heterogeneous File System for Everyone