MEMark Endrei

Mark Endrei

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Mark Endrei has a total of 18 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    A Cache Interaction Graph for Data Locality Optimization

    by Chao Jin, David Abramson, Mo'ATh Qutaish, Mark Endrei, Bronis R. de Supinski


  • Open Access

    Enhancing Research Through FAIR-Compliant Imaging Data Repositories: The PITSCHI – a solution for Large-Scale/multi-user imaging facilities

    by Rubbiya A Ali, Mark Endrei, Tom He Mason, Nishanthi D Mudiyanselage, Jake Carroll, Roger Wepf

    Scientific research produces ever-larger and more complicated datasets because of the advancements of imaging technologies, especially in disciplines that demand multi-modal imaging, spectroscopy and high-resolution microscopy. Repositories that follow the FAIR (Findable, Accessible, Interoperable, and Reusable) data principles are necessary for efficiently managing, sharing, and reusing this data. Maximising the usability and accessibility of imaging data requires us to adhere to FAIR principles, which provide easy data integration, long-term preservation, and discovery across a range of research areas. To address these needs, the Centre for Microscopy and Microanalysis (CMM), in partnership with UQ’s Research Computing Centre (RCC), at the University of Queensland implemented PITSCHI (Particle Imaging depoT using Storage CacHing Infrastructure), which offers a reliable, FAIR-compliant solution for handling large imaging datasets with little manual intervention. PITSCHI is an advanced data repository built to handle large imaging datasets. PITSCHI immediately feeds data into a Clowder-based system [ 1] from any sophisticated scientific instruments, such as mass spectrometry devices, x-ray machines, scanning and transmission electron microscopes, and other microscope techniques. Large-scale microscope datasets may be shared, annotated, organised, and analysed with ease thanks to this integration, which also reduces the need for human data processing. PITSCHI is an end-to-end process for capturing and transferring raw data to storage collections, indexing it in an image repository with automated metadata harvesting, and enabling users to view, search, and process their data on external platforms, [ Fig. 1- 2]. Easy access to UQ’s Research Data Manager collections is made possible by PITSCHI’s complete integration with MeDiCI, a high-performance data storage fabric, and the university’s Research Infrastructure Management System (RIMS). Fig. 1: PITSCHI workflow Fig. 2: PITSCHI Email notifications

  • Statistical and machine learning models for optimizing energy in parallel applications

    by Mark Endrei, Chao Jin, Minh Ngoc Dinh, David Abramson, Heidi Poxon, Luiz DeRose, Bronis R de Supinski

    Rising power costs and constraints are driving a growing focus on the energy efficiency of high performance computing systems. The unique characteristics of a particular system and workload and their effect on performance and energy efficiency are typically difficult for application users to assess and to control. Settings for optimum performance and energy efficiency can also diverge, so we need to identify trade-off options that guide a suitable balance between energy use and performance. We present statistical and machine learning models that only require a small number of runs to make accurate Pareto-optimal trade-off predictions using parameters that users can control. We study model training and validation using several parallel kernels and more complex workloads, including Algebraic Multigrid (AMG), Large-scale Atomic Molecular Massively Parallel Simulator, and Livermore Unstructured Lagrangian Explicit Shock Hydrodynamics. We demonstrate that we can train the models using as few as 12 runs, with prediction error of less than 10%. Our AMG results identify trade-off options that provide up to 45% improvement in energy efficiency for around 10% performance loss. We reduce the sample measurement time required for AMG by 90%, from 13 h to 74 min.


  • Artefacts archive for energy efficiency models for scientific applications on supercomputers

    by Mark Endrei, David Abramson

    The University of Queensland's institutional repository, UQ eSpace, aims to create global visibility and accessibility of UQ’s scholarly research.

  • Energy Efficiency Modeling of Parallel Applications

    by Mark Endrei, Chao Jin, Minh Ngoc Dinh, David Abramson, Heidi Poxon, Luiz DeRose, Bronis R. de Supinski

    Energy efficiency has become increasingly important in high performance computing (HPC), as power constraints and costs escalate. Workload and system characteristics form a complex optimization search space in which optimal settings for energy efficiency and performance often diverge. Thus, we must identify trade-off options for performance and energy efficiency to find the desired balance between them. We present an innovative statistical model that accurately predicts the Pareto optimal performance and energy efficiency trade-off options using only user-controllable parameters. Our approach can also tolerate both measurement and model errors. We study model training and validation using several HPC kernels, then explore the feasibility of applying the model to more complex workloads, including AMG and LAMMPS. We can calibrate an accurate model from as few as 12 runs, with prediction error of less than 10%. Our results identify trade-off options allowing up to 40% improvement in energy efficiency at the cost of under 20% performance loss. For AMG, we reduce the required sample measurement time from 13 hours to 74 minutes (about 90%).


  • HPC Model

    by Mark Endrei, David Abramson

    The University of Queensland's institutional repository, UQ eSpace, aims to create global visibility and accessibility of UQ’s scholarly research.

  • A survey on software methods to improve the energy efficiency of parallel computing

    by Chao Jin, Bronis R de Supinski, David Abramson, Heidi Poxon, Luiz DeRose, Minh Ngoc Dinh, Mark Endrei, Elizabeth R Jessup

    Energy consumption is one of the top challenges for achieving the next generation of supercomputing. Codesign of hardware and software is critical for improving energy efficiency (EE) for future large-scale systems. Many architectural power-saving techniques have been developed, and most hardware components are approaching physical limits. Accordingly, parallel computing software, including both applications and systems, should exploit power-saving hardware innovations and manage efficient energy use. In addition, new power-aware parallel computing methods are essential to decrease energy usage further. This article surveys software-based methods that aim to improve EE for parallel computing. It reviews the methods that exploit the characteristics of parallel scientific applications, including load imbalance and mixed precision of floating-point (FP) calculations, to improve EE. In addition, this article summarizes widely used methods to improve power usage at different granularities, such as the whole system and per application. In particular, it describes the most important techniques to measure and to achieve energy-efficient usage of various parallel computing facilities, including processors, memories, and networks. Overall, this article reviews the state-of-the-art of energy-efficient methods for parallel computing to motivate researchers to achieve optimal parallel computing under a power budget constraint.