MIMahzabeen Islam

Mahzabeen Islam

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Mahzabeen Islam has a total of 82 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    PIM-Potential: Broadening the Acceleration Reach of PIM Architectures

    by Johnathan Alsop, Shaizeen Aga, Mohamed Ibrahim, Mahzabeen Islam, Nuwan Jayasena, Andrew McCrabb


  • Open Access

    T3: Transparent Tracking & Triggering for Fine-grained Overlap of Compute & Collectives

    by Suchita Pati, Shaizeen Aga, Mahzabeen Islam, Nuwan Jayasena, Matthew D. Sinclair

    Large Language Models increasingly rely on distributed techniques for their training and inference. These techniques require communication across devices which can reduce scaling efficiency as the number of devices increases. While some distributed techniques can overlap, and thus, hide this communication with independent computations, techniques such as Tensor Parallelism (TP) inherently serialize communication with model execution. One approach to hide this serialized communication is to interleave it with the producer operation (of the communicated data) in a fine-grained manner. However, this fine-grained interleaving of communication and computation in software can be difficult. Furthermore, as with any concurrent execution, it requires compute and memory resources to be shared between computation and communication, causing resource contention that reduces overlapping efficacy. To overcome these challenges, we propose T3 which applies hardware-software co-design to transparently overlap serialized communication while minimizing resource contention with compute. T3 transparently fuses producer operations with the subsequent communication via a simple configuration of the producer's output address space and requires minor software changes. At the hardware level, T3 adds a lightweight track and trigger mechanism to orchestrate the producer's compute, and communication. It further uses compute-enhanced memories for communication's attendant compute. As a result, T3 reduces resource contention, and efficiently overlaps serialized communication with computation. For important Transformer models like T-NLG, T3 speeds up communication-heavy sublayers by 30% geomean (max 47%) and reduces data movement by 22% geomean (max 36%). Furthermore, T3's benefits persist as models scale: geomean 29% for sublayers in ~500-billion parameter models, PALM and MT-NLG.


  • Open Access

    A Research Retrospective on AMD's Exascale Computing Journey

    by Gabriel H. Loh, Michael J. Schulte, Mike Ignatowski, Vignesh Adhinarayanan, Shaizeen Aga, Derrick Aguren, Varun Agrawal, Ashwin M. Aji, Johnathan Alsop, Paul Bauman, et al.

    The pace of advancement of the top-end supercomputers historically followed an exponential curve similar to (and driven in part by) Moore's Law. Shortly after hitting the petaflop mark, the community started looking ahead to the next milestone: Exascale. However, many obstacles were already looming on the horizon, such as the slowing of Moore's Law, and others like the end of Dennard Scaling had already arrived. Anticipating significant challenges for the overall high-performance computing (HPC) community to achieve the next 1000x improvement, the U.S. Department of Energy (DOE) launched the Exascale Computing Program to enable and accelerate fundamental research across the many technologies needed to achieve exascale computing. AMD had the opportunity to contribute to the so-called "*Forward" programs from the DOE, which were a series of public-private partnerships focused on research and co-design activities covering compute architectures, interconnects, memory systems, chiplets and packaging, software stacks, applications, and more. Some of the research from these programs can now be found in the world's first exascale supercomputer, some were a little ahead of their time and may have an impact in the coming years, and others simply did not pan out. In this paper, we provide a retrospective of AMD's nearly decade-long research journey covering how we tried to predict the architecture of a supercomputer a decade into the future, what we got right, what we got wrong, and some of the insights and learnings that we discovered along the way.


  • Open Access

    On-the-fly Page Migration and Address Reconciliation for Heterogeneous Memory Systems

    by Mahzabeen Islam, Shashank Adavally, Marko Scrbak, Krishna Kavi

    For efficient placement of data in flat-address heterogeneous memory systems consisting of fast (e.g., 3D-DRAM) and slow memories (e.g., NVM), we present a hardware-based page migration technique. Unlike epoch-based approaches that migrate heavily accessed (“hot”) pages from slow to fast memories at each epoch interval, we migrate a page immediately when it becomes hot (“on-the-fly”), using hardware in user-transparent manner and with minimal OS intervention. The management of physical addresses due to page relocation becomes cumbersome and requires costly OS intervention. We use a small hardware remap table to keep track of new physical addresses of the migrated pages. This limits address reconciliation to occur only at periodic evictions of old remap entries. Also, we propose a hardware-orchestrated light-weight address reconciliation process. For our studied heterogeneous memory system, on-the-fly page migration with hardware-assisted address reconciliation provides 74% and 24% IPC improvements, on average for a set of SPEC CPU2006 workloads when compared to a baseline without any page migration and a system with on-the-fly page migration using OS-based address reconciliation, respectively. Furthermore, we present an analytical model for classifying applications as page migration friendly (applications that show performance gains from page migration) or unfriendly based on memory access behavior.


  • HBM-Resident Prefetching for Heterogeneous Memory System

    by Mahzabeen Islam, Krishna M. Kavi, Mitesh Meswani, Soumik Banerjee, Nuwan Jayasena


  • Processing-in-Memory: Exploring the Design Space

    by Marko Scrbak, Mahzabeen Islam, Krishna M. Kavi, Mike Ignatowski, Nuwan Jayasena