MAMohammad Akbari

Mohammad Akbari

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Mohammad Akbari has a total of 45 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Automatic Kernel Generation with Large Language Models: Taxonomies, Evaluations, and Cross-Platform Challenges

    by Kevin Cannons, Ahmad Rezaei, Saeed Ranjbar Alvar, Mohsen Gholami, Zongzuo Wang, Zheng Shen, Boxing Chen, Yong Zhang, Yi Jiang, Mohammad Akbari

    Large language models (LLMs) have recently emerged as a promising tool for automatic generation and optimization of high-performance compute kernels. This rapidly growing area spans across hardware platforms and targets programming models such as AscendC, CUDA, and Triton. However, the literature remains fragmented, with substantial variation in generation workflows, training strategies, evaluation protocols, benchmark assumptions, hardware platforms, and reported metrics. This survey provides a structured review of LLM-based automatic kernel generation. We introduce a taxonomy based on the generation-time workflow, separating direct generation, refinement via structured feedback, search-based generation, and adaptive tool-orchestrating agentic systems. We further analyze orthogonal design axes including learning strategy, input representation, output representation, and hardware platform. Beyond method classification, we examine evaluation practices, including benchmark data, harnesses, integrity policies, as well as performance metrics, and discuss why reported results are often difficult to compare. Finally, we use Ascend NPU kernel generation as a cross-platform case study, highlighting challenges introduced by AscendC, host-device co-design, toolchain constraints, and limited public data. We conclude by identifying open problems in robust evaluation, higher-level task optimization, cross-platform transfer, production deployment, and benchmark design.

  • Open Access

    Blind Watermarking for Tabular Datasets in Machine Learning: A Primary Key-Free Method

    by Xin Che, Mohammad Akbari, Shaoxin Li, David Yue, Yong Zhang, Lingyang Chu

    Watermarking is widely used to protect the ownership of tabular datasets. Comparing to non-blind watermarking that requires the original dataset to detect watermark, blind watermarking is more secure because it can accurately detect watermark without using the original dataset, which restricts public access to the original dataset. Existing blind watermarking methods rely on either a primary key or a virtual primary key to watermark a tabular dataset. However, these watermarks can be easily removed by an attacker with little to no impact on the dataset’s machine learning utility, because a primary key can be significantly modified without affecting the machine learning utility, and a virtual primary key is fragile to slight modifications on the dataset. Can we design a blind watermarking method without relying on a primary key or virtual primary key? In this article, we tackle this challenging task by a novel primary key-free method that embeds a sinusoidal signal as the watermark into a discrete-time signal constructed from the tabular dataset. We theoretically analyzed the robustness of our watermark against six challenging attacks, and empirically validated the outstanding performance of our method through comprehensive experiments on two real-world datasets.

  • Open Access

    Task-Agnostic Language Model Watermarking via High Entropy Passthrough Layers

    by Vaden Masrani, Mohammad Akbari, David Ming Xuan Yue, Ahmad Rezaei, Yong Zhang

    In the era of costly pre-training of large language models, ensuring the intellectual property rights of model owners, and insuring that said models are responsibly deployed, is becoming increasingly important. To this end, we propose model watermarking via passthrough layers, which are added to existing pre-trained networks and trained using a self-supervised loss such that the model produces high-entropy output when prompted with a unique private key, and acts normally otherwise. Unlike existing model watermarking methods, our method is fully task-agnostic, and can be applied to both classification and sequence-to-sequence tasks without requiring advanced access to downstream fine-tuning datasets. We evaluate the proposed passthrough layers on a wide range of downstream tasks, and show experimentally our watermarking method achieves a near-perfect watermark extraction accuracy and false-positive rate in most cases without damaging original model performance. Additionally, we show our method is robust to both downstream fine-tuning, fine-pruning, and layer removal attacks, and can be trained in a fraction of the time required to train the original model. Code is available in the paper.


  • Open Access

    CASP: Compression of Large Multimodal Models Based on Attention Sparsity

    by Mohsen Gholami, Mohammad Akbari, Kevin Cannons, Yong Zhang

    In this work, we propose an extreme compression technique for Large Multimodal Models (LMMs). While previous studies have explored quantization as an efficient post-training compression method for Large Language Models (LLMs), low-bit compression for multimodal models remains under-explored. The redundant nature of inputs in multimodal models results in a highly sparse attention matrix. We theoretically and experimentally demonstrate that the attention matrix's sparsity bounds the compression error of the Query and Key weight matrices. Based on this, we introduce CASP, a model compression technique for LMMs. Our approach performs a data-aware low-rank decomposition on the Query and Key weight matrix, followed by quantization across all layers based on an optimal bit allocation process. CASP is compatible with any quantization technique and enhances state-of-the-art 2-bit quantization methods (AQLM and QuIP#) by an average of 21% on image- and video-language benchmarks.

  • Open Access

    Task-Agnostic Language Model Watermarking via High Entropy Passthrough Layers

    by Vaden Masrani, Mohammad Akbari, David Ming Xuan Yue, Ahmad Rezaei, Yong Zhang

    In the era of costly pre-training of large language models, ensuring the intellectual property rights of model owners, and insuring that said models are responsibly deployed, is becoming increasingly important. To this end, we propose model watermarking via passthrough layers, which are added to existing pre-trained networks and trained using a self-supervised loss such that the model produces high-entropy output when prompted with a unique private key, and acts normally otherwise. Unlike existing model watermarking methods, our method is fully task-agnostic, and can be applied to both classification and sequence-to-sequence tasks without requiring advanced access to downstream fine-tuning datasets. We evaluate the proposed passthrough layers on a wide range of downstream tasks, and show experimentally our watermarking method achieves a near-perfect watermark extraction accuracy and false-positive rate in most cases without damaging original model performance. Additionally, we show our method is robust to both downstream fine-tuning, fine-pruning, and layer removal attacks, and can be trained in a fraction of the time required to train the original model. Code is available in the paper.

  • Primary Key Free Watermarking for Numerical Tabular Datasets in Machine Learning

    by Xin Che, Mohammad Akbari, Shaoxin Li, David Yue, Yong Zhang, Lingyang Chu


  • LaWa: Using Latent Space for In-Generation Image Watermarking

    by Ahmad Rezaei, Mohammad Akbari, Saeed Ranjbar Alvar, Arezou Fatemi, Yong Zhang


  • Open Access

    GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation

    by Mohsen Gholami, Mohammad Akbari, Cindy Hu, Vaden Masrani, Z. Jane Wang, Yong Zhang

    Knowledge distillation from LLMs is essential for the efficient deployment of language models. Prior works have proposed data generation using LLMs for preparing distilled models. We argue that generating data with LLMs is prone to sampling mainly from the center of original content distribution. This limitation hinders the distilled model from learning the true underlying data distribution and to forget the tails of the distributions (samples with lower probability). To this end, we propose GOLD, a task-agnostic data generation and knowledge distillation framework, which employs an iterative out-of-distribution-guided feedback mechanism for the LLM. As a result, the generated data improves the generalizability of distilled models. An energy-based OOD evaluation approach is also introduced to deal with noisy generated data. Our extensive experiments on 10 different classification and sequence-to-sequence tasks in NLP show that GOLD respectively outperforms prior arts and the LLM with an average improvement of 5% and 14%. We will also show that the proposed method is applicable to less explored and novel tasks. The code is available.

  • Open Access

    Learned Image Compression with Gaussian-Laplacian-Logistic Mixture Model and Concatenated Residual Modules

    by Haisheng Fu, Feng Liang, Jianping Lin, Bing Li, Mohammad Akbari, Jie Liang, Guohe Zhang, Dong Liu, Chengjie Tu, Jingning Han

    Recently deep learning-based image compression methods have achieved significant achievements and gradually outperformed traditional approaches including the latest standard Versatile Video Coding (VVC) in both PSNR and MS-SSIM metrics. Two key components of learned image compression are the entropy model of the latent representations and the encoding/decoding network architectures. Various models have been proposed, such as autoregressive, softmax, logistic mixture, Gaussian mixture, and Laplacian. Existing schemes only use one of these models. However, due to the vast diversity of images, it is not optimal to use one model for all images, even different regions within one image. In this paper, we propose a more flexible discretized Gaussian-Laplacian-Logistic mixture model (GLLMM) for the latent representations, which can adapt to different contents in different images and different regions of one image more accurately and efficiently, given the same complexity. Besides, in the encoding/decoding network design part, we propose a concatenated residual blocks (CRB), where multiple residual blocks are serially connected with additional shortcut connections. The CRB can improve the learning ability of the network, which can further improve the compression performance. Experimental results using the Kodak, Tecnick-100 and Tecnick-40 datasets show that the proposed scheme outperforms all the leading learning-based methods and existing compression standards including VVC intra coding (4:4:4 and 4:2:0) in terms of the PSNR and MS-SSIM. The source code is available at \url{https://github.com/fengyurenpingsheng}

  • Open Access

    GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation

    by Mohsen Gholami, Mohammad Akbari, Tianxi Hu, Vaden Masrani, Z. Wang, Yong Zhang

    Mohsen Gholami, Mohammad Akbari, Tianxi Hu, Vaden Masrani, Z. Wang, Yong Zhang. Findings of the Association for Computational Linguistics: NAACL 2024. 2024.