Ahmad Rezaei
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Ahmad Rezaei has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
- Mohammad Akbari
Huawei Technologies Canada Co. Ltd
2 shared publications - Yong Zhang
Huawei Technologies Canada Co., Ltd
2 shared publications - Boxing Chen
Research Center Huawei Noah's Ark Lab Montreal Canada
1 shared publication David Ming Xuan Yue
Huawei Technologies Canada Co. Ltd
1 shared publicationKevin Cannons
Huawei Technologies Canada Co. Ltd
1 shared publication- Kevin Cannons
Huawei Technologies Canada Co., LTD., Canada
1 shared publication
Latest Publications
- Open Access
- 26 October 2026
- Published by MDPI AG in Preprints.org
Automatic Kernel Generation with Large Language Models: Taxonomies, Evaluations, and Cross-Platform Challenges
Large language models (LLMs) have recently emerged as a promising tool for automatic generation and optimization of high-performance compute kernels. This rapidly growing area spans across hardware platforms and targets programming models such as AscendC, CUDA, and Triton. However, the literature remains fragmented, with substantial variation in generation workflows, training strategies, evaluation protocols, benchmark assumptions, hardware platforms, and reported metrics. This survey provides a structured review of LLM-based automatic kernel generation. We introduce a taxonomy based on the generation-time workflow, separating direct generation, refinement via structured feedback, search-based generation, and adaptive tool-orchestrating agentic systems. We further analyze orthogonal design axes including learning strategy, input representation, output representation, and hardware platform. Beyond method classification, we examine evaluation practices, including benchmark data, harnesses, integrity policies, as well as performance metrics, and discuss why reported results are often difficult to compare. Finally, we use Ascend NPU kernel generation as a cross-platform case study, highlighting challenges introduced by AscendC, host-device co-design, toolchain constraints, and limited public data. We conclude by identifying open problems in robust evaluation, higher-level task optimization, cross-platform transfer, production deployment, and benchmark design.
- 26 October 2026
- Published by MDPI AG in Preprints.org
Open AccessAutomatic Kernel Generation with Large Language Models: Taxonomies, Evaluations, and Cross-Platform Challenges
Large language models (LLMs) have recently emerged as a promising tool for automatic generation and optimization of high-performance compute kernels. This rapidly growing area spans across hardware platforms and targets programming models such as AscendC, CUDA, and Triton. However, the literature remains fragmented, with substantial variation in generation workflows, training strategies, evaluation protocols, benchmark assumptions, hardware platforms, and reported metrics. This survey provides a structured review of LLM-based automatic kernel generation. We introduce a taxonomy based on the generation-time workflow, separating direct generation, refinement via structured feedback, search-based generation, and adaptive tool-orchestrating agentic systems. We further analyze orthogonal design axes including learning strategy, input representation, output representation, and hardware platform. Beyond method classification, we examine evaluation practices, including benchmark data, harnesses, integrity policies, as well as performance metrics, and discuss why reported results are often difficult to compare. Finally, we use Ascend NPU kernel generation as a cross-platform case study, highlighting challenges introduced by AscendC, host-device co-design, toolchain constraints, and limited public data. We conclude by identifying open problems in robust evaluation, higher-level task optimization, cross-platform transfer, production deployment, and benchmark design.
- Open Access
CPPO: Contrastive Perception for Vision Language Policy Optimization
We introduce CPPO, a Contrastive Perception Policy Optimization method for finetuning vision-language models (VLMs). While reinforcement learning (RL) has advanced reasoning in language models, extending it to multimodal reasoning requires improving both the perception and reasoning aspects. Prior works tackle this challenge mainly with explicit perception rewards, but disentangling perception tokens from reasoning tokens is difficult, requiring extra LLMs, ground-truth data, forced separation of perception from reasoning by policy model, or applying rewards indiscriminately to all output tokens. CPPO addresses this problem by detecting perception tokens via entropy shifts in the model outputs under perturbed input images. CPPO then extends the RL objective function with a Contrastive Perception Loss (CPL) that enforces consistency under information-preserving perturbations and sensitivity under information-removing ones. Experiments show that CPPO surpasses previous perception-rewarding methods, while avoiding extra models, making training more efficient and scalable.
Open AccessCPPO: Contrastive Perception for Vision Language Policy Optimization
We introduce CPPO, a Contrastive Perception Policy Optimization method for finetuning vision-language models (VLMs). While reinforcement learning (RL) has advanced reasoning in language models, extending it to multimodal reasoning requires improving both the perception and reasoning aspects. Prior works tackle this challenge mainly with explicit perception rewards, but disentangling perception tokens from reasoning tokens is difficult, requiring extra LLMs, ground-truth data, forced separation of perception from reasoning by policy model, or applying rewards indiscriminately to all output tokens. CPPO addresses this problem by detecting perception tokens via entropy shifts in the model outputs under perturbed input images. CPPO then extends the RL objective function with a Contrastive Perception Loss (CPL) that enforces consistency under information-preserving perturbations and sensitivity under information-removing ones. Experiments show that CPPO surpasses previous perception-rewarding methods, while avoiding extra models, making training more efficient and scalable.
- Open Access
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (23) , 24849-24857
Task-Agnostic Language Model Watermarking via High Entropy Passthrough Layers
In the era of costly pre-training of large language models, ensuring the intellectual property rights of model owners, and insuring that said models are responsibly deployed, is becoming increasingly important. To this end, we propose model watermarking via passthrough layers, which are added to existing pre-trained networks and trained using a self-supervised loss such that the model produces high-entropy output when prompted with a unique private key, and acts normally otherwise. Unlike existing model watermarking methods, our method is fully task-agnostic, and can be applied to both classification and sequence-to-sequence tasks without requiring advanced access to downstream fine-tuning datasets. We evaluate the proposed passthrough layers on a wide range of downstream tasks, and show experimentally our watermarking method achieves a near-perfect watermark extraction accuracy and false-positive rate in most cases without damaging original model performance. Additionally, we show our method is robust to both downstream fine-tuning, fine-pruning, and layer removal attacks, and can be trained in a fraction of the time required to train the original model. Code is available in the paper.
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (23) , 24849-24857
Open AccessTask-Agnostic Language Model Watermarking via High Entropy Passthrough Layers
In the era of costly pre-training of large language models, ensuring the intellectual property rights of model owners, and insuring that said models are responsibly deployed, is becoming increasingly important. To this end, we propose model watermarking via passthrough layers, which are added to existing pre-trained networks and trained using a self-supervised loss such that the model produces high-entropy output when prompted with a unique private key, and acts normally otherwise. Unlike existing model watermarking methods, our method is fully task-agnostic, and can be applied to both classification and sequence-to-sequence tasks without requiring advanced access to downstream fine-tuning datasets. We evaluate the proposed passthrough layers on a wide range of downstream tasks, and show experimentally our watermarking method achieves a near-perfect watermark extraction accuracy and false-positive rate in most cases without damaging original model performance. Additionally, we show our method is robust to both downstream fine-tuning, fine-pruning, and layer removal attacks, and can be trained in a fraction of the time required to train the original model. Code is available in the paper.