ApiQ: Finetuning of 2-Bit Quantized Large Language Model
Authors:
Open Access
- 1 January 2024
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 20996-21020
Abstract
Baohao Liao, Christian Herold, Shahram Khadivi, Christof Monz. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and AccelerationGetMobile: Mobile Computing and Communications, 2025
- An empirical study of LLaMA3 quantization: from LLMs to MLLMsVisual Intelligence, 2024
- Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation2022
- Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token2022
- Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning2021
- Are NLP Models really able to Solve Simple Math Word Problems?2021
- WinoGrande: An Adversarial Winograd Schema Challenge at ScaleProceedings of the AAAI Conference on Artificial Intelligence, 2020
- PIQA: Reasoning about Physical Commonsense in Natural LanguageProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015