H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models

Authors:
  • ZZZhenyu Zhang
    Zhenyu Zhang
  • YSYing Sheng
    Ying Sheng
  • TZTianyi Zhou
    Tianyi Zhou
  • TCTianlong Chen
    Tianlong Chen
  • LZLianmin Zheng
    Lianmin Zheng
  • RCRuisi Cai
    Ruisi Cai
  • ZSZhao Song
    Zhao Song
  • YTYuandong Tian
    Yuandong Tian
  • CRChristopher Ré
    Christopher Ré
  • CBClark Barrett
    Clark Barrett
  • ZWZhangyang Atlas Wang
    Zhangyang Atlas Wang
  • BCBeidi Chen
    Beidi Chen
Publisher website
Google Scholar
Abstract
No abstract available