Siddharth Jha
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Siddharth Jha has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
Lutfi Eren Erdogan
2 shared publicationsAmir Gholami
1 shared publication- Amir Gholami
University of California Berkeley
1 shared publication - Coleman Hooper
University of California Berkeley
1 shared publication - Gopala Anumanchipalli
UC Berkeley United States
1 shared publication Kurt Keutzer
1 shared publication
Latest Publications
- Open Access
Characterizing Prompt Compression Methods for Long Context Inference
Long context inference presents challenges at the system level with increased compute and memory requirements, as well as from an accuracy perspective in being able to reason over long contexts. Recently, several methods have been proposed to compress the prompt to reduce the context length. However, there has been little work on comparing the different proposed methods across different tasks through a standardized analysis. This has led to conflicting results. To address this, here we perform a comprehensive characterization and evaluation of different prompt compression methods. In particular, we analyze extractive compression, summarization-based abstractive compression, and token pruning methods. Surprisingly, we find that extractive compression often outperforms all the other approaches, and enables up to 10x compression with minimal accuracy degradation. Interestingly, we also find that despite several recent claims, token pruning methods often lag behind extractive compression. We only found marginal improvements on summarization tasks.
Open AccessCharacterizing Prompt Compression Methods for Long Context Inference
Long context inference presents challenges at the system level with increased compute and memory requirements, as well as from an accuracy perspective in being able to reason over long contexts. Recently, several methods have been proposed to compress the prompt to reduce the context length. However, there has been little work on comparing the different proposed methods across different tasks through a standardized analysis. This has led to conflicting results. To address this, here we perform a comprehensive characterization and evaluation of different prompt compression methods. In particular, we analyze extractive compression, summarization-based abstractive compression, and token pruning methods. Surprisingly, we find that extractive compression often outperforms all the other approaches, and enables up to 10x compression with minimal accuracy degradation. Interestingly, we also find that despite several recent claims, token pruning methods often lag behind extractive compression. We only found marginal improvements on summarization tasks.
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 80-88
TinyAgent: Function Calling at the Edge
Lutfi Eren Erdogan, Nicholas Lee, Siddharth Jha, Sehoon Kim, Ryan Tabrizi, Suhong Moon, Coleman Richard Charles Hooper, Gopala Anumanchipalli, Kurt Keutzer, Amir Gholami. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 80-88
Open AccessTinyAgent: Function Calling at the Edge
Lutfi Eren Erdogan, Nicholas Lee, Siddharth Jha, Sehoon Kim, Ryan Tabrizi, Suhong Moon, Coleman Richard Charles Hooper, Gopala Anumanchipalli, Kurt Keutzer, Amir Gholami. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2024.