Avi Caciularu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Avi Caciularu has a total of 41 co-authors. Here are the top co-authors by number of shared publications:
- David Burshtein
School of Electrical Engineering, Tel-Aviv University, Tel-Aviv 6997801, Israel
3 shared publications - Jacob Goldberger
Faculty of Engineering and Bar-Ilan Institute of Nanotechnology and Advanced Materials (BINA), Bar-Ilan University, Ramat Gan, Israel 5290002
3 shared publications - Arman Cohan
Allen Institute for AI ♡ University of Washington Yale University
2 shared publications Idan Rejwan
Tel Aviv University and AI21 Labs Tel Aviv, Israel
2 shared publications- Ido Dagan
Bar-Ilan University
2 shared publications - Ido Dagan
Independent Researcher, Ramat Gan 5270006, Israel
2 shared publications
Latest Publications
- Open Access
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 2381-2390
Identifying User Goals From UI Trajectories
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 2381-2390
Open AccessIdentifying User Goals From UI Trajectories
- Open Access
Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering
The integration of multi-document pre-training objectives into language models has resulted in remarkable improvements in multi-document downstream tasks. In this work, we propose extending this idea by pre-training a generic multi-document model from a novel cross-document question answering pre-training objective. To that end, given a set (or cluster) of topically-related documents, we systematically generate semantically-oriented questions from a salient sentence in one document and challenge the model, during pre-training, to answer these questions while "peeking" into other topically-related documents. In a similar manner, the model is also challenged to recover the sentence from which the question was generated, again while leveraging cross-document information. This novel multi-document QA formulation directs the model to better recover cross-text informational relations, and introduces a natural augmentation that artificially increases the pre-training data. Further, unlike prior multi-document models that focus on either classification or summarization tasks, our pre-training objective formulation enables the model to perform tasks that involve both short text generation (e.g., QA) and long text generation (e.g., summarization). Following this scheme, we pre-train our model -- termed QAmden -- and evaluate its performance across several multi-document tasks, including multi-document QA, summarization, and query-focused summarization, yielding improvements of up to 7%, and significantly outperforms zero-shot GPT-3.5 and GPT-4.
Open AccessPeek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering
The integration of multi-document pre-training objectives into language models has resulted in remarkable improvements in multi-document downstream tasks. In this work, we propose extending this idea by pre-training a generic multi-document model from a novel cross-document question answering pre-training objective. To that end, given a set (or cluster) of topically-related documents, we systematically generate semantically-oriented questions from a salient sentence in one document and challenge the model, during pre-training, to answer these questions while "peeking" into other topically-related documents. In a similar manner, the model is also challenged to recover the sentence from which the question was generated, again while leveraging cross-document information. This novel multi-document QA formulation directs the model to better recover cross-text informational relations, and introduces a natural augmentation that artificially increases the pre-training data. Further, unlike prior multi-document models that focus on either classification or summarization tasks, our pre-training objective formulation enables the model to perform tasks that involve both short text generation (e.g., QA) and long text generation (e.g., summarization). Following this scheme, we pre-train our model -- termed QAmden -- and evaluate its performance across several multi-document tasks, including multi-document QA, summarization, and query-focused summarization, yielding improvements of up to 7%, and significantly outperforms zero-shot GPT-3.5 and GPT-4.
- 14 January 2023
- Published by Elsevier in International Journal of Electrical Power & Energy Systems
- Vol. 148
Explaining the decisions of power quality disturbance classifiers using latent space features
- 14 January 2023
- Published by Elsevier in International Journal of Electrical Power & Energy Systems
- Vol. 148
Explaining the decisions of power quality disturbance classifiers using latent space features
- Open Access
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 7038-7058
Revisiting Sentence Union Generation as a Testbed for Text Consolidation
Eran Hirsch, Valentina Pyatkin, Ruben Wolhandler, Avi Caciularu, Asi Shefer, Ido Dagan. Findings of the Association for Computational Linguistics: ACL 2023. 2023.
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 7038-7058
Open AccessRevisiting Sentence Union Generation as a Testbed for Text Consolidation
Eran Hirsch, Valentina Pyatkin, Ruben Wolhandler, Avi Caciularu, Asi Shefer, Ido Dagan. Findings of the Association for Computational Linguistics: ACL 2023. 2023.
- Open Access
Long Context Question Answering via Supervised Contrastive Learning
Long-context question answering (QA) tasks require reasoning over a long document or multiple documents. Addressing these tasks often benefits from identifying a set of evidence spans (e.g., sentences), which provide supporting evidence for answering the question. In this work, we propose a novel method for equipping long-context QA models with an additional sequence-level objective for better identification of the supporting evidence. We achieve this via an additional contrastive supervision signal in finetuning, where the model is encouraged to explicitly discriminate supporting evidence sentences from negative ones by maximizing question-evidence similarity. The proposed additional loss exhibits consistent improvements on three different strong long-context transformer models, across two challenging question answering benchmarks -- HotpotQA and QAsper.
Open AccessLong Context Question Answering via Supervised Contrastive Learning
Long-context question answering (QA) tasks require reasoning over a long document or multiple documents. Addressing these tasks often benefits from identifying a set of evidence spans (e.g., sentences), which provide supporting evidence for answering the question. In this work, we propose a novel method for equipping long-context QA models with an additional sequence-level objective for better identification of the supporting evidence. We achieve this via an additional contrastive supervision signal in finetuning, where the model is encouraged to explicitly discriminate supporting evidence sentences from negative ones by maximizing question-evidence similarity. The proposed additional loss exhibits consistent improvements on three different strong long-context transformer models, across two challenging question answering benchmarks -- HotpotQA and QAsper.
- Open Access
CDLM: Cross-Document Language Modeling
We introduce a new pretraining approach geared for multi-document language modeling, incorporating two key ideas into the masked language modeling self-supervised objective. First, instead of considering documents in isolation, we pretrain over sets of multiple related documents, encouraging the model to learn cross-document relationships. Second, we improve over recent long-range transformers by introducing dynamic global attention that has access to the entire input to predict masked tokens. We release CDLM (Cross-Document Language Model), a new general language model for multi-document setting that can be easily applied to downstream tasks. Our extensive analysis shows that both ideas are essential for the success of CDLM, and work in synergy to set new state-of-the-art results for several multi-text tasks. Code and models are available at https://github.com/aviclu/CDLM.
Open AccessCDLM: Cross-Document Language Modeling
We introduce a new pretraining approach geared for multi-document language modeling, incorporating two key ideas into the masked language modeling self-supervised objective. First, instead of considering documents in isolation, we pretrain over sets of multiple related documents, encouraging the model to learn cross-document relationships. Second, we improve over recent long-range transformers by introducing dynamic global attention that has access to the entire input to predict masked tokens. We release CDLM (Cross-Document Language Model), a new general language model for multi-document setting that can be easily applied to downstream tasks. Our extensive analysis shows that both ideas are essential for the success of CDLM, and work in synergy to set new state-of-the-art results for several multi-text tasks. Code and models are available at https://github.com/aviclu/CDLM.
- Open Access
GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps
We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network layers, GAM offers improved visual explanations, when compared to existing alternatives. The algorithmic advantages of GAM are explained in detail, and validated empirically, where it is shown that GAM outperforms its alternatives across various tasks and datasets.
Open AccessGAM: Explainable Visual Similarity and Classification via Gradient Activation Maps
We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network layers, GAM offers improved visual explanations, when compared to existing alternatives. The algorithmic advantages of GAM are explained in detail, and validated empirically, where it is shown that GAM outperforms its alternatives across various tasks and datasets.
- Open Access
On the Evolution of Word Order
Most natural languages have a predominant or fixed word order. For example in English the word order is usually Subject-Verb-Object. This work attempts to explain this phenomenon as well as other typological findings regarding word order from a functional perspective. In particular, we examine whether fixed word order provides a functional advantage, explaining why these languages are prevalent. To this end, we consider an evolutionary model of language and demonstrate, both theoretically and using genetic algorithms, that a language with a fixed word order is optimal. We also show that adding information to the sentence, such as case markers and noun-verb distinction, reduces the need for fixed word order, in accordance with the typological findings.
Open AccessOn the Evolution of Word Order
Most natural languages have a predominant or fixed word order. For example in English the word order is usually Subject-Verb-Object. This work attempts to explain this phenomenon as well as other typological findings regarding word order from a functional perspective. In particular, we examine whether fixed word order provides a functional advantage, explaining why these languages are prevalent. To this end, we consider an evolutionary model of language and demonstrate, both theoretically and using genetic algorithms, that a language with a fixed word order is optimal. We also show that adding information to the sentence, such as case markers and noun-verb distinction, reduces the need for fixed word order, in accordance with the typological findings.
- Open Access
- 1 January 2021
- Published by Assoc. for Computational Linguistics Bulgaria in Proceedings of the Student Research Workshop Associated With Ranlp 2021
- p. 162-166
On the Evolution of Word Order
- 1 January 2021
- Published by Assoc. for Computational Linguistics Bulgaria in Proceedings of the Student Research Workshop Associated With Ranlp 2021
- p. 162-166
Open AccessOn the Evolution of Word Order
- Open Access
- 1 January 2021
- Published by Association for Computational Linguistics (ACL)
- p. 294-301
Denoising Word Embeddings by Averaging in a Shared Space
Avi Caciularu, Ido Dagan, Jacob Goldberger. Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics. 2021.
- 1 January 2021
- Published by Association for Computational Linguistics (ACL)
- p. 294-301
Open AccessDenoising Word Embeddings by Averaging in a Shared Space
Avi Caciularu, Ido Dagan, Jacob Goldberger. Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics. 2021.