Yuqing Xie
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yuqing Xie has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
- Jimmy Lin
University of Waterloo, Waterloo, Canada
4 shared publications - Elman Mansimov
Pioneer Centre for Artificial Intelligence AWS AI Labs IT University of Copenhagen
2 shared publications - Wei Zhong
University of Waterloo
2 shared publications Yi Zhang
AWS AI Labs University of Waterloo
2 shared publicationsYi-An Lai
AWS AI Labs University of Waterloo
2 shared publications- Cynthia Huang
School of Computer Science University of Waterloo
1 shared publication
Latest Publications
- 11 September 2023
- Published by Springer Nature
- p. 209-223
Answer Retrieval for Math Questions Using Structural and Dense Retrieval
- 11 September 2023
- Published by Springer Nature
- p. 209-223
Answer Retrieval for Math Questions Using Structural and Dense Retrieval
- Open Access
Approximating Human-Like Few-shot Learning with GPT-based Compression
In this work, we conceptualize the learning process as information compression. We seek to equip generative pre-trained models with human-like learning capabilities that enable data compression during inference. We present a novel approach that utilizes the Generative Pre-trained Transformer (GPT) to approximate Kolmogorov complexity, with the aim of estimating the optimal Information Distance for few-shot learning. We first propose using GPT as a prior for lossless text compression, achieving a noteworthy compression ratio. Experiment with LLAMA2-7B backbone achieves a compression ratio of 15.5 on enwik9. We justify the pre-training objective of GPT models by demonstrating its equivalence to the compression length, and, consequently, its ability to approximate the information distance for texts. Leveraging the approximated information distance, our method allows the direct application of GPT models in quantitative text similarity measurements. Experiment results show that our method overall achieves superior performance compared to embedding and prompt baselines on challenging NLP tasks, including semantic similarity, zero and one-shot text classification, and zero-shot text ranking.
Open AccessApproximating Human-Like Few-shot Learning with GPT-based Compression
In this work, we conceptualize the learning process as information compression. We seek to equip generative pre-trained models with human-like learning capabilities that enable data compression during inference. We present a novel approach that utilizes the Generative Pre-trained Transformer (GPT) to approximate Kolmogorov complexity, with the aim of estimating the optimal Information Distance for few-shot learning. We first propose using GPT as a prior for lossless text compression, achieving a noteworthy compression ratio. Experiment with LLAMA2-7B backbone achieves a compression ratio of 15.5 on enwik9. We justify the pre-training objective of GPT models by demonstrating its equivalence to the compression length, and, consequently, its ability to approximate the information distance for texts. Leveraging the approximated information distance, our method allows the direct application of GPT models in quantitative text similarity measurements. Experiment results show that our method overall achieves superior performance compared to embedding and prompt baselines on challenging NLP tasks, including semantic similarity, zero and one-shot text classification, and zero-shot text ranking.
- Open Access
Improving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusion
When upgrading neural models to a newer version, new errors that were not encountered in the legacy version can be introduced, known as regression errors. This inconsistent behavior during model upgrade often outweighs the benefits of accuracy gain and hinders the adoption of new models. To mitigate regression errors from model upgrade, distillation and ensemble have proven to be viable solutions without significant compromise in performance. Despite the progress, these approaches attained an incremental reduction in regression which is still far from achieving backward-compatible model upgrade. In this work, we propose a novel method, Gated Fusion, that promotes backward compatibility via learning to mix predictions between old and new models. Empirical results on two distinct model upgrade scenarios show that our method reduces the number of regression errors by 62% on average, outperforming the strongest baseline by an average of 25%.
Open AccessImproving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusion
When upgrading neural models to a newer version, new errors that were not encountered in the legacy version can be introduced, known as regression errors. This inconsistent behavior during model upgrade often outweighs the benefits of accuracy gain and hinders the adoption of new models. To mitigate regression errors from model upgrade, distillation and ensemble have proven to be viable solutions without significant compromise in performance. Despite the progress, these approaches attained an incremental reduction in regression which is still far from achieving backward-compatible model upgrade. In this work, we propose a novel method, Gated Fusion, that promotes backward compatibility via learning to mix predictions between old and new models. Empirical results on two distinct model upgrade scenarios show that our method reduces the number of regression errors by 62% on average, outperforming the strongest baseline by an average of 25%.
- Open Access
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 1010-1022
Improving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusion
Yi-An Lai, Elman Mansimov, Yuqing Xie, Yi Zhang. Findings of the Association for Computational Linguistics: EACL 2023. 2023.
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 1010-1022
Open AccessImproving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusion
Yi-An Lai, Elman Mansimov, Yuqing Xie, Yi Zhang. Findings of the Association for Computational Linguistics: EACL 2023. 2023.
- Open Access
Evaluating Token-Level and Passage-Level Dense Retrieval Models for Math Information Retrieval
With the recent success of dense retrieval methods based on bi-encoders, studies have applied this approach to various interesting downstream retrieval tasks with good efficiency and in-domain effectiveness. Recently, we have also seen the presence of dense retrieval models in Math Information Retrieval (MIR) tasks, but the most effective systems remain classic retrieval methods that consider hand-crafted structure features. In this work, we try to combine the best of both worlds:\ a well-defined structure search method for effective formula search and efficient bi-encoder dense retrieval models to capture contextual similarities. Specifically, we have evaluated two representative bi-encoder models for token-level and passage-level dense retrieval on recent MIR tasks. Our results show that bi-encoder models are highly complementary to existing structure search methods, and we are able to advance the state-of-the-art on MIR datasets.
Open AccessEvaluating Token-Level and Passage-Level Dense Retrieval Models for Math Information Retrieval
With the recent success of dense retrieval methods based on bi-encoders, studies have applied this approach to various interesting downstream retrieval tasks with good efficiency and in-domain effectiveness. Recently, we have also seen the presence of dense retrieval models in Math Information Retrieval (MIR) tasks, but the most effective systems remain classic retrieval methods that consider hand-crafted structure features. In this work, we try to combine the best of both worlds:\ a well-defined structure search method for effective formula search and efficient bi-encoder dense retrieval models to capture contextual similarities. Specifically, we have evaluated two representative bi-encoder models for token-level and passage-level dense retrieval on recent MIR tasks. Our results show that bi-encoder models are highly complementary to existing structure search methods, and we are able to advance the state-of-the-art on MIR datasets.
- Open Access
- 1 January 2022
- Published by Association for Computational Linguistics (ACL)
- p. 1092-1102
Evaluating Token-Level and Passage-Level Dense Retrieval Models for Math Information Retrieval
Wei Zhong, Jheng-Hong Yang, Yuqing Xie, Jimmy Lin. Findings of the Association for Computational Linguistics: EMNLP 2022. 2022.
- 1 January 2022
- Published by Association for Computational Linguistics (ACL)
- p. 1092-1102
Open AccessEvaluating Token-Level and Passage-Level Dense Retrieval Models for Math Information Retrieval
Wei Zhong, Jheng-Hong Yang, Yuqing Xie, Jimmy Lin. Findings of the Association for Computational Linguistics: EMNLP 2022. 2022.
- Open Access
- 1 January 2021
- Published by Association for Computational Linguistics (ACL)
- p. 6589-6602
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates
Yuqing Xie, Yi-An Lai, Yuanjun Xiong, Yi Zhang, Stefano Soatto. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
- 1 January 2021
- Published by Association for Computational Linguistics (ACL)
- p. 6589-6602
Open AccessRegression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates
Yuqing Xie, Yi-An Lai, Yuanjun Xiong, Yi Zhang, Stefano Soatto. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.