Rachel Wicks
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Rachel Wicks has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Matt Post
Microsoft Redmond Washington USA
3 shared publications - Kartik Ravisankar
University of Maryland
1 shared publication - Kevin Duh
Johns Hopkins University
1 shared publication Matt Post
Johns Hopkins University
1 shared publicationPhilipp Koehn
Johns Hopkins University
1 shared publication- Philipp Koehn
University Johns Hopkins University
1 shared publication
Latest Publications
- Open Access
Token-level Ensembling of Models with Different Vocabularies
Model ensembling is a technique to combine the predicted distributions of two or more models, often leading to improved robustness and performance. For ensembling in text generation, the next token's probability distribution is derived from a weighted sum of the distributions of each individual model. This requires the underlying models to share the same subword vocabulary, limiting the applicability of ensembling, since many open-sourced models have distinct vocabularies. In research settings, experimentation or upgrades to vocabularies may introduce multiple vocabulary sizes. This paper proposes an inference-time only algorithm that allows for ensembling models with different vocabularies, without the need to learn additional parameters or alter the underlying models. Instead, the algorithm ensures that tokens generated by the ensembled models \textit{agree} in their surface form. We apply this technique to combinations of traditional encoder-decoder models and decoder-only LLMs and evaluate on machine translation. In addition to expanding to model pairs that were previously incapable of token-level ensembling, our algorithm frequently improves translation performance over either model individually.
Open AccessToken-level Ensembling of Models with Different Vocabularies
Model ensembling is a technique to combine the predicted distributions of two or more models, often leading to improved robustness and performance. For ensembling in text generation, the next token's probability distribution is derived from a weighted sum of the distributions of each individual model. This requires the underlying models to share the same subword vocabulary, limiting the applicability of ensembling, since many open-sourced models have distinct vocabularies. In research settings, experimentation or upgrades to vocabularies may introduce multiple vocabulary sizes. This paper proposes an inference-time only algorithm that allows for ensembling models with different vocabularies, without the need to learn additional parameters or alter the underlying models. Instead, the algorithm ensures that tokens generated by the ensembled models \textit{agree} in their surface form. We apply this technique to combinations of traditional encoder-decoder models and decoder-only LLMs and evaluate on machine translation. In addition to expanding to model pairs that were previously incapable of token-level ensembling, our algorithm frequently improves translation performance over either model individually.
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 9876-9890
Recovering document annotations for sentence-level bitext
Rachel Wicks, Matt Post, Philipp Koehn. Findings of the Association for Computational Linguistics ACL 2024. 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 9876-9890
Open AccessRecovering document annotations for sentence-level bitext
Rachel Wicks, Matt Post, Philipp Koehn. Findings of the Association for Computational Linguistics ACL 2024. 2024.
- Open Access
Identifying Context-Dependent Translations for Evaluation Set Production
A major impediment to the transition to context-aware machine translation is the absence of good evaluation metrics and test sets. Sentences that require context to be translated correctly are rare in test sets, reducing the utility of standard corpus-level metrics such as COMET or BLEU. On the other hand, datasets that annotate such sentences are also rare, small in scale, and available for only a few languages. To address this, we modernize, generalize, and extend previous annotation pipelines to produce CTXPRO, a tool that identifies subsets of parallel documents containing sentences that require context to correctly translate five phenomena: gender, formality, and animacy for pronouns, verb phrase ellipsis, and ambiguous noun inflections. The input to the pipeline is a set of hand-crafted, per-language, linguistically-informed rules that select contextual sentence pairs using coreference, part-of-speech, and morphological features provided by state-of-the-art tools. We apply this pipeline to seven languages pairs (EN into and out-of DE, ES, FR, IT, PL, PT, and RU) and two datasets (OpenSubtitles and WMT test sets), and validate its performance using both overlap with previous work and its ability to discriminate a contextual MT system from a sentence-based one. We release the CTXPRO pipeline and data as open source.
Open AccessIdentifying Context-Dependent Translations for Evaluation Set Production
A major impediment to the transition to context-aware machine translation is the absence of good evaluation metrics and test sets. Sentences that require context to be translated correctly are rare in test sets, reducing the utility of standard corpus-level metrics such as COMET or BLEU. On the other hand, datasets that annotate such sentences are also rare, small in scale, and available for only a few languages. To address this, we modernize, generalize, and extend previous annotation pipelines to produce CTXPRO, a tool that identifies subsets of parallel documents containing sentences that require context to correctly translate five phenomena: gender, formality, and animacy for pronouns, verb phrase ellipsis, and ambiguous noun inflections. The input to the pipeline is a set of hand-crafted, per-language, linguistically-informed rules that select contextual sentence pairs using coreference, part-of-speech, and morphological features provided by state-of-the-art tools. We apply this pipeline to seven languages pairs (EN into and out-of DE, ES, FR, IT, PL, PT, and RU) and two datasets (OpenSubtitles and WMT test sets), and validate its performance using both overlap with previous work and its ability to discriminate a contextual MT system from a sentence-based one. We release the CTXPRO pipeline and data as open source.
- Top Cited PapersOpen Access
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 452-467
Identifying Context-Dependent Translations for Evaluation Set Production
Rachel Wicks, Matt Post. Proceedings of the Eighth Conference on Machine Translation. 2023.
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 452-467
Top Cited PapersOpen AccessIdentifying Context-Dependent Translations for Evaluation Set Production
Rachel Wicks, Matt Post. Proceedings of the Eighth Conference on Machine Translation. 2023.
- Open Access
- 1 January 2022
- Published by Association for Computational Linguistics (ACL)
- p. 148-153
The Effects of Language Token Prefixing for Multilingual Machine Translation
Rachel Wicks, Kevin Duh. Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 2022.
- 1 January 2022
- Published by Association for Computational Linguistics (ACL)
- p. 148-153
Open AccessThe Effects of Language Token Prefixing for Multilingual Machine Translation
Rachel Wicks, Kevin Duh. Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 2022.