Liwen Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Liwen Zhang has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Haifeng Wang
Baidu Inc., Beijing, China
3 shared publications - Hua Wu
School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China
3 shared publications - Pengzhi Gao
Baidu Inc. No 10 Shangdi 10th Street 100085 Beijing China
3 shared publications - Zhongjun He
Baidu Inc. No 10 Shangdi 10th Street 100085 Beijing China
3 shared publications
Latest Publications
- Open Access
Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization
Multilingual sentence representations are the foundation for similarity-based bitext mining, which is crucial for scaling multilingual neural machine translation (NMT) system to more languages. In this paper, we introduce MuSR: a one-for-all Multilingual Sentence Representation model that supports more than 220 languages. Leveraging billions of English-centric parallel corpora, we train a multilingual Transformer encoder, coupled with an auxiliary Transformer decoder, by adopting a multilingual NMT framework with CrossConST, a cross-lingual consistency regularization technique proposed in Gao et al. (2023). Experimental results on multilingual similarity search and bitext mining tasks show the effectiveness of our approach. Specifically, MuSR achieves superior performance over LASER3 (Heffernan et al., 2022) which consists of 148 independent multilingual sentence encoders.
Open AccessLearning Multilingual Sentence Representations with Cross-lingual Consistency Regularization
Multilingual sentence representations are the foundation for similarity-based bitext mining, which is crucial for scaling multilingual neural machine translation (NMT) system to more languages. In this paper, we introduce MuSR: a one-for-all Multilingual Sentence Representation model that supports more than 220 languages. Leveraging billions of English-centric parallel corpora, we train a multilingual Transformer encoder, coupled with an auxiliary Transformer decoder, by adopting a multilingual NMT framework with CrossConST, a cross-lingual consistency regularization technique proposed in Gao et al. (2023). Experimental results on multilingual similarity search and bitext mining tasks show the effectiveness of our approach. Specifically, MuSR achieves superior performance over LASER3 (Heffernan et al., 2022) which consists of 148 independent multilingual sentence encoders.
- Open Access
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization
The multilingual neural machine translation (NMT) model has a promising capability of zero-shot translation, where it could directly translate between language pairs unseen during training. For good transfer performance from supervised directions to zero-shot directions, the multilingual NMT model is expected to learn universal representations across different languages. This paper introduces a cross-lingual consistency regularization, CrossConST, to bridge the representation gap among different languages and boost zero-shot translation performance. The theoretical analysis shows that CrossConST implicitly maximizes the probability distribution for zero-shot translation, and the experimental results on both low-resource and high-resource benchmarks show that CrossConST consistently improves the translation performance. The experimental analysis also proves that CrossConST could close the sentence representation gap and better align the representation space. Given the universality and simplicity of CrossConST, we believe it can serve as a strong baseline for future multilingual NMT research.
Open AccessImproving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization
The multilingual neural machine translation (NMT) model has a promising capability of zero-shot translation, where it could directly translate between language pairs unseen during training. For good transfer performance from supervised directions to zero-shot directions, the multilingual NMT model is expected to learn universal representations across different languages. This paper introduces a cross-lingual consistency regularization, CrossConST, to bridge the representation gap among different languages and boost zero-shot translation performance. The theoretical analysis shows that CrossConST implicitly maximizes the probability distribution for zero-shot translation, and the experimental results on both low-resource and high-resource benchmarks show that CrossConST consistently improves the translation performance. The experimental analysis also proves that CrossConST could close the sentence representation gap and better align the representation space. Given the universality and simplicity of CrossConST, we believe it can serve as a strong baseline for future multilingual NMT research.
- Open Access
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 243-262
Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization
Pengzhi Gao, Liwen Zhang, Zhongjun He, Hua Wu, Haifeng Wang. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track. 2023.
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 243-262
Open AccessLearning Multilingual Sentence Representations with Cross-lingual Consistency Regularization
Pengzhi Gao, Liwen Zhang, Zhongjun He, Hua Wu, Haifeng Wang. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track. 2023.