xPQA: Cross-Lingual Product Question Answering across 12 Languages
Preprint
Abstract
Product Question Answering (PQA) systems are key in e-commerce applications to provide responses to customers' questions as they shop for products. While existing work on PQA focuses mainly on English, in practice there is need to support multiple customer languages while leveraging product information available in English. To study this practical industrial task, we present xPQA, a large-scale annotated cross-lingual PQA dataset in 12 languages across 9 branches, and report results in (1) candidate ranking, to select the best English candidate containing the information to answer a non-English question; and (2) answer generation, to generate a natural-sounding non-English answer based on the selected English candidate. We evaluate various approaches involving machine translation at runtime or offline, leveraging multilingual pre-trained LMs, and including or excluding xPQA training data. We find that (1) In-domain data is essential as cross-lingual rankers trained on other domains perform poorly on the PQA task; (2) Candidate ranking often prefers runtime-translation approaches while answer generation prefers multilingual approaches; (3) Translating offline to augment multilingual models helps candidate ranking mainly on languages with non-Latin scripts; and helps answer generation mainly on languages with Latin scripts. Still, there remains a significant performance gap between the English and the cross-lingual test sets.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Crosslingual Generalization through Multitask Finetuning2023
- Toward Personalized Answer Generation in E-Commerce via Multi-perspective Preference ModelingACM Transactions on Information Systems, 2022
- MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question AnsweringTransactions of the Association for Computational Linguistics, 2021
- Meaningful Answer Generation of E-Commerce Question-AnsweringACM Transactions on Information Systems, 2021
- XOR QA: Cross-lingual Open-Retrieval Question Answering2021
- TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse LanguagesTransactions of the Association for Computational Linguistics, 2020
- From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers2020
- EXAMS: A Multi-subject High School Examinations Dataset for Cross-lingual and Multilingual Question Answering2020
- MLQA: Evaluating Cross-lingual Extractive Question Answering2020
- On the Cross-lingual Transferability of Monolingual Representations2020