Chengyuan Ma
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Chengyuan Ma has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Benjamin Yao
Amazon
2 shared publications - Jialong Han
Tencent AI Lab China
2 shared publications - Kyumin Lee
Computer Science Department Worcester Polytechnic Institute Worcester MA USA
2 shared publications - Yichuan Li
Worcester Polytechnic Institute
2 shared publications - Derek Liu
Worcester Polytechnic Institute Amazon Alexa AI
1 shared publication Xiaohu Liu
1 shared publication
Latest Publications
- Open Access
KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness
In recent years, Pre-trained Language Models (PLMs) have shown their superiority by pre-training on unstructured text corpus and then fine-tuning on downstream tasks. On entity-rich textual resources like Wikipedia, Knowledge-Enhanced PLMs (KEPLMs) incorporate the interactions between tokens and mentioned entities in pre-training, and are thus more effective on entity-centric tasks such as entity linking and relation classification. Although exploiting Wikipedia's rich structures to some extent, conventional KEPLMs still neglect a unique layout of the corpus where each Wikipedia page is around a topic entity (identified by the page URL and shown in the page title). In this paper, we demonstrate that KEPLMs without incorporating the topic entities will lead to insufficient entity interaction and biased (relation) word semantics. We thus propose KEPLET, a novel Knowledge-Enhanced Pre-trained LanguagE model with Topic entity awareness. In an end-to-end manner, KEPLET identifies where to add the topic entity's information in a Wikipedia sentence, fuses such information into token and mentioned entities representations, and supervises the network learning, through which it takes topic entities back into consideration. Experiments demonstrated the generality and superiority of KEPLET which was applied to two representative KEPLMs, achieving significant improvements on four entity-centric tasks.
Open AccessKEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness
In recent years, Pre-trained Language Models (PLMs) have shown their superiority by pre-training on unstructured text corpus and then fine-tuning on downstream tasks. On entity-rich textual resources like Wikipedia, Knowledge-Enhanced PLMs (KEPLMs) incorporate the interactions between tokens and mentioned entities in pre-training, and are thus more effective on entity-centric tasks such as entity linking and relation classification. Although exploiting Wikipedia's rich structures to some extent, conventional KEPLMs still neglect a unique layout of the corpus where each Wikipedia page is around a topic entity (identified by the page URL and shown in the page title). In this paper, we demonstrate that KEPLMs without incorporating the topic entities will lead to insufficient entity interaction and biased (relation) word semantics. We thus propose KEPLET, a novel Knowledge-Enhanced Pre-trained LanguagE model with Topic entity awareness. In an end-to-end manner, KEPLET identifies where to add the topic entity's information in a Wikipedia sentence, fuses such information into token and mentioned entities representations, and supervises the network learning, through which it takes topic entities back into consideration. Experiments demonstrated the generality and superiority of KEPLET which was applied to two representative KEPLMs, achieving significant improvements on four entity-centric tasks.
- Open Access
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 6864-6876
KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness
Yichuan Li, Jialong Han, Kyumin Lee, Chengyuan Ma, Benjamin Yao, Xiaohu Liu. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 6864-6876
Open AccessKEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness
Yichuan Li, Jialong Han, Kyumin Lee, Chengyuan Ma, Benjamin Yao, Xiaohu Liu. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.