Yafang Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yafang Wang has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Gerard de Melo
Hasso Plattner Institute, Potsdam, Germany and University of Potsdam, Potsdam, Germany
1 shared publication - Haitao Mi
School of Cyber Science and Technology Shandong University China
1 shared publication - Jing Zheng
Hasso Plattner Institute / University of Potsdam
1 shared publication Xiang Hu
Hasso Plattner Institute / University of Potsdam
1 shared publication- Yi Su
Hasso Plattner Institute / University of Potsdam
1 shared publication - Zujie Wen
Hasso Plattner Institute / University of Potsdam
1 shared publication
Latest Publications
- Open Access
R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling
Human language understanding operates at multiple levels of granularity (e.g., words, phrases, and sentences) with increasing levels of abstraction that can be hierarchically combined. However, existing deep models with stacked layers do not explicitly model any sort of hierarchical process. This paper proposes a recursive Transformer model based on differentiable CKY style binary trees to emulate the composition process. We extend the bidirectional language model pre-training objective to this architecture, attempting to predict each word given its left and right abstraction nodes. To scale up our approach, we also introduce an efficient pruned tree induction algorithm to enable encoding in just a linear number of composition steps. Experimental results on language modeling and unsupervised parsing show the effectiveness of our approach.
Open AccessR2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling
Human language understanding operates at multiple levels of granularity (e.g., words, phrases, and sentences) with increasing levels of abstraction that can be hierarchically combined. However, existing deep models with stacked layers do not explicitly model any sort of hierarchical process. This paper proposes a recursive Transformer model based on differentiable CKY style binary trees to emulate the composition process. We extend the bidirectional language model pre-training objective to this architecture, attempting to predict each word given its left and right abstraction nodes. To scale up our approach, we also introduce an efficient pruned tree induction algorithm to enable encoding in just a linear number of composition steps. Experimental results on language modeling and unsupervised parsing show the effectiveness of our approach.