Yisong Miao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yisong Miao has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
Min-Yen Kan
Web IR / NLP Group (WING) National University of Singapore
2 shared publications- Hongfu Liu
National University of Singapore
1 shared publication - Min-Yen Kan
School of Computing, National University of Singapore, Singapore, Singapore
1 shared publication - Nancy F. Chen
Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
1 shared publication - Wenqiang Lei
College of Computer Science, Sichuan University, Chengdu, China
1 shared publication
Latest Publications
- Open Access
Discursive Circuits: How Do Language Models Understand Discourse Relations?
Which components in transformer language models are responsible for discourse understanding? We hypothesize that sparse computational graphs, termed as discursive circuits, control how models process discourse relations. Unlike simpler tasks, discourse relations involve longer spans and complex reasoning. To make circuit discovery feasible, we introduce a task called Completion under Discourse Relation (CuDR), where a model completes a discourse given a specified relation. To support this task, we construct a corpus of minimal contrastive pairs tailored for activation patching in circuit discovery. Experiments show that sparse circuits ($\approx 0.2\%$ of a full GPT-2 model) recover discourse understanding in the English PDTB-based CuDR task. These circuits generalize well to unseen discourse frameworks such as RST and SDRT. Further analysis shows lower layers capture linguistic features such as lexical semantics and coreference, while upper layers encode discourse-level abstractions. Feature utility is consistent across frameworks (e.g., coreference supports Expansion-like relations).
Open AccessDiscursive Circuits: How Do Language Models Understand Discourse Relations?
Which components in transformer language models are responsible for discourse understanding? We hypothesize that sparse computational graphs, termed as discursive circuits, control how models process discourse relations. Unlike simpler tasks, discourse relations involve longer spans and complex reasoning. To make circuit discovery feasible, we introduce a task called Completion under Discourse Relation (CuDR), where a model completes a discourse given a specified relation. To support this task, we construct a corpus of minimal contrastive pairs tailored for activation patching in circuit discovery. Experiments show that sparse circuits ($\approx 0.2\%$ of a full GPT-2 model) recover discourse understanding in the English PDTB-based CuDR task. These circuits generalize well to unseen discourse frameworks such as RST and SDRT. Further analysis shows lower layers capture linguistic features such as lexical semantics and coreference, while upper layers encode discourse-level abstractions. Feature utility is consistent across frameworks (e.g., coreference supports Expansion-like relations).
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 32558-32577
Discursive Circuits: How Do Language Models Understand Discourse Relations?
Yisong Miao, Min-Yen Kan. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 32558-32577
Open AccessDiscursive Circuits: How Do Language Models Understand Discourse Relations?
Yisong Miao, Min-Yen Kan. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 6277-6295
Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations
Yisong Miao, Hongfu Liu, Wenqiang Lei, Nancy Chen, Min-Yen Kan. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 6277-6295
Open AccessDiscursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations
Yisong Miao, Hongfu Liu, Wenqiang Lei, Nancy Chen, Min-Yen Kan. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.