Xiaoyu Bai
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Xiaoyu Bai has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Tianwei Yue
Carnegie Mellon University Carnegie Mellon University Columbia University Carnegie Mellon University Carnegie Mellon University Carnegie Mellon University
2 shared publications - Wenping Wang
Carnegie Mellon University Carnegie Mellon University Columbia University Carnegie Mellon University Carnegie Mellon University Carnegie Mellon University
2 shared publications - Chen Liang
School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA
1 shared publication - Cindy Le
Columbia University Carnegie Mellon University Carnegie Mellon University University of Michigan Carnegie Mellon University
1 shared publication - Congrui Hetang
Columbia University Carnegie Mellon University Carnegie Mellon University University of Michigan Carnegie Mellon University
1 shared publication - Dachi Chen
School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA
1 shared publication
Latest Publications
- Open Access
- 21 August 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (16) , 9461
Unlocking Everyday Wisdom: Enhancing Machine Comprehension with Script Knowledge Integration
Harnessing commonsense knowledge poses a significant challenge for machine comprehension systems. This paper primarily focuses on incorporating a specific subset of commonsense knowledge, namely, script knowledge. Script knowledge is about sequences of actions that are typically performed by individuals in everyday life. Our experiments were centered around the MCScript dataset, which was the basis of the SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. As a baseline, we utilized our Three-Way Attentive Networks (TriANs) framework to model the interactions among passages, questions, and answers. Building upon the TriAN, we proposed to: (1) integrate a pre-trained language model to capture script knowledge; (2) introduce multi-layer attention to facilitate multi-hop reasoning; and (3) incorporate positional embeddings to enhance the model’s capacity for event-ordering reasoning. In this paper, we present our proposed methods and prove their efficacy in improving script knowledge integration and reasoning.
- 21 August 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (16) , 9461
Open AccessUnlocking Everyday Wisdom: Enhancing Machine Comprehension with Script Knowledge Integration
Harnessing commonsense knowledge poses a significant challenge for machine comprehension systems. This paper primarily focuses on incorporating a specific subset of commonsense knowledge, namely, script knowledge. Script knowledge is about sequences of actions that are typically performed by individuals in everyday life. Our experiments were centered around the MCScript dataset, which was the basis of the SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. As a baseline, we utilized our Three-Way Attentive Networks (TriANs) framework to model the interactions among passages, questions, and answers. Building upon the TriAN, we proposed to: (1) integrate a pre-trained language model to capture script knowledge; (2) introduce multi-layer attention to facilitate multi-hop reasoning; and (3) incorporate positional embeddings to enhance the model’s capacity for event-ordering reasoning. In this paper, we present our proposed methods and prove their efficacy in improving script knowledge integration and reasoning.
- Open Access
- 30 June 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (13) , 7753
Enhancing Abstractive Summarization with Extracted Knowledge Graphs and Multi-Source Transformers
As the popularity of large language models (LLMs) has risen over the course of the last year, led by GPT-3/4 and especially its productization as ChatGPT, we have witnessed the extensive application of LLMs to text summarization. However, LLMs do not intrinsically have the power to verify the correctness of the information they supply and generate. This research introduces a novel approach to abstractive summarization, aiming to address the limitations of LLMs in that they struggle to understand the truth. The proposed method leverages extracted knowledge graph information and structured semantics as a guide for summarization. Building upon BART, one of the state-of-the-art sequence-to-sequence pre-trained LLMs, multi-source transformer modules are developed as an encoder, which are capable of processing textual and graphical inputs. Decoding is performed based on this enriched encoding to enhance the summary quality. The Wiki-Sum dataset, derived from Wikipedia text dumps, is introduced for evaluation purposes. Comparative experiments with baseline models demonstrate the strengths of the proposed approach in generating informative and relevant summaries. We conclude by presenting our insights into utilizing LLMs with graph external information, which will become a powerful aid towards the goal of factually correct and verified LLMs.
- 30 June 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (13) , 7753
Open AccessEnhancing Abstractive Summarization with Extracted Knowledge Graphs and Multi-Source Transformers
As the popularity of large language models (LLMs) has risen over the course of the last year, led by GPT-3/4 and especially its productization as ChatGPT, we have witnessed the extensive application of LLMs to text summarization. However, LLMs do not intrinsically have the power to verify the correctness of the information they supply and generate. This research introduces a novel approach to abstractive summarization, aiming to address the limitations of LLMs in that they struggle to understand the truth. The proposed method leverages extracted knowledge graph information and structured semantics as a guide for summarization. Building upon BART, one of the state-of-the-art sequence-to-sequence pre-trained LLMs, multi-source transformer modules are developed as an encoder, which are capable of processing textual and graphical inputs. Decoding is performed based on this enriched encoding to enhance the summary quality. The Wiki-Sum dataset, derived from Wikipedia text dumps, is introduced for evaluation purposes. Comparative experiments with baseline models demonstrate the strengths of the proposed approach in generating informative and relevant summaries. We conclude by presenting our insights into utilizing LLMs with graph external information, which will become a powerful aid towards the goal of factually correct and verified LLMs.
- 11 October 2022
- Published by Hindawi Limited in Structural Control and Health Monitoring
- Vol. 29 (11)
Additional Cover
- 11 October 2022
- Published by Hindawi Limited in Structural Control and Health Monitoring
- Vol. 29 (11)
Additional Cover