Trishul Chilimbi
Trishul Chilimbi has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Belinda Zeng
Amazon SP, Seattle, WA, USA
1 shared publication - Carl Yang
Emory University
1 shared publication - Da Zheng
Ant Group, Hangzhou, China
1 shared publication - Han Xie
Emory University, Atlanta, GA, USA
1 shared publication - Houyu Zhang
Amazon Search AI, Seattle, WA, USA
1 shared publication - Jun Ma
Walgreens AI Lab, Bellevue, WA, USA
1 shared publication
Latest Publications
- Open Access
- 4 August 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the Acm Sigkdd Conference on Knowledge Discovery and Data Mining
- Vol. 2023, 5270-5281
Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications
Model pre-training on large text corpora has been demonstrated effective for various downstream applications in the NLP domain. In the graph mining domain, a similar analogy can be drawn for pre-training graph models on large graphs in the hope of benefiting downstream graph applications, which has also been explored by several recent studies. However, no existing study has ever investigated the pre-training of text plus graph models on large heterogeneous graphs with abundant textual information (a.k.a. large graph corpora) and then fine-tuning the model on different related downstream applications with different graph schemas. To address this problem, we propose a framework of graph-aware language model pre-training (GaLM) on a large graph corpus, which incorporates large language models and graph neural networks, and a variety of fine-tuning methods on downstream applications. We conduct extensive experiments on Amazon's real internal datasets and large public datasets. Comprehensive empirical results and in-depth analysis demonstrate the effectiveness of our proposed methods along with lessons learned.
- 4 August 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the Acm Sigkdd Conference on Knowledge Discovery and Data Mining
- Vol. 2023, 5270-5281
Open AccessGraph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications
Model pre-training on large text corpora has been demonstrated effective for various downstream applications in the NLP domain. In the graph mining domain, a similar analogy can be drawn for pre-training graph models on large graphs in the hope of benefiting downstream graph applications, which has also been explored by several recent studies. However, no existing study has ever investigated the pre-training of text plus graph models on large heterogeneous graphs with abundant textual information (a.k.a. large graph corpora) and then fine-tuning the model on different related downstream applications with different graph schemas. To address this problem, we propose a framework of graph-aware language model pre-training (GaLM) on a large graph corpus, which incorporates large language models and graph neural networks, and a variety of fine-tuning methods on downstream applications. We conduct extensive experiments on Amazon's real internal datasets and large public datasets. Comprehensive empirical results and in-depth analysis demonstrate the effectiveness of our proposed methods along with lessons learned.