Mingjie Zhou
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Mingjie Zhou has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Dongmei Zhang
Microsoft Research Asia, Beijing, China
1 shared publication - Huafeng Liu
Beijing Jiaotong University, Beijing, China
1 shared publication - Liping Jing
Beijing Jiaotong University, Beijing, China
1 shared publication - Mengyu Zhou
Microsoft, Beijing, China
1 shared publication - Michael KwokPo Ng
Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong
1 shared publication - Shi Han
Microsoft, Beijing, China
1 shared publication
Latest Publications
- Top Cited PapersOpen Access
- 4 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 645-654
Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study
Large language models (LLMs) are becoming attractive as few-shot reasoners to solve Natural Language (NL)-related tasks. However, there is still much to learn about how well LLMs understand structured data, such as tables. Although tables can be used as input to LLMs with serialization, there is a lack of comprehensive studies that examine whether LLMs can truly comprehend such data. In this paper, we try to understand this by designing a benchmark to evaluate the structural understanding capabilities (SUC) of LLMs. The benchmark we create includes seven tasks, each with its own unique challenges, \eg, cell lookup, row retrieval, and size detection. We perform a series of evaluations on GPT-3.5 and GPT-4. We find that performance varied depending on several input choices, including table input format, content order, role prompting, and partition marks. Drawing from the insights gained through the benchmark evaluations, we proposeself-augmentation for effective structural prompting, such as critical value / range identification using internal knowledge of LLMs. When combined with carefully chosen input choices, these structural prompting methods lead to promising improvements in LLM performance on a variety of tabular tasks, \eg, TabFact(\uparrow2.31%), HybridQA(\uparrow2.13%), SQA(\uparrow2.72%), Feverous(\uparrow0.84%), and ToTTo(\uparrow5.68%). We believe that our open-source (please find code and data at https://github.com/microsoft/TableProvider) benchmark and proposed prompting methods can serve as a simple yet generic selection for future research.
- 4 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 645-654
Top Cited PapersOpen AccessTable Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study
Large language models (LLMs) are becoming attractive as few-shot reasoners to solve Natural Language (NL)-related tasks. However, there is still much to learn about how well LLMs understand structured data, such as tables. Although tables can be used as input to LLMs with serialization, there is a lack of comprehensive studies that examine whether LLMs can truly comprehend such data. In this paper, we try to understand this by designing a benchmark to evaluate the structural understanding capabilities (SUC) of LLMs. The benchmark we create includes seven tasks, each with its own unique challenges, \eg, cell lookup, row retrieval, and size detection. We perform a series of evaluations on GPT-3.5 and GPT-4. We find that performance varied depending on several input choices, including table input format, content order, role prompting, and partition marks. Drawing from the insights gained through the benchmark evaluations, we proposeself-augmentation for effective structural prompting, such as critical value / range identification using internal knowledge of LLMs. When combined with carefully chosen input choices, these structural prompting methods lead to promising improvements in LLM performance on a variety of tabular tasks, \eg, TabFact(\uparrow2.31%), HybridQA(\uparrow2.13%), SQA(\uparrow2.72%), Feverous(\uparrow0.84%), and ToTTo(\uparrow5.68%). We believe that our open-source (please find code and data at https://github.com/microsoft/TableProvider) benchmark and proposed prompting methods can serve as a simple yet generic selection for future research.
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 6212-6222
Doubly Intention Learning for Cold-start Recommendation with Uncertainty-aware Stochastic Meta Process
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 6212-6222
Open AccessDoubly Intention Learning for Cold-start Recommendation with Uncertainty-aware Stochastic Meta Process