SDShuang Dong
Shuang Dong
*This is an algorithmically generated author record based on publicly available information.
Shuang Dong has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Bin Wang
MiLM Plus Xiaomi Inc Beijing China
2 shared publications - Jian Luan
MiLM Plus Xiaomi Inc Beijing China
2 shared publications - Wei Liu
MiLM Plus Xiaomi Inc
2 shared publications - Hanlin Teng
Xiaomi AI Lab
1 shared publication - Hongda Sun
Gaoling School of Artificial Intelligence Renmin University of China
1 shared publication - Rui Yan
School of Artificial Intelligence, Wuhan University, Wuhan, China
1 shared publication
Latest Publications
- 8 October 2023
- Published by Springer Nature
- p. 388-395
Overview of the NLPCC 2023 Shared Task 9: User Feedback Prediction and Response Generation
- 8 October 2023
- Published by Springer Nature
- p. 388-395
Overview of the NLPCC 2023 Shared Task 9: User Feedback Prediction and Response Generation
- Open Access
CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?
We present the Chinese Elementary School Math Word Problems (CMATH) dataset, comprising 1.7k elementary school-level math word problems with detailed annotations, source from actual Chinese workbooks and exams. This dataset aims to provide a benchmark tool for assessing the following question: to what grade level of elementary school math do the abilities of popular large language models (LLMs) correspond? We evaluate a variety of popular LLMs, including both commercial and open-source options, and discover that only GPT-4 achieves success (accuracy $\geq$ 60\%) across all six elementary school grades, while other models falter at different grade levels. Furthermore, we assess the robustness of several top-performing LLMs by augmenting the original problems in the CMATH dataset with distracting information. Our findings reveal that GPT-4 is able to maintains robustness, while other model fail. We anticipate that our study will expose limitations in LLMs' arithmetic and reasoning capabilities, and promote their ongoing development and advancement.Open AccessCMATH: Can Your Language Model Pass Chinese Elementary School Math Test?
We present the Chinese Elementary School Math Word Problems (CMATH) dataset, comprising 1.7k elementary school-level math word problems with detailed annotations, source from actual Chinese workbooks and exams. This dataset aims to provide a benchmark tool for assessing the following question: to what grade level of elementary school math do the abilities of popular large language models (LLMs) correspond? We evaluate a variety of popular LLMs, including both commercial and open-source options, and discover that only GPT-4 achieves success (accuracy $\geq$ 60\%) across all six elementary school grades, while other models falter at different grade levels. Furthermore, we assess the robustness of several top-performing LLMs by augmenting the original problems in the CMATH dataset with distracting information. Our findings reveal that GPT-4 is able to maintains robustness, while other model fail. We anticipate that our study will expose limitations in LLMs' arithmetic and reasoning capabilities, and promote their ongoing development and advancement.