Yiming Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yiming Yang has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Feipeng Chen
School of Linguistic Sciences and Arts, Jiangsu Normal University, China
1 shared publication - Shuguang Yang
School of Psychology, South China Normal University
1 shared publication - Zude Zhu
School of Linguistics Sciences and Arts, and Collaborative Innovation Center for Language Competence, Jiangsu Normal University, Xuzhou, China
1 shared publication
Latest Publications
- Open Access
- 7 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 165-170
A Study on Semantic Understanding of Large Language Models from the Perspective of Ambiguity Resolution
Generative large language models (LLMs) have demonstrated outstanding performance in language understanding and generation tasks. Whether current LLMs can truly understand meaning or not is an open question. Semantic disambiguation is an important indicator to test machine language ability. To this end, three experiments were conducted to test the ambiguity resolution ability. In Experiment 1, the LLMs were asked to make a semantic discrimination on ambiguity phrases. The ChatGPT (May 3 Version) and ChatGPT Plus (May 3 Version) showed a random level performance of discrimination. To make the task easier for the LLMs, ambiguity sentences were presented with context in the Experiment 2. While the LLMs unified the context to fill the position for ambiguity word, the performance of semantic discrimination on these sentences remained at random level. In addition to test the performance of LMMs, prediction is one of the key ability of LLMs. The prediction by unifying the context is one of the key skill for LLMs. Therefore, the Experiment 3 further test the similarity between the human prediction mechanism and the strong generation capabilities of LLMs. Weak similarities were only found under the condition of local constraint features gradually accumulating, specifically, the cosine similarity of predicted sentence semantic vectors and ending word semantic vectors negatively correlated with close probability only. These series of experiments demonstrate that, it is far away to claim that LLMs truly understand human sentences.
- 7 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 165-170
Open AccessA Study on Semantic Understanding of Large Language Models from the Perspective of Ambiguity Resolution
Generative large language models (LLMs) have demonstrated outstanding performance in language understanding and generation tasks. Whether current LLMs can truly understand meaning or not is an open question. Semantic disambiguation is an important indicator to test machine language ability. To this end, three experiments were conducted to test the ambiguity resolution ability. In Experiment 1, the LLMs were asked to make a semantic discrimination on ambiguity phrases. The ChatGPT (May 3 Version) and ChatGPT Plus (May 3 Version) showed a random level performance of discrimination. To make the task easier for the LLMs, ambiguity sentences were presented with context in the Experiment 2. While the LLMs unified the context to fill the position for ambiguity word, the performance of semantic discrimination on these sentences remained at random level. In addition to test the performance of LMMs, prediction is one of the key ability of LLMs. The prediction by unifying the context is one of the key skill for LLMs. Therefore, the Experiment 3 further test the similarity between the human prediction mechanism and the strong generation capabilities of LLMs. Weak similarities were only found under the condition of local constraint features gradually accumulating, specifically, the cosine similarity of predicted sentence semantic vectors and ending word semantic vectors negatively correlated with close probability only. These series of experiments demonstrate that, it is far away to claim that LLMs truly understand human sentences.