Li Liang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Li Liang has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
Bahar Shahrokhian
Arizona State University
2 shared publications- Sebastian Simon
Université Du Maine, Le Mans, France
2 shared publications - Sebastian Strauß
Ruhr-University Bochum
2 shared publications - Sreecharan Sankaranarayanan
Carnegie Mellon University
2 shared publications Amine Hatun Ataş
1 shared publication- Amine Hatun Ataş
Galatasaray University
1 shared publication
Latest Publications
- 15 July 2025
- Published by Springer Nature
- p. 60-73
Comparing a Human’s and a Multi-Agent System’s Thematic Analysis: Assessing Qualitative Coding Consistency
- 15 July 2025
- Published by Springer Nature
- p. 60-73
Comparing a Human’s and a Multi-Agent System’s Thematic Analysis: Assessing Qualitative Coding Consistency
- Open Access
- 11 June 2025
- Published by Center for Open Science in EdArXiv
Bridging Human Intelligence Augmentation (IA) and Classroom Practices via GenAI in Learning Engineering: A Response to Dr. Dede’s Keynote Speeches on IA 2022-2025
This article is written in response to Dr. Christopher Dede’s invitation for public feedback on his keynote speeches on human intelligence augmentation (IA) via learning engineering. The goal here is not for a complex philosophical conversation but to share a down-to-Earth understanding of the significant usefulness of this IA framework on bridging human IA and classroom practices via learning engineering in GenAI age. For the pragmatic application of the IA framework in classroom practices via learning engineering, this constructive dialogue reviews three core pillars of his epistemology synthesized from his keynote speeches 2022-2025, and discusses some major real-world opportunities, pitfalls, challenges and possible solutions to improve the educational ecosystems in GenAI age through a clear roadmap reference for the front-line educators, policy makers, researchers, learning engineers and AI developers, etc. Every reader is advised to spend some time to have a glimpse of Dr. Dede’s open-source website and create their own Hamlet version of user experience (UX) design for human IA to benefit their work and life in GenAI age.
- 11 June 2025
- Published by Center for Open Science in EdArXiv
Open AccessBridging Human Intelligence Augmentation (IA) and Classroom Practices via GenAI in Learning Engineering: A Response to Dr. Dede’s Keynote Speeches on IA 2022-2025
This article is written in response to Dr. Christopher Dede’s invitation for public feedback on his keynote speeches on human intelligence augmentation (IA) via learning engineering. The goal here is not for a complex philosophical conversation but to share a down-to-Earth understanding of the significant usefulness of this IA framework on bridging human IA and classroom practices via learning engineering in GenAI age. For the pragmatic application of the IA framework in classroom practices via learning engineering, this constructive dialogue reviews three core pillars of his epistemology synthesized from his keynote speeches 2022-2025, and discusses some major real-world opportunities, pitfalls, challenges and possible solutions to improve the educational ecosystems in GenAI age through a clear roadmap reference for the front-line educators, policy makers, researchers, learning engineers and AI developers, etc. Every reader is advised to spend some time to have a glimpse of Dr. Dede’s open-source website and create their own Hamlet version of user experience (UX) design for human IA to benefit their work and life in GenAI age.
- Open Access
- 13 May 2025
- Published by Center for Open Science in EdArXiv
Comparing a Human’s and a Multi-Agent System’s Thematic Analysis: Assessing Qualitative Coding Consistency
Large Language Models (LLMs) have demonstrated fluency in text generation and reasoning tasks. Consequently, the field has probed the ability of LLMs to automate qualitative analysis, including inductive thematic analysis (iTA), previously achieved through human reasoning only. Studies using LLMs for iTA have yielded mixed results so far. LLMs have successfully been used for isolated steps of iTA in hybrid setups. With recent advances in multi-agent systems (MAS) enabling complex reasoning and task execution through multiple, cooperating LLM agents, the first results point towards the possibility of automating sequences of the iTA process. However, previous work especially lacks methodological standards for assessing the reliability and validity of LLM-derived iTA themes and outcomes. Therefore, in this paper, we propose a method for assessing the quality of automated iTA systems based on consistency with human coding on a benchmark dataset. We present criteria for benchmark datasets and the comparison of human and AI-generated iTA analysis. We demonstrate the use of both in an expert blind review on two iTA outputs: one iTA conducted by domain experts, and another fully automated with a MAS built on the Claude 3.5 Sonnet LLM. Results indicate a high level of consistency and contribute evidence that complex qualitative analysis methods common in AIED research can be carried out by MAS.
- 13 May 2025
- Published by Center for Open Science in EdArXiv
Open AccessComparing a Human’s and a Multi-Agent System’s Thematic Analysis: Assessing Qualitative Coding Consistency
Large Language Models (LLMs) have demonstrated fluency in text generation and reasoning tasks. Consequently, the field has probed the ability of LLMs to automate qualitative analysis, including inductive thematic analysis (iTA), previously achieved through human reasoning only. Studies using LLMs for iTA have yielded mixed results so far. LLMs have successfully been used for isolated steps of iTA in hybrid setups. With recent advances in multi-agent systems (MAS) enabling complex reasoning and task execution through multiple, cooperating LLM agents, the first results point towards the possibility of automating sequences of the iTA process. However, previous work especially lacks methodological standards for assessing the reliability and validity of LLM-derived iTA themes and outcomes. Therefore, in this paper, we propose a method for assessing the quality of automated iTA systems based on consistency with human coding on a benchmark dataset. We present criteria for benchmark datasets and the comparison of human and AI-generated iTA analysis. We demonstrate the use of both in an expert blind review on two iTA outputs: one iTA conducted by domain experts, and another fully automated with a MAS built on the Claude 3.5 Sonnet LLM. Results indicate a high level of consistency and contribute evidence that complex qualitative analysis methods common in AIED research can be carried out by MAS.