Developing and Validating an Instrument for Assessing Learning Sciences Competence of Doctoral Students in Education in China
Open Access
- 30 June 2024
- journal article
- research article
- Published by MDPI AG in Sustainability
- Vol. 16 (13) , 5607
Abstract
Learning sciences competence refers to a necessary professional competence for educators, which is manifested in their deep understanding of learning sciences knowledge, positive attitudes, and scientific thinking and skills in conducting teaching practice and research. It is of paramount importance for doctoral students in education to develop their competence in the field of learning sciences. This will enhance their abilities to teach and conduct research, and guide their educational research and practice toward greater sustainability. In order to address the shortcomings of current assessment instruments, we constructed a theoretical model for assessing learning sciences competence based on the PISA 2025 framework and Piaget’s theory of knowledge. A three-dimensional assessment framework was designed, along with an initial instrument. Furthermore, the “Delphi method based on large language models (LLM)” was employed to conduct two rounds of expert consultations with the objective of testing and refining the instrument. Throughout this process, we developed a set of guidelines for engaging AI experts to improve interactions with LLM, including an invitation letter to AI experts, the main body of the questionnaire, and the general inquiry about AI experts’ perspectives. In analyzing the results of the Delphi method, we used the “threshold method” to identify and refine the questionnaire items that performed sub-optimally. This resulted in the final assessment instrument for evaluating learning sciences competence among doctoral students in education. The assessment instrument encompasses three dimensions: the knowledge of learning sciences, application of learning sciences, and attitude towards learning sciences, with a total of 40 items. These items integrate Likert scales and scenario-based questions. Furthermore, the study examined potential limitations in the item design, question type selection, and method application of the assessment instrument. The design and development of the assessment instrument provide valuable references for the standardized monitoring and sustainability development of the learning sciences competence of doctoral students in education.Keywords
Funding Information
- National Natural Science Foundation of China (62077035, SYJS202413)
- Shaanxi Normal University 2024 Experimental Technology Research Project (62077035, SYJS202413)
This publication has 42 references indexed in Scilit:
- Leveraging the Potential of Large Language Models in Education Through Playful and Game-Based LearningEducational Psychology Review, 2024
- Prompt Engineering with ChatGPT: A Guide for Academic WritersAnnals of Biomedical Engineering, 2023
- Performance assessment in teacher education research—A scoping review of characteristics of assessment instruments in the DACH regionZeitschrift für Erziehungswissenschaft, 2023
- Development of the Learning Sciences: Theories, Pedagogies, and Technologies2023
- Comparing Self-Report Assessments and Scenario-Based Assessments of Systems Thinking CompetenceJournal of Science Education and Technology, 2023
- Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancementsMethodsX, 2021
- A Delphi approach to develop an evaluation indicator system for the National Food Safety Standards of ChinaFood Control, 2021
- Approaches to measuring use of research evidence in K-12 settings: A systematic reviewEducational Research Review, 2019
- Learning sciencesWIREs Cognitive Science, 2010
- Current Perceptions of the Doctor of Philosophy and Doctor of Education Degrees in Counselor PreparationCounselor Education and Supervision, 1997