Xuetao Tian

In an era of information explosion and overload, individuals face a constant influx of mixed-quality, contradictory data. Navigating this complex landscape requires critical thinking, which enables individuals to dynamically adjust their reasoning and refine judgments. Consequently, accurately measuring this dynamic ability is essential. However, existing assessments predominantly focus on single-round argumentation and fail to capture the iterative process, potentially yielding unrepresentative evaluations of real-world performance. To address this gap, this study develops the Iterative Argumentation Task (IAT), a novel assessment tool that presents participants with a sequence of interrelated passages simulating real-world information flow. By capturing the process of evaluation, position changes, and argument revisions, the IAT evaluates three ability dimensions: single-round argument analysis, inter-argument relationship analysis, and argument integration. We recruited 355 undergraduates to validate the IAT. Confirmatory factor analysis provided evidence for the proposed measurement structure. Furthermore, correlations with established critical thinking and reasoning tests demonstrated convergent and discriminant validity, respectively. Moreover, performance differences between the undergraduates and 33 debaters established known-groups validity. Finally, latent profile analysis identified four distinct profiles across the IAT dimensions. Together, these findings establish the IAT as a theoretically sound and practically effective tool for assessing critical thinking.

Situational Judgment Tests (SJTs) are important tools for evaluating individual abilities by presenting specific scenarios and assessing the responses. Thi

Introduction: Adolescent suicide is a critical public health concern worldwide, necessitating effective methods for early detection of high suicidal ideation. Traditional detection methods, such as self-report scales, suffer from limited accuracy and are susceptible to personal concealment. Automatic methods based on artificial intelligence techniques are more accurate, while they often lack scalability due to strict data requirements. In order to achieve a balance between accuracy and scalability, this paper introduces the Tree-Drawing Test (TDT) as an effective tool for suicidal ideation detection, and proposes a novel graph learning approach to enable its automatic application.Methods: The proposed method first constructs a semantic graph based on psychological features annotated automatically from tree-drawing images, and leverages a Graph Convolutional Network (GCN) model to realize individual suicidal ideation detection. To evaluate this method, a real dataset of 806 students from primary and secondary school in Shaanxi Province, China, is collected, and some metrics including macro-F1, G-mean, and false positive rate are used.Results: The results demonstrate that the proposed method significantly outperforms traditional machine learning and convolution neural network approaches. The ablation study demonstrates the effectiveness of feature “leaves and fruits” in detecting suicidal ideation. Further experiments demonstrate that the proposed method remains stable even when the model is disturbed, such as when a tree-drawing image cannot be fully represented.Discussion: The proposed method highlights its effectiveness in large-scale suicidal ideation screening, as it not only achieves high detection performance but also maintains model stability while remaining flexible and adaptable.

The scarcity of high-quality large-scale labeled datasets poses a huge challenge for employing deep learning models in video deception detection. To address this issue, inspired by the psychological theory on the relation between deception and expressions, we propose a novel method called AFFAKT in this paper, which enhances the classification performance by transferring useful and correlated knowledge from a large facial expression dataset. Two key challenges in knowledge transfer arise: 1) how much knowledge of facial expression data should be transferred and 2) how to effectively leverage transferred knowledge for the deception classification model during inference. Specifically, the optimal relation mapping between facial expression classes and deception samples is firstly quantified using proposed H-OTKT module and then transfers knowledge from the facial expression dataset to deception samples. Moreover, a correlation prototype within another proposed module SRKB is well designed to retain the invariant correlations between facial expression classes and deception classes through momentum updating. During inference, the transferred knowledge is fine-tuned with the correlation prototype using a sample-specific re-weighting strategy. Experimental results on two deception detection datasets demonstrate the superior performance of our proposed method. The interpretability study reveals high associations between deception and negative affections, which coincides with the theory in psychology.

Situational Judgment Tests (SJTs) are important tools for evaluating individual abilities by presenting specific scenarios and assessing the responses. This study explores the potential of Large Language Models (LLMs) in developing SJTs, with emotional regulation as an example. We proposed and evaluated two approaches using ERNIE 4.0, an advanced Chinese LLM: generating new items from scratch (Approach 1) and adapting items from existing examples (Approach 2). The quality of generated items was assessed through expert evaluations and empirical testing with 93 psychology undergraduates and 184 participants from a general population pool. Results indicate that items generated by ERNIE 4.0, particularly new items from Approach 1, met or exceeded the quality of existing tests in both expert evaluations and empirical validations, while adapted items showed some limitations. This research demonstrates LLMs' potential for developing valid assessments, offering insights for future applications.

Student evaluation of teaching (SET) is a vital component of educational enhancement, yet conventional assessment tools face inherent limitations. While open-ended questions provide a platform for students to convey authentic sentiments, the absence of automated labeling tools poses a challenge in the case of large-scale applications. In response, this study undertakes a comprehensive exploration, centered on the utilization of ChatGPT for capturing fine-grained teacher performance from SET. Based on a collected dataset and manual coding, the performance of ChatGPT with various strategies including zero-shot and few-shot, and some supervised models, including CNN, LSTM and BERT, are evaluated and compared. As a result, ChatGPT exhibits the promise of achieving commendable performance with a small number of labeled samples. This approach reduces the dependency on extensive labeled data, offering an effective solution. However, in terms of performance, a discernible margin persists in comparison to advanced supervised models, BERT. Our study also acknowledges there are various factors, such as task complexity and prompt clarity, influencing ChatGPT’s performance and consistency. In summation, while the integration of ChatGPT into practical SET applications holds significant promise, further explorations are imperative to ensure the alignment of its capabilities with the intricate demands. Cited as: Zhang, B., & Tian, X. (2024). Capturing fine-grained teacher performance from student evaluation of teaching via ChatGPT. Education andLifelong Development Research, 1(4), 156-169. https://doi.org/10.46690/elder.2024.04.01 References Alnaqbi, N.M., & Fouda, W. (2023). Exploring the role of chatgpt and social media in enhancing student evaluation of teaching styles in higher education using neutrosophic sets. International Journal of Neutrosophic Science, 20(4), 181-190. Alwaely, S.A., El-Zeiny, M.E., Alqudah, H., Alamarnih, E.F.M., Salman, O.K.I., Halim, M., Khasawneh, M.A.S. (2023). The impact of teacher evaluation on professional development and student achievement. Revista de Gestao Social e Ambiental, 17(7), e03484-e03484. Ambady, N., & Rosenthal, R. (1993). Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology, 64(3), 431. Annan, S., Tratnack, S., Rubenstein, C., Sawin, E., & Hulton, L. (2013). An integrative review of student evaluations of teaching: Implications for evaluation of nursing faculty. Journal of Professional Nursing, 29(5), e10-e24. Beran, T., & Rokosh, J. (2009). Instructors’ perspectives on the utility of student ratings of instruction. Instructional Science, 37, 171-184. Beran, T., Violato, C., Kline, D. (2007). What’s the “use” of student ratings of instruction for administrators? one university’s experience. Canadian Journal of Higher Education, 37, 27-43. Brockx, B., Van Roy, K., Mortelmans, D. (2012). The student as a commentator: Students’ comments in student evaluations of teaching. Procedia - Social and Behavioral Sciences, 69, 1122-1133. Chantamuang, O., Polpinij, J., Vorakitphan, V., Luaphol, B. (2022). Sentence-level sentiment analysis for student feedback relevant to teaching process assessment. In Multi-disciplinary Trends in Artificial Intelligence (p. 156-168). Berlin Heidelberg: Springer-Verlag. Chong, C., Sheikh, U.U., Samah, N., Sha’ameri, A. (2020). Analysis on reflective writing using natural language processing and sentiment analysis. IOP Conference Series: Materials Science and Engineering, 884, 012069. Clayson, D., & Haley, D. (2011). Are students telling us the truth? a critical look at the student evaluation of teaching. Marketing Education Review, 21, 101-112. Denson, N., Loveday, T., Dalton, H. (2010). Student evaluation of courses: What predicts satisfaction? Higher Education Research & Development, 29, 339-356. Devlin, J., Chang, M., Lee, K., Toutanova, K. (2019). BERT: pre-training of deep bidirectional transformers for language understanding, J. Burstein, C. Doran, & T. Solorio (Eds.), 2019 Conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 4171–4186). Association for Computational Linguistics. Donnon, T., Delver, H., Beran, T. (2010). Student and teaching characteristics related to ratings of instruction in medical sciences graduate programs. Medical Teacher, 32, 327-332. Eccles, J. S., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, C. (1983). Expectancies, Values, and Academic Behaviors. In J. T. Spence (Ed.), Achievement and Achievement Motivation (pp. 75-146). San Francisco, CA: W. H. Freeman. Emerson, R., & Records, K. (2007). Design and testing of classroom and clinical teaching evaluation tools for nursing education. International Journal of Nursing Education Scholarship, 4, Article12. Fiske, S. T., & Taylor, S. E. (2013). Social cognition: From brains to culture (2nd ed.). Sage Publications. Hammond, I., Taylor, J., Mcmenamin, P. (2003). Value of a structured participant evaluation questionnaire in the development of a surgical education program. The Australian & New Zealand Journal of Obstetrics & Gynaecology, 43, 115-118. Hazarika, D., Konwar, G., Deb, S., Bora, D. J. (2020). Sentiment Analysis on Twitter by Using TextBlob for Natural Language Processing. Proceedings of the International Conference on Research in Management & Technovation (pp. 63-67). Hodges, L., & Stanton, K. (2007). Translating comments on student evaluations into the language of learning. Innovative Higher Education, 31, 279-286. Hommel, B. E. (2023). Expanding the methodological toolbox: Machine-based item desirability ratings as an alternative to human-based ratings. Personality and Individual Differences, 213, 112307. Hoon, A., Oliver, E., Szpakowska, K., Newton, P. (2015). Use of the...

The scarcity of high-quality large-scale labeled datasets poses a huge challenge for employing deep learning models in video deception detection. To address this issue, inspired by the psychological theory on the relation between deception and expressions, we propose a novel method called AFFAKT in this paper, which enhances the classification performance by transferring useful and correlated knowledge from a large facial expression dataset. Two key challenges in knowledge transfer arise: 1) \textit{how much} knowledge of facial expression data should be transferred and 2) \textit{how to} effectively leverage transferred knowledge for the deception classification model during inference. Specifically, the optimal relation mapping between facial expression classes and deception samples is firstly quantified using proposed H-OTKT module and then transfers knowledge from the facial expression dataset to deception samples. Moreover, a correlation prototype within another proposed module SRKB is well designed to retain the invariant correlations between facial expression classes and deception classes through momentum updating. During inference, the transferred knowledge is fine-tuned with the correlation prototype using a sample-specific re-weighting strategy. Experimental results on two deception detection datasets demonstrate the superior performance of our proposed method. The interpretability study reveals high associations between deception and negative affections, which coincides with the theory in psychology.