Rianne Conijn
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Rianne Conijn has a total of 61 co-authors. Here are the top co-authors by number of shared publications:
- Luuk Van Waes
Department of Management, University of Antwerp, 2000 Antwerp, Belgium
4 shared publications Chris Snijders
Eindhoven Artificial Intelligence Systems Institute Human-Technology Interaction group Human-Technology Interaction group Eindhoven University of Technology Eindhoven University
3 shared publicationsEmily Dux Speltz
Department of English, Iowa State University, Ames, USA
3 shared publicationsMenno van Zaanen
South African Centre of Digital Language Resources, Potchefstroom, South Africa
3 shared publications- Menno van Zaanen
South African Centre for Digital Language Resources
3 shared publications - Chris Snijders
Human-Technology Interaction Group, Eindhoven University of Technology, Eindhoven, The Netherlands, The Netherlands
2 shared publications
Latest Publications
- Open Access
- 7 September 2026
- Published by MDPI AG in Preprints.org
Moving Explanations Closer to Humans: A Systematic Review of Comparisons of Explainable AI Implementations
As AI increasingly supports human decision-making, designing clear and usable explanations has become a key challenge. Explainable Artificial Intelligence (XAI) aims to address this challenge, yet it remains unclear which explanation characteristics contribute to effective human-AI interaction. Following PRISMA guidelines, this systematic review synthesizes evidence from 161 empirical studies that directly compared multiple XAI implementations using human-centered evaluation metrics. Studies were analyzed across four explanation aspects: method, content, format, and interaction. Results were highly heterogeneous and often showed limited or non-significant differences between explanation implementations. Variations in explanation methods and content rarely produced consistent effects on user outcomes. In contrast, user-facing design choices, particularly explanation format and interaction, showed more consistent effects on user-related outcomes. Overall, the findings suggest that explanation effectiveness depends less on the underlying algorithm and more on how explanations are presented and experienced, highlighting the importance of a user-centered design perspective for XAI research and practice.
- 7 September 2026
- Published by MDPI AG in Preprints.org
Open AccessMoving Explanations Closer to Humans: A Systematic Review of Comparisons of Explainable AI Implementations
As AI increasingly supports human decision-making, designing clear and usable explanations has become a key challenge. Explainable Artificial Intelligence (XAI) aims to address this challenge, yet it remains unclear which explanation characteristics contribute to effective human-AI interaction. Following PRISMA guidelines, this systematic review synthesizes evidence from 161 empirical studies that directly compared multiple XAI implementations using human-centered evaluation metrics. Studies were analyzed across four explanation aspects: method, content, format, and interaction. Results were highly heterogeneous and often showed limited or non-significant differences between explanation implementations. Variations in explanation methods and content rarely produced consistent effects on user outcomes. In contrast, user-facing design choices, particularly explanation format and interaction, showed more consistent effects on user-related outcomes. Overall, the findings suggest that explanation effectiveness depends less on the underlying algorithm and more on how explanations are presented and experienced, highlighting the importance of a user-centered design perspective for XAI research and practice.
- Open Access
- 21 August 2026
- Published by Taylor & Francis in Creativity Research Journal
- p. 1-17
Semantic Alignment is Not Enough: Evaluating Creativity in Iterative AI Image Generation
Creativity is a complex and multifaceted phenomenon studied extensively across disciplines such as psychology, philosophy, and the arts (Runco, 2004; Said-Metwaly et al., 2017). Broadly defined, it...
- 21 August 2026
- Published by Taylor & Francis in Creativity Research Journal
- p. 1-17
Open AccessSemantic Alignment is Not Enough: Evaluating Creativity in Iterative AI Image Generation
Creativity is a complex and multifaceted phenomenon studied extensively across disciplines such as psychology, philosophy, and the arts (Runco, 2004; Said-Metwaly et al., 2017). Broadly defined, it...
- Open Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 610-620
The Blind Spots in Automated Feedback Generation for Academic Writing
Machine learning-based automated essay scoring (AES) and feedback generation (AFG) tools have been developed since the 1960s, with some commercially deployed. Such systems are expected to lighten the labor-intensive task of essay scoring and help provide timely feedback to students. However, these tools have not been used effectively enough in practice despite the long history of this research field. We aim to determine how accurately currently available models can make the necessary corrections and detect improvable segments of text. Latest attempts of AFG utilize state-of-the-art generative large language models (LLMs) for writing evaluation. To the best of our knowledge, however, none of the past studies have included generative LLM-based models in a fine-grained sentence-to-sentence comparison with human feedback or among AI tools. To fill these gaps, we conduct an experimental comparison of human feedback and three AI tools developed at different stages of technological advancement. Findings indicate that the overlap between human and AI feedback is predominantly limited to surface-level linguistic features and that generative AI-augmented tools demonstrate a markedly higher capability than a tool based on conventional rule-based AI.
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 610-620
Open AccessThe Blind Spots in Automated Feedback Generation for Academic Writing
Machine learning-based automated essay scoring (AES) and feedback generation (AFG) tools have been developed since the 1960s, with some commercially deployed. Such systems are expected to lighten the labor-intensive task of essay scoring and help provide timely feedback to students. However, these tools have not been used effectively enough in practice despite the long history of this research field. We aim to determine how accurately currently available models can make the necessary corrections and detect improvable segments of text. Latest attempts of AFG utilize state-of-the-art generative large language models (LLMs) for writing evaluation. To the best of our knowledge, however, none of the past studies have included generative LLM-based models in a fine-grained sentence-to-sentence comparison with human feedback or among AI tools. To fill these gaps, we conduct an experimental comparison of human feedback and three AI tools developed at different stages of technological advancement. Findings indicate that the overlap between human and AI feedback is predominantly limited to surface-level linguistic features and that generative AI-augmented tools demonstrate a markedly higher capability than a tool based on conventional rule-based AI.
- Open Access
- 18 November 2025
- Published by Center for Open Science in PsyArXiv
Assessing students’ DRIVE: A framework to evaluate learning through students’ interactions with generative AI
As generative AI (GenAI) transforms how students learn and work, higher education must rethink its assessment strategies. This paper presents a taxonomy and conceptual framework (DRIVE) to evaluate student learning from GenAI interactions (prompting strategies), focusing on cognitive engagement (Directive Reasoning Interaction) and knowledge infusion (Visible Expertise). Despite extensive research mapping student GenAI writing behaviors, practical tools for assessing domain-specific learning remain underexplored. This paper shows how GenAI interactions inform such learning in authentic classroom contexts, moving beyond technical skills or low-stakes assignments. We conducted multi-methods analysis of GenAI interaction annotations (n=1450) from graded essays (n=70) in STEM writing courses. A strong positive correlation was found between high-quality GenAI interactions and final essay scores, validating the feasibility of this assessment approach. Furthermore, our taxonomy revealed distinct interaction profiles: High essay scores correlated with a ”Targeted Improvement Partnership” focused on text refinement, while high interaction scores were linked to a ”Collaborative Intellectual Partnership” centered on idea development. Conversely, below-average performance was associated with ”Basic Information Retrieval” or ”Passive Task Delegation”. These findings demonstrate how the assessment method (output vs. process focus) may shape students’ GenAI usage and learning depth. These findings demonstrate that the assessment method (output vs. process) shapes student AI use. Traditional assessment can reinforce text optimization, while process-focused evaluation may reward the exploratory partnership crucial for deeper learning. The DRIVE framework and related taxonomy offers educators a practical tool to design assessments that capture authentic learning in AI-integrated classrooms.
- 18 November 2025
- Published by Center for Open Science in PsyArXiv
Open AccessAssessing students’ DRIVE: A framework to evaluate learning through students’ interactions with generative AI
As generative AI (GenAI) transforms how students learn and work, higher education must rethink its assessment strategies. This paper presents a taxonomy and conceptual framework (DRIVE) to evaluate student learning from GenAI interactions (prompting strategies), focusing on cognitive engagement (Directive Reasoning Interaction) and knowledge infusion (Visible Expertise). Despite extensive research mapping student GenAI writing behaviors, practical tools for assessing domain-specific learning remain underexplored. This paper shows how GenAI interactions inform such learning in authentic classroom contexts, moving beyond technical skills or low-stakes assignments. We conducted multi-methods analysis of GenAI interaction annotations (n=1450) from graded essays (n=70) in STEM writing courses. A strong positive correlation was found between high-quality GenAI interactions and final essay scores, validating the feasibility of this assessment approach. Furthermore, our taxonomy revealed distinct interaction profiles: High essay scores correlated with a ”Targeted Improvement Partnership” focused on text refinement, while high interaction scores were linked to a ”Collaborative Intellectual Partnership” centered on idea development. Conversely, below-average performance was associated with ”Basic Information Retrieval” or ”Passive Task Delegation”. These findings demonstrate how the assessment method (output vs. process focus) may shape students’ GenAI usage and learning depth. These findings demonstrate that the assessment method (output vs. process) shapes student AI use. Traditional assessment can reinforce text optimization, while process-focused evaluation may reward the exploratory partnership crucial for deeper learning. The DRIVE framework and related taxonomy offers educators a practical tool to design assessments that capture authentic learning in AI-integrated classrooms.
- Open Access
- 7 October 2025
- Published by ARLE (International Association for Research in L1 Education) in Journal of Writing Research
- Vol. 17 (2) , 339-369
Phase to phase: Developing an automated procedure to identify and visualize phases in writing sessions using keystroke data
Understanding the temporal organization of writing is key to studying writing processes. Existing methods to segment writing into phases often rely on arbitrary rules, extensive manual annotation, or focus on numerous transitions. This study aimed to develop an automated segmentation method to detect distinctive transition in the dominant writing process, particularly the transition from first draft to revision. For this, keystroke data (source-based L1 writing (N = 80) and text simplification in L2 (N = 88)) were manually annotated. The BEAST algorithm was applied for Bayesian change point detection, based on five characteristics derived from the annotation criteria: (1) percentage of the final text written so far, (2) distance between typed and remaining characters, (3) relative cursor position, (4) source use, and (5) pause timings. The first three features proved most effective in identifying change points. A rule-based approach was further applied to select one final change point, which resulted in mediocre accuracy ranging from 31% exact agreement to 49% agreement within 60 seconds. To conclude, the BEAST algorithm is useful in detecting a variety of change points in writing processes, yet connecting them to meaningful phases is still quite complex.
- 7 October 2025
- Published by ARLE (International Association for Research in L1 Education) in Journal of Writing Research
- Vol. 17 (2) , 339-369
Open AccessPhase to phase: Developing an automated procedure to identify and visualize phases in writing sessions using keystroke data
Understanding the temporal organization of writing is key to studying writing processes. Existing methods to segment writing into phases often rely on arbitrary rules, extensive manual annotation, or focus on numerous transitions. This study aimed to develop an automated segmentation method to detect distinctive transition in the dominant writing process, particularly the transition from first draft to revision. For this, keystroke data (source-based L1 writing (N = 80) and text simplification in L2 (N = 88)) were manually annotated. The BEAST algorithm was applied for Bayesian change point detection, based on five characteristics derived from the annotation criteria: (1) percentage of the final text written so far, (2) distance between typed and remaining characters, (3) relative cursor position, (4) source use, and (5) pause timings. The first three features proved most effective in identifying change points. A rule-based approach was further applied to select one final change point, which resulted in mediocre accuracy ranging from 31% exact agreement to 49% agreement within 60 seconds. To conclude, the BEAST algorithm is useful in detecting a variety of change points in writing processes, yet connecting them to meaningful phases is still quite complex.
- Open Access
- 3 July 2025
- Published by Wiley in Journal of Computer Assisted Learning
- Vol. 41 (4)
Service Design for the Improvement of Intelligent Tutoring Systems: A Case Study
Background: Recently human‐centred design plays an increasing role in learning analytics, however this approach is mostly lacking in the design and evaluation of Intelligent Tutoring Systems (ITSs). A potential way to incorporate human‐centred design principles in ITS development is by adopting a service design approach.Objectives: This article aims to identify how a service design approach can contribute to ITS development.Methods: A specific case study is implemented on Waterproof, a system for helping students learn how to write mathematical proofs. Three substudies are conducted, namely a co‐creative service blueprinting session; a usability test which focuses on the broad usage timeline; and an analysis of trace data.Results and Conclusions: The case study's results showed several concrete steps to improve the system and its embedding in the course, including for the onboarding process and the presentation of feedback in the form of error messages. Service design was shown to be useful in providing more a comprehensive design and evaluation of the ITS, going beyond the mere tool, by incorporating multiple stakeholders as well as the educational context.
- 3 July 2025
- Published by Wiley in Journal of Computer Assisted Learning
- Vol. 41 (4)
Open AccessService Design for the Improvement of Intelligent Tutoring Systems: A Case Study
Background: Recently human‐centred design plays an increasing role in learning analytics, however this approach is mostly lacking in the design and evaluation of Intelligent Tutoring Systems (ITSs). A potential way to incorporate human‐centred design principles in ITS development is by adopting a service design approach.Objectives: This article aims to identify how a service design approach can contribute to ITS development.Methods: A specific case study is implemented on Waterproof, a system for helping students learn how to write mathematical proofs. Three substudies are conducted, namely a co‐creative service blueprinting session; a usability test which focuses on the broad usage timeline; and an analysis of trace data.Results and Conclusions: The case study's results showed several concrete steps to improve the system and its embedding in the course, including for the onboarding process and the presentation of feedback in the form of error messages. Service design was shown to be useful in providing more a comprehensive design and evaluation of the ITS, going beyond the mere tool, by incorporating multiple stakeholders as well as the educational context.
- Open Access
- 25 February 2025
- Published by Center for Open Science in OSF Preprints
Typing in tandem: language planning in multi-sentence text production is fundamentally parallel
Classical serial models view the process of producing a text as a chain of discrete pauses during which the next span of text is planned, and bursts of activity during which this text is output onto the page or computer screen. In contrast, parallel models assume that by default planning of the next text unit is performed in parallel with previous execution. We instantiated these two views as Bayesian mixed-effects models across six sets of keystroke data from child and adult writers composing different types of multi-sentence text. We modelled interkey intervals with a single distribution, hypothesised by the serial processing account, and with a two-distribution mixture model that is hypothesised by the parallel-processing account. We analysed intervals occuring before-sentence, before word, and within word. Model comparisons demonstrated strong evidence in favour of the parallel view across all datasets. When pausing occurred, sentence initial inter-keystroke intervals were longer than word initial pauses. This is consistent with the idea that edges of larger linguisticunits are associated with higher level planning. However, we found – across populations –that interkey intervals at word and even at sentence boundaries were often too brief toplausibly represent time to plan what was written next. Our results cannot be explained bythe serial processing but are in line with the parallel view of multi-sentence text composition.
- 25 February 2025
- Published by Center for Open Science in OSF Preprints
Open AccessTyping in tandem: language planning in multi-sentence text production is fundamentally parallel
Classical serial models view the process of producing a text as a chain of discrete pauses during which the next span of text is planned, and bursts of activity during which this text is output onto the page or computer screen. In contrast, parallel models assume that by default planning of the next text unit is performed in parallel with previous execution. We instantiated these two views as Bayesian mixed-effects models across six sets of keystroke data from child and adult writers composing different types of multi-sentence text. We modelled interkey intervals with a single distribution, hypothesised by the serial processing account, and with a two-distribution mixture model that is hypothesised by the parallel-processing account. We analysed intervals occuring before-sentence, before word, and within word. Model comparisons demonstrated strong evidence in favour of the parallel view across all datasets. When pausing occurred, sentence initial inter-keystroke intervals were longer than word initial pauses. This is consistent with the idea that edges of larger linguisticunits are associated with higher level planning. However, we found – across populations –that interkey intervals at word and even at sentence boundaries were often too brief toplausibly represent time to plan what was written next. Our results cannot be explained bythe serial processing but are in line with the parallel view of multi-sentence text composition.
- Open Access
- 16 January 2025
- Published by Center for Open Science in OSF Preprints
Service design for the improvement of Intelligent Tutoring Systems: a case study
Background: Recently human-centred design plays an increasing role in learning analytics, however this approach is mostly lacking in the design and evaluation of Intelligent Tutoring Systems (ITS). A potential way to incorporate human-centred design principles in ITS development is by adopting a service design approach.Objectives: This article aims to identify how a service design approach can contribute to ITS development.Methods: A specific case study is implemented on Waterproof, a system for helping students learn how to write mathematical proofs. Three substudies are conducted, namely a co-creative service blueprinting session; a usability test which focuses on the broad usage timeline; and an analysis of trace data.Results and Conclusions: The case study’s results showed several concrete steps to improve the system and its embedding in the course, including for the onboarding process and the presentation of feedback in the form of error messages. Service design was shown to be useful in providing more a comprehensive design and evaluation of the ITS, going beyond the mere tool, by incorporating multiple stakeholders as well as the educational context.
- 16 January 2025
- Published by Center for Open Science in OSF Preprints
Open AccessService design for the improvement of Intelligent Tutoring Systems: a case study
Background: Recently human-centred design plays an increasing role in learning analytics, however this approach is mostly lacking in the design and evaluation of Intelligent Tutoring Systems (ITS). A potential way to incorporate human-centred design principles in ITS development is by adopting a service design approach.Objectives: This article aims to identify how a service design approach can contribute to ITS development.Methods: A specific case study is implemented on Waterproof, a system for helping students learn how to write mathematical proofs. Three substudies are conducted, namely a co-creative service blueprinting session; a usability test which focuses on the broad usage timeline; and an analysis of trace data.Results and Conclusions: The case study’s results showed several concrete steps to improve the system and its embedding in the course, including for the onboarding process and the presentation of feedback in the form of error messages. Service design was shown to be useful in providing more a comprehensive design and evaluation of the ITS, going beyond the mere tool, by incorporating multiple stakeholders as well as the educational context.
- Top Cited PapersOpen Access
- 10 January 2025
- Published by Elsevier in Computers & Education
- Vol. 228
Dynamics of self-regulated learning: The effectiveness of students’ strategies across course periods
- 10 January 2025
- Published by Elsevier in Computers & Education
- Vol. 228
Top Cited PapersOpen AccessDynamics of self-regulated learning: The effectiveness of students’ strategies across course periods
- 13 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Learning Technologies
- Vol. 17, 2129-2140
Boundary Conditions of Generalizing Predictive Models for Academic Performance: Within Cohort Versus Within Course
Much of Learning Analytics research has focused on factors influencing model generalizability of predictive models for academic performance. The degree of model generalizability across courses may depend on aspects such as the similarity of the course setup, course material, the student cohort, or the teacher. Which of these contextual factors affect generalizability and to what extent is yet unclear. The current study explicitly compares model generalizability within course versus within cohort of predictive models. This study considered 66 behavioral indicators which are commonly used in the literature. Indicators regarding frequency and duration of online study time, accessing study material, time management, assignments and quizzes, and weekly measures were extracted from the university's learning management system. Numerical and binary predictive models were generated via recursive feature selection. Model generalizability was evaluated in terms of both model stability and model performance. The results showed that model stability was better for numerical models generalized within course compared to models generalized within cohort or across course and across cohort. Nevertheless, model stability was low for the binary models and only moderate for numerical models under all conditions. Concerning model performance, the increase in estimation error after model generalizability depends on the initial model performance for models generalized within course and within cohort. Contrary to previous research, with respect to performance we found no difference between model generalizability within cohort and within course. We suspect that performance reduction after any form of model generalizability depends on initial performance.
- 13 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Learning Technologies
- Vol. 17, 2129-2140
Boundary Conditions of Generalizing Predictive Models for Academic Performance: Within Cohort Versus Within Course
Much of Learning Analytics research has focused on factors influencing model generalizability of predictive models for academic performance. The degree of model generalizability across courses may depend on aspects such as the similarity of the course setup, course material, the student cohort, or the teacher. Which of these contextual factors affect generalizability and to what extent is yet unclear. The current study explicitly compares model generalizability within course versus within cohort of predictive models. This study considered 66 behavioral indicators which are commonly used in the literature. Indicators regarding frequency and duration of online study time, accessing study material, time management, assignments and quizzes, and weekly measures were extracted from the university's learning management system. Numerical and binary predictive models were generated via recursive feature selection. Model generalizability was evaluated in terms of both model stability and model performance. The results showed that model stability was better for numerical models generalized within course compared to models generalized within cohort or across course and across cohort. Nevertheless, model stability was low for the binary models and only moderate for numerical models under all conditions. Concerning model performance, the increase in estimation error after model generalizability depends on the initial model performance for models generalized within course and within cohort. Contrary to previous research, with respect to performance we found no difference between model generalizability within cohort and within course. We suspect that performance reduction after any form of model generalizability depends on initial performance.