RCRianne Conijn

Rianne Conijn

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Rianne Conijn has a total of 61 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Moving Explanations Closer to Humans: A Systematic Review of Comparisons of Explainable AI Implementations

    by Mieke Ronckers, Rianne Conijn, Chris Snijders

    As AI increasingly supports human decision-making, designing clear and usable explanations has become a key challenge. Explainable Artificial Intelligence (XAI) aims to address this challenge, yet it remains unclear which explanation characteristics contribute to effective human-AI interaction. Following PRISMA guidelines, this systematic review synthesizes evidence from 161 empirical studies that directly compared multiple XAI implementations using human-centered evaluation metrics. Studies were analyzed across four explanation aspects: method, content, format, and interaction. Results were highly heterogeneous and often showed limited or non-significant differences between explanation implementations. Variations in explanation methods and content rarely produced consistent effects on user outcomes. In contrast, user-facing design choices, particularly explanation format and interaction, showed more consistent effects on user-related outcomes. Overall, the findings suggest that explanation effectiveness depends less on the underlying algorithm and more on how explanations are presented and experienced, highlighting the importance of a user-centered design perspective for XAI research and practice.

  • Open Access

    Semantic Alignment is Not Enough: Evaluating Creativity in Iterative AI Image Generation

    by R. P. W. Schmidt, R. Conijn, P.-Y. Lin, P. Pauwels, H. Hofmeyer

    Creativity is a complex and multifaceted phenomenon studied extensively across disciplines such as psychology, philosophy, and the arts (Runco, 2004; Said-Metwaly et al., 2017). Broadly defined, it...

  • Open Access

    The Blind Spots in Automated Feedback Generation for Academic Writing

    by Toru Sasaki, Rianne Conijn, Martijn C. Willemsen

    Machine learning-based automated essay scoring (AES) and feedback generation (AFG) tools have been developed since the 1960s, with some commercially deployed. Such systems are expected to lighten the labor-intensive task of essay scoring and help provide timely feedback to students. However, these tools have not been used effectively enough in practice despite the long history of this research field. We aim to determine how accurately currently available models can make the necessary corrections and detect improvable segments of text. Latest attempts of AFG utilize state-of-the-art generative large language models (LLMs) for writing evaluation. To the best of our knowledge, however, none of the past studies have included generative LLM-based models in a fine-grained sentence-to-sentence comparison with human feedback or among AI tools. To fill these gaps, we conduct an experimental comparison of human feedback and three AI tools developed at different stages of technological advancement. Findings indicate that the overlap between human and AI feedback is predominantly limited to surface-level linguistic features and that generative AI-augmented tools demonstrate a markedly higher capability than a tool based on conventional rule-based AI.

  • Open Access

    Assessing students’ DRIVE: A framework to evaluate learning through students’ interactions with generative AI

    by Manuel J. B. Oliveira, Carlos Zednik, Gunter Bombaerts, Bert Sadowski, Rianne Conijn

    As generative AI (GenAI) transforms how students learn and work, higher education must rethink its assessment strategies. This paper presents a taxonomy and conceptual framework (DRIVE) to evaluate student learning from GenAI interactions (prompting strategies), focusing on cognitive engagement (Directive Reasoning Interaction) and knowledge infusion (Visible Expertise). Despite extensive research mapping student GenAI writing behaviors, practical tools for assessing domain-specific learning remain underexplored. This paper shows how GenAI interactions inform such learning in authentic classroom contexts, moving beyond technical skills or low-stakes assignments. We conducted multi-methods analysis of GenAI interaction annotations (n=1450) from graded essays (n=70) in STEM writing courses. A strong positive correlation was found between high-quality GenAI interactions and final essay scores, validating the feasibility of this assessment approach. Furthermore, our taxonomy revealed distinct interaction profiles: High essay scores correlated with a ”Targeted Improvement Partnership” focused on text refinement, while high interaction scores were linked to a ”Collaborative Intellectual Partnership” centered on idea development. Conversely, below-average performance was associated with ”Basic Information Retrieval” or ”Passive Task Delegation”. These findings demonstrate how the assessment method (output vs. process focus) may shape students’ GenAI usage and learning depth. These findings demonstrate that the assessment method (output vs. process) shapes student AI use. Traditional assessment can reinforce text optimization, while process-focused evaluation may reward the exploratory partnership crucial for deeper learning. The DRIVE framework and related taxonomy offers educators a practical tool to design assessments that capture authentic learning in AI-integrated classrooms.

  • Open Access

    Phase to phase: Developing an automated procedure to identify and visualize phases in writing sessions using keystroke data

    by Rianne Conijn, Alessandra Rossetti, Nina Vandermeulen, Luuk Van Waes

    Understanding the temporal organization of writing is key to studying writing processes. Existing methods to segment writing into phases often rely on arbitrary rules, extensive manual annotation, or focus on numerous transitions. This study aimed to develop an automated segmentation method to detect distinctive transition in the dominant writing process, particularly the transition from first draft to revision. For this, keystroke data (source-based L1 writing (N = 80) and text simplification in L2 (N = 88)) were manually annotated. The BEAST algorithm was applied for Bayesian change point detection, based on five characteristics derived from the annotation criteria: (1) percentage of the final text written so far, (2) distance between typed and remaining characters, (3) relative cursor position, (4) source use, and (5) pause timings. The first three features proved most effective in identifying change points. A rule-based approach was further applied to select one final change point, which resulted in mediocre accuracy ranging from 31% exact agreement to 49% agreement within 60 seconds. To conclude, the BEAST algorithm is useful in detecting a variety of change points in writing processes, yet connecting them to meaningful phases is still quite complex.


  • Open Access

    Service Design for the Improvement of Intelligent Tutoring Systems: A Case Study

    by Jelle Wemmenhove, Dorina Bór, Rianne Conijn, Jim Portegies

    Background: Recently human‐centred design plays an increasing role in learning analytics, however this approach is mostly lacking in the design and evaluation of Intelligent Tutoring Systems (ITSs). A potential way to incorporate human‐centred design principles in ITS development is by adopting a service design approach.Objectives: This article aims to identify how a service design approach can contribute to ITS development.Methods: A specific case study is implemented on Waterproof, a system for helping students learn how to write mathematical proofs. Three substudies are conducted, namely a co‐creative service blueprinting session; a usability test which focuses on the broad usage timeline; and an analysis of trace data.Results and Conclusions: The case study's results showed several concrete steps to improve the system and its embedding in the course, including for the onboarding process and the presentation of feedback in the form of error messages. Service design was shown to be useful in providing more a comprehensive design and evaluation of the ITS, going beyond the mere tool, by incorporating multiple stakeholders as well as the educational context.

  • Open Access

    Typing in tandem: language planning in multi-sentence text production is fundamentally parallel

    by Jens Roeser, Rianne Conijn, Evgeny Chukharev, Gunn Helen Ofstad, Mark Torrance

    Classical serial models view the process of producing a text as a chain of discrete pauses during which the next span of text is planned, and bursts of activity during which this text is output onto the page or computer screen. In contrast, parallel models assume that by default planning of the next text unit is performed in parallel with previous execution. We instantiated these two views as Bayesian mixed-effects models across six sets of keystroke data from child and adult writers composing different types of multi-sentence text. We modelled interkey intervals with a single distribution, hypothesised by the serial processing account, and with a two-distribution mixture model that is hypothesised by the parallel-processing account. We analysed intervals occuring before-sentence, before word, and within word. Model comparisons demonstrated strong evidence in favour of the parallel view across all datasets. When pausing occurred, sentence initial inter-keystroke intervals were longer than word initial pauses. This is consistent with the idea that edges of larger linguisticunits are associated with higher level planning. However, we found – across populations –that interkey intervals at word and even at sentence boundaries were often too brief toplausibly represent time to plan what was written next. Our results cannot be explained bythe serial processing but are in line with the parallel view of multi-sentence text composition.

  • Open Access

    Service design for the improvement of Intelligent Tutoring Systems: a case study

    by Jelle Wemmenhove, Dorina Bór, Rianne Conijn, J.W.P Ortegies

    Background: Recently human-centred design plays an increasing role in learning analytics, however this approach is mostly lacking in the design and evaluation of Intelligent Tutoring Systems (ITS). A potential way to incorporate human-centred design principles in ITS development is by adopting a service design approach.Objectives: This article aims to identify how a service design approach can contribute to ITS development.Methods: A specific case study is implemented on Waterproof, a system for helping students learn how to write mathematical proofs. Three substudies are conducted, namely a co-creative service blueprinting session; a usability test which focuses on the broad usage timeline; and an analysis of trace data.Results and Conclusions: The case study’s results showed several concrete steps to improve the system and its embedding in the course, including for the onboarding process and the presentation of feedback in the form of error messages. Service design was shown to be useful in providing more a comprehensive design and evaluation of the ITS, going beyond the mere tool, by incorporating multiple stakeholders as well as the educational context.

  • Top Cited Papers
    Open Access

    Dynamics of self-regulated learning: The effectiveness of students’ strategies across course periods

    by T.S. Cristea, S. Heikkinen, C. Snijders, M. Saqr, U. Matzat, R. Conijn, A. Kleingeld


  • Boundary Conditions of Generalizing Predictive Models for Academic Performance: Within Cohort Versus Within Course

    by Sonja Kleter, Uwe Matzat, Rianne Conijn

    Much of Learning Analytics research has focused on factors influencing model generalizability of predictive models for academic performance. The degree of model generalizability across courses may depend on aspects such as the similarity of the course setup, course material, the student cohort, or the teacher. Which of these contextual factors affect generalizability and to what extent is yet unclear. The current study explicitly compares model generalizability within course versus within cohort of predictive models. This study considered 66 behavioral indicators which are commonly used in the literature. Indicators regarding frequency and duration of online study time, accessing study material, time management, assignments and quizzes, and weekly measures were extracted from the university's learning management system. Numerical and binary predictive models were generated via recursive feature selection. Model generalizability was evaluated in terms of both model stability and model performance. The results showed that model stability was better for numerical models generalized within course compared to models generalized within cohort or across course and across cohort. Nevertheless, model stability was low for the binary models and only moderate for numerical models under all conditions. Concerning model performance, the increase in estimation error after model generalizability depends on the initial model performance for models generalized within course and within cohort. Contrary to previous research, with respect to performance we found no difference between model generalizability within cohort and within course. We suspect that performance reduction after any form of model generalizability depends on initial performance.