Detmar Meurers

Educational technologies are being used more and more in secondary school settings. This increases the amount of students’ learning related data produced and stored. To keep up with this rise and to get most out of the collected data, teachers need digital tools that support and facilitate their pedagogical decision-making process. Learning analytics dashboards can be a good source to provide teachers with necessary insights into their students’ learning processes. However, for such tools to be effective and actionable, they have to be aligned with teachers’ needs and thus, provide and visualize data in a concise and structured way. We therefore conducted a survey study with 11 English teachers from K-12 secondary schools in Germany who evaluated the assumed usefulness of possible dashboard features. Based on these findings, we developed a teacher dashboard incorporating the most desired functionalities, such as a quickly accessible summary of strengths, weaknesses and support needs, or an overview of current misconceptions and competencies alongside additional metrics in order to support multiple teaching practices. The implementation and the underlying calculations are described, focusing on the importance of learners’ process data to provide teachers with a detailed and revealing view on their students’ and class learning states. In an evaluation study of the dashboard’s prototype with mock data, teachers (n=6) gave high ratings for the dashboard’s usability.

Interactive digital tools increasingly used for language learning can provide detailed system logs (e.g., number of attempts, responses submitted), and thereby a window into the user’s learning processes. To date, SLA researchers have made little use of such data to understand the relationships between learning conditions, processes, and outcomes. To fill this gap, we analyzed and interpreted detailed logs from an ICALL system used in a randomized controlled field study where 205 German learners of English in secondary school received either general or specific corrective feedback on grammar exercises. In addition to explicit pre-/post-test results, we derived 19 learning process variables from the system log. Exploratory factor analysis revealed three latent factors underlying these process variables: effort, accuracy focus, and time on task. Accuracy focus and finish time (a process variable that did not load well on any factors) significantly predicted pre-/post-test gain scores with a medium effect size. We then clustered learners based on their process patterns and found that the specific feedback group tended to demonstrate particular learning processes and that these patterns moderate the advantage of specific feedback. We discuss the implications of analyzing system logs for SLA, CALL, and education researchers and call for more collaboration.

Macro-adaptive systems aim to assign practice exercises to language learners that match their proficiency levels. While learner-dependent parameters of exercise difficulty need to be considered online, learner-independent parameters can inform an exercise’s difficulty level in a resource-efficient offline procedure. We present an evaluation of learners’ responses to form-based grammar exercises that aims to identify learner-independent exercise parameters affecting exercise complexity. The results indicate that the exercise type can yield coarse-grained complexity estimates, whereas exercise type specific features can inform more fine-grained estimates. For fine-grained estimates, we show that syntactic variants significantly impact exercise difficulty. Since there is strong variation between learning targets and learners with respect to the impact of different exercise parameters on a learner’s performance, exercise difficulty can only be reliably determined if the exercises are created in a systematic way and by also considering characteristics of the learner.