Magdalena Andrzejewska

Computational thinking is a key universal competence, often taught using methods specific to computer science. One step towards achieving it is learning to analyse and create algorithms. Researchers have long been trying to establish how the form of representation of algorithms (pseudocode versus flowchart) affects its understanding and have reached varying, sometimes conflicting results. This article presents findings that provide objective new data on this topic. In our experiment, we used two different types of algorithmic tasks with three levels of complexity and a group of 114 research participants with varying programming skills. In addition, we used an eye tracking technique that allowed us to collect detailed information about the subjects’ attention distribution during analysis of algorithms. Our results show that subjects took significantly less time to analyse flowcharts (than they did with pseudocode), made much fewer errors, and had higher confidence in the correctness of their solution. Based on eye tracking data, a reduced number of both re-analyses of the algorithm and input data re-referencing was observed for graphically presented tasks. The difference in favour of flowcharts was revealed (with few exceptions) for all levels of algorithm complexity (simple, medium, complex), while regarding the duration of analysis the advantage of flowcharts increased with the growing complexity of algorithms. For complex algorithms, a significant relationship was observed between algorithm presentation and level of programming skills versus the duration of task solving and confidence level. Our study strongly supports the idea of using graphic representation of algorithms both when learning to code and in acquiring computational thinking skills.

Prediction of visual attention is a new and challenging subject, and to the best of our knowledge, there are not many pieces of research devoted to the anticipation of students’ cognition when solving tests. The aim of this paper is to propose, implement, and evaluate a machine learning method that is capable of predicting saliency maps of students who participate in a learning task in the form of quizzes based on quiz questionnaire images. Our proposal utilizes several deep encoder–decoder symmetric schemas which are trained on a large set of saliency maps generated with eye tracking technology. Eye tracking data were acquired from students, who solved various tasks in the sciences and natural sciences (computer science, mathematics, physics, and biology). The proposed deep convolutional encoder–decoder network is capable of producing accurate predictions of students’ visual attention when solving quizzes. Our evaluation showed that predictions are moderately positively correlated with actual data with a coefficient of 0.547 ± 0.109. It achieved better results in terms of correlation with real saliency maps than state-of-the-art methods. Visual analyses of the saliency maps obtained also correspond with our experience and expectations in this field. Both source codes and data from our research can be downloaded in order to reproduce our results.

The article discusses the findings of longitudinal studies (three stages spanning 6 months) which were to investigate the process of acquiring the ability to comprehension program code by the computer science students having started to learn to program. The studies were conducted with the use of a knowledge measurement test, the diagnostic survey, and eye tracking technology that enabled the recording of movement of the subjects' eyes and an analysis of the patterns of information processing during solving programming problems. The obtained results have shown that the students solved the tasks most effectively in the last stage of the research during which they obtained the highest indicator of correct answers in the significantly shortest time. In the last stage of the research the dominant form of the algorithmic problem analysis was code, in two previous it was flowchart. The eye tracking data have shown that regardless of the research stage the code analysis was definitely connected with a greater number of fixations, with very near values of time devoted to solving those two forms of the algorithm. The participants who increased their competences in a scope of the program code analysis had a significantly greater saccade amplitude average (SAA) and a significantly shorter fixation duration average (FDA) in the last stage of the research comparing to previous ones. The results suggest that the FDA and SAA are parameters sensitive to the development of program comprehension skills.

Programming as a cognitive activity requires the utilization of various kinds of mental models that involve different cognitive loads while students learn to program. The article discusses the results of an experiment aimed at answering the following question: are eye tracking based measures related to the intrinsic cognitive load (ICL) connected with program comprehension? Thirty one students of computer science took part in the experiment. They analyzed two program codes written in the C++ language to search for (1) logical errors (LER) and (2) syntax errors (SER). ICL was measured by subjective rating of the difficulty of each task. There were significant differences found for the subjective measures of intrinsic load, the effectiveness and the time of tasks performance, and the values of eye tracking parameters: fixation duration average (FDA) and saccade amplitude average (SAA) in two experiment conditions. Longer fixation and shorter saccades were associated with higher ICL. The results obtained suggest that FDA and SAA are eye tracking measures sensitive of intrinsic cognitive load.

In their work, educators are increasingly relying on the knowledge of information processing mechanisms such as perception, attention or memory to explain the processes involved in learning and teaching. This article discusses the results of eye tracking experiments on external cognitive load (related to perception or attention control) when solving algorithmic tasks. The analysis of eye tracking indicators has determined which factors reduce the cognitive effort. However, there were no significant differences in the parameters analyzed, which led to the assertion that the distracters presented in the task did not increase the cognitive load, which may indicate that in the case of difficult tasks, the internal cognitive load plays a dominant role. It was noted that the low efficiency of solving algorithmic tasks constitutes a major impediment to statistical inference.

In this article, we attempted to examine the issue of the existence of differences in eye move-ment of school-age students as they solve tasks of different difficulty levels in the sciences and natural sciences (computer science, mathematics, physics, biology). Categories of the task’s difficulty level were established on the basis of two types of criteria: subjective (an evaluation made by the subjects) and behavioural (connected to the correctness of their solution). The relationships of these criteria with the visual activity parameters, which were considered to be indicators of mental effort, were identified. An analysis of the data obtained enabled the observation of discrepancies in categorizing difficulties of the tasks on the basis of subjective and behavioural criteria. A significant and strong correlation was noticed between task difficulty level, determined by the percentage of correct answers, and the fixation parameters, although such a relationship with the blink parameters was not found. There was no correlation of the activity of the eye movement parameters, considered to be mental effort indicators, with a student’s opinion about the task’s difficulty level. On the basis of the investigations made, it can be stated that the fixation duration average can be taken as an index of the difficulty level of the task being solved.

The results of qualitative and quantitative investigations conducted with individuals who learned algorithms in school are presented in this article. In these investigations, eye-tracking technology was used to follow the process of solving algorithmic problems. The algorithmic problems were presented in two comparable variants: in a pseudocode and flowchart. The eye-tracking data resulted in both qualitative (films registering the gaze paths) and quantitative measures, which allowed the detection and interpretation of the differences in the task-solving strategies between those who found the correct answer and those that did not. The results confirmed a hypothesis that use of the formal notation characteristic of a programming language for presenting algorithms is often a practical difficulty in the process of solving even simple tasks. This study opens a new direction of research; they show that eye-tracking technology can be used to optimise the educational process of learning programming.