Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Juan Bartolomé has a total of 16 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Game On: Exploring the Potential for Soft Skill Development Through Video Games

    by Juan Bartolomé, Idoya del Río, Aritz Martínez, Andoni Aranguren, Ibai Laña, Sergio Alloza

    Soft skills remain fundamental for employability and sustainable human development in an increasingly technology-driven society. These interpersonal and cognitive competencies—such as communication, adaptability, and critical thinking—represent uniquely human capabilities that current Artificial Intelligence (AI) systems cannot replicate. However, assessing and developing these skills consistently remains a challenge due to the lack of standardized evaluation frameworks. This study explores the potential of commercial video games as engaging environments for soft skills enhancement and introduces an AI-based assessment methodology to quantify such improvement. Using player data collected from the Steam platform, we designed and validated an AI model based on Gradient Boosting Regressor (GBR) to estimate participants’ soft skill progression. The model achieved high predictive performance (R2 ≈ 0.9; MAE/RMSE ≈ 1), demonstrating strong alignment between gameplay behavior and soft skill improvement. The results highlight that video game-based data analysis can provide a reliable, non-intrusive alternative to traditional testing methods, reducing test-related anxiety while maintaining assessment validity. This approach supports the integration of video games into educational and professional training frameworks as a scalable and data-driven tool for soft skills development.


  • Open Access

    Using Eye-Tracking Data to Examine Response Processes in Digital Competence Assessment for Validation Purposes

    by Juan Bartolomé, Pablo Garaizar, Erlantz Loizaga, Leire Bastida

    Background: When measuring complex cognitive constructs, it is crucial to correctly design the evaluation items in order to trigger the intended knowledge and skills. Furthermore, assessing the validity of an assessment requires considering not only the content of the evaluation tasks, but also how examinees perform by engaging construct-relevant response processes. Objectives: We used eye-tracking techniques to examine item response processes in the assessment of digital competence. The eye-tracking observations helped to fill an ‘explanatory gap’ by providing data on the variation in response processes that cannot be captured by other common sources. Method: Specifically, we used eye movement data to validate the inferences made between claimed and observed behavior. This allowed us to interpret how participants processed the information in the items in terms of Area Of Interest (their size, placement, and order). Results and Conclusions: The gaze data provide detailed information about response strategies at the item level, profiling the examinees according to their engagement, response processes and performance/success rate. The presented evidence confirms that the response patterns of the participants who responded well do not represent an alternative to the interpretation of the results that would undermine the assessment criteria. Takeaways: Gaze-based evidence has great potential to provide complementary data about the response processes performed by examinees, thereby contributing to the validity argument.


  • Open Access

    Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

    by Juan Bartolomé, Pablo Garaizar

    Until recently, most of the digital literacy frameworks have been based on assessment frameworks used by commercial entities. The release of the DigComp framework has allowed the development of tailored implementations for the evaluation of digital competence. However, the majority of these digital literacy frameworks are based on self-assessments, measuring only low-order cognitive skills. This paper reports on a study to develop and validate an assessment instrument, including interactive simulations to assess citizens’ digital competence. These formats are particularly important for the evaluation of complex cognitive constructs such as digital competence. Additionally, we selected two different approaches for designing the tests based on their scope, at the competence or competence area level. Their overall and dimensional validity and reliability were analysed. We summarise the issues addressed in each phase and key points to consider in new implementations. For both approaches, items present satisfactory difficulty and discrimination indicators. Validity was ensured through expert validation, and the Rasch analysis revealed good EAP/PV reliabilities. Therefore, the tests have sound psychometric properties that make them reliable and valid instruments for measuring digital competence. This paper contributes to an increasing number of tools designed to evaluate digital competence and highlights the necessity of measuring higher-order cognitive skills.


  • A Pragmatic Approach for Evaluating and Accrediting Digital Competence of Digital Profiles: A Case Study of Entrepreneurs and Remote Workers

    by Juan Bartolomé, Pablo Garaizar, Xabier Larrucea

    During the last decades, digital competence has become essential at workplace. Nowadays, it is difficult to find a job where no ICT skills are required. At the same time, there is a lack of ecosystems for adult reskilling in digital competence. Moreover, most of them do not use of a common language and terminology, decreasing the possibilities of being used by a wider public. In addition, the assessment of digital competence cannot be done using simple self-assessment tests, but more complex tools such as simulations or other activities based on real scenarios. Considering this, we designed a performance-based evaluation system following a pragmatic approach based on DigComp framework. We carried out a needs analysis based on expert consultation (63 teleworkers and 82 entrepreneurs) to create an assessment syllabus and implement the assessment modules. Then, we conducted an analysis by experts (n = 21) of the relationship between the content of the tests and the construct it was intended to measure. After refinement, the system was piloted by end-users all over Europe (n = 525). Results confirmed that DigComp was the most appropriate reference when considering the transversality of digital competence, providing researchers with well-defined clear criteria.


  • DEVELOPING A DIGITAL COMPETENCE ASSESSMENT AND ACCREDITATION PLATFORM FOR DIGITAL PROFILES

    by Juan Bartolomé, Iker Martínez de Soria, Mara Jakobsone, Antonio Fernández, Gabriela Ruseva, Panagiotis Koutoudis, Debbie Merrigan, Mertxe Vaquero

    Information about the paper titled "DEVELOPING A DIGITAL COMPETENCE ASSESSMENT AND ACCREDITATION PLATFORM FOR DIGITAL PROFILES" at IATED Digital Library