Juan Bartolomé
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Juan Bartolomé has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Pablo Garaizar
Faculty of Engineering, University of Deusto, Av. Universidades, 24, 48007 Bilbao, Spain
3 shared publications - Andoni Aranguren
TECNALIA, Basque Research and Technology Alliance (BRTA), Parque Científico y Tecnológico de Bizkaia, Astondo Bidea, Edificio 700, E-48160 Derio, Bizkaia, Spain
1 shared publication Antonio Fernández
1 shared publication- Aritz. D. Martinez
TECNALIA Basque Research & Technology Alliance (BRTA) 48160 Derio Spain
1 shared publication Debbie Merrigan
1 shared publication- Erlantz Loizaga
TECNALIA, Basque Research and Technology Alliance (BRTA), Arteaga Auzoa 25, 48160 Derio, Spain
1 shared publication
Latest Publications
- Open Access
- 20 October 2025
- Published by MDPI AG in Information
- Vol. 16 (10) , 918
Game On: Exploring the Potential for Soft Skill Development Through Video Games
Soft skills remain fundamental for employability and sustainable human development in an increasingly technology-driven society. These interpersonal and cognitive competencies—such as communication, adaptability, and critical thinking—represent uniquely human capabilities that current Artificial Intelligence (AI) systems cannot replicate. However, assessing and developing these skills consistently remains a challenge due to the lack of standardized evaluation frameworks. This study explores the potential of commercial video games as engaging environments for soft skills enhancement and introduces an AI-based assessment methodology to quantify such improvement. Using player data collected from the Steam platform, we designed and validated an AI model based on Gradient Boosting Regressor (GBR) to estimate participants’ soft skill progression. The model achieved high predictive performance (R2 ≈ 0.9; MAE/RMSE ≈ 1), demonstrating strong alignment between gameplay behavior and soft skill improvement. The results highlight that video game-based data analysis can provide a reliable, non-intrusive alternative to traditional testing methods, reducing test-related anxiety while maintaining assessment validity. This approach supports the integration of video games into educational and professional training frameworks as a scalable and data-driven tool for soft skills development.
- 20 October 2025
- Published by MDPI AG in Information
- Vol. 16 (10) , 918
Open AccessGame On: Exploring the Potential for Soft Skill Development Through Video Games
Soft skills remain fundamental for employability and sustainable human development in an increasingly technology-driven society. These interpersonal and cognitive competencies—such as communication, adaptability, and critical thinking—represent uniquely human capabilities that current Artificial Intelligence (AI) systems cannot replicate. However, assessing and developing these skills consistently remains a challenge due to the lack of standardized evaluation frameworks. This study explores the potential of commercial video games as engaging environments for soft skills enhancement and introduces an AI-based assessment methodology to quantify such improvement. Using player data collected from the Steam platform, we designed and validated an AI model based on Gradient Boosting Regressor (GBR) to estimate participants’ soft skill progression. The model achieved high predictive performance (R2 ≈ 0.9; MAE/RMSE ≈ 1), demonstrating strong alignment between gameplay behavior and soft skill improvement. The results highlight that video game-based data analysis can provide a reliable, non-intrusive alternative to traditional testing methods, reducing test-related anxiety while maintaining assessment validity. This approach supports the integration of video games into educational and professional training frameworks as a scalable and data-driven tool for soft skills development.
- Open Access
- 24 January 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (3) , 1215
Using Eye-Tracking Data to Examine Response Processes in Digital Competence Assessment for Validation Purposes
Background: When measuring complex cognitive constructs, it is crucial to correctly design the evaluation items in order to trigger the intended knowledge and skills. Furthermore, assessing the validity of an assessment requires considering not only the content of the evaluation tasks, but also how examinees perform by engaging construct-relevant response processes. Objectives: We used eye-tracking techniques to examine item response processes in the assessment of digital competence. The eye-tracking observations helped to fill an ‘explanatory gap’ by providing data on the variation in response processes that cannot be captured by other common sources. Method: Specifically, we used eye movement data to validate the inferences made between claimed and observed behavior. This allowed us to interpret how participants processed the information in the items in terms of Area Of Interest (their size, placement, and order). Results and Conclusions: The gaze data provide detailed information about response strategies at the item level, profiling the examinees according to their engagement, response processes and performance/success rate. The presented evidence confirms that the response patterns of the participants who responded well do not represent an alternative to the interpretation of the results that would undermine the assessment criteria. Takeaways: Gaze-based evidence has great potential to provide complementary data about the response processes performed by examinees, thereby contributing to the validity argument.
- 24 January 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (3) , 1215
Open AccessUsing Eye-Tracking Data to Examine Response Processes in Digital Competence Assessment for Validation Purposes
Background: When measuring complex cognitive constructs, it is crucial to correctly design the evaluation items in order to trigger the intended knowledge and skills. Furthermore, assessing the validity of an assessment requires considering not only the content of the evaluation tasks, but also how examinees perform by engaging construct-relevant response processes. Objectives: We used eye-tracking techniques to examine item response processes in the assessment of digital competence. The eye-tracking observations helped to fill an ‘explanatory gap’ by providing data on the variation in response processes that cannot be captured by other common sources. Method: Specifically, we used eye movement data to validate the inferences made between claimed and observed behavior. This allowed us to interpret how participants processed the information in the items in terms of Area Of Interest (their size, placement, and order). Results and Conclusions: The gaze data provide detailed information about response strategies at the item level, profiling the examinees according to their engagement, response processes and performance/success rate. The presented evidence confirms that the response patterns of the participants who responded well do not represent an alternative to the interpretation of the results that would undermine the assessment criteria. Takeaways: Gaze-based evidence has great potential to provide complementary data about the response processes performed by examinees, thereby contributing to the validity argument.
- Open Access
- 14 March 2022
- Published by MDPI AG in Sustainability
- Vol. 14 (6) , 3392
Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations
Until recently, most of the digital literacy frameworks have been based on assessment frameworks used by commercial entities. The release of the DigComp framework has allowed the development of tailored implementations for the evaluation of digital competence. However, the majority of these digital literacy frameworks are based on self-assessments, measuring only low-order cognitive skills. This paper reports on a study to develop and validate an assessment instrument, including interactive simulations to assess citizens’ digital competence. These formats are particularly important for the evaluation of complex cognitive constructs such as digital competence. Additionally, we selected two different approaches for designing the tests based on their scope, at the competence or competence area level. Their overall and dimensional validity and reliability were analysed. We summarise the issues addressed in each phase and key points to consider in new implementations. For both approaches, items present satisfactory difficulty and discrimination indicators. Validity was ensured through expert validation, and the Rasch analysis revealed good EAP/PV reliabilities. Therefore, the tests have sound psychometric properties that make them reliable and valid instruments for measuring digital competence. This paper contributes to an increasing number of tools designed to evaluate digital competence and highlights the necessity of measuring higher-order cognitive skills.
- 14 March 2022
- Published by MDPI AG in Sustainability
- Vol. 14 (6) , 3392
Open AccessDesign and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations
Until recently, most of the digital literacy frameworks have been based on assessment frameworks used by commercial entities. The release of the DigComp framework has allowed the development of tailored implementations for the evaluation of digital competence. However, the majority of these digital literacy frameworks are based on self-assessments, measuring only low-order cognitive skills. This paper reports on a study to develop and validate an assessment instrument, including interactive simulations to assess citizens’ digital competence. These formats are particularly important for the evaluation of complex cognitive constructs such as digital competence. Additionally, we selected two different approaches for designing the tests based on their scope, at the competence or competence area level. Their overall and dimensional validity and reliability were analysed. We summarise the issues addressed in each phase and key points to consider in new implementations. For both approaches, items present satisfactory difficulty and discrimination indicators. Validity was ensured through expert validation, and the Rasch analysis revealed good EAP/PV reliabilities. Therefore, the tests have sound psychometric properties that make them reliable and valid instruments for measuring digital competence. This paper contributes to an increasing number of tools designed to evaluate digital competence and highlights the necessity of measuring higher-order cognitive skills.
- 29 April 2021
- Published by Springer Nature in Technology, Knowledge and Learning
- Vol. 27 (3) , 843-878
A Pragmatic Approach for Evaluating and Accrediting Digital Competence of Digital Profiles: A Case Study of Entrepreneurs and Remote Workers
During the last decades, digital competence has become essential at workplace. Nowadays, it is difficult to find a job where no ICT skills are required. At the same time, there is a lack of ecosystems for adult reskilling in digital competence. Moreover, most of them do not use of a common language and terminology, decreasing the possibilities of being used by a wider public. In addition, the assessment of digital competence cannot be done using simple self-assessment tests, but more complex tools such as simulations or other activities based on real scenarios. Considering this, we designed a performance-based evaluation system following a pragmatic approach based on DigComp framework. We carried out a needs analysis based on expert consultation (63 teleworkers and 82 entrepreneurs) to create an assessment syllabus and implement the assessment modules. Then, we conducted an analysis by experts (n = 21) of the relationship between the content of the tests and the construct it was intended to measure. After refinement, the system was piloted by end-users all over Europe (n = 525). Results confirmed that DigComp was the most appropriate reference when considering the transversality of digital competence, providing researchers with well-defined clear criteria.
- 29 April 2021
- Published by Springer Nature in Technology, Knowledge and Learning
- Vol. 27 (3) , 843-878
A Pragmatic Approach for Evaluating and Accrediting Digital Competence of Digital Profiles: A Case Study of Entrepreneurs and Remote Workers
During the last decades, digital competence has become essential at workplace. Nowadays, it is difficult to find a job where no ICT skills are required. At the same time, there is a lack of ecosystems for adult reskilling in digital competence. Moreover, most of them do not use of a common language and terminology, decreasing the possibilities of being used by a wider public. In addition, the assessment of digital competence cannot be done using simple self-assessment tests, but more complex tools such as simulations or other activities based on real scenarios. Considering this, we designed a performance-based evaluation system following a pragmatic approach based on DigComp framework. We carried out a needs analysis based on expert consultation (63 teleworkers and 82 entrepreneurs) to create an assessment syllabus and implement the assessment modules. Then, we conducted an analysis by experts (n = 21) of the relationship between the content of the tests and the construct it was intended to measure. After refinement, the system was piloted by end-users all over Europe (n = 525). Results confirmed that DigComp was the most appropriate reference when considering the transversality of digital competence, providing researchers with well-defined clear criteria.
- 1 March 2018
- Published by IATED Academy in INTED Proceedings
- p. 4552-4559
DEVELOPING A DIGITAL COMPETENCE ASSESSMENT AND ACCREDITATION PLATFORM FOR DIGITAL PROFILES
Information about the paper titled "DEVELOPING A DIGITAL COMPETENCE ASSESSMENT AND ACCREDITATION PLATFORM FOR DIGITAL PROFILES" at IATED Digital Library
- 1 March 2018
- Published by IATED Academy in INTED Proceedings
- p. 4552-4559
DEVELOPING A DIGITAL COMPETENCE ASSESSMENT AND ACCREDITATION PLATFORM FOR DIGITAL PROFILES
Information about the paper titled "DEVELOPING A DIGITAL COMPETENCE ASSESSMENT AND ACCREDITATION PLATFORM FOR DIGITAL PROFILES" at IATED Digital Library