Assessment Instruments for Generative AI Literacy in Higher Education: A Systematic Review of Measurement Approaches, Psychometric Properties, and Implications for SDG 4
Authors:
- Willy Adauto-MedinaWAWilly Adauto-Medina
- César León-VelardeCLCésar León-Velarde
- Silvia Fernández-FloresSFSilvia Fernández-Flores
- José Antonio Arévalo-TuestaJAJosé Antonio Arévalo-Tuesta
- Irma Aybar-BellidoIAIrma Aybar-Bellido
- Maritza AronesMAMaritza Arones
- Beatriz Caycho-SalasBCBeatriz Caycho-Salas
- Adrián Quispe-AndíaAQAdrián Quispe-Andía
Open Access
- 29 September 2026
- journal article
- review article
- Published by MDPI AG in Sustainability
- Vol. 18 (19) , 9952
Abstract
Assessment of generative artificial intelligence (GenAI) literacy in higher education draws on both GenAI-specific instruments and broader artificial intelligence (AI) literacy measures, but these evidence sources are not interchangeable. This systematic review examined the GenAI-literacy assessment landscape while distinguishing direct GenAI-specific evidence from general AI-literacy evidence. Following PRISMA 2020 and a registered INPLASY protocol, four electronic information sources were searched for peer-reviewed English-language studies published from 2022 through 26 July 2026. Thirty-one studies met the eligibility criteria: 23 (74.2%) assessed general AI literacy, whereas eight (25.8%) used GenAI-specific instruments. Self-report Likert-type scales predominated (n = 28; 90.3%); performance-based and hybrid approaches were uncommon. Psychometric evidence centered on internal consistency and factor structure, while temporal stability, measurement invariance, external validity, and item-level functioning were rarely examined. Potential implications were mapped to digital competencies, quality education, and responsible AI use, but inclusive learning appeared in only 11 studies (35.5%). Conclusions about GenAI-specific assessment are therefore limited to eight studies. The evidence indicates a need for GenAI-specific hybrid instruments, broader validation, and accessible performance tasks. Such assessment can inform education but constitutes a diagnostic contribution to SDG 4 rather than evidence of its achievement.Keywords
This publication has 83 references indexed in Scilit:
- A theory-driven scale for assessing text-based generative AI literacy from a self-efficacy perspective (T-GASE)Education and Information Technologies, 2026
- Discipline-Specific AI literacy (DiSAIL): a theoretical framework for situated engagement with generative AI in educationInternational Journal of Technology and Design Education, 2026
- Effects of a prompt engineering intervention on undergraduate students' AI self‐efficacy, AI knowledge and prompt engineering ability: A mixed methods studyBritish Educational Research Journal, 2025
- AILIS 1.0: A new framework to measure AI literacy in library and information science (LIS)The Journal of Academic Librarianship, 2025
- Evaluating the role of Artificial Intelligence in sustainable development goals with an emphasis on “quality education”Discover Sustainability, 2024
- Taxonomy of competence models based on an integrative literature reviewEducation and Information Technologies, 2024
- What do university students know about Artificial Intelligence? Development and validation of an AI literacy testComputers and Education: Artificial Intelligence, 2023
- Artificial intelligence literacy in higher and adult education: A scoping literature reviewComputers and Education: Artificial Intelligence, 2022
- Ethics of AI in Education: Towards a Community-Wide FrameworkInternational Journal of Artificial Intelligence in Education, 2022
- A Comparison of Reliability Coefficients for Ordinal Rating ScalesJournal of Classification, 2021