Cultural Domain in Development and Validation of Tests on Arts to Assess the Skills of Student Graphic Designers
Authors:
- 3 January 2022
- journal article
- research article
- Published by World Scientific and Engineering Academy and Society (WSEAS) in WSEAS TRANSACTIONS ON ENVIRONMENT AND DEVELOPMENT
- Vol. 18, 1-9
Abstract
The purpose of the study was to examine whether the presence of items that covered cultural questions in the test оn arts for student graphic designers influenced the fairness of the test across different ethnic and social groups. The reason for the study was to address the gap in the development and validation of tests оn arts that include a cultural domain to assess the skills of undergraduate graphic designers. The research design was based on DIF and DTF methods to examine the factorial structure of test data and to identify whether the factorial structure of the test was the same across ethnic and social groups. A one-factor CFA model was applied to perform measurements for categorised ethnic and social status groups to identify whether the factorial structure was similar or identical for them. The goodness-of-fit measures were calculated using the chi-square statistics, CFI, TLI, and RMSEA to identify how the obtained data is consistent with the hypothesised model. The inclusion of local culture-related questions in the tests оn arts to assess the skills of student graphic designers influences the individuals' latent traits which lead to an increase in DIF values. Though there were detected seven items with the DIF for the whole test, the DTF measurements showed that the DIF effect eliminated each out at test level which is related to the fact the certain ethnic groups performed better in answering some specific test items, while certain status groups performed better in answering some other test items. It is noteworthy that DTF for the Ukrainian art (miscellaneous) section was between 0.07 and 0.14 meaning a moderate DTF effect. However, the DTF variance values for the sections of principles of design and principles of art were lower than 0.07 meaning a small effect. Therefore, it could be concluded that small DTF effects found in both the whole test and each test section separately indicated that DIF effects eliminate each other at the test level.This publication has 24 references indexed in Scilit:
- Cultural Bias in TestingUpdate: Applications of Research in Music Education, 2016
- Item Response Theory (IRT) Models for Dichotomous Data2014
- Solving the measurement invariance anchor item problem in item response theory.Journal of Applied Psychology, 2012
- A taxonomy of effect size measures for the differential functioning of items and scales.Journal of Applied Psychology, 2010
- Application of Think Aloud Protocols for Examining and Confirming Sources of Differential Item Functioning Identified by Expert ReviewsEducational Measurement: Issues and Practice, 2010
- Educational Measurement by Brennan, R. L. (Ed.)Journal of Educational Measurement, 2008
- Generalized Mantel-Haenszel Methods for Differential Item Functioning DetectionEducational and Psychological Measurement, 2008
- Test Fairness, Test Bias, and DIFLanguage Assessment Quarterly, 2007
- Examining the Effects of Differential Item (Functioning and Differential) Test Functioning on Selection Decisions: When Are Statistically Significant Effects Practically Important?Journal of Applied Psychology, 2004
- The Accuracy of Multilevel Structural Equation Modeling With Pseudobalanced Groups and Small SamplesStructural Equation Modeling: A Multidisciplinary Journal, 2001