Tianyou Wang
Tianyou Wang has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Michael J. Kolen
The University of Iowa
3 shared publications - Walter P. Vispoel
University of Iowa
3 shared publications Bradley A. Hanson
2 shared publications- Deborah J. Harris
ACT, Inc
2 shared publications - Timothy Bleiler
State University of New York at Buffalo
2 shared publications - Bradley A. Hanson
CTB/McGraw‐Hill
1 shared publication
Latest Publications
- 1 September 2000
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 24 (3) , 195-210
The Effectiveness of Circular Equating as a Criterion for Evaluating Equating
Circular equating—equating a test form to itself through a chain of equatings—has been widely used as a criterion to evaluate the adequacy of equating. In this paper, analytical methods and simulations showed that this criterion is generally invalid for evaluating the adequacy of equating. Three different designs were studied: (1) the random groups design implemented in the same year, (2) the random groups design implemented across different years, and (3) the common-item-nonequivalent-groups design. For Design 1, it was shown analytically that circular equating always resulted in the identity function (i.e., the perfect result), even with the presence of random and systematic equating errors. For Design 2, a heuristic argument showed that circular equating generally deviates from the identity function by some random sampling error. A simulation study for this design also showed that expected values of circular equating might deviate slightly from the identity function, but those deviations do not reflect the systematic error (bias) embedded in the equating. For Design 3, a simulation study again showed thatcircular equating cannot reflect the systematic error in equating. systematic error in equating.
- 1 September 2000
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 24 (3) , 195-210
The Effectiveness of Circular Equating as a Criterion for Evaluating Equating
Circular equating—equating a test form to itself through a chain of equatings—has been widely used as a criterion to evaluate the adequacy of equating. In this paper, analytical methods and simulations showed that this criterion is generally invalid for evaluating the adequacy of equating. Three different designs were studied: (1) the random groups design implemented in the same year, (2) the random groups design implemented across different years, and (3) the common-item-nonequivalent-groups design. For Design 1, it was shown analytically that circular equating always resulted in the identity function (i.e., the perfect result), even with the presence of random and systematic equating errors. For Design 2, a heuristic argument showed that circular equating generally deviates from the identity function by some random sampling error. A simulation study for this design also showed that expected values of circular equating might deviate slightly from the identity function, but those deviations do not reflect the systematic error (bias) embedded in the equating. For Design 3, a simulation study again showed thatcircular equating cannot reflect the systematic error in equating. systematic error in equating.
- 1 June 2000
- Published by Wiley in Journal of Educational Measurement
- Vol. 37 (2) , 141-162
Psychometric Properties of Scale Scores and Performance Levels for Performance Assessments Using Polytomous IRT
With a focus on performance assessments, this paper describes procedures for calculating conditional standard error of measurement (CSEM) and reliability of scale scores and classification consistency of performance levels. Scale scores that are transformations of total raw scores are the focus of these procedures, although other types of raw scores are considered as well. Polytomous IRT models provide the psychometric foundation for the procedures that are described. The procedures are applied using test data from ACT's Work Keys Writing Assessment to demonstrate their usefulness. Two polytomous IRT models were compared, as were two different procedures for calculating scores. One simulation study was done using one of the models to evaluate the accuracy of the proposed procedures. The results suggest that the procedures provide quite stable estimates and have the potential to be useful in a variety of performance assessment situations.
- 1 June 2000
- Published by Wiley in Journal of Educational Measurement
- Vol. 37 (2) , 141-162
Psychometric Properties of Scale Scores and Performance Levels for Performance Assessments Using Polytomous IRT
With a focus on performance assessments, this paper describes procedures for calculating conditional standard error of measurement (CSEM) and reliability of scale scores and classification consistency of performance levels. Scale scores that are transformations of total raw scores are the focus of these procedures, although other types of raw scores are considered as well. Polytomous IRT models provide the psychometric foundation for the procedures that are described. The procedures are applied using test data from ACT's Work Keys Writing Assessment to demonstrate their usefulness. Two polytomous IRT models were compared, as were two different procedures for calculating scores. One simulation study was done using one of the models to evaluate the accuracy of the proposed procedures. The results suggest that the procedures provide quite stable estimates and have the potential to be useful in a variety of performance assessment situations.
- 1 September 1999
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 23 (3) , 263-278
Reducing Bias in CAT Trait Estimation: A Comparison of Approaches
The use of a beta prior in trait estimation was extended to the maximum expected a posteriori (MAP) method of Bayesian estimation. This new method, called essentially unbiased MAP (EU-MAP), was compared with MAP (using a standard normal prior), essentially unbiased expected a posteriori, weighted likelihood, and maximum likelihood estimation methods. Comparisons were made based on the effects that the shape of prior distributions, different item bank characteristics, and practical constraints had on bias, standard error, and root-mean-square error (RMSE). Overall, EU-MAP performed best. This new method significantly reduced bias in fixed-length tests (though with a slight increase in RMSE) and performed reasonably well when a fixed posterior variance termination rule was used. Practical constraints had little effect on the bias of this method.
- 1 September 1999
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 23 (3) , 263-278
Reducing Bias in CAT Trait Estimation: A Comparison of Approaches
The use of a beta prior in trait estimation was extended to the maximum expected a posteriori (MAP) method of Bayesian estimation. This new method, called essentially unbiased MAP (EU-MAP), was compared with MAP (using a standard normal prior), essentially unbiased expected a posteriori, weighted likelihood, and maximum likelihood estimation methods. Comparisons were made based on the effects that the shape of prior distributions, different item bank characteristics, and practical constraints had on bias, standard error, and root-mean-square error (RMSE). Overall, EU-MAP performed best. This new method significantly reduced bias in fixed-length tests (though with a slight increase in RMSE) and performed reasonably well when a fixed posterior variance termination rule was used. Practical constraints had little effect on the bias of this method.
- 1 June 1999
- Published by Wiley in Journal of Educational Measurement
- Vol. 36 (2) , 141-157
Can Examinees Use a Review Option to Obtain Positively Biased Ability Estimates on a Computerized Adaptive Test?
Part of the controversy about allowing examinees to review and change answers to previous items on computerized adaptive tests (CATs) centers on a strategy for obtaining positively biased ability estimates attributed to Wainer (1993) in which examinees intentionally answer items incorrectly before review and to the best of their abilities upon review. Our results, based on both simulated and live testing data, showed that there were instances in which the Wainer strategy yielded inflated ability estimates as well as instances in which it yielded deflated ability estimates. The success of the strategy in inflating ability estimates depended on the ability estimation method used (maximum likelihood versus Bayesian), the examinee's true ability level, the standard error of the ability estimate, the examinee's ability to implement the strategy, and the type of decision made from the ability estimate. We discuss approaches to dealing with the Wainer strategy in operational CAT settings.
- 1 June 1999
- Published by Wiley in Journal of Educational Measurement
- Vol. 36 (2) , 141-157
Can Examinees Use a Review Option to Obtain Positively Biased Ability Estimates on a Computerized Adaptive Test?
Part of the controversy about allowing examinees to review and change answers to previous items on computerized adaptive tests (CATs) centers on a strategy for obtaining positively biased ability estimates attributed to Wainer (1993) in which examinees intentionally answer items incorrectly before review and to the best of their abilities upon review. Our results, based on both simulated and live testing data, showed that there were instances in which the Wainer strategy yielded inflated ability estimates as well as instances in which it yielded deflated ability estimates. The success of the strategy in inflating ability estimates depended on the ability estimation method used (maximum likelihood versus Bayesian), the examinee's true ability level, the standard error of the ability estimate, the examinee's ability to implement the strategy, and the type of decision made from the ability estimate. We discuss approaches to dealing with the Wainer strategy in operational CAT settings.
- 1 June 1998
- Published by Wiley in Journal of Educational Measurement
- Vol. 35 (2) , 109-135
Properties of Ability Estimation Methods in Computerized Adaptive Testing
- 1 June 1998
- Published by Wiley in Journal of Educational Measurement
- Vol. 35 (2) , 109-135
Properties of Ability Estimation Methods in Computerized Adaptive Testing
- 1 March 1997
- Published by Wiley in Journal of Educational Measurement
- Vol. 34 (1) , 43-63
Computerized Adaptive and Fixed‐Item Testing of Music Listening Skill: A Comparison of Efficiency, Precision, and Concurrent Validity
We evaluated the efficiency, precision, and concurrent validity of results obtained from adaptive and fired‐item music listening tests in three studies: (a) a computer simulation study in which each of 2,200 simulees completed a computerized adaptive tonal memory test, a computerized fired‐item tonal memory test constructed from items in the adaptive test pool and two standardized group‐administered tonal memory tests; (b) a live testing study in which each of 204 examinees took the computerized adaptive test and the standardized tests; and (c) a live testing study in which randomly equivalent groups took either the computerized adaptive test (n = 86) or the computerized fired‐item test (n = 86). The adaptive music test required 50% to 93% fewer items to match the reliability and concurrent validity of the fired‐item tests, and it yielded higher levels of reliability and concurrent validity than the fired‐item tests when test length was held constant. These findings suggest that computerized adaptive tests, which typically have been limited to visually produced items, may also be well suited for measuring skills that require aurally produced items.
- 1 March 1997
- Published by Wiley in Journal of Educational Measurement
- Vol. 34 (1) , 43-63
Computerized Adaptive and Fixed‐Item Testing of Music Listening Skill: A Comparison of Efficiency, Precision, and Concurrent Validity
We evaluated the efficiency, precision, and concurrent validity of results obtained from adaptive and fired‐item music listening tests in three studies: (a) a computer simulation study in which each of 2,200 simulees completed a computerized adaptive tonal memory test, a computerized fired‐item tonal memory test constructed from items in the adaptive test pool and two standardized group‐administered tonal memory tests; (b) a live testing study in which each of 204 examinees took the computerized adaptive test and the standardized tests; and (c) a live testing study in which randomly equivalent groups took either the computerized adaptive test (n = 86) or the computerized fired‐item test (n = 86). The adaptive music test required 50% to 93% fewer items to match the reliability and concurrent validity of the fired‐item tests, and it yielded higher levels of reliability and concurrent validity than the fired‐item tests when test length was held constant. These findings suggest that computerized adaptive tests, which typically have been limited to visually produced items, may also be well suited for measuring skills that require aurally produced items.