| Abstract: | The current study aims to evaluate the performance of three non-IRT procedures (i.e., normal approximation, Livingston-Lewis, and compound multinomial) for estimating classification indices when the observed score distribution shows atypical patterns: (a) bimodality, (b) structural (i.e., systematic) bumpiness, or (c) structural zeros (i.e., no frequencies). Under a bimodal distribution, the normal approximation procedure produced substantially large bias. For a distribution with structural bumpiness, the compound multinomial procedure tended to introduce larger bias. Under a distribution with structural zeroes, the relative performance of selected estimation procedures depended on cut score location and the sample-size conditions. In general, the differences in estimation errors among the three procedures were not substantially large. |
