Effects of Data-Collection Designs in the Comparison of Computer-Based and Paper-Based Tests |
| |
| Authors: | Alvaro J Arce-Ferrer Okan Bulut |
| |
| Institution: | 1. Pearson, San Antonio, TX, USA;2. Alvaro.Arce-Ferrer@pearson.com;4. University of Alberta, Edmonton, Alberta, Canada |
| |
| Abstract: | This study investigated the performance of four widely used data-collection designs in detecting test-mode effects (i.e., computer-based versus paper-based testing). The experimental conditions included four data-collection designs, two test-administration modes, and the availability of an anchor assessment. The test-level and item-level results were analyzed with inferential statistics and multidimensional scaling, respectively. The test-level results supported the superiority of the single-group counterbalanced design and the random-groups design over the single-group design without counterbalancing and the anchor test design in the recovery of the actual test-mode effects. Analysis at the item level revealed the presence of a two-dimensional solution, with the data-collection design contributing significant variability over and beyond the test mode. |
| |
| Keywords: | Comparability study computer-based testing data-collection designs test mode validity |
|
|
