共查询到20条相似文献，搜索用时 15 毫秒
1.
Howard Wainer Charles Lewis Bruce Kaplan James Braswell 《Journal of Educational Measurement》1991,28(4):311-323
Earlier (Wainer & Lewis, 1990), we reported the initial development of a testlet-based algebra test. In this account, we provide the details of this excursion into the use of testlets. A pretest of two 15–item algebra tests was carried out in which examinees' performance on a 4-item subset of each test (a 4–item testlet) was used to predict performance on the entire test. Two models for constructing the testlets were considered: hierarchical (adaptive) and linear (fixed format). These models are compared with each other. It was found on cross–validation that, although an adaptive testlet is superior to a fixed format testlet, this superiority is modest, whereas the potential cost of that superiority is considerable. It was concluded that in circumstances similar to those we report a fixed format testlet that uses the best items in a pool can do almost as well as the optimal adaptive testlet of equal length from that same pool. 相似文献
2.
Christine E. DeMars 《Journal of Educational Measurement》2006,43(2):145-168
Four item response theory (IRT) models were compared using data from tests where multiple items were grouped into testlets focused on a common stimulus. In the bi-factor model each item was treated as a function of a primary trait plus a nuisance trait due to the testlet; in the testlet-effects model the slopes in the direction of the testlet traits were constrained within each testlet to be proportional to the slope in the direction of the primary trait; in the polytomous model the item scores were summed into a single score for each testlet; and in the independent-items model the testlet structure was ignored. Using the simulated data, reliability was overestimated somewhat by the independent-items model when the items were not independent within testlets. Under these nonindependent conditions, the independent-items model also yielded greater root mean square error (RMSE) for item difficulty and underestimated the item slopes. When the items within testlets were instead generated to be independent, the bi-factor model yielded somewhat higher RMSE in difficulty and slope. Similar differences between the models were illustrated with real data. 相似文献
3.
How should we think about the concept of the testlet? How can testlets be better incorporated into test score analysis? Can there be a one‐item testlet? 相似文献
4.
It is not always convenient or appropriate to construct tests in which individual items are fungible. There are situations in which small clusters of items (testlets) are the units that are assembled to create a test. Using data from a test of reading comprehension constructed of four passages with several questions following each passage, we show that local independence fails at the level of the individual questions. The questions following each passage, however, constitute a testlet. We discuss the application to testlet scoring of some multiple-category models originally developed for individual items, In the example examined, the concurrent validity of the testlet scoring equaled or exceeded that of individual-item-level scoring 相似文献
5.
Natalie A. Koziol 《教育实用测度》2013,26(3):184-195
ABSTRACTTestlets, or groups of related items, are commonly included in educational assessments due to their many logistical and conceptual advantages. Despite their advantages, testlets introduce complications into the theory and practice of educational measurement. Responses to items within a testlet tend to be correlated even after controlling for latent ability, which violates the assumption of conditional independence made by traditional item response theory models. The present study used Monte Carlo simulation methods to evaluate the effects of testlet dependency on item and person parameter recovery and classification accuracy. Three calibration models were examined, including the traditional 2PL model with marginal maximum likelihood estimation, a testlet model with Bayesian estimation, and a bi-factor model with limited-information weighted least squares mean and variance adjusted estimation. Across testlet conditions, parameter types, and outcome criteria, the Bayesian testlet model outperformed, or performed equivalently to, the other approaches. 相似文献
6.
Frank Rijmen 《Journal of Educational Measurement》2010,47(3):361-372
Testlet effects can be taken into account by incorporating specific dimensions in addition to the general dimension into the item response theory model. Three such multidimensional models are described: the bi-factor model, the testlet model, and a second-order model. It is shown how the second-order model is formally equivalent to the testlet model. In turn, both models are constrained bi-factor models. Therefore, the efficient full maximum likelihood estimation method that has been established for the bi-factor model can be modified to estimate the parameters of the two other models. An application on a testlet-based international English assessment indicated that the bi-factor model was the preferred model for this particular data set. 相似文献
7.
Hong Jiao Akihito Kamata Shudong Wang Ying Jin 《Journal of Educational Measurement》2012,49(1):82-100
The applications of item response theory (IRT) models assume local item independence and that examinees are independent of each other. When a representative sample for psychometric analysis is selected using a cluster sampling method in a testlet‐based assessment, both local item dependence and local person dependence are likely to be induced. This study proposed a four‐level IRT model to simultaneously account for dual local dependence due to item clustering and person clustering. Model parameter estimation was explored using the Markov Chain Monte Carlo method. Model parameter recovery was evaluated in a simulation study in comparison with three other related models: the Rasch model, the Rasch testlet model, and the three‐level Rasch model for person clustering. In general, the proposed model recovered the item difficulty and person ability parameters with the least total error. The bias in both item and person parameter estimation was not affected but the standard error (SE) was affected. In some simulation conditions, the difference in classification accuracy between models could go up to 11%. The illustration using the real data generally supported model performance observed in the simulation study. 相似文献
8.
9.
Using a New Statistical Model for Testlets to Score TOEFL 总被引：1，自引：0，他引：1
Standard item response theory (IRT) models fit to examination responses ignore the fact that sets of items (testlets) often are matched with a single common stimulus (e.g., a reading comprehension passage). In this setting, all items given to an examinee are unlikely to be conditionally independent (given examinee proficiency). Models that assume conditional independence will overestimate the precision with which examinee proficiency is measured. Overstatement of precision may lead to inaccurate inferences as well as prematurely ended examinations in which the stopping rule is based on the estimated standard error of examinee proficiency (e.g., an adaptive test). The standard three parameter IRT model was modified to include an additional random effect for items nested within the same testlet (Wainer, Bradlow, & Du, 2000). This parameter, γ characterizes the amount of local dependence in a testlet.
We fit 86 TOEFL testlets (50 reading comprehension and 36 listening comprehension) with the new model, and obtained a value for the variance of γ for each testlet. We compared the standard parameters (discrimination (a), difficulty (b) and guessing (c)) with what is obtained through traditional modeling. We found that difficulties were well estimated either way, but estimates of both a and c were biased if conditional independence is incorrectly assumed. Of greater import, we found that test information was substantially over-estimated when conditional independence was incorrectly assumed. 相似文献
We fit 86 TOEFL testlets (50 reading comprehension and 36 listening comprehension) with the new model, and obtained a value for the variance of γ for each testlet. We compared the standard parameters (discrimination (a), difficulty (b) and guessing (c)) with what is obtained through traditional modeling. We found that difficulties were well estimated either way, but estimates of both a and c were biased if conditional independence is incorrectly assumed. Of greater import, we found that test information was substantially over-estimated when conditional independence was incorrectly assumed. 相似文献
10.
A series of computer simulations were run to measure the relationship between testlet validity and the factors of item pool size and testlet length for both adaptive and linearly constructed testlets. We confirmed the generality of earlier empirical findings (Wainer, Lewis, Kaplan, & Braswell, 1991) that making a testlet adaptive yields only modest increases in aggregate validity because of the peakedness of the typical proficiency distribution. 相似文献
11.
This study demonstrated the equivalence between the Rasch testlet model and the three‐level one‐parameter testlet model and explored the Markov Chain Monte Carlo (MCMC) method for model parameter estimation in WINBUGS. The estimation accuracy from the MCMC method was compared with those from the marginalized maximum likelihood estimation (MMLE) with the expectation‐maximization algorithm in ConQuest and the sixth‐order Laplace approximation estimation in HLM6. The results indicated that the estimation methods had significant effects on the bias of the testlet variance and ability variance estimation, the random error in the ability parameter estimation, and the bias in the item difficulty parameter estimation. The Laplace method best recovered the testlet variance while the MMLE best recovered the ability variance. The Laplace method resulted in the smallest random error in the ability parameter estimation while the MCMC method produced the smallest bias in item parameter estimates. Analyses of three real tests generally supported the findings from the simulation and indicated that the estimates for item difficulty and ability parameters were highly correlated across estimation methods. 相似文献
12.
Howard Wainer Stephen G. Sireci David Thissen 《Journal of Educational Measurement》1991,28(3):197-219
It is sometimes sensible to think of the fundamental unit of test construction as being larger than an individual item. This unit, dubbed the testlet, must pass muster in the same way that items do. One criterion of a good item is the absence of DIF–the item must function in the same way in all important subpopulations of examinees. In this article, we define what we mean by testlet DIF and provide a statistical methodology to detect it. This methodology parallels the IRT-based likelihood ratio procedures explored previously by Thissen, Steinberg, and Wainer (1988, in press). We illustrate this methodology with analyses of data from a testlet-based experimental version of the Scholastic Aptitude Test (SAT). 相似文献
13.
The presence of nuisance dimensionality is a potential threat to the accuracy of results for tests calibrated using a measurement model such as a factor analytic model or an item response theory model. This article describes a mixture group bifactor model to account for the nuisance dimensionality due to a testlet structure as well as the dimensionality due to differences in patterns of responses. The model can be used for testing whether or not an item functions differently across latent groups in addition to investigating the differential effect of local dependency among items within a testlet. An example is presented comparing test speededness results from a conventional factor mixture model, which ignores the testlet structure, with results from the mixture group bifactor model. Results suggested the 2 models treated the data somewhat differently. Analysis of the item response patterns indicated that the 2-class mixture bifactor model tended to categorize omissions as indicating speededness. With the mixture group bifactor model, more local dependency was present in the speeded than in the nonspeeded class. Evidence from a simulation study indicated the Bayesian estimation method used in this study for the mixture group bifactor model can successfully recover generated model parameters for 1- to 3-group models for tests containing testlets. 相似文献
14.
Yi-Hung Lin Mark Wilson Ching-Lin Cheng 《European Journal of Psychology of Education - EJPE》2013,28(4):1141-1161
In teaching, representations are used as ways to illustrate the concepts underlying a specific topic. For example, use symbols (e.g., 1?+?2?=?3) to express the concept of addition. To compare students’ abilities to interpret different representations in mathematics, the symbolic representation (SR) test and the pictorial representation (PR) test were designed, and then administered to 681 sixth graders in Taipei, Taiwan. This study adopts two different modeling perspectives, the testlet perspective and the multi-ability perspective, to analyze this SR and PR test data in the context of item response theory. The main results show that:
- Students scored on average significantly higher on the SR test than the PR test.
- The effects of the item stem testlets could be large, but they are statistically non-significant; however, the influence of the number of items in the testlet should also be considered.
- The nature of the option representations, SR and PR, represents two different mathematics abilities.
- The main factor that influences students’ item responses is students’ abilities to interpret SR and PR, and the testlet effects generated from the shared item stem can be ignored.
- Regarding the parameter estimates of the best-fitting model: (a) the person ability variance estimates show that the ability distributions on the SR and PR dimension may not be the same, (b) the correlation estimate between the SR and PR dimension indicates that these two abilities are moderately correlated, and (c) the item difficulty estimates for different models are similar.
15.
In this study, the effectiveness of detection of differential item functioning (DIF) and testlet DIF using SIBTEST and Poly-SIBTEST
were examined in tests composed of testlets. An example using data from a reading comprehension test showed that results from
SIBTEST and Poly-SIBTEST were not completely consistent in the detection of DIF and testlet DIF. Results from a simulation
study indicated that SIBTEST appeared to maintain type I error control for most conditions, except in some instances in which
the magnitude of simulated DIF tended to increase. This same pattern was present for the Poly-SIBTEST results, although Poly-SIBTEST
demonstrated markedly less control of type I errors. Type I error control with Poly-SIBTEST was lower for those conditions
for which the ability was unmatched to test difficulty. The power results for SIBTEST were not adversely affected, when the
size and percent of simulated DIF increased. Although Poly-SIBTEST failed to control type I errors in over 85% of the conditions
simulated, in those conditions for which type I error control was maintained, Poly-SIBTEST demonstrated higher power than
SIBTEST. 相似文献
16.
Gautam Puhan Timothy P. Moses Mary C. Grant Frederick McHale 《Journal of Educational Measurement》2009,46(3):344-362
A single-group (SG) equating with nearly equivalent test forms (SiGNET) design was developed by Grant to equate small-volume tests. Under this design, the scored items for the operational form are divided into testlets or mini tests. An additional testlet is created but not scored for the first form. If the scored testlets are testlets 1–6 and the unscored testlet is testlet 7, then the first form is composed of testlets 1–6 and the second form is composed of testlets 2–7. The seven testlets are administered as a single administered form, and when a sufficient number of examinees have taken the administered form, the second form (testlets 2–7) is equated to the first form (testlets 1–6) using an SG equating design. As evident, this design facilitates the use of an SG equating and allows for the accumulation of data, both of which may reduce equating error. This study compared equatings under the SiGNET and common-item equating designs and found lower equating error for the SiGNET design in very small sample size conditions (e.g., N = 10). 相似文献
17.
Okan Bulut Ming Lei Qi Guo 《International Journal of Research & Method in Education》2018,41(2):169-183
Item positions in educational assessments are often randomized across students to prevent cheating. However, if altering item positions results in any significant impact on students’ performance, it may threaten the validity of test scores. Two widely used approaches for detecting position effects – logistic regression and hierarchical generalized linear modelling – are often inconvenient for researchers and practitioners due to some technical and practical limitations. Therefore, this study introduced a structural equation modeling (SEM) approach for examining item and testlet position effects. The SEM approach was demonstrated using data from a computer-based alternate assessment designed for students with cognitive disabilities from three grade bands (3–5, 6–8, and high school). Item and testlet position effects were investigated in the field-test (FT) items that were received by each student at different positions. Results indicated that the difficulty of some FT items in grade bands 3–5 and 6–8 differed depending on the positions of the items on the test. Also, the overall difficulty of the field-test task in grade bands 6–8 increased as students responded to the field-test task in later positions. The SEM approach provides a flexible method for examining different types of position effects. 相似文献
18.
王开元 《江苏广播电视大学学报》2012,(6):69-72
自西学传入以来,自由便成为学者们探讨的对象,对儒家自由内涵的考察也从未止步。但是,儒家的自由有着自身特定的文化背景和民族传统,其在《论语》中也更多地表现出作为一种心灵体验的内涵。这与西方传统自由观念不同,西方把自由作为道德的前提,并给予自由极高的地位;而儒家则把道德视为达成自由体验的条件。中西不同的自由观造成了两者不同的文化,但通过逐渐深入的交流互动,二者可以相互借鉴,以彼之所长补己之所短。 相似文献
19.
本文对项目反应理论（IRT）局部独立性问题的主要研究成果进行了文献梳理。在此基础上,阐释局部独立性假设的定义。文章同时就局部独立性与测验维度的关系,局部依赖的甄别与计算、起因和控制程序,以及局部依赖对测量实践的影响进行讨论,并探讨了题组中局部题目依赖问题的解决策略。 相似文献
20.
LEE Yeaann 《Educational Philosophy and Theory》2019,51(9):912-923
AbstractIn recent decades in Korea, many significant changes in political, social and cultural dimensions have been held by the citizen’s initiative, where the revitalization of citizenship and strong civic unity have played a role. Yet, in regard to the characteristic of Korean citizenship, it seems that the aspect of individual subject has not been fully matured or issued; that is, there is a dissymmetry between the strong civic unity and a weak individual subject. This paper attempts to explore a possible historical account of why this has been the case by examining the historical development of the concept of enlightenment in modern Korea and Japan. ‘Enlightenment’, as a modern concept in Korea, was imported via Japan in the period from the late nineteenth century to the early twentieth century as in many other new concepts such as ‘democracy’ or ‘nation’. However, by comparison to the Western idea of the Enlightenment, its modern concept, Korean or Japanese, developed a different meaning in each own context, while lacking its original meaning essential to the creation of the ‘modern individual subject’ as a ‘citizen’. Hence, in modern Korea and Japan, the word ‘enlightenment’ is regarded as a historical concept with no contemporary relevance. 相似文献