共查询到20条相似文献，搜索用时 31 毫秒
1.
Cognitive diagnosis models (CDMs) typically assume skill attributes with discrete (often binary) levels of skill mastery, making the existence of skill continuity an anticipated form of model misspecification. In this article, misspecification due to skill continuity is argued to be of particular concern for several CDM applications due to the lack of invariance it yields in CDM skill attribute metrics, or what in this article are viewed as the “thresholds” applied to continuous attributes in distinguishing masters from nonmasters. Using the deterministic input noisy and (DINA) model as an illustration, the effects observed in real data are found to be systematic, with higher thresholds for mastery tending to emerge in higher ability populations. The results are shown to have significant implications for applications of CDMs that rely heavily upon the parameter invariance properties of the models, including, for example, applications toward the measurement of growth and differential item functioning analyses. 相似文献
2.
A Note on the Invariance of the DINA Model Parameters 总被引：1，自引：0，他引：1
Cognitive diagnosis models (CDMs), as alternative approaches to unidimensional item response models, have received increasing attention in recent years. CDMs are developed for the purpose of identifying the mastery or nonmastery of multiple fine-grained attributes or skills required for solving problems in a domain. For CDMs to receive wider use, researchers and practitioners need to understand the basic properties of these models. The article focuses on one CDM, the deterministic inputs, noisy "and" gate (DINA) model, and the invariance property of its parameters. Using simulated data involving different attribute distributions, the article demonstrates that the DINA model parameters are absolutely invariant when the model perfectly fits the data. An additional example involving different ability groups illustrates how noise in real data can contribute to the lack of invariance in these parameters. Some practical implications of these findings are discussed . 相似文献
3.
Compared to unidimensional item response models (IRMs), cognitive diagnostic models (CDMs) based on latent classes represent examinees' knowledge and item requirements using discrete structures. This study systematically examines the viability of retrofitting CDMs to IRM‐based data with a linear attribute structure. The study utilizes a procedure to make the IRM and CDM frameworks comparable and investigates how estimation accuracy is affected by test diagnosticity and the match between the true and fitted models. The study shows that comparable results can be obtained when highly diagnostic IRM data are retrofitted with CDM, and vice versa, retrofitting CDMs to IRM‐based data in some conditions can result in considerable examinee misclassification, and model fit indices provide limited indication of the accuracy of item parameter estimation and attribute classification. 相似文献
4.
This article used the Wald test to evaluate the item‐level fit of a saturated cognitive diagnosis model (CDM) relative to the fits of the reduced models it subsumes. A simulation study was carried out to examine the Type I error and power of the Wald test in the context of the G‐DINA model. Results show that when the sample size is small and a larger number of attributes are required, the Type I error rate of the Wald test for the DINA and DINO models can be higher than the nominal significance levels, while the Type I error rate of the A‐CDM is closer to the nominal significance levels. However, with larger sample sizes, the Type I error rates for the three models are closer to the nominal significance levels. In addition, the Wald test has excellent statistical power to detect when the true underlying model is none of the reduced models examined even for relatively small sample sizes. The performance of the Wald test was also examined with real data. With an increasing number of CDMs from which to choose, this article provides an important contribution toward advancing the use of CDMs in practical educational settings. 相似文献
5.
The DINA (deterministic input, noisy, and gate) model has been widely used in cognitive diagnosis tests and in the process of test development. The outcomes known as slip and guess are included in the DINA model function representing the responses to the items. This study aimed to extend the DINA model by using the random‐effect approach to allow examinees to have different probabilities of slipping and guessing. Two extensions of the DINA model were developed and tested to represent the random components of slipping and guessing. The first model assumed that a random variable can be incorporated in the slipping parameters to allow examinees to have different levels of caution. The second model assumed that the examinees’ ability may increase the probability of a correct response if they have not mastered all of the required attributes of an item. The results of a series of simulations based on Markov chain Monte Carlo methods showed that the model parameters and attribute‐mastery profiles can be recovered relatively accurately from the generating models and that neglect of the random effects produces biases in parameter estimation. Finally, a fraction subtraction test was used as an empirical example to demonstrate the application of the new models. 相似文献
6.
Assessment of Differential Item Functioning Under Cognitive Diagnosis Models: The DINA Model Example 
下载免费PDF全文
下载免费PDF全文 The assessment of differential item functioning (DIF) is routinely conducted to ensure test fairness and validity. Although many DIF assessment methods have been developed in the context of classical test theory and item response theory, they are not applicable for cognitive diagnosis models (CDMs), as the underlying latent attributes of CDMs are multidimensional and binary. This study proposes a very general DIF assessment method in the CDM framework which is applicable for various CDMs, more than two groups of examinees, and multiple grouping variables that are categorical, continuous, observed, or latent. The parameters can be estimated with Markov chain Monte Carlo algorithms implemented in the freeware WinBUGS. Simulation results demonstrated a good parameter recovery and advantages in DIF assessment for the new method over the Wald method. 相似文献
7.
AbstractThe present study compared the performance of six cognitive diagnostic models (CDMs) to explore inter skill relationship in a reading comprehension test. To this end, item responses of about 21,642 test-takers to a high-stakes reading comprehension test were analyzed. The models were compared in terms of model fit at both test and item levels, classification consistency and accuracy, and proportion of skill mastery profiles. The results showed that the G-DINA performed the best and the C-RUM, NC-RUM, and ACDM showed the closest affinity to the G-DINA. In terms of some criteria, the DINA showed comparable performance to the G-DINA. The test-level results were corroborated by the item-level model comparison, where DINA, DINO, and ACDM variously fit some of the items. The results of the study suggested that relationships among the subskills of reading comprehension might be a combination of compensatory and non-compensatory. Therefore, it is suggested that the choice of the CDM be carried out at item level rather than test level. 相似文献
8.
Cognitive diagnosis models provide profile information about a set of latent binary attributes, whereas item response models yield a summary report on a latent continuous trait. To utilize the advantages of both models, higher order cognitive diagnosis models were developed in which information about both latent binary attributes and latent continuous traits is available. To facilitate the utility of cognitive diagnosis models, corresponding computerized adaptive testing (CAT) algorithms were developed. Most of them adopt the fixed‐length rule to terminate CAT and are limited to ordinary cognitive diagnosis models. In this study, the higher order deterministic‐input, noisy‐and‐gate (DINA) model was used as an example, and three criteria based on the minimum‐precision termination rule were implemented: one for the latent class, one for the latent trait, and the other for both. The simulation results demonstrated that all of the termination criteria were successful when items were selected according to the Kullback‐Leibler information and the posterior‐weighted Kullback‐Leibler information, and the minimum‐precision rule outperformed the fixed‐length rule with a similar test length in recovering the latent attributes and the latent trait. 相似文献
9.
认知诊断通过分析被试的项目作答反应,推断被试的认知属性掌握状态,为学习困难学生设计补救教学提供了非常有价值的信息。本文作者在探讨了小学生多位数乘法计算能力的认知属性、编制了2份相同考核模式的认知诊断测验后,选择江西某小学310名高年级学生为被试,先施测第1份认知诊断测验,采用DINA模型,自编参数估计程序进行诊断,得到了每一个被试的属性掌握模式分类及全体被试在各个属性上的掌握情况。然后设计和实施补救教学,在实施补救教学后再施测第2份认知诊断测验以检验补救效果。研究发现：（1）该小学高年级学生对0XN运算法则、多位数乘以两位数的运算程序、乘法进位认知属性的掌握不太理想,特别是乘法进位。（2）属性掌握模式中属全部掌握模式的被试人数占86．47％,其余被试均分类于存在各种认知不足的掌握模式。（3）比较两份认知诊断测验报告,结果表明在认知诊断指导下的补救教学有针对性,补救后被试正确作答项目增多,属性掌握个数也有所增加,补救效果良好。 相似文献
10.
Fuzhong Li Terry E. Duncan Peter Harmer Alan Acock Mike Stoolmiller 《Structural equation modeling》2013,20(3):294-306
Confirmatory factor analysis (CFA) is often used in the social sciences to estimate a measurement model in which multiple measurement items are hypothesized to assess a particular latent construct. This article presents the utility of multilevel CFA (MCFA; Muthén, 1991, 1994) and hierarchical linear modeling (HLM; Raudenbush, Rowan, & Kang, 1991) methods in testing measurement models in which the underlying attribute may vary as a function of various levels of observation. An illustrative example using a real dataset is provided in which an unconditional model specification and parameter estimates from the MCFA and HLM are shown. The article demonstrates the comparability of the two methods in estimating measurement parameters of interest (i.e., true variance at levels the measures are used and measurement errors). 相似文献
11.
In cognitive diagnostic models (CDMs), a set of fine-grained attributes is required to characterize complex problem solving and provide detailed diagnostic information about an examinee. However, it is challenging to ensure reliable estimation and control computational complexity when The test aims to identify the examinee's attribute profile in a large-scale map of attributes. To address this problem, this study proposes a cognitive diagnostic multistage testing by partitioning hierarchically structured attributes (CD-MST-PH) as a multistage testing for CDM. In CD-MST-PH, multiple testlets can be constructed based on separate attribute groups before testing occurs, which retains the advantages of multistage testing over fully adaptive testing or the on-the-fly approach. Moreover, testlets are offered sequentially and adaptively, thus improving test accuracy and efficiency. An item information measure is proposed to compute the discrimination power of an item for each attribute, and a module assembly method is presented to construct modules anchored at each separate attribute group. Several module selection indices for CD-MST-PH are also proposed by modifying the item selection indices used in cognitive diagnostic computerized adaptive testing. The results of simulation study show that CD-MST-PH can improve test accuracy and efficiency relative to the conventional test without adaptive stages. 相似文献
12.
Cognitive diagnostic assessment (CDA) allows for diagnosing second language (L2) learners’ strengths and weaknesses of attributes in a specific domain. Exploring the little-known territory of CDA, the current study retrofitted the reading section of the International English Language Testing System (IELTS) with a cognitive diagnostic model (CDM). It aimed to identify the attributes involved in successfully implementing IELTS reading, analyze the overall and individual test-takers’ reading performance, and, finally, explore the IELTS reading differences of Iranian students in engineering and veterinary domains. Based on think-aloud protocols and expert judgement, an initial Q-matrix was developed. Using R package CDM, the generalized deterministic inputs, noisy “and” gate (G-DINA) model was applied to IELTS reading data to refine and validate the initial Q-matrix and estimate the mastery probabilities of 1025 test-takers on each attribute. The final Q-matrix consisted of 6 attributes assumed to be involved in IELTS reading. Moreover, the overall test-takers and the individuals demonstrated different mastery/non-mastery across the 6 IELTS reading attributes on both macro and micro levels. Further, significant differences were found between IELTS reading performances of Iranian engineering and veterinary students. The findings supported the assumption that CDA can provide instructors and IELTS candidates with detailed diagnostic feedback to promote test-takers’ IELTS reading performance. 相似文献
13.
Dual‐Objective Item Selection Criteria in Cognitive Diagnostic Computerized Adaptive Testing 下载免费PDF全文
下载免费PDF全文 The development of cognitive diagnostic‐computerized adaptive testing (CD‐CAT) has provided a new perspective for gaining information about examinees' mastery on a set of cognitive attributes. This study proposes a new item selection method within the framework of dual‐objective CD‐CAT that simultaneously addresses examinees' attribute mastery status and overall test performance. The new procedure is based on the Jensen‐Shannon (JS) divergence, a symmetrized version of the Kullback‐Leibler divergence. We show that the JS divergence resolves the noncomparability problem of the dual information index and has close relationships with Shannon entropy, mutual information, and Fisher information. The performance of the JS divergence is evaluated in simulation studies in comparison with the methods available in the literature. Results suggest that the JS divergence achieves parallel or more precise recovery of latent trait variables compared to the existing methods and maintains practical advantages in computation and item pool usage. 相似文献
14.
The traditional kappa statistic in assessing interrater agreement is not adequate when multiraters and multiattributes are involved. In this article, latent trait models are proposed to assess the multirater multiattribute (MRMA) agreement. Data from the Third International Mathematics and Science Studies (TIMSS) are used to illustrate the application of the latent trait models. Results showed that among four possible latent trait models, the correlated uniqueness model had the best fit to assess the MRMA agreement. Furthermore, in coding a set of different attributes, the coding accuracy within the same rater may differ across attributes. Likewise, when different raters rate the same attribute, the accuracy in rating varies among the raters. Thus, the latent models provide us with a more refined and accurate assessment of interrater agreement. The application of the latent trait models is important in school psychology research and intervention because accurate assessment of children's functioning is fundamental in designing effective intervention strategies. © 2007 Wiley Periodicals, Inc. Psychol Schs 44: 515–525, 2007. 相似文献
15.
Item response theory (IRT) procedures have been used extensively to study normal latent trait distributions and have been shown to perform well; however, less is known concerning the performance of IRT with non-normal latent trait distributions. This study investigated the degree of latent trait estimation error under normal and non-normal conditions using four latent trait estimation procedures and also evaluated whether the test composition, in terms of item difficulty level, reduces estimation error. Most importantly, both true and estimated item parameters were examined to disentangle the effects of latent trait estimation error from item parameter estimation error. Results revealed that non-normal latent trait distributions produced a considerably larger degree of latent trait estimation error than normal data. Estimated item parameters tended to have comparable precision to true item parameters, thus suggesting that increased latent trait estimation error results from latent trait estimation rather than item parameter estimation. 相似文献
16.
Most of the existing classification accuracy indices of attribute patterns lose effectiveness when the response data is absent in diagnostic testing. To handle this issue, this article proposes new indices to predict the correct classification rate of a diagnostic test before administering the test under the deterministic noise input “and” gate (DINA) model. The new indices include an item‐level expected classification accuracy (ECA) for attributes and a test‐level ECA for attributes and attribute patterns, and both of them are calculated based solely on the known item parameters and Q ‐matrix. Theoretical analysis showed that the item‐level ECA could be regarded as a measure of correct classification rates of attributes contributed by an item. This article also illustrates how to apply the item‐level ECA for attributes to estimate the correct classification rate of attributes patterns at the test level. Simulation results showed that two test‐level ECA indices, ECA_I_W (an index based on the independence assumption and the weighted sum of the item‐level ECAs) and ECA_C_M (an index based on Gaussian Copula function that incorporates the dependence structure of the events of attribute classification and the simple average of the item‐level ECAs), could make an accurate prediction for correct classification rates of attribute patterns. 相似文献
17.
We present a logistic function of a monotonic polynomial with a lower asymptote, allowing additional flexibility beyond the three‐parameter logistic model. We develop a maximum marginal likelihood‐based approach to estimate the item parameters. The new item response model is demonstrated on math assessment data from a state, and a computationally efficient strategy for choosing the order of the polynomial is demonstrated. Finally, our approach is tested through simulations and compared to response function estimation using smoothed isotonic regression. Results indicate that our approach can result in small gains in item response function recovery and latent trait estimation. 相似文献
18.
To diagnose the English as a Foreign Language (EFL) reading ability of Chinese high-school students, the study explored how an educational theory, the revised taxonomy of educational objectives, could be used to create the attribute list. Q-matrices were proposed and refined qualitatively and quantitatively. The final Q-matrix specified the relationship between 53 reading items and 9 cognitive attributes. Thereafter, 978 examinees’ responses were calibrated by cognitive diagnosis models (CDMs) to explore their strengths and weaknesses in EFL reading. Results showed strengths and weaknesses on the 9 attributes of the sampled population, examinees at three proficiency levels and individual learners. A diagnostic score report was also developed to communicate multi-layered information to various stakeholders. The goodness of fit of the selected CDM was evaluated from multiple measures. The results provide empirical evidence for the utility of educational theories in cognitive diagnosis, and the feasibility of retrofitting non-diagnostic tests for diagnostic purposes in language testing. In addition, the study also demonstrates procedures of model selection and a post-hoc approach of model verification in language diagnosis. 相似文献
19.
20.
Robert Henson Jonathan Templin Jeffrey Douglas 《Journal of Educational Measurement》2007,44(4):361-376
Consider test data, a specified set of dichotomous skills measured by the test, and an IRT cognitive diagnosis model (ICDM). Statistical estimation of the data set using the ICDM can provide examinee estimates of mastery for these skills, referred to generally as attributes. With such detailed information about each examinee, future instruction can be tailored specifically for each student, often referred to as formative assessment. However, use of such cognitive diagnosis models to estimate skills in classrooms can require computationally intensive and complicated statistical estimation algorithms, which can diminish the breadth of applications of attribute level diagnosis. We explore the use of sum-scores (each attribute measured by a sum-score) combined with estimated model-based sum-score mastery/nonmastery cutoffs as an easy-to-use and intuitive method to estimate attribute mastery in classrooms and other settings where simple skills diagnostic approaches are desirable. Using a simulation study of skills diagnosis test settings and assuming a test consisting of a model-based calibrated set of items, correct classification rates (CCRs) are compared among four model-based approaches for estimating attribute mastery, namely using full model-based estimation and three different methods of computing sum-scores (simple sum-scores, complex sum-scores, and weighted complex sum-scores) combined with model-based mastery sum-score cutoffs. In summary, the results suggest that model-based sum-scores and mastery cutoffs can be used to estimate examinee attribute mastery with only moderate reductions in CCRs in comparison with the full model-based estimation approach. Certain topics are mentioned that are currently being investigated, especially applications in classroom and textbook settings. 相似文献