共查询到20条相似文献，搜索用时 31 毫秒
1.
Machine learning has been frequently employed to automatically score constructed response assessments. However, there is a lack of evidence of how this predictive scoring approach might be compromised by construct-irrelevant variance (CIV), which is a threat to test validity. In this study, we evaluated machine scores and human scores with regard to potential CIV. We developed two assessment tasks targeting science teacher pedagogical content knowledge (PCK); each task contains three video-based constructed response questions. 187 in-service science teachers watched the videos with each had a given classroom teaching scenario and then responded to the constructed-response items. Three human experts rated the responses and the human-consent scores were used to develop machine learning algorithms to predict ratings of the responses. Including the machine as another independent rater, along with the three human raters, we employed the many-facet Rasch measurement model to examine CIV due to three sources: variability of scenarios, rater severity, and rater sensitivity of the scenarios. Results indicate that variability of scenarios impacts teachers’ performance, but the impact significantly depends on the construct of interest; for each assessment task, the machine is always the most severe rater, compared to the three human raters. However, the machine is less sensitive than the human raters to the task scenarios. This means the machine scoring is more consistent and stable across scenarios within each of the two tasks. 相似文献
2.
跨文化副语言交际策略的教学实效性研究 总被引：5，自引：0，他引：5
社会语言学把伴随言语交际过程的辅助表达行为称为副语言行为。它的伴随语言特征和超语言特征决定其在语言交际和教学中的各种功能。副语言交际策略是指人们传递信息的特定的态势和手段。其教学实效性具体表现为信息补偿效应、刺激强化效应、和谐沟通效应、审美愉悦效应及社会文化效应。 相似文献
3.
The concept of validity in theory and practice 总被引：1，自引：1，他引：0
Simon Wolming Christina Wikström 《Assessment in Education: Principles, Policy & Practice》2010,17(2):117-132
The concept of validity, as described in the literature, has changed over time to become a broad and rather complex issue. The purpose of this paper is to investigate if practice has followed theory, or if there is a gap between validity in theory and validity in practice. It compares the theoretical development of the concept of validity with the methodology adopted in validity studies over time. Important phases in the history of validity, and also common arguments for and against traditional and modern validity perspectives, are presented and discussed. Thereafter, three Swedish research projects aiming to validate instruments used for selection to higher education are described. The idea is to use these projects as examples of contemporary practice, and to compare their designs, research questions and outcomes with how validity was theoretically described during their specific period of time. The conclusions from these comparisons are that practices seem to have followed theory when it comes to how the validity research programmes have been designed, but not when it comes to how they then were carried out in practice. This gap between theory and practice seems to have increased with the introduction of broader and more modern validity perspectives. The scope of the research is more extensive but results are fragmented and there is no evidence of a ‘unified’ validity argument, which has been one of the central aspects in modern validity theory. This supports the arguments that validity theory is difficult to put into practice and that there is a need for guidance on how to prioritise validity questions and interpret validity evidence. 相似文献
4.
温志 《四川教育学院学报》2012,28(4):82-85
以Cyril J.Weir的效度整体观为基础,以全国英语应用能力A级考试为研究对象,对基于理论的效度、环境效度、评分效度、效标关联效度和后果效度等五个方面的效度证据进行了分析。研究表明,A级考试整体而言有较高效度,但也存在较大的改进空间。 相似文献
5.
效度问题是人类测量活动中最重要也是最困难的一个问题。本文首先从效度的概念与分类、效度的理论公式以及效度的评估方法等三个方面讨论了经典效度理论存在的弊端，然后针对这三个方面提出了相应的改进意见，在一定程度上实现了效度理论的重建。 相似文献
6.
左瑞红 《黑龙江教育学院学报》2007,26(7):13-15
郑燕祥对教育效能的分析指出了学校效能的多元性和复杂性,但对学校效能差异性的认识不全面。学校效能改进模型是对郑燕祥教育效能观的补充,指出学校效能的静态差异及学校效能改进的动态差异,对学校效能改进具有一定启示。 相似文献
7.
Melinda R. Snodgrass Bryan G. Cook Lysandra Cook 《Learning disabilities research & practice》2023,38(4):311-319
This article is part of a special LDRP research-to-practice series introducing key concepts to enable special education practitioners and other nonresearchers to be more informed research consumers. In the article, we explore how social validity is assessed in special education research and how to interpret social validity assessments. Rather than focusing on measuring intervention effects, social validity involves assessing the social importance of the goals, procedures, and outcomes of interventions and programs. We define social validity, provide questions to consider when examining assessments of social validity in research papers, review approaches commonly used to assess social validity with examples from the research literature, and make recommendations for reconciling findings of positive intervention effects on targeted outcomes but absent or negative findings related to social validity in a study. Our take-home message is that considering social validity assessments helps research consumers interpret study findings and informs how to apply findings in practice. 相似文献
8.
法的要素是由法律规范、概念和原则构成的，其中法律规范是最主要、最基本的要素。法律规范除具有本身的含义、逻辑结构范式和种类外，有效性则是贯穿其始终的关键所在。法律规范的有效性应包括应然和实然两方面。应然有效性是正义和秩序的综合体，就实然有效性而言，如果一项法律规范本质上与应然有效性同一，则法律规范有效（或生效）。反之，则法律规范无效（或失效）。在法的要素中，为确保法律规范具有效性。应做到法律规范应然与突然、本质与形式有效的完美结合。 相似文献
9.
Leonidas Kyriakides 《Assessment in Education: Principles, Policy & Practice》2004,11(2):143-165
This paper argues for an expanded conception of test validity, in which teachers, as end-users of tests, contribute a distinctive perspective on validity, referred to as inferential validity. It also offers a methodology that could be adopted in order to subject this dimension of validity to scrutiny. An investigation conducted into the meanings constructed by teachers of a literacy test, the Emergent Literacy Baseline Assessment (ELBA), is reported to illustrate the methodology. In the first section of the paper, current conceptions of validity are discussed. It is argued that the validation process for tests should include the clarification and justification of the interpretations and uses of observed scores. This argument is illustrated from the methodology for investigating the validity of the ELBA. Self-assessment questionnaires and focus-group interviews provided data on teachers' views about the validity of the ELBA. Arguments in favour of investigating the validity of large-scale tests by taking into account teachers' perspectives are provided. 相似文献
10.
Per‐Erik Lyrén 《Scandinavian Journal of Educational Research》2013,57(6):565-581
This article reviews ten predictive validity studies of the Swedish Scholastic Assessment Test (SweSAT). A primary result is that the predictive validity of the SweSAT seems to be highly dependent upon the study programme being examined; that is, the predictive validity is better at some programmes than others. When compared with the upper‐secondary school grade point average, the predictive validity of the SweSAT seems to be fairly good, but there are major differences between study programmes in this case as well. However, it is suggested that the validity of the results is to some extent threatened by methodological issues. A general conclusion is, therefore, that there is room for improving the test itself, as well as the way that predictive validity studies are carried out. 相似文献
11.
This article reports university/school partnership in research and development over a decade of unprecedented change in England. The programme of work evolved through three distinct phases in response to formative evaluations of each stage and changing circumstances. This could be conceived as action research on three levels: the classroom, the school and the partnership. The success of the collaboration is evaluated by reference to Anderson & Herr's (1999) five validity criteria for practitioner research: outcome validity, process validity, democratic validity, catalytic validity and dialogic validity 相似文献
12.
熊薇薇 《深圳信息职业技术学院学报》2011,9(2):40-44
针对专门用途英语教学发展很快,而相应测试较少的情况,开发专门用途英语测试非常有必要。任务型测试比较适合专门用途英语测试改革的方向。本文介绍了深圳信息职业技术学院秘书职业英语测试改革探索的经验,并从情景效度、理论效度、评分效度、外部效度等方面对秘书职业英语任务型测试的效度进行了验证,研究证明该测试具有较高效度,可以进行推广。 相似文献
13.
:教育行动研究是教育理论和教育实践结合的有效途径 ,效度则是决定这种结合的程度的主要因素。基于不同理论基础的研究效度具有不同的特点 ,教育行动研究的效度既不能以量化研究的效度来衡量 ,也不完全等同于质化研究的效度 ,而要从教育行动研究的目的和过程来把握 相似文献
14.
法律的规范有效性之源——解读罗尔斯与哈贝马斯道德哲学在法哲学中的延伸 总被引：3，自引：0，他引：3
肖小芳 《湖南师范大学社会科学学报》2007,36(5):77-80
法律的规范有效性从何而来?罗尔斯遵循契约模式的程序建构路向,以其正义论为基础,认为法律的规范有效性根源于正义原则.哈贝马斯则遵循商谈模式的程序建构路向,认为法律的规范有效性来源于民主的立法程序,主张法律的有效性是事实性与规范有效性的内在统一.该文主要梳理了罗尔斯和哈贝马斯在法律的规范有效性问题上的解释路径,并对之作出简要评析. 相似文献
15.
周赟赟 《佳木斯教育学院学报》2012,(4):267+284
全国翻译专业资格(水平)考试(CATTI)是为加强我国外语翻译专业人才建设于2003年形成的一项新兴考试。本文选取2010年下半年至2011年下半年三次二级笔译试题为研究对象,从内在效度、外在效度、使用效度这三个方面对该试题进行效度分析,以期对提高试卷质量有一些帮助。 相似文献
16.
Assessments that function close to classroom teaching and learning can play a powerful role in fostering academic achievement. Unfortunately, however, relatively little attention has been given to discussion of the design and validation of such assessments. The present article presents a framework for conceptualizing and organizing the multiple components of validity applicable to assessments intended for use in the classroom to support ongoing processes of teaching and learning. The conceptual framework builds on existing validity concepts and focuses attention on three components: cognitive validity, instructional validity, and inferential validity. The goal in presenting the framework is to clarify the concept of validity, including key components of the interpretive argument, while considering the types and forms of evidence needed to construct a validity argument for classroom assessments. The framework's utility is illustrated by presenting an application to the analysis of the validity of assessments embedded within an elementary mathematics curriculum. 相似文献
17.
The psychometric literature is replete with comprehensive discussions of test validity, test validation, and the characteristics of quality assessment programs. The most authoritative source for guidance regarding sound test development and evaluation practices is the Standards for Educational and Psychological Testing. However, the Standards are not legally binding. In this article, we review the way in which validity is conceptualized in the Standards and compare this conceptualization with validity evidence presented in specific court cases involving legal challenges to tests. Our review indicates that, in general, there is strong congruence between the Standards and how validity is viewed in the courts, and that testing agencies that conform to these guidelines are likely to withstand legal scrutiny. However, the courts have taken a more practical, less theoretical view on validity and tend to emphasize evidence based on test content and testing consequences. 相似文献
18.
Content validity is an important part of language testing.In this paper,the content validity of the CET-4 fast reading test is analyzed in terms of expected response and text input.The result of final research shows that the content validity of the fast reading test is high with some limitations proposed. 相似文献
19.
The Student Interest-in-the-Arts Questionnaire was designed to measure elementary school students’ interest in dance, drama, music, and the visual arts. We collected data providing evidence for reliability, content validity, construct validity, and convergent and discriminant validity. We describe the development of the method and the collection and analysis of the validity data. The brief instrument is easy to administer, fills a gap in the compendium of available instruments, and is useful in a variety of settings with a variety of research and evaluation designs. 相似文献
20.
Students' scores on questionnaires concerning their approaches to studying in higher education exhibit reasonable stability over time, moderate convergent validity with their scores on other questionnaires, and reasonable levels of discriminating power and criterion-related validity. Nevertheless, the internal consistency of the constituent scales and the construct validity of these instruments are variable, their content validity within contemporary higher education is open to question, and their wording may need to be revised when they are used with students from different social or cultural groups. Future research should investigate the possibility of response bias in such instruments and the validity of self-reports concerning study behavior. 相似文献