Sources of difficulty in assessment: example of PISA science items |
| |
| Authors: | Florence Le Hebel Pascale Montpied Andrée Tiberghien Valérie Fontanieu |
| |
| Institution: | 1. ESPE, Université Lyon 1, Lyon, France;2. Interactions Corpus Apprentissage Representations, ENS Lyon, Université Lyon 2, CNRS, Lyon, France;3. Laboratoire de l'Education, Ecole Normale superieure de Lyon, Lyon, UMS 3773, Franceflorence.le-hebel@ens-lyon.fr;5. Interactions Corpus Apprentissage Representations, ENS Lyon, Université Lyon 2, CNRS, Lyon, France;6. Ecole normale superieure de Lyon, IFE, Lyon, France |
| |
| Abstract: | ABSTRACTThe understanding of what makes a question difficult is a crucial concern in assessment. To study the difficulty of test questions, we focus on the case of PISA, which assesses to what degree 15-year-old students have acquired knowledge and skills essential for full participation in society. Our research question is to identify PISA science item characteristics that could influence the item’s proficiency level. It is based on an a-priori item analysis and a statistical analysis. Results show that only the cognitive complexity and the format out of the different characteristics of PISA science items determined in our a-priori analysis have an explanatory power on an item’s proficiency levels. The proficiency level cannot be explained by the dependence/independence of the information provided in the unit and/or item introduction and the competence. We conclude that in PISA, it appears possible to anticipate a high proficiency level, that is, students’ low scores for items displaying a high cognitive complexity. In the case of a middle or low cognitive complexity level item, the cognitive complexity level is not sufficient to predict item difficulty. Other characteristics play a crucial role in item difficulty. We discuss anticipating the difficulties in assessment in a broader perspective. |
| |
| Keywords: | Students’ proficiencies assessment scientific literacy |
|
|
