Applying computerized-scoring models of written biological explanations across courses and colleges: prospects and limitations     

Ha M  Nehm RH  Urban-Lurain M  Merrill JE 《CBE life sciences education》2011,10(4):379-393

Our study explored the prospects and limitations of using machine-learning software to score introductory biology students’ written explanations of evolutionary change. We investigated three research questions: 1) Do scoring models built using student responses at one university function effectively at another university? 2) How many human-scored student responses are needed to build scoring models suitable for cross-institutional application? 3) What factors limit computer-scoring efficacy, and how can these factors be mitigated? To answer these questions, two biology experts scored a corpus of 2556 short-answer explanations (from biology majors and nonmajors) at two universities for the presence or absence of five key concepts of evolution. Human- and computer-generated scores were compared using kappa agreement statistics. We found that machine-learning software was capable in most cases of accurately evaluating the degree of scientific sophistication in undergraduate majors’ and nonmajors’ written explanations of evolutionary change. In cases in which the software did not perform at the benchmark of “near-perfect” agreement (kappa > 0.80), we located the causes of poor performance and identified a series of strategies for their mitigation. Machine-learning software holds promise as an assessment tool for use in undergraduate biology education, but like most assessment tools, it is also characterized by limitations.  相似文献   

Harnessing technology to improve formative assessment of student conceptions in STEM: forging a national network   总被引：2，自引：0，他引：2  

Haudek KC  Kaplan JJ  Knight J  Long T  Merrill J  Munn A  Nehm R  Smith M  Urban-Lurain M 《CBE life sciences education》2011,10(2):149-155

Concept inventories, consisting of multiple-choice questions designed around common student misconceptions, are designed to reveal student thinking. However, students often have complex, heterogeneous ideas about scientific concepts. Constructed-response assessments, in which students must create their own answer, may better reveal students' thinking, but are time- and resource-intensive to evaluate. This report describes the initial meeting of a National Science Foundation-funded cross-institutional collaboration of interdisciplinary science, technology, engineering, and mathematics (STEM) education researchers interested in exploring the use of automated text analysis to evaluate constructed-response assessments. Participants at the meeting shared existing work on lexical analysis and concept inventories, participated in technology demonstrations and workshops, and discussed research goals. We are seeking interested collaborators to join our research community.  相似文献   

A validated search assessment tool: assessing practice-based learning and improvement in a residency program     

Gurpreet K Rana  Doreen R Bradley  Stanley J Hamstra  Paula T Ross  Robert E Schumacher  John G Frohna  Hilary M Haftel  Monica L Lypson 《Journal of the Medical Library Association》2011,99(1):77-81

Objective:

The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center.

Method:

Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident''s longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies.

Results:

Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts.

Conclusion:

The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

Highlights

• The University of Michigan MEDLINE Search Assessment tool can be used to assess search skills in residency education.
• Five elements were identified as critical elements in the development of an effective MEDLINE search strategy: inclusion of all search concepts, appropriate use of Medical Subject Headings, appropriate use of search limits, successful combination of all concepts, and search efficiency.

Implications

• This validated assessment tool can serve as an effective means to measure improvements in residents'' information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

  相似文献   

Measure of opportunity: assessing equitable conditions to learn twenty-first century thinking skills     

Anderson  Ross C.  Beach  Paul 《Learning Environments Research》2022,25(3):741-774

Learning Environments Research - Many school change efforts aim to foster conditions for the development of students’ twenty-first century skills in an equitable and comprehensive way, but...  相似文献   

Classical Test Theory in Historical Perspective     

Ross E. Traub 《Educational Measurement》1997,16(4):8-14

What were the historic origins of classical test theory? What have been major milestones in its development?  相似文献   

Effect of three classroom listening conditions on speech intelligibility     

M Ross  T G Giolas 《American annals of the deaf》1971,116(6):580-584

  相似文献   

Differences and Perceived Differences in Teacher Education Programmes at a CAE and a University     

Ross Telfer 《Asia-Pacific Journal of Teacher Education》1982,10(2):9-21

  相似文献   

The pricing of books: a market view     

Neill Ross 《Learned Publishing》1988,1(3):105-115

  相似文献   

Analysis of reference transactions using packaged computer programs.     

N Calabretta  R Ross 《Medical reference services quarterly》1984,3(3):23-47

Motivated by a continuing education class attended by the authors on the measurement of reference desk activities, the reference department at Scott Memorial Library initiated a project to gather data on reference desk transactions and to analyze the data by using packaged computer programs. The programs utilized for the project were SPSS (Statistical Package for the Social Sciences) and SAS (Statistical Analysis System). The planning, implementation and development of the project are described.  相似文献   

Horace Kalien's Two Conceptions of Cultural Pluralism     

Spencer J. Maxcy 《Educational theory》1979,29(1):31-39

  相似文献