Analyzing student essay responses from the Geoscience Literacy exam

Presenter Information

Thomas Olson

Document Type

Oral Presentation

Campus where you would like to present

SURC Ballroom C/D

Start Date

15-5-2014

End Date

15-5-2014

Keywords

InTeGrate, Earth Systems, Undergraduate Education

Abstract

The Geoscience Literacy Exam (GLE), designed by the InTeGrate project, assesses Earth, climate, atmospheric, and ocean science literacy as defined in documents developed by those communities. The GLE includes three levels of questions: single-answer multiple choice, multiple-answer multiple choice, and short essay. While the multiple choice questions had been validated and are in use, the essay questions had been pilot-tested but the answers not analyzed for validity. The goal of this project was to examine 364 student responses to an essay about complex systems to see if the question is valid and to better define a rubric for scoring. Students in online (n = 45) and face-to-face (n = 150) versions of an introductory Earth Science class for non-majors at the University of Akron were given the question in two exams. Responses were scored and analyzed in an Excel spreadsheet using a four-point rubric that addressed all components of the question. In addition to scoring student responses, misconceptions, understanding of the question, and additional details were noted. In both exams and both classes, a consistent bimodal pattern appears in the score distribution, with one mode near 2.5 and the other at 0. We interpret this to suggest that a large percentage of students (who contribute to the 0 mode) didn’t understand the question. Two common misconceptions were equating "system" and "cycle" and including the exosphere as one of the major Earth systems. In conclusion the question should be revised to give students a better opportunity to answer the question well.

Poster Number

47

Faculty Mentor(s)

Egger, Anne

Additional Mentoring Department

Geological Sciences

This document is currently not available here.

Share

COinS
 
May 15th, 8:30 AM May 15th, 11:00 AM

Analyzing student essay responses from the Geoscience Literacy exam

SURC Ballroom C/D

The Geoscience Literacy Exam (GLE), designed by the InTeGrate project, assesses Earth, climate, atmospheric, and ocean science literacy as defined in documents developed by those communities. The GLE includes three levels of questions: single-answer multiple choice, multiple-answer multiple choice, and short essay. While the multiple choice questions had been validated and are in use, the essay questions had been pilot-tested but the answers not analyzed for validity. The goal of this project was to examine 364 student responses to an essay about complex systems to see if the question is valid and to better define a rubric for scoring. Students in online (n = 45) and face-to-face (n = 150) versions of an introductory Earth Science class for non-majors at the University of Akron were given the question in two exams. Responses were scored and analyzed in an Excel spreadsheet using a four-point rubric that addressed all components of the question. In addition to scoring student responses, misconceptions, understanding of the question, and additional details were noted. In both exams and both classes, a consistent bimodal pattern appears in the score distribution, with one mode near 2.5 and the other at 0. We interpret this to suggest that a large percentage of students (who contribute to the 0 mode) didn’t understand the question. Two common misconceptions were equating "system" and "cycle" and including the exosphere as one of the major Earth systems. In conclusion the question should be revised to give students a better opportunity to answer the question well.