Analyzing student essay responses from the Geoscience Literacy exam

The Geoscience Literacy Exam (GLE), designed by the InTeGrate project, assesses Earth, climate, atmospheric, and ocean science literacy as defined in documents developed by those communities. The GLE includes three levels of questions: single-answer multiple choice, multiple-answer multiple choice, and short essay. While the multiple choice questions had been validated and are in use, the essay questions had been pilot-tested but the answers not analyzed for validity. The goal of this project was to examine 364 student responses to an essay about complex systems to see if the question is valid and to better define a rubric for scoring. Students in online (n = 45) and face-to-face (n = 150) versions of an introductory Earth Science class for non-majors at the University of Akron were given the question in two exams. Responses were scored and analyzed in an Excel spreadsheet using a four-point rubric that addressed all components of the question. In addition to scoring student responses, misconceptions, understanding of the question, and additional details were noted. In both exams and both classes, a consistent bimodal pattern appears in the score distribution, with one mode near 2.5 and the other at 0. We interpret this to suggest that a large percentage of students (who contribute to the 0 mode) didn’t understand the question. Two common misconceptions were equating "system" and "cycle" and including the exosphere as one of the major Earth systems. In conclusion the question should be revised to give students a better opportunity to answer the question well.