Published In
Journal of Chemical Education
Document Type
Post-Print
Publication Date
2-11-2019
Subjects
Organic chemistry -- Study and teaching (Higher), Organic chemistry -- Curriculum -- Assessment, Science education -- Evaluation
Abstract
Quasi-experiments are common in studies that estimate the effect of instructional interventions on student performance outcomes. In this type of research, the nature of the experimental design, the choice in assessment, the selection of comparison groups, and the statistical methods used to analyze the comparison data dictate the validity of causal inferences. Therefore, gathering and reporting validity evidence in causal studies is of utmost importance, especially when conclusions have real policy implications for students and faculty, among other stakeholders. This review examines 24 articles that reported quantitative investigations of the effect of instructional interventions on performance-based outcomes conducted within undergraduate chemistry courses. Specifically, we examined four aspects of conducting such evaluations, including: (1) the type of quasi-experimental design used to study the relationship between interventions, students, outcomes, and settings; (2) the metrics used to measure performance outcomes; (3) the type of groups used to contrast outcomes across experimental conditions; and (4) the statistical methods used to analyze the comparison data. Through the examination of these four aspects of causal studies, together with a validity typology for quasi-experimental designs, we catalogued the metrics and methods used to compare student performance outcomes across varied instructional contexts. Recommendations for researchers and practitioners planning quasi-experimental investigations and for interpreting results from causal studies in chemistry education are provided.
Rights
Copyright © 2019 American Chemical Society and Division of Chemical Education, Inc.
DOI
10.1021/acs.jchemed.8b00713
Persistent Identifier
https://archives.pdx.edu/ds/psu/34559
Citation Details
Published as Michael M. Mack, Cory Hensen, and Jack Barbera, “Metrics and Methods Used to Compare Student Performance Data in Chemistry Education Research Articles,” Journal of Chemical Education, 2019, 96, 3, 401-413, DOI:10.1021/acs.jchemed.8b00713.
List of Sampled Articles (Table S1); Reading Protocol; Data Matrices (Tables S2-S5); and Code Descriptions
Description
This is the author’s version of a work that was accepted for publication in The Journal of Chemical Education. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in The Journal of Chemical Education 2019, 96, 3, 401-413, and can be found online at: http://dx.doi.org/10.1021/acs.jchemed.8b00713.