Published In

Journal of Chemical Education

Document Type


Publication Date



Organic chemistry -- Study and teaching (Higher), Organic chemistry -- Curriculum -- Assessment, Science education -- Evaluation


Quasi-experiments are common in studies that estimate the effect of instructional interventions on student performance outcomes. In this type of research, the nature of the experimental design, the choice in assessment, the selection of comparison groups, and the statistical methods used to analyze the comparison data dictate the validity of causal inferences. Therefore, gathering and reporting validity evidence in causal studies is of utmost importance, especially when conclusions have real policy implications for students and faculty, among other stakeholders. This review examines 24 articles that reported quantitative investigations of the effect of instructional interventions on performance-based outcomes conducted within undergraduate chemistry courses. Specifically, we examined four aspects of conducting such evaluations, including: (1) the type of quasi-experimental design used to study the relationship between interventions, students, outcomes, and settings; (2) the metrics used to measure performance outcomes; (3) the type of groups used to contrast outcomes across experimental conditions; and (4) the statistical methods used to analyze the comparison data. Through the examination of these four aspects of causal studies, together with a validity typology for quasi-experimental designs, we catalogued the metrics and methods used to compare student performance outcomes across varied instructional contexts. Recommendations for researchers and practitioners planning quasi-experimental investigations and for interpreting results from causal studies in chemistry education are provided.


Copyright © 2019 American Chemical Society and Division of Chemical Education, Inc.


This is the author’s version of a work that was accepted for publication in The Journal of Chemical Education. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in The Journal of Chemical Education 2019, 96, 3, 401-413, and can be found online at:



Persistent Identifier

MackHensenBarbera_PerformanceMetrics_SI_1-19-2019.pdf (235 kB)
List of Sampled Articles (Table S1); Reading Protocol; Data Matrices (Tables S2-S5); and Code Descriptions