Published In
Journal of Chemical Education
Document Type
Post-Print
Publication Date
7-6-2018
Subjects
Chemistry -- Study and teaching (Higher) -- Research, Chemistry -- Curriculum -- Assessment, Chemometrics
Abstract
This methodological paper examines current conceptions of reliability in chemistry education research (CER) and provides recommendations for moving beyond the current reliance on reporting coefficient alpha (α) as reliability evidence without regard to its appropriateness for the research context. To help foster a better understanding of reliability and the assumptions that underlie reliability coefficients, reliability is first described from a conceptual framework, drawing on examples from measurement in the physical sciences; then classical test theory is used to frame a discussion of how reliability evidence for psychometric measurements is commonly examined in CER, primarily in the form of single-administration reliability coefficients. Following this more conceptual introduction to reliability, the paper transitions to a more mathematical treatment of reliability using a factor analysis framework with emphasis on the assumptions underlying coefficient alpha and other single-administration reliability coefficients, such as omega (ω) and coefficient H, which are recommended as successors to alpha in CER due to their more broad applicability to a variety of factor models. The factor analysis-based reliability discussion is accompanied by R code that demonstrates the mathematical relations underlying single-administration reliability coefficients and provides interested readers the opportunity to compute coefficients beyond alpha for their own data.
Note: Supporting information (R Code for Reliability Calculations) is available at the ACS publications website at DOI 10.1021/acs.jchemed.8b00220.
Rights
Copyright © 2018 American Chemical Society and Division of Chemical Education, Inc.
DOI
10.1021/acs.jchemed.8b00220
Persistent Identifier
https://archives.pdx.edu/ds/psu/27217
Citation Details
Published as Regis Komperda, Thomas Pentecost, and Jack Barbera, “Moving Beyond Alpha: A Primer on Alternative Sources of Single-Administration Reliability Evidence for Quantitative Chemistry Education Research,” Journal of Chemical Education, 2018, 95, 9, 1477-1491, 35 DOI:10.1021/acs.jchemed.8b00220.
Description
This is the author’s version of a work that was accepted for publication in The Journal of Chemical Education. Changes resulting from the publishing process, such as editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in The Journal of Chemical Education 2018, 95, 9, 1477-1491, 35, and can be found online at: http://dx.doi.org/10.1021/acs.jchemed.8b00220.