Estimating and Testing for Differential Treatment Effects on Outcomes When the Outcome Variances Differ

Authors

Todd E. Bodner, Portland State UniversityFollow

Document Type

Citation

Publication Date

2018

Abstract

Researchers working in the context of randomized trials routinely estimate and test for treatment effects on the study outcomes. This article discusses the merits of assessing differential treatment effects across outcomes and proposes a multivariate approach using standardized outcomes for this purpose. This multivariate approach extends prior approaches to an arbitrary number of treatment groups and outcomes and does not require that the within-group covariance matrix have particular properties (e.g., sphericity). Theoretical analyses articulate the inferential basis for earlier recommendations for data standardization prior to analysis and demonstrate that inferential procedures (e.g., null hypothesis significance tests and confidence intervals) can exhibit poor operating characteristics when unstandardized outcome data are used for analysis and differential standardized treatment effects are the conceptual, intended target of inference. This article explores these and other issues (e.g., statistical power to detect and confidence intervals for differential standardized treatment effects) and demonstrates the proposed approach using data from a published experiment. The theoretical utility of differential treatment-effect evidence is considered from a construct validity perspective for randomized trials. The proposed approach provides inferential procedures to evaluate theoretically motivated predictions for differential treatment effects on the outcomes; failure to support such predictions either calls the construct validity of the randomized trial into question or the underlying theory. The proposed approach also enables the detection of differences in treatment effects on the outcomes that are not theoretically expected; such results, especially if replicated, would motivate the need for theoretical refinements.

Description

© American Psychological Association, 2018

Locate the Document

http://dx.doi.org/10.1037/met0000158

DOI

10.1037/met0000158

Persistent Identifier

http://archives.pdx.edu/ds/psu/25148

Citation Details

Bodner, T. E. (2018). Estimating and testing for differential treatment effects on outcomes when the outcome variances differ. Psychological Methods, 23(1), 125-137.