Unnecessary Ping-Pong: Illustrations of Why Previous Findings Should Be Taken Into Account When Evaluating New Data Sets

Reference:

O'Connor, B. P. & Ermacora, D. (2021, Online First). Unnecessary ping-pong: Illustrations of why previous findings should be taken into account when evaluating new data sets. Canadian Journal of Behavioral Science, https://doi.org/10.1037/cbs0000259

Abstract:

Eighteen data sets for a wide selection of research topics were used to illustrate why previous findings should be taken into account when attempting to reach a conclusion about an effect when evaluating a new data set. Traditional data analysis methods that neglect previous findings can result in high rates of incorrect or conflicting conclusions while, for the same data sets, the rates of conflicting findings for methods that incorporate previous data (updating meta-analyses or Bayesian analyses) are dramatically lower. The benefits of incorporating previous findings are particularly evident when the effect size for a phenomenon is small. It is hoped that the illustrations will assist in a shift away from the still-common tendency to neglect the incorporation of effect sizes from previous studies when evaluating new data sets.

Click here for an R package that has all of the commands and datasets

Brian P. O'Connor
Department of Psychology
University of British Columbia - Okanagan
Kelowna, British Columbia, Canada
brian.oconnor@ubc.ca