SPSS, SAS, and MATLAB Programs for Generalizability Theory Analyses

Reference:

Mushquash, C., & O'Connor, B. P. (2006). SPSS, SAS, and MATLAB programs for generalizability theory analyses. Behavior Research Methods, 38(3), 542-547.

The identification and reduction of measurement errors is a major challenge in psychological testing. Most investigators rely solely on classical test theory for assessing reliability, whereas most experts have long recommended using generalizability theory instead. One reason for the common neglect of generalizability theory is the absence analytic facilities for this purpose in popular statistical software packages. This article provides a brief introduction to generalizability theory, describes easy to use SPSS, SAS, and MATLAB programs for conducting the recommended analyses, and provides an illustrative example using data (N = 329) for the Rosenberg Self-Esteem Scale. Program output includes variance components, relative and absolute errors and generalizability coefficients, coefficients for D studies, and graphs of D study results.

The "G1" programs below can be used for six possible one- or two-facet designs. They use the ANOVA method and they require balanced data.

The "G2" programs below use the SPSS VARCOMPS or SAS VARCOMP procedure to compute the variance components, which are then read and processed by the program for G theory analyses. The "G2" programs can be used for any design that can be processed by the SPSS VARCOMPS or SAS VARCOMP procedures; there are no limitations on the number of facets; and they can process unbalanced data. The following four kinds of variance components can be produced by the SPSS VARCOMPS and SAS VARCOMP procedure and then processed by the G2 programs: ML, REML, MINQUE, and ANOVA.

Brian P. O'Connor
Department of Psychology
University of British Columbia - Okanagan
Kelowna, British Columbia, Canada
brian.oconnor@ubc.ca