LAUSR

The reproducibility crisis has captivated psychologists since the Open Science Collaboration (2015) failed to replicate several influential psychological studies. Replication efforts in this area have focused exclusively on human-generated datasets. To spark discourse on a neglected aspect of the methodological literature, we attempted exact replications of three prominent factor analytic simulation studies. Briggs and MacCallum (2003) investigated sampling error and model-approximation error on the ability of ordinary least squares (OLS) and maximum likelihood (ML) factor analysis to recover weak common factors. Our exact replication confirmed that OLS outperformed ML in these conditions. However, important discrepancies were found between the original and replicated results. Namely, Briggs and MacCallum reported factor recovery RMSE values that were far larger than ours. Follow-up investigations revealed that the original authors used an incorrect RMSE formula. de Winter, Dodou, and Wieringa (2009) investigated minimum sample size requirements for recovering population factor loadings. These models differed in their factor-loading strength, number of factors, and number of factor indicators. The resulting minimum sample sizes were cross-validated in a second simulation with five performance criteria. We replicated the original findings to a high degree of accuracy. MacCallum, Widaman, Zhang, and Hong (1999) investigated the influence of sample size, communality strength, and the ratio of factor indicators to common factors in the recovery of population factor models. Model recovery was quantified by congruence coefficients between the sample and population loadings matrices and the RMSE values between the sample and sample average solution for each model. We replicated the congruence coefficient results. However, the original RMSE findings were not published and thus we were not able to determine replication success for this criterion.