LAUSR

Cumulative data analysts typically rely on two frameworks: meta-analysis (MA) of aggregate data (AD, i.e., summary statistics) and integrative data analysis (IDA) of individual participant data (IPD). In practice, AD may be available from some studies whereas IPD are available from others. AD and IPD can be pooled using two-stage and one-stage multilevel modeling (MLM) approaches (e.g., Riley et al., 2008). Although jointly modeling AD and IPD may allow for more generalizable and precise estimates, these models remain surprisingly unused in psychological science. Furthermore, analytical and simulation study of the performance of these models is limited. We conducted a simulation study to evaluate 12 combinations of available data (only AD; only IPD; both AD and IPD), imputation strategy (pseudo-IPD imputation strategy for AD studies, Papadimitropoulou et al., 2019; no imputation for AD studies), models (random-effects MA; fixed-intercepts/ random-slopes MLM, Riley et al., 2008; empirically reference-group mean-centered MLM; randomintercepts/random-slopes MLM) for cumulative analysis of treatment differences (see Table 1). We evaluated the estimation accuracy, type I error rate, and power of these 12 approaches. Data were generated from a random-intercepts/random-slopes MLM for treatment differences with a dichotomous (0/1) predictor and a normally distributed outcome. Estimate bias for the average treatment difference lh was ignorable in all conditions. Estimates of the between-study variance of treatment effects s2 could be negatively biased when a random-intercepts/random-slopes MLM was fit to IPD or imputed pseudoIPD when the study-level residual correlation was negative. The empirical reference-group mean-centered MLM overestimated s2: Type I error rates for lh were well controlled unless random-effects MA was used; MA was conservative with fewer than six (s2 1⁄4 0) or 28 studies (“medium” s2). The results suggest that modeling all available data can improve the estimation precision for lh and power for testing lh: Further, modeling (pseudo-) IPD can yield greater precision and power than MA