TY - JOUR
T1 - Comparing cross-classified mixed effects and Bayesian structural equations modeling for stimulus sampling designs
T2 - A simulation study
AU - Wickham, Robert E.
AU - Hardy, Kristin K.
AU - Buckman, Holly L.
AU - Lepovic, Elan
N1 - Publisher Copyright:
© 2020 Elsevier Inc.
PY - 2021/1
Y1 - 2021/1
N2 - Researchers examining a wide range of psychological phenomena, including interpersonal perception, attitude formation, and stereotype activation, apply stimulus sampling designs (SSD). The standard SSD study requires participant raters to provide evaluations of a series of target stimuli (e.g., photographs, media clips, vignettes), and the constituent responses are simultaneously nested within participants and stimuli, yielding a cross-classified data structure. Prior methodological work has illustrated the application of both cross-classified mixed effects, and cross-classified structural equation modeling to accommodate the corresponding dependency structure. Despite their widespread application, little is known about how sample size for both participants and stimuli is associated with inferential power and coverage in SSDs. Even less is known about the feasibility of random slopes, or whether the frequentist (maximum likelihood) and Bayesian (MCMC) estimators differ in accuracy or efficiency under the design conditions typically observed in SSD studies. We conducted a Monte Carlo simulation study to better understand parameter bias, statistical power, and confidence or credible interval coverage, as a function of the number of participant raters and target stimuli, effect size, as well as the presence of random slopes, and modeling framework. Recommendations for future research are provided.
AB - Researchers examining a wide range of psychological phenomena, including interpersonal perception, attitude formation, and stereotype activation, apply stimulus sampling designs (SSD). The standard SSD study requires participant raters to provide evaluations of a series of target stimuli (e.g., photographs, media clips, vignettes), and the constituent responses are simultaneously nested within participants and stimuli, yielding a cross-classified data structure. Prior methodological work has illustrated the application of both cross-classified mixed effects, and cross-classified structural equation modeling to accommodate the corresponding dependency structure. Despite their widespread application, little is known about how sample size for both participants and stimuli is associated with inferential power and coverage in SSDs. Even less is known about the feasibility of random slopes, or whether the frequentist (maximum likelihood) and Bayesian (MCMC) estimators differ in accuracy or efficiency under the design conditions typically observed in SSD studies. We conducted a Monte Carlo simulation study to better understand parameter bias, statistical power, and confidence or credible interval coverage, as a function of the number of participant raters and target stimuli, effect size, as well as the presence of random slopes, and modeling framework. Recommendations for future research are provided.
KW - Bayesian analysis
KW - Cross-classified
KW - Mixed effects model
KW - Stimulus sampling
KW - Structural equation models
UR - http://www.scopus.com/inward/record.url?scp=85093652676&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85093652676&partnerID=8YFLogxK
U2 - 10.1016/j.jesp.2020.104062
DO - 10.1016/j.jesp.2020.104062
M3 - Article
AN - SCOPUS:85093652676
SN - 0022-1031
VL - 92
JO - Journal of Experimental Social Psychology
JF - Journal of Experimental Social Psychology
M1 - 104062
ER -