How to analyze data from quantitative vignette studies

Speaker(s):

Thom Baguley, NTU Psychology, Nottingham Trent University

Abstract:

This workshop outlines the key characteristics of vignette studies and considers how the design of the study and the construction or selection of the vignettes impacts on the appropriate statistical model to employ, as this depends on subtle details of the design, selection and allocation of vignettes to participants. The ideal design will balance the fidelity of the vignettes to the real world phenomena being studied, the complexity of the allocation (including problems such as aliasing arising in fractional factorial designs) and statistical generalizability. Notably many published vignette studies have unmodeled heterogeneity between vignettes. This leads to underestimation of error variance in the statistical model and hence problems such as Type I error inflation. Multilevel models are proposed as a general approach to handling nested and crossed designs including unbalanced and fractional designs.