Estimating and correcting for multiple types of measurement errors in longitudinal studies
Survey data, in its myriad forms, is essential in modern societies for policy development, business and marketing, as well as academic research. Longitudinal datasets (where the same individuals are interviewed on repeated occasions over time), are particularly important because they enable the analysis of 'within-individual change which is increasingly recognised as essential for developing causal explanations of the social world. For example, longitudinal data have played a key role in understanding the impact of long term poverty on children's educational and labour market prospects, the effect of smoking and alcohol consumption during pregnancy on child and parent outcomes, and on the factors underpinning intergenerational social mobility.
Yet even though the importance of longitudinal data is widely recognised, research on the quality and accuracy of longitudinal data is surprisingly sparse. Certain issues have been especially neglected, such as how to detect and correct for errors in the measurement of key concepts, so-called 'measurement error. Measurement error can be thought of as the difference between the conceptual 'true' score of an individual on a particular characteristic and what is actually measured in a survey. For instance, a particular survey respondent might actually weigh 81 kilos, yet in the survey data is recorded as weighing 80 kilos. There are a number of different reasons why the difference between the measured value and the true score might arise, which can be thought of collectively as different forms of measurement error. Measurement error is particularly problematic in longitudinal data, because it can incorporate multiple types of errors that appear at different time points. If such errors are present in a data set - and the existing research evidence suggests that their presence is widespread – then analyses and conclusions will be negatively affected. In short, we run the risk of drawing incorrect conclusions relating to key areas of public policy.
This proposal is for a project that will develop a new type of model to enable estimation of, and correction for, multiple types of errors in longitudinal data. This model will make it possible to estimate and correct simultaneously for:
- Method effects (where the response scale used for the question biases answers);
- Acquiescence (where respondents tend to agree to questions regardless of their content);
- Social desirability (where respondents provide answers in ways that are considered socially desirable);
- Cross-cultural effects (where measurement errors vary cross-culturally).
In addition to controlling for these different measurement errors the methods to be developed will also enable the evaluation of how they change over time. This will allow researchers to differentiate between measurement errors that are stable in time from those that are time specific, thus enabling powerful correction of error in longitudinal data. Additionally, the project will contribute to important debates in psychology, political science and survey methodology regarding the relationship between stable traits and errors in surveys. Similarly, the new model will enable researchers to estimate how the stability and changes in errors influence substantive survey answers in longitudinal data.
People
Dr Alexandru Cernat, University of Manchester
Partners
Daniel Oberski (University of Utrecht)
Understanding Society Innovation Panel (ISER, University of Essex)
German Socio-Economic Panel – Innovation Sample (DIW)
