Quality of Longitudinal Survey Data

The project focused on the quality of longitudinal survey data in general, but specifically upon:

timing of data collection waves and quality of retrospective recall data
the role of interviewers in longitudinal survey data quality
handling missing data in the analysis of longitudinal survey data

Intervals Between Waves

Most longitudinal surveys attempt to build up histories of the sample units with respect to key substantive dimensions. Examples might include fertility histories, household structure histories and employment histories. However, observation cannot be continuous. Instead, sample persons are interviewed at regular intervals and asked to supply data relating to a past period of time, typically the time since the previous interview. An important design question then is how frequently the interviews should take place. There are constraints on the interval, primarily concerning respondent burden (and possible consequent sample attrition), recall accuracy and survey costs. It is therefore important in this context to understand what determines the ability of survey respondents to accurately recall and report past events and behaviour. Mediating factors that affect respondents' ability to recall events include not only the length of elapsed time since the event, but also the saliency of the event and the occurrence of other events in the interim that may distort the memory of the event in question. Respondents' ability to recall events can be improved with the help of appropriate "cues" or memory aids. These can take various forms, including Event History Calendars (EHCs). But with any method, some degree of memory decay will remain and data quality will decrease the longer the survey reference period. The challenge for researchers is to identify the most appropriate frequency of data collection, considering recall accuracy, costs and burden.

The Role of Interviewers

The ways in which interviewers are deployed on longitudinal surveys may affect data quality in important ways. First is the question of the extent to which it is desirable for the same interviewer to interview a survey respondent over several waves of a survey. There may be both advantages and disadvantages in interviewer continuity, but the current reality is that interviewer allocation on longitudinal surveys is done on pragmatic and intuitive grounds, with no explicit regard for effects on data quality. Interviewer continuity may affect response rates and could also affect measurement error. This could either work through improved rapport and trust over repeated contacts with the same interviewer, leading to more considered and perhaps honest answers, or through reduced social distance leading to less honest answers due to social desirability effects. Theories of respondent motivation suggest that the effect is likely to depend on the nature of the data to be collected as well as the cognitive ability and predisposition of the respondent. But there is a paucity of empirical evidence in this area.

Missing Data

The nature of missing data, and the consequences for analysts, are quite distinct in the context of a longitudinal survey. Missing data can arise from unit, wave or item non-response and in a variety of patterns. As well as the patterns of missing data being complex and the levels potentially high, the reasons for missing data can also be different on a longitudinal survey, with consequences for the likely ignorability of the missing data mechanism.

Network Activities

To achieve these aims, four main activities were conducted:

An expert seminar on optimum timing of data collection waves and quality of retrospective recall data to be held in July 2007 at the University of Essex.
An expert seminar on the role of interviewers in longitudinal survey data quality to be held in Edinburgh on 20 February 2008. The seminar programme and presentations are available from.
A workshop on handling missing data in the analysis of longitudinal survey data to be held on 11 December 2007 in London. Information on the workshop programme is available from the Centre for Longitudinal Studies website;
A junior researcher exchange between the Centre for Longitudinal Studies (Institute of Education) and the UK Longitudinal Studies Centre (University of Essex).

For further information please contact:
Professor Peter Lynn
University of Essex

Email: plynn@essex.ac.uk