Cognitive Interviewing for Testing Survey Questions

Description:

This course runs over three afternoons.

Introduction/overview

Cognitive interviewing explores how the respondent proceeds through the stages of comprehension, recall, judgement, and response when answering a survey question. It allows the researcher to pinpoint survey question problems that may not be identified with other testing methods. Cognitive interviewing itself encompasses several methods. The most popular ones include think-alouds, probing, and observation. In addition to learning about what cognitive interviewing is, participants will learn how to do it and how to analyse the resulting data.

The course has an initial breakout room session where participants practice cognitive interviewing on questions that Dr. Campanelli provides. There are also breakout sessions on analysing cognitive interview transcript data (which has been summarised into key points) and on revising a survey question based on cognitive interview findings. In addition, there is a stream of breakout sessions where participants first critique each other’s questions, then design cognitive probes, conduct cognitive interviewing on these questions and after that meet up to share what was learned from that process.

Topics

• Problems with the standard field pilot: Cognitive interviewing as the solution
• What is cognitive interviewing?
• Main cognitive interviewing techniques
• Mini Appendix - Peripheral cognitive interviewing techniques
• Structuring and conducting the cognitive interview: Some key principles
• Recruiting and sampling respondents
• Analysis of cognitive interview data (from simple to sophisticated)
• From cognitive interviewing analysis to question changes with examples of cognitive interviewing results and revised questions
• Reporting guidelines
• Mini Appendix - Variations and other uses of cognitive interviewing: For example, for documents other than questionnaires, for translations, after the main survey for exploring unusual findings, cognitive interviewing for establishment surveys and controversy in cognitive interviewing
• Six breakout room sessions

Course requirements

• Participants need to have familiarity with questionnaires.

IMPORTANT: 3 of the breakout sessions are based on participants submitting 3 survey questions 2 weeks prior to the start of the course. These questions can be from any source (e.g., from a questionnaire that you wrote, or from another questionnaire, or a report or journal article). These could be 3 separate questions or 3 sub-parts of a grid. If you have any questions, please contact: Lindsay.Adams@the-sra.org.uk

Learning Objectives

By the end of the course, participants will:

• Have an understanding of the range of techniques which make up a cognitive interview
• Have increased their own ability to do cognitive interviewing
• Have insights into the issues of selecting and recruiting respondents and analysing cognitive interview data

Who will benefit?

Participants wanting to pretest their questions or examine unusual findings after the survey is complete. Cognitive interviewing can also be used for critiquing existing surveys.

Course tutor

Dr Pamela Campanelli is a Survey Methods Consultant and runs The Survey Coach business. She is also a Chartered Statistician and Fellow of the Academy of Social Sciences. She has worked at the University of Michigan, the U.S. Bureau of the Census, ISER at the University of Essex, and NatCen Social Research. She has both led and been a team member on ESRC grants (one on survey nonresponse and one on measurement error in mixed mode surveys). She regularly teaches short courses in the UK and abroad for government departments, survey research companies, universities, as well as for various other institutions and businesses. She believes in delivering courses that are lively and engaging and foster an informal and interactive atmosphere.

Cost:

£360 for SRA members, £470 for non-members

Website and registration:

Register for this course

Region:

International

Keywords:

Frameworks for Research and Research Designs, Data Collection, Data Quality and Data Management , Qualitative Data Handling and Data Analysis, Quantitative Data Handling and Data Analysis, Research Management and Impact, Research Skills, Communication and Dissemination

Related publications and presentations from our eprints archive:

Frameworks for Research and Research Designs
Data Collection
Data Quality and Data Management
Qualitative Data Handling and Data Analysis
Quantitative Data Handling and Data Analysis
Research Management and Impact
Research Skills, Communication and Dissemination

Back to the training database