The 3 Stages of Weighting Probability Surveys

Description:

This course runs over 2 afternoons.

Introduction/Overview

Comment from a recent participant:

“Our trainer was brilliant! Very knowledgeable and willing to go above and beyond to help us . . . Good value for money too - a very wide range of topics covered.”

Focusing on probability samples, this course covers different weighting methods for survey data for each of the 3 stages of the weighting process (design weight, nonresponse weight and calibration weight). It covers all aspects of each method so that participants can make an informed decision about what will be best practice for them. The course starts with some simple weighting class examples to facilitate understanding of how weighing works, and moves through to more sophisticated methods and combinations of methods. It presents the debates in the weighting literature as well as being very practical. It includes real-life examples, but it does NOT use a practice dataset. This is because a practice dataset would tie the course to one software package and there are many packages that do weighting which are used by researchers.  Instead, the course focuses on the decisions to be made in choosing and implementing an appropriate method. Breakout sessions will help to put learning into practice.

Note that differences between surveys of households / individuals and businesses are pointed out throughout the course.

Topics

• Background information at the start of the course: How your weighting plan is reflective of your sample design,  defining nonresponse bias (mini appendix on study designs for determining who the nonrespondents are and correlates of nonresponse bias) and a short refresher on the statistics used on the course
• Two simple weighting class examples to understand how weighting works (with Workshop)
• Four other things to keep in mind: Combining weights, re-scaling (normalising) weights, the effect of weighting, trimming (truncating) weights
• 3 common stages of weighting a probability survey:
  • design weights and adjusting for unknown eligibility,
  • nonresponse weights
  • calibration to population values
  • and why not every survey will use all three.
• Using post-stratification for nonresponse adjustment (Bulgarian example)
• Workshop – Did the post-stratification used in airport noise study adjust for nonresponse bias?
• Using propensity score weighting with logistic regression or classification trees for nonresponse adjustment
• Workshop – Participants create the weights for an asthma study
• What is calibration?
• Post-stratification as a calibration method along with raking, GREG weighting, and the ONS General Estimation System
  • Mini appendix with detailed example of how to do raking in R
• Calibration correcting for nonresponse or a separate nonresponse adjustment phase?
• British Household Panel Survey example
  • Mini appendix with examples of weighting the English Housing Survey
  • Mini appendix with example of weighting a push to web survey
  • Mini appendix with example of weighting a business survey
• Summary of methods and discussion of software

Course requirements

Participants need to be familiar with survey sampling concepts, confidence intervals, the terms standard error and sampling variance, and regression analysis.

Learning Objectives

By the end of the course, participants will:

• Have knowledge about the purpose of the three stages of weighting
• Be aware of a variety of weighting methods, knowing how and where to use them
• Be aware of the pros and cons of each method
• Be able to make informed choices about which particular weighting method(s) are best for their own data

Who will benefit

Both participants who want to use weighting strategies on their data and those participants who want to knowledge of weighting for critiquing others’ work.

Course tutor

Dr Pamela Campanelli is a Survey Methods Consultant and runs The Survey Coach business. She is also a Chartered Statistician and Fellow of the Academy of Social Sciences. She has worked at the University of Michigan, the U.S. Bureau of the Census, ISER at the University of Essex, and NatCen Social Research. She has both led and been a team member on ESRC grants (one on survey nonresponse and one on measurement error in mixed mode surveys). She regularly teaches short courses in the UK and abroad for government departments, survey research companies, universities, as well as for various other institutions and businesses. She believes in delivering courses that are lively and engaging and foster an informal and interactive atmosphere.

 

Cost:

£165 for SRA members, £220 for non-members

Website and registration:

Register for this course

Region:

International

Keywords:

Frameworks for Research and Research Designs, Data Collection, Data Quality and Data Management , Quantitative Data Handling and Data Analysis, Research Management and Impact, Research Skills, Communication and Dissemination

Related publications and presentations from our eprints archive:

Frameworks for Research and Research Designs
Data Collection
Data Quality and Data Management
Quantitative Data Handling and Data Analysis
Research Management and Impact
Research Skills, Communication and Dissemination

Back to the training database