Validation studies of survey data are typically limited to a very small number of survey items, to cross-sectional estimates, and to particular sub-populations for which access to records happens to be available. We report here on a validation study carried out in the UK in 2003 with large numbers of validated items (20+ for all respondents, 40+ for employed respondents, 60+ for some), longitudinal data, and based upon a large national sample. However, the representativeness of the validation sample obtained depends on the co-operation of both survey respondents and the providers of validation data and on error in the matching process. In the UK, matching survey data with administrative records is not common practice. In this paper, we investigate several aspects of the feasibility of validation studies. We focus on the validation of income and employment data. Two validation sources were used: Department for Work and Pensions (DWP) benefit data and employers records. The former provided histories of benefit receipt and tax credits (e.g. for child / disability / housing / unemployment benefits, pensions and income support). The latter provided information on occupation and employment status, gross and net pay, membership of company pension schemes and industry sector. In the survey interview, respondents were asked for written permission both to obtain their DWP records and to contact their employer. They were also asked to provide information that would facilitate the validation: National Insurance number (NINO) and employer contact details. Subsequently, DWP records were extracted using a non-hierarchical matching strategy, based on different combinations of identifying variables obtained in the survey (NINO, sex, date of birth, name and postcode), and a survey of employers was carried out (mail, with telephone follow-up). We report permission rates, proportions providing matching items, match rates for the DWP data and response rates to the employer survey. We identify correlates of these measures of success at each stage of the validation process in terms of substantive characteristics of the survey respondents. Variation by subgroups is identified and implications for the representativeness of the validation sample are discussed. The analyses will be extended to address potential bias in conclusions regarding validity of survey reports.

Jäckle, A., Sala, E., Jenkins, P., Lynn, P. (2004). Validating Survey Data: Experiences using Employer Records and Government Benefit (Transfer) Data in the UK. In Proceedings of the Survey Research Methods Section of the American Statistical Association (pp.4802-4810).

Validating Survey Data: Experiences using Employer Records and Government Benefit (Transfer) Data in the UK

SALA, EMANUELA MARIA;
2004

Abstract

Validation studies of survey data are typically limited to a very small number of survey items, to cross-sectional estimates, and to particular sub-populations for which access to records happens to be available. We report here on a validation study carried out in the UK in 2003 with large numbers of validated items (20+ for all respondents, 40+ for employed respondents, 60+ for some), longitudinal data, and based upon a large national sample. However, the representativeness of the validation sample obtained depends on the co-operation of both survey respondents and the providers of validation data and on error in the matching process. In the UK, matching survey data with administrative records is not common practice. In this paper, we investigate several aspects of the feasibility of validation studies. We focus on the validation of income and employment data. Two validation sources were used: Department for Work and Pensions (DWP) benefit data and employers records. The former provided histories of benefit receipt and tax credits (e.g. for child / disability / housing / unemployment benefits, pensions and income support). The latter provided information on occupation and employment status, gross and net pay, membership of company pension schemes and industry sector. In the survey interview, respondents were asked for written permission both to obtain their DWP records and to contact their employer. They were also asked to provide information that would facilitate the validation: National Insurance number (NINO) and employer contact details. Subsequently, DWP records were extracted using a non-hierarchical matching strategy, based on different combinations of identifying variables obtained in the survey (NINO, sex, date of birth, name and postcode), and a survey of employers was carried out (mail, with telephone follow-up). We report permission rates, proportions providing matching items, match rates for the DWP data and response rates to the employer survey. We identify correlates of these measures of success at each stage of the validation process in terms of substantive characteristics of the survey respondents. Variation by subgroups is identified and implications for the representativeness of the validation sample are discussed. The analyses will be extended to address potential bias in conclusions regarding validity of survey reports.
slide + paper
Validation studies, data linkage, BHPS
English
Conference of the American Statistical Association (Survey Research Methods Section)
2004
Proceedings of the Survey Research Methods Section of the American Statistical Association
2004
4802
4810
http://www.amstat.org/sections/srms/Proceedings/y2004/files/Jsm2004-000269.pdf
none
Jäckle, A., Sala, E., Jenkins, P., Lynn, P. (2004). Validating Survey Data: Experiences using Employer Records and Government Benefit (Transfer) Data in the UK. In Proceedings of the Survey Research Methods Section of the American Statistical Association (pp.4802-4810).
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/25861
Citazioni
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
Social impact