The European Journal of Public Health Advance Access originally published online on November 5, 2007
The European Journal of Public Health 2008 18(2):189-194; doi:10.1093/eurpub/ckm103
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Health Monitoring |
The effect of non-response on estimates of health care utilisation: linking health surveys and registers
Jens Gundgaard1, Ola Ekholm2, Ebba Holme Hansen3 and Niels Kr. Rasmussen2
1 Institute of Public Health – Health Economics, University of Southern Denmark, Odense, Denmark
2 National Institute of Public Health, University of Southern Denmark, Copenhagen, Denmark
3 Faculty of Pharmaceutical Sciences, Section for Social Pharmacy, Copenhagen University, Copenhagen, Denmark
Correspondence: Jens Gundgaard, Institute of Public Health – Health Economics, University of Southern Denmark, J.B. Winsløwsvej 9, 1, 5000 Odense C, Denmark, tel: +45 6550 3965, fax: +45 6550 3880, e-mail: jgu{at}sam.sdu.dk
Received July 4, 2007, accepted October 4, 2007
 |
Abstract |
|---|
 Top Abstract Material and methodsResultsDiscussionConclusionAcknowledgementsReferences |
|---|
Background: Non-response in health surveys may lead to bias in estimates of health care utilisation. The magnitude, direction and composition of the bias are usually not well known. When data from health surveys are merged with data from registers at the individual level, analyses can reveal non-response bias. Our aim was to estimate the composition, direction and magnitude of non-response bias in the estimation of health care costs in two types of health interview surveys. Methods: The surveys were (1) a national personal interview survey of 22 484 Danes (2) a telephone interview survey of 5000 Danes living in Funen County. Data were linked with register information on health care utilisation in hospitals and primary care. Health care utilisation was estimated for respondents and non-respondents, and the difference was explained by a decomposition method of bias components. Results: The surveys produced the same pattern of non-response, but with slight differences in non-response bias. Response rates for the interview and telephone surveys were 75 and 69%, respectively. Refusal was the most frequent reason for non-response (22 and 20% of those sampled, respectively), whereas illness, non-contact, and other reasons were less frequent. Respondents used 3–6% less health care than non-respondents at the aggregate level, but the opposite was true for some specific types of health care. Non-response due to illness was the main contributor to non-response bias. Conclusions: Different types of non-response have different bias effects. However, the magnitude of the bias encourages the continued use of interview health surveys.
Keywords: non-response, health care, health survey, registers, bias
Health interview surveys contribute important information on health status, health behaviour and health care utilisation in the population. A health survey is often the only source of information when variables of interest are not available from registers. The problem with surveys is, of course, that not all the people who are invited to participate do so. Failure to participate can lead to bias in estimates on the population. If missing data are randomly unavailable for unknown reasons, the non-response might not interfere with the survey's intention to be representative. However, when non-response is systematically related to the variables of interest, the potential effect of non-response needs scrutiny.1–3 This may well be the case for health surveys, as illness can be one of the reasons for not participating.4–6 The bias of the non-response is determined by the multiplicative effect of the proportion of non-respondents and the difference between the respondents and the non-respondents.7 Furthermore, different types of non-response may have different or off-setting effects on the magnitude and direction of the bias.7 The magnitude, direction and composition of the bias are usually not well known and the problem of non-response bias tends to be ignored. This is in keeping with the purpose of conducting a population survey, which is to collect information not available from other sources. Although the issue of non-response is as old as survey research itself, its importance does not appear to decrease.8 Although not unequivocal, there are signs that it is increasingly difficult to obtain high response rates.7–9 The climate for taking surveys may deteriorate due to societal changes such as increased urbanisation and changing demographic and family structures, and decreasing norms for civic duty.7 In addition, over-surveying in some countries and public debates on invasion of privacy and confidentiality may worsen conditions even more.7,8
There is no general agreement about which mode of interview (face-to-face or telephone) results in the highest response rates, and both have been mentioned.10,11 The two modes are not usually directly comparable, as different design features are applied, in addition to the mode of interview.
Previous studies support the presumption that non-response is a complex phenomenon made up of various effects, especially when the focus is on health care utilisation. Concerning hospital admissions, some studies showed higher utilisation among non-respondents,12–17 while other studies showed no differences between respondents and non-respondents.4,18–20 A Danish study showed that the non-respondents in general had the same admission rates as the respondents, but the admission rates were higher among non-respondents around the interview period. The pattern is even more diverse for primary health care. Some studies registered lower health care utilisation for non-respondents,4,18,19 whereas other studies showed higher health care utilisation rates.21,22 The present study focuses on the representativeness of estimates of health care utilisation among respondents in sampled surveys by obtaining information on health care utilisation from registers for respondents as well as non-respondents. The aim of the article is 2-fold: (1) to estimate the magnitude, direction and composition of non-response bias in health care utilisation and (2) to compare the non-response bias from two sources of health surveys: a personal interview survey and a telephone interview survey.
|
Material and methods |
|---|
|
TopAbstractMaterial and methodsResultsDiscussionConclusionAcknowledgementsReferences |
|---|
Personal interview data
The personal interview data were obtained from The Danish Health Interview Survey 2000 (DHIS).23,24 22 484 Danish citizens aged 16 and above were drawn from the Centralised Civil Register to participate in a national health survey on health care utilisation, health status, morbidity, health behaviour, environmental and occupational health risks and health resources. The sample was stratified to include at least 1000 respondents from each of the 15 Danish counties, and data were weighted by the reciprocals of the sampling probabilities. Data were collected from personal interviews by trained interviewers in three rounds in February, May and September 2000, respectively. 16 688 individuals participated, corresponding to a weighted response rate of 74.5%.
Telephone interview data
The telephone interview data were obtained from The Funen County Health Survey (FCHS). Five thousand people living in Funen County, Denmark aged 16–80 were drawn from the Centralised Civil Register to participate in a health survey on health status, health behaviour and socio-economic background. The sample was stratified with respect to municipalities, and the data were weighted by the reciprocals of the sampling probabilities. Data were gathered through telephone interviews from October 2000 through April 2001.25 A total of 3421 individuals participated, corresponding to a weighted response rate of 69.2%
Register-based data
Data from each survey were merged with data from individual-level computerised registers including: all somatic hospital visits, services in the primary health care sector such as GP visits, physiotherapy, specialist treatment, dentistry, and dispensed prescription medicine in 2000 and 2001. The use of registers to extract information on health care utilisation makes it possible to obtain exact information on health care services over a long period of time. The registers also make it possible to distinguish between different types of health care, and the different types of services can be added by measuring health care utilisation in monetary terms. Health care services were measured as the long-run costs of the services and approximated by prices, charges or fees.
Hospital visits were extracted from the National Patient Register and the Funen County Patient Administrative System (FPAS) and linked to DHIS and FCHS, respectively. These registers include records on somatic inpatient stays, ambulatory and emergency room visits. Each hospital admission was described by an estimated charge based on the 2002 Danish case-mix system of Diagnosis Related Groups (DRGs). The case-mix system covers inpatient hospital stays, whereas ambulatory and emergency room visits are described by a similar but simpler system. Capital costs are not included in the case-mix system. All charges were adjusted to 2003 price level for hospital treatments.26
The services in the primary health care sector were extracted from the Registry of Public Health Insurance and linked to DHIS and FCHS. This registry includes all partly or fully reimbursed health services in primary health care, i.e. from general practitioners, physiotherapists, dentists and specialists. Each service is described with a reimbursement fee. As considerable co-payment exists for dental health care and physiotherapy, these fees were adjusted to reflect the full amount (reimbursement + co-payment). Expert judgments were used to adjust the dental care fees to the approximate average level of dental care fees, whereas the relevant fees for physiotherapy were adjusted by dividing the reimbursement fees by the proportion of reimbursement. General practitioners are partly financed through capitation (about one-third of GP revenue), and the GP reimbursement fees were scaled up by this amount. All reimbursement fees were adjusted for inflation up to 2003 by the price index for physicians and physiotherapists.26
Medicine data were extracted from the Register of Medicinal Product Statistics (RMPS) and Odense University Pharmacoepidemiologic Database (OPED) and linked to DHIS and FCHS, respectively.27 These databases consist of all prescription refunds from Denmark and Funen County, respectively. The dispensed medicine is described by the total purchase price in the RMPS, whereas pharmacy retail prices including Value added tax (VAT) are used for the OPED. A further difference is that OPED includes prescription medicine entitled to reimbursement, whereas RMPS includes all prescribed medicines. Neither of the databases contains information on over-the-counter-medicine. Medicines dispensed in hospitals are included in the hospital charges. The medicine prices were adjusted for inflation to the 2003 level by the index for pharmaceutical products and equipment.26
The Danish Data Protection Agency has approved the linking of the registers and the survey data and all local confidentiality and privacy requirements have been met.
Statistical analysis
As non-response can have many causes, non-respondents were classified into different types of non-response along the lines of Kjøller and Thoning.9 The following categories were used: Interviewed, Refusals, Illness, Non-contacts, and a residual category of Other (table 1). The category of interviewed consisted of partially or completely interviewed respondents. That is, the item non-response is included in the category of interviewed, as can be seen in the table. People living outside Funen County or registered as dead were regarded as not belonging to the parent population under investigation and were excluded from the full sample.28
|
Differences in health care utilisation between respondents and non-respondents were analysed with ANOVA. The effect of different types of non-response bias on the estimates of health care utilisation has been analysed in a decomposition framework. Groves and Couper suggest a decomposition of the survey estimate into components of bias from the different types of non-response:7 µinterviewed = µn + (mrefusals/n) (µinterviewed – µrefusals) + (millness/n)(µinterviewed – µillness) + (mnon-contacts/n)(µinterviewed – µnon-contacts) + (mother/n) (µinterviewed – µother), where µj is the mean health care utilisation for the different categories of non-response, mj is the weighted number of people in the jth non-response category, and n is the weighted sample size. This formula makes it possible to quantify the effect on bias from the proportion of (the specific type of) non-respondents and the difference between the respondents and (the specific type of) the non-respondents. Each of the differences in means was tested by a t-test for comparing two means.
|
Results |
|---|
|
TopAbstractMaterial and methodsResultsDiscussionConclusionAcknowledgementsReferences |
|---|
Table 1 displays the categorisation of respondents and non-respondents for the two sources. The response rate was higher for the personal interview survey (74.5%) than the telephone survey (69.2%). The most frequent reason for non-response was refusal (22.1 and 20.3%, respectively). The non-contacts constituted the second largest category, which was larger in the telephone survey than in the interview survey (8% vs. 2%). Illness and hospital visits played a relatively minor role in the magnitude of non-response, although some refusals may have been due to health-related disabilities.
The mean health care costs for respondents and non-respondents are shown in table 2. The first column shows the mean health care costs for the overall representative samples for all sampled individuals. The figures are remarkably similar for both the interview and the telephone surveys.
|
The resulting non-response bias could be derived by comparing the first and the second columns. Health care costs were a bit smaller for respondents at the overall level [DHIS: Denmark Kroner (DKK) 19 966; FCHS: DKK 19 073] than for the whole sampled population (DHIS: DKK 20 510; FCHS: DKK 20 254) (1 USD = 8.33 DKK, 2005 PPP),29 An F-test for the differences between the respondents and non-respondents was only significant at a 10% level in the FCHS (F-statistics unavailable in the DHIS at the aggregate level). The interviewed had a significantly lower level of hospital utilisation and medicines, and a higher level of dentistry in DHIS at a 5% level of significance. In the telephone survey, which is a smaller sample, the interviewed had a lower level of hospital utilisation and medicine use at a 5% level of significance, and a higher level of specialist care and dentistry at a 10% level of significance.
The magnitude and composition of the non-response bias in health care utilisation is illustrated in table 3 by applying Groves and Couper's decomposition.7 The size of non-response bias for each type of health care consists of the proportion of the specific type of non-respondents (which is constant with respect to health care) and the difference between the interviewed and the specific type of non-respondents. The biggest contributor to non-response bias was illness. Although the ill or disabled constituted only about 1%, the large differential in health care utilisation between the interviewed and the ill or disabled contributed to a considerable non-response bias component for the aggregate level (DHIS: DKK –615; FCHS: DKK –608). Refusal was the dominating reason for non-response, but the effect on non-response bias was not large, as the differential between the interviewed and those who refused was small.
|
At the aggregate level, all bias components were negative for the telephone survey, indicating that all types of non-response led to an underestimation of the magnitude of health care utilisation. For the interview survey, the bias components were positive except for illness. The picture was less clear for the specific types of health care, and there was bias in different directions. The absolute sizes were rather small, although significant in some cases.
|
Discussion |
|---|
|
TopAbstractMaterial and methodsResultsDiscussionConclusionAcknowledgementsReferences |
|---|
The main findings of this study were that refusal was the most frequent reason for non-response, but that illness was the most important contributor to non-response bias. However, the non-response bias was small and insignificant in most cases. At the aggregate level, respondents used slightly less health care than the full sample. For some specific types of health care, typically in primary health care, the opposite was true. The analyses showed that non-response is a diversified matter. Different types of non-response have varied effects on the non-response bias. The ill/disabled constitute a small group among non-responders, but nevertheless consume a much higher level of health care than the interviewed. This should be expected, as health care utilisation is directly influenced by this reason for non-response. This group of people may not be able to participate because they are too ill, i.e. the same reason that they need health care. Therefore, this pattern is most pronounced for types of health care such as hospitalisation, GP visits and prescription medicine. The two modes of interview produced almost the same pattern of non-response, although the proportion of respondents was higher in the interview survey. However, some bias components had opposite effects in the two modes of interview.
Although the present study is not directly comparable with most of the studies in the literature, the results are in accordance with the main tendencies from previous findings: that hospital utilisation is higher among non-respondents or that there are no differences between respondents and non-respondents.4,12–20 As for the primary care utilisation, the less clear pattern is also in accordance with the literature, with some studies showing lower and some studies higher health care utilisation for non-respondents.4,18,19,21,22
Specific strengths of this study were that it used representative data from the general population and not only from a patient group or an insurance group, and investigated two modes of interview data. Linkage to registers made it possible to extract exact information on several types of health care in primary and secondary health care sectors for a long period of time. It also made it possible to measure health care utilisation in monetary terms by applying fees and charges as proxies for the costs of the services. The advantage of using a common unit of measurement is that different types of health care services can be aggregated. Furthermore, the cost of the service reflects the complexity of the procedure. A visit to the doctor is not just a visit. A surgical procedure is quite different from a check-up, for example, and this would not have been reflected in health care measured by the number of visits or by a dichotomised variable for use/no-use, which is the normal unit of measurement in many health surveys. Furthermore, several services are often provided during one visit. Hence the number of visits, or a dichotomised variable for health care utilisation, would not reflect the number of health care services.
Our study also has some limitations: although the analyses cover health care services from the primary care sector and hospital sector, some types of health care are not included in the analyses. For example, mental health care was not included with respect to hospital treatment. Furthermore, only services subject to partial or complete reimbursement by the Public Health Insurance system were included in the analyses. Private for-profit supply of some health care services does exist in Denmark, but plays a marginal although increasing role (e.g. <1% of hospital beds are private). For dentistry, however, a considerable share of services is not reimbursed at all, and these services are not included in the registers. For medicine, only prescription medicine in the primary health care sector is in the database at the individual level (and in FCHS, only prescription medicine entitled to reimbursement and bought from pharmacies in Funen County). In addition, medicine used in hospitals, and medicine purchased illegally over the internet is not covered by the database (medicine used in hospitals is included in hospital charges). Furthermore, even though the two sources of data are sampled from general population groups, the FCHS is only a sample from one Danish county, whereas DHIS covers all of Denmark (although the county of Funen is generally supposed to be representative of the Danish population).23,30 Some of the differences could be explained by regional differences.
The purpose of the paper was to quantify the overall non-response bias for general populations. However, the direction of bias for the different health care types points towards the possibility that non-response-bias could be further explained by socio-demographic characteristics (such as age and gender) or underlying socio-economic status variables that are correlated with health status and health behaviour in the population. In order to provide a deeper understanding of non-response bias, future research could investigate the possibilities of incorporating additional explanatory variables in the decomposition. Using an Oaxaca-type decomposition and appropriate statistical specifications, the decomposition could be further decomposed into bias components due to socio-demographic/economic differences across non-response groups and bias components due to different utilisation patterns across non-response groups.31 This would give new insight into the composition of non-response bias.
|
Conclusion |
|---|
|
TopAbstractMaterial and methodsResultsDiscussionConclusionAcknowledgementsReferences |
|---|
The two modes of interview (personal and telephone) produce the same pattern of non-response, but do not produce the same non-response bias with respect to health care utilisation. Refusal is the most frequent reason for non-response, whereas illness is the most important contributor to non-response bias.
There seems to be a trend towards decreasing participation rates in health interview surveys. However, the general validity of the results may not be threatened by the bias produced by non-response, although the underreporting caused by non-participating ill people is an important component that should be taken into consideration in surveys based on general populations.
|
Acknowledgements |
|---|
|
TopAbstractMaterial and methodsResultsDiscussionConclusionAcknowledgementsReferences |
|---|
The authors would like to thank Funen County and OPED for supplying data for the analyses of the Funen population. The Funen County Health Survey was partly funded by a research grant from The Health Insurance Foundation, Denmark (Sygekassernes Helsefond). The National Health Interview Survey was funded by the National Institute of Public Health and The Ministry of Interior and Health. The authors alone are responsible for the contents of the article.
Sources of support: The data collection was partly funded by a research grant from The Health Insurance Foundation, Denmark (Sygekassernes Helsefond), and by the National Institute of Public Health and The Ministry of Interior and Health.
Conflicts of interest: None declared.
Key points
|
|
References |
|---|
|
TopAbstractMaterial and methodsResultsDiscussionConclusionAcknowledgementsReferences |
|---|
1 Greene WH. Econometric Analysis (2000) 4. Upper Saddle River, NJ: Prentice Hall.
2 Perneger TV, Chamot E, Bovier PA. Nonresponse bias in a survey of patient perceptions of hospital care. Med Care (2005) 43:374–80.[CrossRef][Web of Science][Medline]
3 Cohen G, Duffy JC. Are nonrespondents to health surveys less healthy than respondents. J Off Stat (2002) 18:13–23.
4 Lamers LM. Medical consumption of respondents and non-respondents to a mailed health survey. Eur J Public Health (1997) 7:267–71.
5 Boersma F, Eefsting JA, van den Brink W, van Tilburg W. Characteristics of non-responders and the impact of non-response on prevalence estimates of dementia. Int J Epidemiol (1997) 26:1055–62.
6 Macera CA, Jackson KL, Davis DR, Kronenfeld JJ, et al. Patterns of non-response to a mail survey. J Clin Epidemiol (1990) 43:1427–30.[CrossRef][Web of Science][Medline]
7 Groves RM, Couper MP. Nonresponse in Household Interview Surveys (1998) New York: Wiley Interscience.
8 de Heer W. International response trends: results of an international survey. J Off Stat (1999) 15:129–42.
9 Kjoller M, Thoning H. Characteristics of non-response in the Danish health interview surveys, 1987–1994. Eur J Public Health (2005) 15:528–35.
10 Bowling B. Research Methods in Health: Investigating Health and Health Services (2002) 2nd. Buckingham, Philadelphia: Open University Press.
11 Frankfort-Nachmias C, Nachmias D. Research Methods in the Social Sciences (1996) 5th. London: Arnold.
12 Osler M, Schroll M. Differences between participants and nonparticipants in a population study on nutrition and health in the elderly. Eur J Clin Nutr (1992) 46:289–95.[Web of Science][Medline]
13 Vestbo J, Rasmussen FV. Baseline characteristics are not sufficient indicators of non-response bias follow up studies. J Epidemiol Community Health (1992) 46:617–9.
14 Paganini-hill A, Hsu G, Chao A, Ross RK. Comparison of early and late respondents to a postal health survey questionnaire. Epidemiology (1993) 4:375–9.[Web of Science][Medline]
15 Jackson R, Chambless LE, Yang K, et al. Differences between respondents and nonrespondents in a multicenter community-based study vary by gender ethnicity. The atherosclerosis risk in communities (ARIC) study investigators. J Clin Epidemiol (1996) 49:1441–6.[CrossRef][Web of Science][Medline]
16 Janzon L, Hanson BS, Isacsson SO, Lindell SE, et al. Factors influencing participation in health surveys. Results from prospective population study ‘Men born in 1914’ in Malmo, Sweden. J Epidemiol Community Health (1986) 40:174–7.
17 Drivsholm T, Eplov LF, Davidsen M, et al. Representativeness in population-based studies: A detailed description of non-response in a Danish cohort study. Scand J Public Health (2006) 34:623–31.
18 Reijneveld SA, Stronks K. The impact of response bias on estimates of health care utilization in a metropolitan area: the use of administrative data. Int J Epidemiol (1999) 28:1134–40.
19 Etter JF, Perneger TV. Analysis of non-response bias in a mailed health survey. J Clin Epidemiol (1997) 50:1123–8.[CrossRef][Web of Science][Medline]
20 Rupp I, Triemstra M, Boshuizen HC, Jacobi CE. Selection bias due to non-response in a health survey among patients with rheumatoid arthritis. Eur J Public Health (2002) 12:131–5.
21 Grotzinger KM, Stuart BC, Ahern F. Assessment and control of nonresponse bias in a survey of medicine use by the elderly. Med Care (1994) 32:989–1003.[Web of Science][Medline]
22 Korkeila K, Suominen S, Ahvenainen J, et al. Non-response and related factors in a nation-wide health survey. Eur J Epidemiol (2001) 17:991–9.[CrossRef][Web of Science][Medline]
23 Kjøller M, Rasmussen NK. Danish Health and Morbidity Survey 2000 ... & trends since 1989 (in Danish). (2002) Copenhagen: National Institute of Public Health.
24 Davidsen M, Kjoller M. The Danish health and morbidity survey 2000 – design and analysis. Statistics in Transition (2002) 5:927–42.
25 Gundgaard J, Sørensen J. Evaluation of the prevention strategy in Funen County: baseline survey on behaviour with respect to tobacco, alcohol, diet and exercise (in Danish). (2002) Funen County.
26 Statistics Denmark. StatBank Denmark, PRIS6. accessed 28-10-2003. www.dst.dk.
27 Hallas J. Conducting pharmacoepidemiologic research in Denmark. Pharmacoepidemiol Drug Saf (2001) 10:619–23.[CrossRef][Web of Science][Medline]
28 Section for Survey Statistics and Swedish Statistician Society. Standard for Non-response Calculations (in Swedish). (2005).
29 OECD. Purchasing Power Parities – Comparative Price Levels. In: Main Economic Indicators. (2006).
30 Bjerrum L, Sogaard J, Hallas J, Kragstrup J. Polypharmacy in general practice: differences between practitioners. Br J Gen Pract (1999) 49:195–8.[Web of Science][Medline]
31 Oaxaca RL. Male-female wage discriminatory behaviour and the estimation of wage discrimination. Int Econ Rev (1973) 14:693–709.[CrossRef]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  O. Ekholm, U. Hesse, M. Davidsen, and M. Kjoller The study design and characteristics of the Danish national health interview surveys Scand J Public Health, September 1, 2009; 37(7): 758 - 765. [Abstract] [PDF]
|
|
|
|
|
|
J. Sorensen, M. Davidsen, C. Gudex, K. M. Pedersen, and H. Bronnum-Hansen Danish EQ-5D population norms Scand J Public Health, July 1, 2009; 37(5): 467 - 474. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||