OUP user menu

Representativeness of participants in a cross-sectional health survey by time of day and day of week of data collection

Jennifer Mindell, Maria Aresu, Laia Bécares, Hanna Tolonen
DOI: http://dx.doi.org/10.1093/eurpub/ckr093 364-369 First published online: 29 September 2011

Abstract

Background: General population health examination surveys (HESs) provide a reliable source of information to monitor the health of populations. A number of countries across Europe are currently planning their first HES, or the first after a significant gap, and some of these intend offering appointments only during office hours and/or weekdays, raising concerns about representativeness of survey participants. It is important to ascertain whether personal characteristics of participants vary by time of day and day of week of data collection, in order to determine the association between time and day of interview and physical examination on the results of data collected in HES. Methods: Multivariable regression models were applied to national HES in England to examine socio-demographic and health variations in three combined day–time periods of interview and physical examination: weekday daytime; weekday evening; and weekend. Results: The characteristics of participants interviewed or visited by a nurse varied by both time of day and day of the week for age, ethnicity, marital status, income, socio-economic group, economic activity and deprivation. People seen during weekday working hours had higher rates of poor self-reported health, limiting longstanding illness and obesity, and higher alcohol consumption, BMI and systolic blood pressure; adjustment for socio-demographic characteristics eliminated or substantially reduced these differences. Conclusion: People responsible for planning surveys should be aware of participant preference for the timing of data collection and ensure flexibility and choice in times and days offered to optimise participation rates and representativeness.

Introduction

Population-level information about health and health-related risk factors in different population groups are needed to plan and target health promotion activities and health care. The only reliable source of such information is to conduct a general population health examination survey (HES), a population study in which randomly selected people answer a number of questionnaire items, have physical examinations and provide biological samples. In England, this type of survey has been conducted annually since 1991.

The representativeness of the results obtained depends on the participation rate: survey participants differ from those who do not participate.1–4 Non-participants tend to have worse health behaviours2,4 and excess mortality5–7 than participants. Therefore, low participation rates give biased results that do not provide a true picture about the population’s health. Where participants and non-participants do not differ from each other, the participation rate should ideally be around 70%;8,9 where non-participation is not random, higher participation rates are needed for accurate estimates.10 In recent national HESs in Europe, the participation rate varied from 21% to 85%.11 Globally, participation rates in population surveys has been declining,12,13 diminishing the representativeness of the results.

For the working age population, it may be difficult to take part in a survey during the day (working hours). Previous American studies showed that people are best contacted at home in the evenings after 17:00 and during the day at weekends.14,15 At the weekend, potential participants were most likely to be at home during the morning and afternoon hours on Saturday and in the evening hours on Sunday.14 However, previous studies were based on telephone interviews, while HESs require face-to-face contact and reported information only for the population as a whole: findings may differ by socio-demographic group or health status.

Until recently, few HESs have been conducted at a national scale.11 Several countries across Europe are currently planning their first general population HES or the first after a significant gap. Some intend offering appointments only during office hours and/or only on weekdays. Given international concern about falling response rates11–13 and under-representativeness of survey participants,1–7 we have used data from a well-established general population HES in England to examine representativeness of respondents by time and day of data collection. First, this article considers whether, and how, personal demographic and socio-economic characteristics of participants vary by time of day and day of week of data collection. Second, we have ascertained the effect of this variation on the results of subjective and objective health and risk factor data collected at personal household visits.

Method

Participants and data collection

The Health Survey for England (HSE) is an annual, cross-sectional, general population HES. Each year, a new nationally representative random probability sample is selected using a two-stage process. The sampling frame is the small address Postcode Address File, which lists all non-commercial addresses in the country. The primary sampling unit, postcode sectors (each containing around 3000 addresses, except in rural areas) are randomly selected after stratification by region and the percentage of non-manual (‘white collar’) households. A fixed number of addresses are then selected within each sampled postcode sector. The sample is designed to be nationally representative in each quarter of the year to avoid seasonal effects.

An advance letter was sent to each selected address providing introductory information about the survey and notifying the household that an interviewer would visit. All adults aged ≥16 years (up to a maximum of 10) within the household were eligible to participate, as well as up to two children aged 0–15 years (randomly selected). Interviewers chose when they first visited their addresses; some chose weekday working hours, but some chose evenings or weekends, which can be more productive. Interviewers were expected to visit the address on at least four occasions, including different times of the day and different days of the week (including weekends), before logging the address as ‘unable to make contact’. Interviewers made an average of 7.6 calls at non-contact addresses.

Those living at the sampled address who agreed to take part were interviewed using CAPI, Computer-Aided Personal Interviewing and were asked to answer a self-completion questionnaire and have their height and weight measured [to calculate body mass index (BMI; weight/height2)]. When convenient for participants, the interviewer conducted the interview(s) immediately. In other cases, the interviewer recruited the household and arranged to return to conduct the interviews at a more convenient time, or to interview other eligible members of the household who were not present at the time.

Towards the end of the interview, participants were given information about a nurse visit. Those who agreed to see a nurse were then given an appointment at a mutually convenient time for the survey nurse to call. The nurse visit included a second, short interview; more physical measurements (waist and hip circumference, blood pressure) taken using standard protocols;16 and biological sampling (urine and blood samples). Approval was obtained from the London Multicentre Research Ethics Committee. More details of the sampling, recruitment, data collection and measurement protocols have been reported elsewhere.16

Variables used

We used self-reported marital status, educational level and ethnicity. Respondents’ socio-economic status (using National Statistics Socio-Economic Classification, NS-SEC17) was analysed using the three category version: professional and managerial; intermediate; and routine and manual. Self-reported economic status was categorized into ‘in employment’; ‘ILO unemployed’ (not in paid work, available to work and actively seek paid employment); retired and ‘other unemployed’ (not actively seeking work, e.g. home-makers, students). Household income was adjusted for the number of adults and children living in the home and analysed as tertiles of equivalized household income. Area deprivation was measured using the Index of Multiple Deprivation (IMD2004), a score of deprivation applied at small geographical levels. IMD2004 is based on a weighted average of 37 indicators across 7 domains: income deprivation; employment deprivation; health deprivation and disability; education, skills and training deprivation; barriers to housing and services; living environment and crime.18 This was geocoded to participants’ postcode and analysed in tertiles.

Information collected at the interview included questions on self-reported general health, dichotomized into ‘very good/good’ vs. ‘fair/bad/very bad’ for analysis and on longstanding illness. Those that responded positively were then asked whether the illness/es or disability/disabilities limit their activity in any way, with the responses dichotomized as ‘limiting longstanding illness’ or ‘non-limiting or no longstanding illness’. Information was collected on lifestyle risk behaviours including smoking status (never smoker, ex-smoker or current smoker) and alcohol consumption. The amount consumed on the heaviest drinking day in the previous 7 days was dichotomized into exceeding the recommended daily limit (>4 U for men, >3 U for women) or less than that (including those who drank no alcohol in that period). The questionnaires are available in the survey report.16

Analysis

The purpose of the present study was to examine demographic, socio-economic and health differences, depending on time and day of interview, to inform the planning of other national surveys. Given this, we did not apply non-response weighting to the analyses.

The proportion of each subgroup seen by an interviewer or a nurse was assessed by time of day and by day of week for sex, age group, ethnicity, marital status, educational achievement, socio-economic classification, equivalized household income, economic status and area deprivation. Categories of demographic and socio-economic variables with small numbers were combined with another category. For example, ‘mixed ethnicity’ was added to ‘other ethnic group’.

We created three combined day and time periods: Monday–Friday before 17.00 (‘weekday daytime’); Monday–Thursday 17.00 onwards (‘weekday evening’) and Friday 17.00 onwards to Sunday (‘weekend’). The analyses were conducted for each of these periods. We originally planned to consider weekend daytime and weekend evenings separately but the number in each was small so were combined.

In the first stage, descriptive procedures were conducted to investigate the associations between day and time period and demographic and socio-economic variables. We then used univariable logistic regression to assess odds ratio (OR) associated with day and time period and each predictor (Supplementary table S1). Independent covariates showing statistically significant associations with the dependent variable were included in the final multivariable model.

In the second stage, exemplar variables were chosen to investigate the effects of day and time period on self-reported health and disease status (self-reported general health, longstanding illness); self-reported lifestyle risk behaviours (current cigarette smoking prevalence, alcohol consumption above recommended limits); objective physical measurements (BMI, waist circumference, blood pressure); and objective laboratory analytes (total cholesterol, glycated haemoglobin). To assess variation in health-related variables by day–time of the interviewer (self-reported information and BMI) or nurse visit (waist circumference, blood pressure and blood analytes), we conducted multivariate regression models (logistic regression for categorical outcomes and linear regression for continuous variables). Unadjusted results are presented, followed by analyses adjusting for demographic and socio-economic variables included in the final multivariable regression from the first stage. All analyses were conducted in Stata11.

Results

Response rates

Overall, 68% of the sampled eligible households participated in HSE 2006. Within co-operating households, 88% of the adults (aged ≥16 years) were interviewed (14 142 adults) and 66% of the adults in co-operating households saw a nurse (60 and 45%, respectively, of the estimated adult population sampled). Late afternoon/early evening on Monday–Thursday was the most likely time to see an interviewer or nurse (see Supplementary figure S1).

Characteristics of participants by day and time

Several characteristics of participants interviewed and visited by a nurse varied by both time of day and day of the week. These included age, ethnicity, marital status, education, income, NS-SEC, economic activity and area deprivation. Gender differences were observed by time of day, but not by day of week. Older people were most likely to be seen on a weekday during the day, as were those who were retired or otherwise economically inactive but not unemployed, those in routine–manual occupations and those with no educational qualifications (table 1). Evening visits were particularly useful for seeing those under the age of 65 years, cohabiting, with A level (advanced high school) education or above, and higher income or in employment. Weekend evenings were particularly important for visiting participants from non-White ethnic groups with Asian participants more likely to be seen during weekend days.

View this table:
Table 1

Characteristics of HSE 2006 participants by day-time period of interviewer or nurse visit

InterviewerNurse visit
CharacteristicsnMon–Fri before 17:00 (%)Mon–Thu 17:00 onwards (%)Fri 17:00 onwards and weekend (%)nMon–Fri before 17:00 (%)Mon–Thu 17:00 onwards (%)Fri 17:00 onwards and weekend (%)
All (aged ≥ 16 years)13 90054.330.415.310 48947.736.116.3
Sex
    Male621250.133.416.6470844.238.417.4
    Female768857.728.114.2578150.534.215.3
Age (years)
    16–34338340.740.718.6235230.448.521.1
    35–64726447.535.017.4569439.742.517.8
    ≥65325383.59.57.1244382.99.08.1
Marital status
    Single256345.634.919.4174137.343.719.1
    Married or civil partnership756754.730.714.6587147.735.616.7
    Separated or divorced119659.726.014.392455.330.214.5
    Widowed118179.910.79.384979.711.39.0
    Cohabiting139041.941.316.8110332.850.716.5
Index of multiple deprivation
    Least deprived tertile274951.935.212.9218344.440.015.6
    Middle tertile302655.030.414.6228949.137.813.2
    Most deprived tertile300752.431.116.5235646.837.415.8
Equivalized household income tertile
    <£16 852410669.419.211.4315362.523.613.7
    £16 852–£25 114231654.031.614.4185350.933.715.4
    ≥ £25 114473537.743.518.8375332.049.218.8
Education (aged ≥18 years)
    Degree equivalent or above266140.8538.0321.12203134.843.521.8
    A level or equivalent/higher education below degree333345.937.7416.35259939.444.416.2
    GCSE or equivalent or foreign qualification398854.0931.5214.39304346.737.415.8
    None387070.8817.9111.21280765.821.412.8
Economic status
    In employment749038.142.519.4568929.950.419.8
    ILO unemployed57249.634.116.342037.643.119.3
    Retired349483.49.96.7266282.39.87.9
    Other economically inactive230463.821.814.4171055.727.516.8
Ethnicity
    White12 61055.630.613.9969548.936.214.9
    Mixed10944.033.022.98542.431.825.9
    Asian or Asian British68641.129.030.242632.434.033.6
    Black or Black British32938.632.528.920228.235.236.6
    Chinese or other ethnic group13145.822.931.37935.434.230.4
  • Tests on association using Pearson's chi-squared were carried out: results showed highly significant associations between predictor and day–time interview variables.

Results of the univariable regressions showed very similar findings for both interviewer and nurse data (Supplementary table S1). The day–time period in which the visit occurred was associated with age, sex, marital status, ethnicity, education level, income, economic activity and area deprivation.

Results of multivariable regression with mutual adjustment showed that, when compared with weekday daytime visits, participants interviewed during weekday evenings were more likely to be male; employed; widowed or married/in civil partnership; Black or Asian; with higher household income; and living in an area in the middle deprivation tertile (table 2). After adjusting for the other predictor variables, older people were most likely to be seen during the day on weekdays (odds of being seen per 10-year age increase, RRR = 0.78 for weekday evenings, RRR = 0.87 at the weekend compared with day time on weekdays). Those interviewed at the weekend had similar socio-demographic characteristics to participants seen during weekday evenings except for marital status.

View this table:
Table 2

Multivariable regression of association of demographic and socio-economic variables with day-time period

Time and day of interview
Mon–Thu 17:00 onwardsaFri 17:00 onwards and weekenda
RRR (95% CI)RRR (95% CI)
Sex
    Male11
    Female0.80 (0.73–0.88)0.82 (0.73–0.92)**
    Age0.98 (0.97–0.98)0.99 (0.98–0.99)
Marital status
    Single11
    Married or civil partnership1.21 (1.04–1.39)*0.92 (0.77–1.10)
    Separated or divorced1.03 (0.83–1.26)0.87 (0.68–1.12)
    Widowed1.27 (0.95–1.69)1.15 (0.83–1.62)
    Cohabiting1.16 (0.98–1.39)0.86 (0.69–1.07)
Economic activity
    In employment11
    ILO unemployment0.58 (0.46–0.73)0.54 (0.40–0.73)
    Retired0.26 (0.22–0.31)0.30 (0.24–0.37)
    Other0.42 (0.37–0.49)0.55 (0.46–0.65)
Equivalized household income
    Lowest tertile (<£16 852)11
    Middle tertile (£16 852–£25 114)1.62 (1.41–1.85)1.50 (1.26–1.77)
    Highest tertile (≥ £25 114)2.30 (2.04–2.59)2.16 (1.86–2.50)
Area deprivation
    Least deprived tertile11
    Middle tertile1.06 (0.94–1.19)1.31 (1.13–1.52)
    Most deprived tertile0.84 (0.75–0.94)**1.05 (0.92–1.21)
Ethnicity
    White11
    Asian or Asian British1.44 (1.13–1.85)**3.29 (2.58–4.22)
    Black or Black British1.57 (1.12–2.20)**2.97 (2.10–4.20)
    Other0.91 (0.62–1.32)2.28 (1.58–3.30)
  • *P < 0.05, **P < 0.01, P < 0.001.

  • a: Monday–Friday before 17:00 = reference value

Associations with the time and day of the nurse visit were very similar. Nurses were more likely to visit married participants and those in civil partnerships in the evening or weekend and those who were cohabiting in the evening (Supplementary table S2).

Variation in health of participants by day and time

Unadjusted coefficients show that participants interviewed or having measurements taken during the day on weekdays were substantially less healthy than those interviewed or visited by a nurse at other times (tables 3 and 4). Those seen in the daytime on weekdays had higher rates of poor self-reported health, limiting longstanding illness and higher mean BMI and systolic blood pressure but were least likely to exceed the recommended daily limits for alcohol. Current cigarette smoking, mean glycated haemoglobin and total cholesterol levels were not related to the day–time period. Mean waist circumference was significantly higher among those seen on weekday evenings, but was not associated with being seen at weekends, in comparison with weekday daytime visits.

View this table:
Table 3

Multivariable regression of self-reported health and health-related behaviours, by day–time period, crude and adjusted for demographic and socio-economic variables

Poor self-rated healthLimiting longstanding illnessCurrent cigarette smokingExceeded recommended alcohol intakea
CrudeAdjustedbCrudeAdjustedbCrudeAdjustedbCrudeAdjustedb
OR (95% CI)OR (95% CI)OR (95% CI)OR (95% CI)OR (95% CI)OR (95% CI)OR (95% CI)OR (95% CI)
Mon–Fri before 17:00c11111111
Mon–Thu 17:00 onwards0.30 (0.25–0.37)0.81 (0.64–1.02)0.40 (0.36–0.43)0.89* (0.79–0.99)1.03 (0.94–1.13)0.96 (0.85–1.07)1.92 (1.77–2.08)1.17** (1.06–1.30)
Friday 17:00 onwards and all weekend0.46 (0.37–0.57)0.86 (0.65–1.12)0.51 (0.45–0.57)0.97 (0.84–1.24)1.12 (0.99–1.25)1.09 (0.94–1.25)1.31 (1.18–1.46)0.86 (0.79–1.02)
  • *P < 0.05, **P < 0.01, P < 0.001.

  • a: Drank more than 4 units (male) or 3 units (female) of alcohol on heaviest drinking day in the previous week

  • b: Adjusted for age, sex, marital status, ethnicity, economic activity, income and area deprivation

  • c: Reference category: Monday–Friday before 17:00

View this table:
Table 4

Multivariable regression of objective measures of health, day-time period, unadjusted and adjusted for demographic and socio-economic variable

BMI (kg/m2)Waist circumference (cm)Systolic blood pressure (mm Hg)Total cholesterol (mmol/l)Glycated haemoglobin (%)
Unadjusted coeff (SE)Adjusteda coeff (SE)Unadjusted coeff (SE)Adjusteda coeff (SE)Unadjusted coeff (SE)Adjusteda coeff (SE)Unadjusted coeff (SE)Adjusteda coeff (SE)Unadjusted coeff (SE)Adjusteda coeff (SE)
Monday to Friday before 17:00b1111111111
Monday to Thursday 17:00 onwards0.368** (0.11)0.234 (0.12)1.537** (0.47)0.209 (0.53)−7.871 (1.10)−0.256 (1.32)0.056 (0.07)−0.008 (0.08)−0.066 (0.07)−0.040 (0.08)
Friday 17:00 onwards and all weekend−0.570 (0.14)−0.040 (0.15)−1.304 (0.61)−0.212 (0.65)−8.298 (1.43)−2.440 (1.61)0.130 (0.09)0.131 (0.10)(0.094) (0.09)0.170 (0.10)
  • **P < 0.01, P < 0.001.

  • a: Adjusted for age, sex, marital status, ethnicity, economic activity, income and area deprivation

  • b: Reference category

Tables 3 and 4 also show associations of day–time period with health-related variables after adjusting for the significant socio-economic and demographic variables shown in table 2 (interviewer visit) and Supplementary table S2 (supplementary data; nurse visit). After adjustment, the differences in alcohol consumption and limiting longstanding illness were attenuated for weekday evenings and became non-significant for participants seen at the weekend. The day–time effects were eliminated, or greatly reduced and non-significant, for self-reported health, BMI and systolic blood pressure for both weekday evenings and weekends.

Discussion

This study examined socio-demographic and health differences depending on time and day of interview. Our results show that there are marked demographic and socio-economic differences across participants by the time and day of data collection in the HSE. These differences affect results for self-reported health, health behaviour and physical measurements, although not for the two biological analytes examined.

Strengths and limitations

This article has two main strengths. First, the data analysed come from one of the largest general population HESs in the world. It includes the largest number of interviews and nurse visits both in day–time and evening and weekdays and weekends,11 so results are unlikely to be due to chance. Second, the HSE has been running annually from 1991 to 2010, so this article draws on the considerable experience among both the fieldstaff and the survey organisers. Both the researchers planning the survey and the field staff visiting participants are experienced at maximizing response rates through survey design and personal skills. Thus, variations in participants’ characteristics by day and time is more likely to reflect underlying preferences.

The data on day and time of the start of the household interview includes people who agreed to be interviewed opportunistically at the time of the interviewer’s call, as well as from those who arranged appointments for the interviewer to return to conduct the interview when it was more convenient for participants. However, virtually all the nurse visits were arranged in advance for mutually convenient times (particularly paying attention to participants’ preferences). The times and days of the nurse visit, therefore, reflect participants’ preferences to a greater extent than the interview data but the profiles hardly differed. This lack of difference in day–time of interview and nurse visits is not surprising: the times when it would be convenient for participants to have a pre-arranged visit are generally the times when they could see an interviewer who arrived by chance. An interviewer will often visit one person opportunistically while arranging to visit again to see other members of the household, whereas the nurse usually arranges to visit adults in a household on the same visit. This probably explains the stronger association between marital status and nurse than interviewer visit.

This study cannot predict the effects on data collection of limiting times and days in which data collection can occur: an unknown proportion of those being seen in HSE outside ‘office hours’ would probably be willing to be seen at a different time and/or day if that were the only option. The HSE collects all data from home visits; most HESs conduct the examination in a central clinic. There are differences between in-home and clinic appointments over and above time and day, relating to ability and willingness to travel and cultural differences in some countries in the acceptability of allowing strangers into the home. However, it is likely that a significant proportion of individuals would not be available at other times or days, particularly if it meant taking time off work at a time of recession and job insecurity. There may not be a problem with availability during ‘office hours’ in cultures in which an official government-endorsed invitation is taken very seriously, or where employers are likely to allow employees to keep appointments with a survey team during working hours without adverse consequences. However, this will often not be the case.

One study limitation is the lack of information on, and preferences of, those who did not participate. Although response rates are lower than is desirable, ineligible people (e.g. lacking mental capacity to give informed consent) and those who could not be contacted are included as non-responders in HSE (some studies in Europe exclude such individuals from the denominator19). The main limitation of this study is that the findings are not necessarily generalizable to other countries, where HES examinations are conducted at fixed clinics rather than during home visits. However, home visits are an option in many countries for those who cannot attend the clinic. Some of these constraints will also apply to people who were asked to attend a central survey location, albeit to a different, and perhaps for some groups a lesser, extent. Cultural differences across Europe affect how people accept home visits and value the costs and benefits of attending an examination centre, and how employers allow participation in surveys during work time.

It is important for those planning surveys to be aware of participant preferences about the timing of data collection and to ensure flexibility and choice in times and days offered, to optimize response rates and representativeness. For most data, items collected in a health survey, e.g. smoking status, variation by day–time reflects the characteristics of participants interviewed at those times, not real temporal variation. This variation is likely to go unnoticed but can be dealt with by adjusting for participant characteristics.

Similarly, it is important that non-response weighting adjusts for systematic differences between the demographic and socio-economic characteristics of participants compared with those sampled or the general population, to ensure that results do not over-represent those who are easily available during the day, which includes those who are less healthy.

Previous studies of response days/times examined participation in telephone surveys;14,15 this study adds knowledge regarding personal household visits. If limited times are offered, the response rates will probably be lower. Our results should help inform those arranging general population surveys to consider how survey options could optimize representative participation and by those using survey results to check for bias in the results.

Supplementary data

Supplementary data are available at EURPUB online.

Acknowledgements

We thank the HSE participants, Natcen’s interviewers and nurses, and colleagues at UCL, NatCen and the NHS Information Centre for their help with the survey, and Rachel Craig for helpful comments on the article. J.M. and H.T. had the initial idea; M.A., L.B. and J.M. designed the analyses, which were conducted by all four authors, particularly L.B. and M.A.; J.M. and H.T. wrote the first draft; all authors contributed to interpreting the results and writing the final draft. All authors approved the final manuscript.

Conflicts of interest: None declared.

References

View Abstract