The European Journal of Public Health Advance Access originally published online on September 8, 2005
The European Journal of Public Health 2005 15(5):448-453; doi:10.1093/eurpub/cki146
Injuries |
Determining the cost effectiveness of a smoke alarm give-away program using data from a randomized controlled trial
Laura Ginnelly1, Mark Sculpher1, Chris Bojke1, Ian Roberts2, Angie Wade3 and Carolyn Diguiseppi4
1 Centre for Health Economics, University of York, York, UK
2 London School of Hygiene and Tropical Medicine, London, UK
3 Centre for Paediatric Epidemiology and Biostatistics, Institute of Child Health, University College London, London, UK
4 Department of Preventive Medicine and Biometrics, University of Colorado Health Sciences Center, Denver, CO, USA
Correspondence: Laura Ginnelly, MSc, Centre for Health Economics, University of York, York YO10 5DD, UK, tel: +44 1904 433643, fax: +44 1904 433644, e-mail: lg116{at}york.ac.uk
Received October 15, 2003, accepted February 17, 2004
 |
Abstract |
|---|
 Top Abstract MethodsResultsDiscussionReferences |
|---|
Background: In 2001, 486 deaths and 17 300 injuries occurred in domestic fires in the UK. Domestic fires represent a significant cost to the UK economy, with the value of property loss alone estimated at £375 million in 1999. In 2001 in the US, there were 383 500 home fires, resulting in 3110 deaths, 15 200 injuries and $5.5 billion in direct property damage. Methods: A cluster RCT was conducted to determine whether a smoke alarm give-away program, directed to an inner-city UK population, is effective and cost-effective in reducing the risk of fire-related deaths/injuries. Forty areas were randomized to the giveaway or control group. The number of injuries/deaths and the number of fires in each ward were collected prospectively. Cost-effectiveness analysis was undertaken to relate the number of deaths/injuries to resource use (damage, fire service, healthcare and giveaway costs). Analytical methods were used which reflected the characteristics of the trial data including the cluster design of the trial and a large number of zero costs and effects. Results: The mean cost for a household in a give-away ward, including the cost of the program, was £12.76, compared to £10.74 for the control ward. The total mean number of deaths and injuries was greater in the intervention wards then the control wards, 6.45 and 5.17. When an injury/death avoided is valued at £1000, a smoke alarm give-away has a probability of being cost effective of 0.15. Conclusions: A smoke alarm give-away program, as administered in the trial, is unlikely to represent a cost-effective use of resources.
Keywords: cluster-randomized trial, cost effectiveness analysis, smoke alarm give-away program
In 2001, local authority fire services attended over 1 million fires or false alarms in the UK, 64 600 of which were domestic fires.1 The British Crime Survey (BCS)2 also estimates that only 13–26% of domestic fires are reported to the Fire Service. In 2001 in the UK, it was estimated that 486 people died in fires in the home, and approximately 17 300 people were injured.1 There is a steep social class gradient in the risk of fire-related deaths among children aged 1–15. In the UK, this risk is 15 times higher in the lowest income groups compared to the highest income groups.3 The risk of having a residential fire is 25% higher in inner city areas compared to non-inner city areas.4
Fires detected by smoke alarms tend to be discovered more rapidly and are associated with a reduced risk of death (three versus nine deaths per 1000 fires) and non-fatal casualty (137 versus 178 per 1000 fires),5 and less property damage.6 For several years, the UK government has conducted annual publicity campaigns to increase the percentage of households that have a fitted operational smoke alarm. The National Community Fire Safety Centre estimated that, in 1998, 82% of households owned a smoke alarm. However, materially deprived households are less likely to own a smoke alarm.7
An observational study, conducted in Oklahoma City, distributed smoke alarms to target areas using various give-away strategies.8 Results of this study showed an 80% decline in serious injuries in the follow up period. The non-randomized design of this study, however, may have biased the results of the study.9 Furthermore, it is unclear whether the results of this study can be generalized from a small mid-western US city to an urban, materially deprived, ethnically diverse, British population.
A randomized-controlled trial was conducted to determine whether a smoke alarm give-away program, directed to such a population, is effective and cost-effective in reducing the risk of fires and fire-related deaths/injuries. This showed that giving away free smoke alarms did not reduce the number of fire related injuries or deaths.9 Despite these results, the presence of a smoke alarm in a house may reduce the cost of a fire, as the Fire Service may be called to the scene quicker. If this is the case, the program may still be cost-effective. The net cost of the program will depend on the cost of giving away alarms and any savings generated, as a result of a reduction in the incidence and/or severity of fires. Cost effectiveness will involve relating this net cost to the effectiveness of the program in terms of deaths/injuries avoided.
|
Methods |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
Intervention design
The give-away program, called the ‘Let’s Get alarmed!' Initiative,10 distributed 20 050 smoke alarms, batteries and fire safety brochures. Fitting of alarms by trained staff was offered to all recipients, although only 8% of households used this service.
Trial design
Full details of the trial design have been published elsewhere.9 A cluster-randomized controlled trial was conducted in the inner London Boroughs of Camden and Islington. Households were categorized into administrative units (wards), based on geographical location. Wards included were those with above average material deprivation, defined as a Jarman Under Privileged Area score 
20.11 Wards were pair matched according to Jarman score. Random allocation to either intervention (smoke alarm give-away) or control (no smoke alarm give-away) was then undertaken within the matched pairs.
Forty wards, averaging 3683 households in each, were randomized to intervention and control status. Free smoke alarms and fire safety information was distributed to intervention wards. The aim was to increase the proportion of households with smoke alarms in the intervention wards from 47% to 72% (national prevalence, as of 1996).10 The number of alarms distributed and costs of the program from a societal perspective were recorded.10
Data collected in the trial
Details relating to the number of injuries and deaths, the number of fires and resource use, in each ward, were collected prospectively in intervention and control wards for between 22.9 and 25 months (mean 23.9). The number of fires occurring in each ward was restricted to those reported by the Fire Service. Details of each fire were recorded, which facilitated subsequent costing. These data included room of ignition, details of spread of fire, length of time spent at scene by fire service, police presence and number of persons involved in the fire. Injuries were defined as those that resulted in an accident and emergency (A&E) attendance, hospitalization or death. These data were collected from local A&E department registers, which included details of treatments given, referrals to other departments, and hospitalizations and from hospital case records. Deaths occurring as a result of fire were recorded in the A&E and follow-up forms or in coroners' reports. Data necessary to cost ambulance attendance were routinely recorded by the ambulance service. Full details of the trial have been published elsewhere.12
Cost analysis
Resource use data collected in the trial were used to evaluate the net cost of the give-away program, that is, the cost of providing alarms to appropriate households, net of any savings resulting from fewer and/or less serious fires in the smoke-alarms wards. The cost analysis adopted a societal perspective. Unit cost data was taken from a number of sources and is detailed in table 1. There were four components to the cost analysis, each of which is detailed below.
|
The give-away program
The total cost of the give-away was £157 823, including the one-year reminder postcards.10 A total of 20 050 alarms were distributed to 73 399 intervention households; therefore, the mean cost per household of the give-away program was £2.15.
Fire service and police
Fire service costs were calculated by multiplying the time spent by the Fire Service at the scene of a fire, by a cost per pump-hour.13 For fires with more than four pumps present, a Fire Investigation Unit is required, which was costed at £57 per hour.13 In cases where it was apparent that injuries had occurred in the fire, and more than four pumps were used, the Assistant Divisional Officer (ADO) was required to attend, costed at £47 per hour.13 Police presence at the scene of a fire was assigned a fixed cost of £110 (Strathclyde Fire Brigade, personal communication, 2000).
Property damage
Costs of property damage were estimated from the 1996 BCS.2 Fires were defined as one of four types: (i) inside/no spread; (ii) inside/spread; (iii) outside/spread; and (iv) outside/no spread. This definition is consistent with the categorization used in the BCS from which cost data were available. Respondents to the BCS, who had experienced a fire in the last year, indicated into which of the four categories their fire fell and provided an estimate of the cost of property damage. These responses were used to derive a mean cost of damage for the four types of fire. Insurance administration costs were estimated as 51.5% of the gross cost of the claim.14
Health service
The duration of ambulance journeys to A&E was measured from the time of the first call to the time the ambulance returned to base, and costed based on a published source.15 Helicopter emergency ambulance journeys were assigned an average cost.16 All cases that presented at A&E were assigned a cost of £41,14 representing the average cost of an A&E visit in the UK. Hospital admission costs, follow-up costs, visits to family doctors and surgery were added based on data collected from medical notes.
Deaths resulting from fires also incurred a cost (£1730) representing the average cost of a funeral, coroners' and autopsy costs (Cooperative Funeral Services, personal communication, 2000).
Effectiveness
The measure of effect for the cost-effectiveness analysis was fire-related deaths and injuries over the follow-up period. These are based on the incidence of injuries related to fires resulting in attendance at an A&E department, hospitalization, or death during the follow-up period.
Cost-effectiveness
Two measures of cost-effectiveness are derived from trial data. Firstly, standard decision rules17 are used to establish the incremental cost-effectiveness of the give-away program based on mean differential costs and effects. Unless one arm of the trial dominates the other (less costly and more effective), an incremental cost-effectiveness ratio is calculated to indicate the additional cost of each death/injury avoided within the more effective arm of the trial.
Secondly, the uncertainty surrounding mean expected costs and deaths/injuries can be represented as a cost-effectiveness acceptability curve.18 These show the probability that the intervention is cost-effective for different threshold values that the decision maker might be willing to pay to avoid a fire-related death/injury. When this threshold is set to £0, this is equivalent to reporting the probability that the giveaway program generates societal cost savings because, at this threshold, avoiding deaths/injuries is given no value. Threshold values ranging between £0 and £50 000 per death/injury averted are used in the analysis.
Statistical analysis
Particular characteristics of the trial required non-standard approaches to statistical analysis. Firstly the cluster-randomized design of the trial. As the unit of analysis was the individual household, whilst the unit of randomization was the administrative ward, statistical methods were required to allow for the potential correlation in costs and effects between households within wards. The second feature of the trial was the large number of households within the trial that did not experience a fire during the two years of follow-up, that is, they were characterized by zero costs and effects. This generated heavily right-skewed cost and effect distributions making the assumption of normality invalid.
To deal with these characteristics, data were analysed using an econometric model. Because of the large proportion of non-events, a two-stage model was used.19 The first stage modelled the probability of a fire occurring using a logit model, and the second stage modelled the expected cost and numbers of injuries/deaths conditional on a fire occurring. Since the cost and injury data are not normally distributed and because predictions of these outcomes are required, they were modelled using a generalized linear model with the appropriate best fitting link function and distribution (gamma for cost and Poisson for injuries).19 The expected costs and injuries for a control and intervention household were calculated by multiplying the relevant probability of a fire by the appropriate expected cost/injuries given that there has been a fire. For both parts of the model, the independent variables were whether the ward/household was in the give-away or control group, the borough in which the ward was located and Jarman score. A robust estimator of variance20 was used to overcome the clustered nature of the trial.
To generate cost-effectiveness acceptability curves, the following methods were used. Firstly, household-level costs and effects were placed onto a single monetary scale using the principles of net monetary benefit (NMB),21,22 which values effects in monetary terms according to a threshold willingness to pay per unit of effect. Secondly, distributions of mean NMB for the control and intervention groups were calculated using Monte Carlo simulation allowing for relevant correlation between parameters. The third stage involved calculating the proportion of the simulations for which the mean NMB was higher for the intervention group than for the control. This represents one point on the cost-effectiveness acceptability curve corresponding to a particular threshold willingness to pay value for a unit of effect. The full curve was plotted by varying this threshold and re-running the simulation.
|
Results |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
Fires
The predicted expected number of fires, for intervention and control wards, can be seen in table 2. This is based on an average ward consisting of 3686 households, with an average Jarman score of 34.55. Intervention households have a higher probability of having a fire than control household over 2 years: 0.007885 compared to 0.007067.
|
Resource use measured in trial
Table 3 shows resource use associated with fire events in the trial. Control households experiencing a fire were more likely to call the Fire Service, and had fires resulting in more extensive property damage (inside fires spreading to more than one room).
|
Deaths and injuries
Injuries and deaths were more common in fires that occurred in intervention households compared to control households (table 2). The expected number of deaths and injuries in an average ward over the 24-month follow-up period was 6.45 and 5.17 in intervention and control wards, respectively. This equates to an expected number of injuries/deaths per household experiencing a fire of 0.222 and 0.199 in the intervention and control households, respectively. The expected number of deaths or injuries due to fire for a household is, therefore, 0.0014 and 0.0018 for control and intervention households, respectively (table 2).
Costs
For those households experiencing a fire in the follow-up period, the expected cost of that fire was lower in the intervention households (£1345) than in the control households (£1520) (table 2). Property damage contributed the most to the cost of a fire: £1008 for control and £887 for intervention households. Extinction costs were also lower in intervention households: £282 and £396 for intervention and control, respectively. However, mean injury costs were higher in intervention households experiencing fires: £151 compared to £105 in control households.
When the cost of the give-away program, at £2.15 per household, is not included, the mean cost of fires across all intervention wards is marginally lower than that in control wards: £10.61 compared to £10.74. When the cost of the give-away program itself is added to the intervention wards, on the basis of mean costs and outcomes, the give-away program has both higher overall expected costs and more deaths/injuries; that is, it is a dominated intervention.
Reflecting uncertainty in cost-effectiveness
To examine the uncertainty in mean costs and effects, a cost-effectiveness acceptability curve is shown in figure 1. When the willingness to pay for avoidance of a death/injury is £0, the probability that the give-away program is cost-effective (cost saving) is 18%. As the willingness to pay increases, the probability that a smoke alarm give-away program is cost effective decreases because the intervention group experienced more deaths and injuries. When an averted injury/death is valued at £1000, a smoke alarm give-away has a probability of being cost effective of 0.15. When an averted injury/death is valued at £50 000, the probability of a smoke alarm give-away program being cost effective is 0.11.
|
|
Discussion |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
Overall, the results of this study indicate that, as used in the ‘Let’s Get alarmed!' Initiative,10 a smoke alarm give-away program is not a cost-effective use of societal resources. From a policy perspective, however, it is important to emphasize that the results of this study do not suggest that a smoke alarm give-away program can never be cost-effective. The results shown in the trial probably reflect the fact that too few alarms were distributed, installed and maintained over the study period. Inspection of a random sample of council-owned homes suggested that the prevalence of functioning smoke alarms 12–18 months after the program was nearly identical in intervention and control households.23 Hence the distribution of 20 000 free alarms in an area of 73 400 homes was insufficient to increase the overall prevalence of functioning alarms. A further trial24 was undertaken to assess whether providing different types of smoke alarm (from different power sources) can increase the prevalence of functioning alarms. This trial found that nearly half of installed, battery-operated alarms did not work when tested 15 months after installation, and that alarm type and power source were important determinants of function.
There are a number of methodological issues emerging from this study. Firstly, this is the first randomized-controlled trial of a smoke-alarm give-away program, and shows that it is feasible to use experimental designs to evaluate important public health initiatives. The results seen in the trial are markedly different to those in the observational study conducted in Oklahoma upon which the give-away intervention was modelled,8 which concluded that, as well as 20 fewer fatalities and 24 fewer non-fatal injuries, the intervention generated savings of $15 million from a societal perspective.25 In part, this may reflect population differences, for example, fewer owner-occupiers, fewer English speakers and a materially poorer, less literate population in the UK study. However, the non-experimental design in Oklahoma City might also explain the differences in results.
A second point is that the use of a regression model to predict expected costs and outcomes using trial data has several potential benefits. Firstly, it provides a range of solutions to statistical problems, which may be unavoidable in any given pragmatic trial. In the context of this trial, these methods have allowed for the clustered nature of the trial. Secondly, by explicitly modelling the effects of observable covariates such as the Jarman score, we are able to counter the effects of slightly unbalanced control and intervention groups. Furthermore, knowledge of the effects of differing Jarman scores will facilitate sub-group analysis.
A third methodological point related to the use of a combined outcome measure of deaths and injuries. If the study had shown that the intervention program was more costly and effective than the control, it would have been important to deal differently with outcomes in order to assess whether the additional cost of the program was worth incurring for the extra benefits. This could have been achieved by looking at the two types of outcome separately, and assessing expected net benefit (and the probability of cost-effectiveness) assuming an averted death is valued in monetary terms to the level used by policy makers in transport.26 Alternatively, the two outcomes could have been combined into a measure such as quality-adjusted life-years which is used by, amongst others, the National Institute for Clinical Excellence.27
In conclusion, this analysis indicates that a smoke alarm give-away program, as administered in the trial, is unlikely to represent a cost-effective use of societal resources. Further research is required to identify effective and efficient ways of getting appropriate smoke alarms into high-risk homes.
Key points
|
|
Acknowledgments |
|---|
This economic evaluation was funded by the UK Medical Research Council. The trial was additionally funded by the Home Office, Department of Health, Camden and Islington Councils, British Medical Association and the Camden and Islington Health Authority. Mark Sculpher is supported by a Career Award in Public Health funded by the NHS Research and Development programme.
|
References |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
1 Office of the Deputy Prime Minister. Fire Statistics, United Kingdom 2001. London, 2003.
2 Hales JNS. 1996 British Crime Survey: Social and Community Planning Research, 1998.
3 Roberts ICP. Does the decline in child injury mortality vary by social class? A comparison of class specific mortality in 1981 and 1991. Br Med J 1996;313:784–6.
4 Budd T, Mayhew P. Fires In The Home In 1995: Results From The British Crime Survey. Home Office Statistical Bulletin 9/97. London: Government Statistical Service, 1997.
5 Watson L, Gamble J. Fire Statistics: United Kingdom 1999. London: the Home Office, 2000.
6 Roy D. The cost of fires a review of the information available. London: the Home Office, 1997.
7 Roberts I. Smoke alarm use: prevalence and household predictors. Injury Prevent 1996;2:263–5.[Abstract]
8 Mallonee S, Istre GR, Rosenberg M, et al. Surveillance and prevention of residential of residential-fire injuries. N Engl J Med 1996;335:27–31.
9 DiGuiseppi C, Roberts I, Wade A, et al. Incidence of fires and related injuries after giving out free smoke alarms: cluster randomised controlled trial. Br Med J 2002;325:995.
10 DiGuiseppi C, Slater S, Roberts I, et al. The ‘Lets Get alarmed’ initiative: a smoke alarm giveaway programme. Injury Prevent 1999;5:177–82.
11 Jarman R. Under privileged area: validation and distribution of scores. Br Med J 1984;289:1582–92.[ISI][Medline]
12 DiGuiseppi C, Edwards P, Godward C, Roberts IAW. Residential fire and flame injuries in an urban community: a population-based study. Injury Prevent 2000;6:250–4.
13 London Fire and Civil Defence Authority (LFCDA). Fire Service Costs, London 1999.
14 Association of British Insurers. Insurance statistics yearbook: 1998–1998. 1998.
15 Netten A, Dennett JJK. The unit costs of health and social care. Personal Social Services Research Unit, Kent, Department of Health 1999.
16 Brazier J, Nicholl JHS. The cost and effectiveness of the London Helicopter Emergency Medical Service. J Health Services Res Policy 1996;1:232–7.[Medline]
17 Johannesson M, Weinstein M. On the decision rules of cost-effectiveness analysis. J Health Economics 1993;12:459–67.[CrossRef][ISI][Medline]
18 Van Hout BA, Al MJ, Gordon GS, et al. Costs, effects and c/e-ratios alongside a clinical trial. Health Economics 1994;3:309–19.[ISI][Medline]
19 Blough D, Madden C, Hornbrook M. Modeling Risk Using Generalized Linear Models. J Health Economics 1999;18:153–71.[CrossRef][ISI][Medline]
20 Stata Reference Manual volume Volume User Manual: Release 7 Edition: Stata Press, Texas 2001.
21 Claxton KPJ. An economic approach to clinical trial design and research priority-setting. Health Economics 1996;5:513–24.[CrossRef][ISI][Medline]
22 Stinnett AJM. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decision Making 1998;18 (Suppl):S68–80.
23 DiGuiseppi C, Roberts I, Speirs N. Smoke alarm installation and function in inner London council housing. Arch Dis Child 1999;81:400–3.
24 Rowland D, DiGuiseppi C, Roberts I, et al. Prevalence of working smoke alarms in local authority inner city housing: randomised controlled trial. Br Med J 2002;325:998–1001.
25 Haddix AC, Mallonee S, Waxweiller R, Douglas, MR. Cost effectiveness analysis of a smoke alarm giveaway program in Oklahoma City, Oklahoma. Injury Prevent 2001;7:276–81.
26 Department for Transport. 2 year pilot partnership study, London 2003.
27 London National Institute for Clinical Excellence. Guidance for manufacturers and sponsors, July 2001.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||