OUP user menu

Determining the cost effectiveness of a smoke alarm give-away program using data from a randomized controlled trial

Laura Ginnelly , Mark Sculpher , Chris Bojke , Ian Roberts , Angie Wade , Carolyn Diguiseppi
DOI: http://dx.doi.org/10.1093/eurpub/cki146 448-453 First published online: 8 September 2005

Abstract

Background: In 2001, 486 deaths and 17 300 injuries occurred in domestic fires in the UK. Domestic fires represent a significant cost to the UK economy, with the value of property loss alone estimated at £375 million in 1999. In 2001 in the US, there were 383 500 home fires, resulting in 3110 deaths, 15 200 injuries and $5.5 billion in direct property damage. Methods: A cluster RCT was conducted to determine whether a smoke alarm give-away program, directed to an inner-city UK population, is effective and cost-effective in reducing the risk of fire-related deaths/injuries. Forty areas were randomized to the giveaway or control group. The number of injuries/deaths and the number of fires in each ward were collected prospectively. Cost-effectiveness analysis was undertaken to relate the number of deaths/injuries to resource use (damage, fire service, healthcare and giveaway costs). Analytical methods were used which reflected the characteristics of the trial data including the cluster design of the trial and a large number of zero costs and effects. Results: The mean cost for a household in a give-away ward, including the cost of the program, was £12.76, compared to £10.74 for the control ward. The total mean number of deaths and injuries was greater in the intervention wards then the control wards, 6.45 and 5.17. When an injury/death avoided is valued at £1000, a smoke alarm give-away has a probability of being cost effective of 0.15. Conclusions: A smoke alarm give-away program, as administered in the trial, is unlikely to represent a cost-effective use of resources.

  • cluster-randomized trial
  • cost effectiveness analysis
  • smoke alarm give-away program

In 2001, local authority fire services attended over 1 million fires or false alarms in the UK, 64 600 of which were domestic fires.1 The British Crime Survey (BCS)2 also estimates that only 13–26% of domestic fires are reported to the Fire Service. In 2001 in the UK, it was estimated that 486 people died in fires in the home, and approximately 17 300 people were injured.1 There is a steep social class gradient in the risk of fire-related deaths among children aged 1–15. In the UK, this risk is 15 times higher in the lowest income groups compared to the highest income groups.3 The risk of having a residential fire is 25% higher in inner city areas compared to non-inner city areas.4

Fires detected by smoke alarms tend to be discovered more rapidly and are associated with a reduced risk of death (three versus nine deaths per 1000 fires) and non-fatal casualty (137 versus 178 per 1000 fires),5 and less property damage.6 For several years, the UK government has conducted annual publicity campaigns to increase the percentage of households that have a fitted operational smoke alarm. The National Community Fire Safety Centre estimated that, in 1998, 82% of households owned a smoke alarm. However, materially deprived households are less likely to own a smoke alarm.7

An observational study, conducted in Oklahoma City, distributed smoke alarms to target areas using various give-away strategies.8 Results of this study showed an 80% decline in serious injuries in the follow up period. The non-randomized design of this study, however, may have biased the results of the study.9 Furthermore, it is unclear whether the results of this study can be generalized from a small mid-western US city to an urban, materially deprived, ethnically diverse, British population.

A randomized-controlled trial was conducted to determine whether a smoke alarm give-away program, directed to such a population, is effective and cost-effective in reducing the risk of fires and fire-related deaths/injuries. This showed that giving away free smoke alarms did not reduce the number of fire related injuries or deaths.9 Despite these results, the presence of a smoke alarm in a house may reduce the cost of a fire, as the Fire Service may be called to the scene quicker. If this is the case, the program may still be cost-effective. The net cost of the program will depend on the cost of giving away alarms and any savings generated, as a result of a reduction in the incidence and/or severity of fires. Cost effectiveness will involve relating this net cost to the effectiveness of the program in terms of deaths/injuries avoided.

Methods

Intervention design

The give-away program, called the ‘Let’s Get alarmed!' Initiative,10 distributed 20 050 smoke alarms, batteries and fire safety brochures. Fitting of alarms by trained staff was offered to all recipients, although only 8% of households used this service.

Trial design

Full details of the trial design have been published elsewhere.9 A cluster-randomized controlled trial was conducted in the inner London Boroughs of Camden and Islington. Households were categorized into administrative units (wards), based on geographical location. Wards included were those with above average material deprivation, defined as a Jarman Under Privileged Area score ≥20.11 Wards were pair matched according to Jarman score. Random allocation to either intervention (smoke alarm give-away) or control (no smoke alarm give-away) was then undertaken within the matched pairs.

Forty wards, averaging 3683 households in each, were randomized to intervention and control status. Free smoke alarms and fire safety information was distributed to intervention wards. The aim was to increase the proportion of households with smoke alarms in the intervention wards from 47% to 72% (national prevalence, as of 1996).10 The number of alarms distributed and costs of the program from a societal perspective were recorded.10

Data collected in the trial

Details relating to the number of injuries and deaths, the number of fires and resource use, in each ward, were collected prospectively in intervention and control wards for between 22.9 and 25 months (mean 23.9). The number of fires occurring in each ward was restricted to those reported by the Fire Service. Details of each fire were recorded, which facilitated subsequent costing. These data included room of ignition, details of spread of fire, length of time spent at scene by fire service, police presence and number of persons involved in the fire. Injuries were defined as those that resulted in an accident and emergency (A&E) attendance, hospitalization or death. These data were collected from local A&E department registers, which included details of treatments given, referrals to other departments, and hospitalizations and from hospital case records. Deaths occurring as a result of fire were recorded in the A&E and follow-up forms or in coroners' reports. Data necessary to cost ambulance attendance were routinely recorded by the ambulance service. Full details of the trial have been published elsewhere.12

Cost analysis

Resource use data collected in the trial were used to evaluate the net cost of the give-away program, that is, the cost of providing alarms to appropriate households, net of any savings resulting from fewer and/or less serious fires in the smoke-alarms wards. The cost analysis adopted a societal perspective. Unit cost data was taken from a number of sources and is detailed in table 1. There were four components to the cost analysis, each of which is detailed below.

View this table:
Table 1

Unit costs used in the analysis in UK sterling at 1999 prices

ResourceUnit measuredUnit costSource
Police and fire services
    PoliceAttendance at scene of fire£110a
    Fire Service ExtinctionCost per pump, per minute£2.63LFCDA13
    Fire Investigation UnitCost per hour£57LFCDA13
    Assistant Divisional OfficerCost per hour£47LFCDA13
Self extinction
    Fire blanketPer blanket£13.95b
    9 litre water extinguisherPer extinguisher£55.50c
    2 kg carbon dioxide extinguisherPer extinguisher£83.50c
    9 litre foam AFFF extinguisherPer extinguisher£64.50c
    Halogen extinguisher (1 kg)Per extinguisher£46.81d
    Halogen extinguisher (1.5 kg)Per extinguisher£63.83d
    Halogen extinguisher (2 kg)Per extinguisher£85.11d
    Dry powder extinguisher (1 kg)Per extinguisher£27.50c
    45 litre Aqueous fire fighting foamPer unit£506.19e
Property damage
    Inside/no spreadPer fire£231.14BCS2
    Inside/spreadPer fire£3103BCS2
    Outside/spreadPer fire£543.4BCS2
    Outside/no spreadPer fire£69.19BCS2
    Insurance administrationCost per fire claim51.5% of the value of the claimf
Injury costs
    Ambulance servicesCost per minute£4.53Netten and Dennett16
    Helicopter emergency medical serviceCost per mission£3624.08Brazier and Nicholl17
    Accident and emergency attendanceCost per visit£41Netten and Dennett16
    General practitioner attendanceCost per visit£23.50Netten and Dennett16
    Outpatient clinicCost per visit£37Netten and Dennett16
    Accident and emergency wardCost per day£359Netten and Dennett16
    Hospital generic wardCost per day£222Netten and Dennett16
    Intensive care unitCost per day£1845.72g
    Burns unitCost per day£826.32g
    Plastic surgeryCost per graft£2076.45g
    TracheotomyCost per procedure£214.75g
    Hyperbolic oxygen therapyCost per session£1000g
    FuneralCost per service£1600h
    Coroners costsCost per death£30i
    AutopsyCost per procedure£100j
  • a: Strathclyde Fire Brigade, personal communication, 2000

  • b: SP Services L. 2000

  • c: FPS Fire Protection Limited. 2000

  • d: Fire Master Extinguisher Limited L. 2000

  • e: Wireless Alarms. 2000

  • f: Association of British Insurers. Insurance: Facts Figures and Trends, 1999

  • g: University College London. Extra Contractual Referral Prices, 1997–1998, 1998

  • h: Cooperative Funeral Services, personal communication, 2000

  • i: HomeOffice. Coroners service survey: Home Office Research and Statistics Directorate, 1998

  • j: Leicester University Pathology Department, personal communication, 2000

The give-away program

The total cost of the give-away was £157 823, including the one-year reminder postcards.10 A total of 20 050 alarms were distributed to 73 399 intervention households; therefore, the mean cost per household of the give-away program was £2.15.

Fire service and police

Fire service costs were calculated by multiplying the time spent by the Fire Service at the scene of a fire, by a cost per pump-hour.13 For fires with more than four pumps present, a Fire Investigation Unit is required, which was costed at £57 per hour.13 In cases where it was apparent that injuries had occurred in the fire, and more than four pumps were used, the Assistant Divisional Officer (ADO) was required to attend, costed at £47 per hour.13 Police presence at the scene of a fire was assigned a fixed cost of £110 (Strathclyde Fire Brigade, personal communication, 2000).

Property damage

Costs of property damage were estimated from the 1996 BCS.2 Fires were defined as one of four types: (i) inside/no spread; (ii) inside/spread; (iii) outside/spread; and (iv) outside/no spread. This definition is consistent with the categorization used in the BCS from which cost data were available. Respondents to the BCS, who had experienced a fire in the last year, indicated into which of the four categories their fire fell and provided an estimate of the cost of property damage. These responses were used to derive a mean cost of damage for the four types of fire. Insurance administration costs were estimated as 51.5% of the gross cost of the claim.14

Health service

The duration of ambulance journeys to A&E was measured from the time of the first call to the time the ambulance returned to base, and costed based on a published source.15 Helicopter emergency ambulance journeys were assigned an average cost.16 All cases that presented at A&E were assigned a cost of £41,14 representing the average cost of an A&E visit in the UK. Hospital admission costs, follow-up costs, visits to family doctors and surgery were added based on data collected from medical notes.

Deaths resulting from fires also incurred a cost (£1730) representing the average cost of a funeral, coroners' and autopsy costs (Cooperative Funeral Services, personal communication, 2000).

Effectiveness

The measure of effect for the cost-effectiveness analysis was fire-related deaths and injuries over the follow-up period. These are based on the incidence of injuries related to fires resulting in attendance at an A&E department, hospitalization, or death during the follow-up period.

Cost-effectiveness

Two measures of cost-effectiveness are derived from trial data. Firstly, standard decision rules17 are used to establish the incremental cost-effectiveness of the give-away program based on mean differential costs and effects. Unless one arm of the trial dominates the other (less costly and more effective), an incremental cost-effectiveness ratio is calculated to indicate the additional cost of each death/injury avoided within the more effective arm of the trial.

Secondly, the uncertainty surrounding mean expected costs and deaths/injuries can be represented as a cost-effectiveness acceptability curve.18 These show the probability that the intervention is cost-effective for different threshold values that the decision maker might be willing to pay to avoid a fire-related death/injury. When this threshold is set to £0, this is equivalent to reporting the probability that the giveaway program generates societal cost savings because, at this threshold, avoiding deaths/injuries is given no value. Threshold values ranging between £0 and £50 000 per death/injury averted are used in the analysis.

Statistical analysis

Particular characteristics of the trial required non-standard approaches to statistical analysis. Firstly the cluster-randomized design of the trial. As the unit of analysis was the individual household, whilst the unit of randomization was the administrative ward, statistical methods were required to allow for the potential correlation in costs and effects between households within wards. The second feature of the trial was the large number of households within the trial that did not experience a fire during the two years of follow-up, that is, they were characterized by zero costs and effects. This generated heavily right-skewed cost and effect distributions making the assumption of normality invalid.

To deal with these characteristics, data were analysed using an econometric model. Because of the large proportion of non-events, a two-stage model was used.19 The first stage modelled the probability of a fire occurring using a logit model, and the second stage modelled the expected cost and numbers of injuries/deaths conditional on a fire occurring. Since the cost and injury data are not normally distributed and because predictions of these outcomes are required, they were modelled using a generalized linear model with the appropriate best fitting link function and distribution (gamma for cost and Poisson for injuries).19 The expected costs and injuries for a control and intervention household were calculated by multiplying the relevant probability of a fire by the appropriate expected cost/injuries given that there has been a fire. For both parts of the model, the independent variables were whether the ward/household was in the give-away or control group, the borough in which the ward was located and Jarman score. A robust estimator of variance20 was used to overcome the clustered nature of the trial.

To generate cost-effectiveness acceptability curves, the following methods were used. Firstly, household-level costs and effects were placed onto a single monetary scale using the principles of net monetary benefit (NMB),21,22 which values effects in monetary terms according to a threshold willingness to pay per unit of effect. Secondly, distributions of mean NMB for the control and intervention groups were calculated using Monte Carlo simulation allowing for relevant correlation between parameters. The third stage involved calculating the proportion of the simulations for which the mean NMB was higher for the intervention group than for the control. This represents one point on the cost-effectiveness acceptability curve corresponding to a particular threshold willingness to pay value for a unit of effect. The full curve was plotted by varying this threshold and re-running the simulation.

Results

Fires

The predicted expected number of fires, for intervention and control wards, can be seen in table 2. This is based on an average ward consisting of 3686 households, with an average Jarman score of 34.55. Intervention households have a higher probability of having a fire than control household over 2 years: 0.007885 compared to 0.007067.

View this table:
Table 2

Overall cost and effect results predicted by the two-stage model and relating to the average ward of 3686 households, in the London Borough of Camden, and with an average Jarman score of 34.55

Control wardIntervention ward
Expected number of fires per ward over 24 months26.03 (24.92–27.19)29.04 (27.67–30.48)
Probability of household having a fire in 24 months0.00707 (0.00676–0.00738)0.00789 (0.00751–0.00828)
Expected number of fire-related injuries or deaths per ward over 24 months5.172 (4.492–5.964)6.455 (5.627–7.418)
Expected number of fire-related injuries or deaths in a household given that there has been a fire0.1987 (0.1738–0.2272)0.2223 (0.1955–0.2528)
Expected number of fire-related injuries per household over 24 months0.0014 (0.0012–0.0016)0.0018 (0.0015–0.0020)
Expected total cost of a fire given that there has been a fire1519.98 (1371.75–1684.24)1344.99 (1215.89–1487.80)
Total made up of:
    Extinction costs395.61 (357.75–437.47)282.06 (255.34–311.58)
    Injury costs104.50 (79.89–136.68)151.21 (122.80–186.18)
    Property damage costs1008.40 (877.98–1158.19)886.68 (771.50–1019.05)
Mean total cost per household over 24 months£10.74 (9.60–12.02)£10.61 (9.48–11.87)
Give-away cost per household£0£2.15
Mean costs per household over 24 months (including give-away cost)£10.74 (9.60–12.02)£12.76 (11.63–14.02)
  • Figures in parentheses are inter-quartile ranges

Resource use measured in trial

Table 3 shows resource use associated with fire events in the trial. Control households experiencing a fire were more likely to call the Fire Service, and had fires resulting in more extensive property damage (inside fires spreading to more than one room).

View this table:
Table 3

Resource use associated with fire events collected in the trial

Item of resource useControl wardIntervention ward
Proportion of fires requiring Fire Service attendance0.94990.91826
Proportion of fires resulting in an A&E attendance0.15020.1704
Proportion of fires resulting in extensive property damagea0.38530.3321
  • a: Defined as fires that started inside of a property and spread to more than one room

  • The table is based on raw data collected in the trial and shows the proportion of fires displaying certain characteristics

Deaths and injuries

Injuries and deaths were more common in fires that occurred in intervention households compared to control households (table 2). The expected number of deaths and injuries in an average ward over the 24-month follow-up period was 6.45 and 5.17 in intervention and control wards, respectively. This equates to an expected number of injuries/deaths per household experiencing a fire of 0.222 and 0.199 in the intervention and control households, respectively. The expected number of deaths or injuries due to fire for a household is, therefore, 0.0014 and 0.0018 for control and intervention households, respectively (table 2).

Costs

For those households experiencing a fire in the follow-up period, the expected cost of that fire was lower in the intervention households (£1345) than in the control households (£1520) (table 2). Property damage contributed the most to the cost of a fire: £1008 for control and £887 for intervention households. Extinction costs were also lower in intervention households: £282 and £396 for intervention and control, respectively. However, mean injury costs were higher in intervention households experiencing fires: £151 compared to £105 in control households.

When the cost of the give-away program, at £2.15 per household, is not included, the mean cost of fires across all intervention wards is marginally lower than that in control wards: £10.61 compared to £10.74. When the cost of the give-away program itself is added to the intervention wards, on the basis of mean costs and outcomes, the give-away program has both higher overall expected costs and more deaths/injuries; that is, it is a dominated intervention.

Reflecting uncertainty in cost-effectiveness

To examine the uncertainty in mean costs and effects, a cost-effectiveness acceptability curve is shown in figure 1. When the willingness to pay for avoidance of a death/injury is £0, the probability that the give-away program is cost-effective (cost saving) is 18%. As the willingness to pay increases, the probability that a smoke alarm give-away program is cost effective decreases because the intervention group experienced more deaths and injuries. When an averted injury/death is valued at £1000, a smoke alarm give-away has a probability of being cost effective of 0.15. When an averted injury/death is valued at £50 000, the probability of a smoke alarm give-away program being cost effective is 0.11.

Figure 1

Cost-effectiveness acceptability curve showing the probability that the give-away program is cost-effective given the threshold willingness to pay: value per death/injury averted

Discussion

Overall, the results of this study indicate that, as used in the ‘Let’s Get alarmed!' Initiative,10 a smoke alarm give-away program is not a cost-effective use of societal resources. From a policy perspective, however, it is important to emphasize that the results of this study do not suggest that a smoke alarm give-away program can never be cost-effective. The results shown in the trial probably reflect the fact that too few alarms were distributed, installed and maintained over the study period. Inspection of a random sample of council-owned homes suggested that the prevalence of functioning smoke alarms 12–18 months after the program was nearly identical in intervention and control households.23 Hence the distribution of 20 000 free alarms in an area of 73 400 homes was insufficient to increase the overall prevalence of functioning alarms. A further trial24 was undertaken to assess whether providing different types of smoke alarm (from different power sources) can increase the prevalence of functioning alarms. This trial found that nearly half of installed, battery-operated alarms did not work when tested 15 months after installation, and that alarm type and power source were important determinants of function.

There are a number of methodological issues emerging from this study. Firstly, this is the first randomized-controlled trial of a smoke-alarm give-away program, and shows that it is feasible to use experimental designs to evaluate important public health initiatives. The results seen in the trial are markedly different to those in the observational study conducted in Oklahoma upon which the give-away intervention was modelled,8 which concluded that, as well as 20 fewer fatalities and 24 fewer non-fatal injuries, the intervention generated savings of $15 million from a societal perspective.25 In part, this may reflect population differences, for example, fewer owner-occupiers, fewer English speakers and a materially poorer, less literate population in the UK study. However, the non-experimental design in Oklahoma City might also explain the differences in results.

A second point is that the use of a regression model to predict expected costs and outcomes using trial data has several potential benefits. Firstly, it provides a range of solutions to statistical problems, which may be unavoidable in any given pragmatic trial. In the context of this trial, these methods have allowed for the clustered nature of the trial. Secondly, by explicitly modelling the effects of observable covariates such as the Jarman score, we are able to counter the effects of slightly unbalanced control and intervention groups. Furthermore, knowledge of the effects of differing Jarman scores will facilitate sub-group analysis.

A third methodological point related to the use of a combined outcome measure of deaths and injuries. If the study had shown that the intervention program was more costly and effective than the control, it would have been important to deal differently with outcomes in order to assess whether the additional cost of the program was worth incurring for the extra benefits. This could have been achieved by looking at the two types of outcome separately, and assessing expected net benefit (and the probability of cost-effectiveness) assuming an averted death is valued in monetary terms to the level used by policy makers in transport.26 Alternatively, the two outcomes could have been combined into a measure such as quality-adjusted life-years which is used by, amongst others, the National Institute for Clinical Excellence.27

In conclusion, this analysis indicates that a smoke alarm give-away program, as administered in the trial, is unlikely to represent a cost-effective use of societal resources. Further research is required to identify effective and efficient ways of getting appropriate smoke alarms into high-risk homes.

Key points

  • A smoke alarm give-away program, as administered in the trial, is unlikely to represent a cost-effective use of resources.

  • It is important to emphasize that the results of this study do not suggest that a smoke alarm give-away program can never be cost-effective.

  • Regressions model to predict expected costs and outcomes using trial data have several potential benefits.

Acknowledgments

This economic evaluation was funded by the UK Medical Research Council. The trial was additionally funded by the Home Office, Department of Health, Camden and Islington Councils, British Medical Association and the Camden and Islington Health Authority. Mark Sculpher is supported by a Career Award in Public Health funded by the NHS Research and Development programme.

References

View Abstract