The European Journal of Public Health Advance Access originally published online on August 26, 2005
The European Journal of Public Health 2006 16(4):346-353; doi:10.1093/eurpub/cki157
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Health Inequalities |
Evaluating the effect of policies and interventions to address inequalities in health: lessons from a Dutch programme
Karien Stronks1 and Johan P. Mackenbach2
1 Department of Social Medicine, Academic Medical Centre/University of Amsterdam, The Netherlands
2 Department of Public Health, Erasmus MC, University Medical Centre Rotterdam, Rotterdam, The Netherlands
Correspondence: Dr K. Stronks, Department of Social Medicine, Academic Medical Centre/University of Amsterdam, Meibergdreef 15, 1105 AZ Amsterdam, The Netherlands, tel: +31 20 5664892, fax: +31 20 6972316, e-mail: k.stronks{at}amc.uva.nl
Received October 6, 2004, accepted June 7, 2005
 |
Abstract |
|---|
 Top Abstract MethodsResults of the evaluation...DiscussionReferences |
|---|
Objectives: Many initiatives have been taken in European countries that are designed to reduce inequalities in health. However, the effects of only a very few of these initiatives have been assessed. The main aim of a Dutch research and development programme was to systematically investigate and evaluate interventions aimed at reducing inequalities in health. In this paper, we report on this investigation, and draw lessons from the methodology used to evaluate such interventions. Approach: The programme included 12 evaluation studies, focusing on the wider determinants of inequalities in health (n = 2), behavioural determinants (n = 4), working conditions (n = 3) and health care (n = 3). Results: An experimental design was applied in two evaluation studies. The studies provided evidence of a positive effect. A quasi-experimental design appeared to be the only attainable option in seven studies. Five of these provided sufficient evidence for a positive effect, but two interventions appeared not to be successful. In three studies, no experimental or quasi-experimental design could be applied. Conclusions: The programme showed that it is possible to apply experimental or quasi-experimental studies to complex public health interventions. The Programme Committee steering the programme considered that the evidence generated by the experimental and quasi-experimental studies justified the implementation of the interventions on a wider scale, accompanied by further evaluation studies. Further development of the methodology of public health interventions is necessary. These include non-experimental designs such as international comparisons and time trend studies, especially in order to be able to evaluate broader policy measures.
Keywords: evaluation studies, health, socioeconomic status
Socioeconomic inequalities in health are present in all European countries. People in lower socioeconomic groups on average have a lower life expectancy, a worse perceived health status and higher morbidity rates.1 In several European countries, such as the UK, Sweden and The Netherlands, national committees have developed comprehensive sets of measures to improve the health status of people in lower socioeconomic groups. Examples of possible interventions and policies, proposed as well as in progress, include anti-poverty measures, tax on tobacco and health-care interventions tailored to the needs of lower socioeconomic groups.2
There is a widespread agreement that a policy aimed at the reduction of inequalities in health should be based on evidence, including evidence on their effectiveness.3 In reality, very few of the potential measures are assessed as to their effect. Instead, most of the proposed measures seem to be based only upon knowledge of the factors explaining the association between socioeconomic position and health, such as working conditions, income or smoking. Little is known, however, about the extent to which measures addressing these factors contribute to achieving a reduction of these inequalities in health.3–7 This applies to ‘downstream’ interventions, focusing on specific determinants such as smoking, as well as for ‘upstream’ interventions or broader policies such as income policies.
The lack of studies on the effectiveness of interventions to reduce inequalities in health is without doubt partly because the evaluation of these interventions and policies is difficult for a number of reasons that involve methodological, practical and ethical issues.8–10 These include the frequently described problem that more complex policy measures cannot be evaluated within a classic study design such as the randomized controlled trial (RCT) or the community intervention trial (CIT).
In view of the lack of evidence on the effectiveness of interventions to address inequalities in health and the difficulties in generating this evidence, the Dutch Ministry of Health has funded a 6-year national research and development programme. The main aim of this programme was to investigate and evaluate interventions specifically aimed at reducing inequalities in health. In this paper we describe the experiences of the Dutch programme, and draw lessons from the methodology used to evaluate such interventions.
|
Methods |
|---|
|
TopAbstractMethodsResults of the evaluation...DiscussionReferences |
|---|
Possible strategies to reduce inequalities in health
In light of the background to socioeconomic inequalities in health, the Committee steering the programme distinguished four possible strategies to reduce inequalities in health.11,12 These include interventions and policies:
- targeting socioeconomic disadvantage, such as antipoverty policies and social benefit schemes;
- reducing the effect of health on socioeconomic disadvantage, including benefit levels for long-term inability to work and adaptation of working conditions for chronically ill;
- targeting factors mediating the effect of socioeconomic disadvantage on health, including the promotion of healthy behaviour and healthy working conditions; and
- improving the accessibility and quality of health care provided to the lower socioeconomic groups, including maintaining good financial accessibility of health care.
Selection of evaluation studies
Two sets of criteria were applied in selecting the evaluation studies to be funded. The first applied to the proposed intervention: the factors targeted by the proposed intervention (e.g. working conditions, income, smoking). These factors should be known to contribute to the explanation of inequalities in health, and the proposed intervention should be likely to diminish the exposure of lower socioeconomic groups to these factors. The second set of criteria concerned the scientific quality of the proposed evaluation study. A scientific subcommittee reviewed the scientific quality of the proposals by applying a set of standards that had to be met.8 The standard allowed for study designs other than the RCT, requiring less complex measurements, including RCTs that include people in lower socioeconomic groups only. The researchers had to argue that the assumptions to be made in that case were valid. The scientific quality of the proposals was judged prior to deciding whether they were relevant. Therefore, only the proposals that met the methodological requirements, were subsequently judged in relation to their relevance.
Twelve evaluation studies were carried out. Three projects resulted from a top-down approach, whereas nine were selected in two open tender rounds. The open tenders were invited during 1996 and 1997, not only from researchers engaged in health (care) research, but also in areas such as socioeconomic research. The first call for tenders elicited 21 preliminary applications for possible projects, the second tender elicited 20. Those applications that proposed to study an intervention that seemed relevant for the reduction of socioeconomic inequalities in health, and that were likely to develop into a rigorous scientific evaluation, were invited to submit a detailed proposal. We received six in 1996 and nine in 1997. Ultimately, four from the first round and five from the second were accepted.
Relevance of the intervention
At the start of the programme, the Committee steering the programme identified a number of intervention areas that are of crucial importance for the reduction of socioeconomic inequalities in health, according to the four possible strategies specified above. Additionally, the Committee identified some interventions where there was only a small possibility that evaluation studies would spontaneously arise in the open tender rounds. These were therefore selected top-down. The interventions that were selected are described in table 1.
|
The three projects that were selected top-down include an intervention aimed at addressing poverty among children (#1), an educational campaign aimed at the promotion of the periconceptional use of folic acid (#3) and a community-based intervention (#4).
The other nine projects were selected from the open tender rounds. These included an absentee counselling project (#2) and an anti-smoking intervention in secondary schools (#5), collective teeth brushing at primary schools (#6), a new working method for a team of bricklayers (#7), task rotation among dustmen (#8), self-organizing teams in production organizations (#9), a nurse practitioner in general practice (#10), the introduction of peer education in general practice (#11) and local care networks in deprived areas (#12).
Ultimately, most of the 12 interventions follow the third strategy mentioned before, i.e. reducing the effect of socioeconomic status on health, by addressing specific determinants, in particular behavioural determinants (#3–6) and working conditions (#7–9). Three interventions focused on the fourth strategy of improving the accessibility and effectiveness of health care (#10–12). The remaining two interventions fit into in the first and second strategy respectively (#1 and #2).
Scientific quality of the evaluation study
The scientific criteria were developed by a group of experts.8 Basically, they started from the model of the RCT, or, as interventions to address inequalities in health often imply the group being unit of allocation, the CIT. They required measurement in one or more control and experimental populations, of changes over time in the size of inequalities in health. In case this design was not feasible, alternative designs were accepted, under the condition that the assumptions upon which these designs were based were clarified, and that arguments were provided to support these assumptions. These included for example the exclusion of the higher socioeconomic groups in case they were not targeted by the intervention (e.g. the intervention was targeted at those living in poverty), or the use of intermediate (instead of health) outcome measures, in case this intermediate factor had been shown to affect health (such as smoking).8
The design of each study is summarized in table 2. All but one included an effect evaluation, whereas a process evaluation was part of all studies. In this paper we will focus on the effect evaluations in particular. The 11 evaluation studies that included an effect evaluation all proposed to compare the level of health (or its determinants) as assessed in experimental groups, with that in control groups. In two of these (#5 and #6), both targeted at health behaviour, the groups were randomly allocated to the intervention or the control condition (experimental study). In the other nine studies, allocation did not occur randomly (quasi-experimental) (#2–4 and #7–12). Moreover, the original proposals for all effect evaluation studies included pre- and post-measurements. Seven experimental and quasi-experimental studies included people in lower socioeconomic groups only, partly because the intervention was not relevant for higher socioeconomic groups (i.e. the two interventions for specific occupational groups and the local care networks), or because the assumption was that the intervention, if targeted to higher socioeconomic groups, would have a similar effect in higher socioeconomic groups (anti-smoking intervention, community-based intervention, peer education in general practice) (#5, #7, #8 and #10–12).
|
|
Results of the evaluation studies |
|---|
|
TopAbstractMethodsResults of the evaluation...DiscussionReferences |
|---|
The evaluation studies were set up as experimental or quasi-experimental studies for nine interventions (#3–8 and #10–12). For the remaining three interventions, no experimental or quasi-experimental design could be applied (table 2) (#1, #2 and #9).
Experimental and quasi-experimental studies
Seven studies indicated a positive effect of the interventions. These included two school-based interventions targeted at health-related behaviour, smoking and teeth-brushing respectively (#5 and #6). Both interventions were evaluated in an experimental design. In the first project, including a smoke-free class competition, the number of pupils starting smoking was lower in the experimental compared with the control schools. In addition, a significant difference was observed between the intervention and control groups regarding the change in social pressure of class mates. In the longer term, however, the difference between the intervention and control schools was no longer statistically significant.18 In the second project, on teeth brushing, the interim assessment found that after the intervention, the well-known difference in brushing behaviour was no longer evident between children from higher and lower socioeconomic schools. Despite this, the final assessment showed an increase in the socioeconomic gradient in the prevalence of caries. This was probably due to the fact that the intervention period of 3 years was too short for the intervention to have an effect on caries.19
Two interventions focused on working conditions, more specifically on the exposure to physically strenuous labour (#7 and #8). Both interventions, among refuse collectors and bricklayers, were evaluated in a quasi-experimental design. In the first study, three companies with three teams of refuse collectors participated. As it was hard to find volunteers for this study, there was a risk of the experimental group being more motivated than the control group. It was therefore decided to apply the principle of job rotation to each of the participating teams, using different working schemes of job rotation (e.g. between days and during a day).21 In the second study, the experimental group consisted of companies that had decided to switch to the new working method, whereas companies that worked with the traditional methods formed the control group. A random assignment of groups was impossible because of the necessary investment that the companies would have to make.20 Both studies provided indications for a positive effect of the interventions on physical workload.20,21 In the study among bricklayers health problems were also measured, but no reduction in health problems could be observed. This may have been to do with the relatively long latency period of health problems in relation to the relatively short follow-up period of 1 year.
The programme included three interventions related to health care: nurse practitioners and peer educators in general practice, and local care networks (#10–12). All interventions were carried out in deprived neighbourhoods.
To date, the introduction of a nurse practitioner has been evaluated by comparing the compliance with inhalation medication and the number of exacerbations before and after the intervention in the intervention practices.23 The results showed a positive change in these indicators. Further analyses among the control group should yield more evidence as to whether this change can actually be attributed to the introduction of the nurse. In addition, the evaluation did not give sufficient insight into the element of the intervention that was responsible for the observed change (more extensive feedback?, treatment more frequently delivered in accordance with the guidelines?, etc.).
In the study on peer educators for Turkish diabetes patients, randomization was considered to be impossible for practical reasons, including the fact that the bicultural educators were already working in the intervention practices. Probable differences in routine diabetes care between the intervention and control practices were controlled for on the basis of information from the medical record (e.g. number of diabetes-related referrals). The results of this study elicited weak evidence for a positive effect of intense counselling on health behaviour.24,25 An effect on glycaemic control, however, could be demonstrated only among female patients. This raises the question of which conditions these measures must comply in order to be effective: should, for example, the educator also belong to the same sex in order to have an effect on health? The evaluation study did not answer these questions.
Finally, also in the study on local care networks, the selection of intervention neighbourhoods was not within the control of the researcher, therefore making random assignment impossible. The intervention and control neighbourhoods were matched on indicators for level of social deprivation and the organization of health care.26 The evaluation study showed a decline in escalated problems such as evictions in the experimental as compared with the control neighbourhoods. A detailed process evaluation indicated that important features of this network are the cooperation between different parties from various sectors, as well as the ‘outreach function’.
Unlike the interventions described above, two interventions, both evaluated in a quasi-experimental study, appeared not to be successful (#3 and #4). These included an intervention to promote periconceptional use of folic acid. In this study, the effect of a strategy of a local educational campaign, which supplemented a national campaign, was evaluated. The evaluation was performed by comparing the trend in the socioeconomic gradient in the use of folic acid in two regions where the local campaign was in operation with those in two other regions where only the national campaign was in force. The results in these regions were similar, leading to the conclusion that the regional campaign had no additional effect.15,16 On the basis of the process evaluation, the authors could not draw a conclusion as to whether this result indicated that the local educational campaign had no extra value at all, or whether the activities of the local campaign were too few to make a difference.
The second study that showed no effect related to a community-based intervention. This intervention was evaluated by comparing the intervention and control neighbourhoods with respect to health indicators as well as to three specific health problems, i.e. safety, parenting and stress. These outcome measures were determined by the intervention team, based on discussions with community members and local workers. The evaluation indicated no systematic effect of the intervention on the three themes nor on perceived health status.17 One explanation for this might be that the intervention itself was not effective enough.
Observational studies
In three studies, no experimental or quasi-experimental design could be applied (table 2) (#1, #2 and #9).
In the case of the anti-poverty intervention, it was clear from the beginning that it was impossible to evaluate its health effect within the time frame of the programme, as this would be visible only after a lengthy time period. Therefore, the study proposal included a process evaluation only.13
In the case of the absentee counselling project at secondary schools, the proposed quasi-experimental study with pre- and post-measurements could not be carried out for logistical reasons. After the start of the study, it was not possible for the Juvenile Health Service staff member to carry out the planned measurement before the intervention for practical reasons. Therefore, the actual design was changed to a post-test-only design.14 The study on self-organizing teams was also planned as a quasi-experimental study, but this also failed for practical reasons mainly due to the fact that it was not possible to find a sufficient number of control groups.22 As a consequence, no conclusions on the effect of these two interventions could be drawn.
|
Discussion |
|---|
|
TopAbstractMethodsResults of the evaluation...DiscussionReferences |
|---|
The aim of this paper was to report on the experience of evaluating the effect of interventions to address socioeconomic inequalities in health within a comprehensive Dutch research programme. In comparison with national research programmes in other European countries, the Dutch programme is unique for its emphasis on the evaluation of possible interventions.2,27 For nine (out of a total of 12) interventions that have been evaluated within a experimental or quasi-experimental design, the evaluation studies provided reasonable evidence of their (in)effectiveness. In the remaining three studies no conclusions could be drawn as no experimental or quasi-experimental design could be applied.
What can be learned from the Dutch programme when it comes to generating evidence on the effectiveness of interventions to reduce inequalities in health, as a basis for making policy decisions?
First, RCTs are widely accepted in health care as the gold standard for assessing effectiveness. Doubts have been raised as to whether it is feasible to apply this standard to social interventions such as those included in the Dutch programme, given, for example, their complex character and the complexity of social reality in which they are implemented.28 The results of the Dutch programme illustrate that it is possible to apply this standard to public health interventions implemented in a non-laboratorium setting, at least as far as relatively limited educational interventions, such as those on teeth brushing and non-smoking, are concerned.
Secondly, although the methodological requirements that were applied to the programme started from the assumption that the experimental design is the most valid design for effect evaluations, it was clear from the beginning that it would not be very fruitful to adhere to this standard, as this implied that most of the interventions could not be evaluated at all. In more than half of the interventions studied, randomization was not possible because the implementation of the intervention was not under the control of the researcher. In some studies, the intervention was already in preparation (bicultural educators, nurse practitioner in general practices), whereas in others the implementation of the intervention required investment by an organization itself (bricklayers).
For the Programme Committee that had developed the recommendations on the basis of the results of the evaluation studies, this implied that they had to weigh the validity of the evidence generated in these studies. The greatest threat to the validity of the results of non-randomized studies arises from the possibility of selection bias, i.e. a biased allocation of patients to a given intervention.29 In view of this, in some studies intervention and control groups were matched on possible confounders (e.g. care networks in deprived neighbourhoods), whereas in others possible confounders were controlled for in a statistical analysis (e.g. in the study on Turkish peer educators). Even then, however, the results of non-experimental studies are shown to be different from those of experimental studies.29 The question for the Committee was therefore: how sure does one have to be before recommending implementation? In reaction to this the Committee decided to recommend that further evaluation studies should accompany the implementation of interventions for those studies that indicated positive results. It should be mentioned, however, that the greatest doubts on the effect of these studies were not so much a consequence of the quasi-experimental character of the evaluation studies, but rather that there was too little information on the process of the intervention to identify those elements that were responsible for its effect. This applied for example, to the study on peer education for Turkish patients as well as for the nurse practitioners. Although a process evaluation was part of both studies, the information gathered did not give enough insight to obtain a valid interpretation of the effect. This illustrates that the evidence on the effect of public health interventions as generated by experimental or quasi-experimental studies should be supplemented by extensive evidence on the process as a basis for policy decisions.10 In addition, further evaluation studies need to test the assumptions that were made in specific interventions, such as the effect of a intervention being equal for people in lower and higher socioeconomic groups.
The third issue that requires attention relates to the absence of evaluation studies in upstream interventions, including income and educational policies. It is generally agreed that when addressing inequalities in health, a broad range of intervention strategies should be considered. As we have previously argued, the randomized experimental study could rightly be considered the gold standard for the evaluation of specific interventions, such as the teeth brushing experiment. Broader policy interventions might less easily be evaluated in an experimental design, however.28,30 As the selection of interventions within the Dutch programme started from the assumption that the experimental design was the most valid design, and the methodological requirements were a more important selection mechanism than the type of intervention, the selection process automatically elicited rather specific, downstream interventions. In hindsight, we consider the lack of studies on broader policy measures, mainly related to the strict methodological criteria applied when selecting the studies, as the most important limitation of the programme. In order to be able to evaluate broader policy measures, designs other than those used in the Dutch programme should be considered, including international comparisons and time trend studies.31–33
Fourthly, the Dutch programme underlines the view that the evaluation design should not only be adapted to the type of the intervention, but also to the phase of development of an intervention. For example, at the start of the programme, the anti-poverty intervention was in an early stage of development. Following the approach of Campbell et al.,34 the evaluation study of this intervention could be classified as a phase I study, which in itself does not test the effect of the intervention, but instead explores the components that are relevant for the intervention, and tries to find potential barriers for the implementation of the intervention. The research questions that are central to this phase do not ask for a randomized controlled trial. Instead a broad range of descriptive techniques might be used, including qualitative research.34,35
The final lesson relates to the time frame of the programme. For most interventions, a period of 6 years appeared to be too short to establish valid health effects. This applies for example to the interventions in the workplace, which were able to demonstrate a positive effect on physical workload, but not on health. In addition, for some interventions that appeared to be effective, the follow-up was too short to see whether the effect also lasted in the long run (e.g. anti-smoking intervention). In hindsight, the short time frame should have been taken into consideration more explicitly in the process of the selection of research proposals. More importantly, in future studies it seems wise to have a long follow-up period when evaluating social interventions. In the case of social factors that have very long latency periods, such as an anti-poverty intervention, this might even imply that the health effect cannot be demonstrated in a randomized design.36
When drawing lessons from the Dutch programme, the ‘proof of the pudding’ is of course its actual influence on health policy and practice. The answer to this question is mixed. At the one hand, specific interventions that have been proven to be effective have now been implemented on a larger scale. These include the programme to prevent school children starting smoking and the local care networks for chronic psychiatric patients. On the other hand, at the national level, little progress has been made in developing a comprehensive package of policy measures to reduce inequalities. However, the reason for this lack of progress is related to the political constellation after the research programme, rather than to the evidence base of policies and interventions.37
In this paper we have attempted to draw lessons from our experiences of a research programme on the evaluation of interventions to reduce socioeconomic inequalities in health. Although the studies within the Dutch programme have contributed to the evidence base in this area, they also show that further development of the methodology of evaluation studies of complex interventions is necessary. We hope that our experience might serve as a source of inspiration for this.
Key points
|
|
Acknowledgments |
|---|
Dr Karien Stronks acted as co-ordinator of the research and development program and wrote this paper. Professor Johan Mackenbach acted as secretary of the Programme Committee, and provided comments on drafts of this paper. The Programme Committee was chaired by Professor W. Albeda. Members (in addition to the authors of this paper) were: Professor Dr H. Dupuis, Professor Dr H. F. L. Garretsen, Professor Dr P. J. van der Maas, Dr M. Mootz and Dr R. W. Welschen. The research and development program was funded by the Ministry of Health, Welfare and Sports, through the Health Research and Development Council of The Netherlands.
|
References |
|---|
|
TopAbstractMethodsResults of the evaluation...DiscussionReferences |
|---|
1 Mackenbach JP, Kunst AE, Cavelaars AEJM, et al. Socioeconomic inequalities in morbidity and mortality in Western Europe. Lancet 1997;349:1655–9.[CrossRef][ISI][Medline]
2 Mackenbach JP, Bakker MJ, for the European Network on Interventions and Policies to Reduce Inequalities in Health. Tackling socioeconomic inequalities in health: analysis of European experiences. Lancet 2003;362:1409–14.[CrossRef][ISI][Medline]
3 Macintyre S, Chalmers I, Horton R, et al. Using evidence to inform health policy: case study. BMJ 2001;322:222–5.
4 Gepkens A, Gunning-Schepers LJ. Interventions to reduce socioeconomic inequalities differences: a review of the international literature. Eur J Public Health 1996;6:218–26.
5 NHS Centre for Reviews and Dissemination. Review of the research on the effectiveness of Health Service interventions to reduce variations in health. Report 3. York: NHS Centre for Reviews and Dissemination, 1995.
6 Macintyre S. Evidence based policy making. BMJ 2003;326:5–6.
7 Petticrew M, Whitehead M, Macintyre SJ, et al. Evidence for public health policy on inequalities: 1: The reality according to policy makers. J Epidemiol Community Health 2004;58:811–6.
8 Mackenbach JP, Gunning-Schepers LJ. How should interventions to reduce inequalities in health be evaluated? J Epidemiol Community Health 1997;51:359–64.[Abstract]
9 Thomson H, Hoskins R, Petticrew M, et al. Evaluating the health effects of social interventions. BMJ 2004;328:282–5.
10 Victora CG, Habicht JP, Bryce J. Evidence-based public health: moving beyond Randomized Trials. Am J Public Health 2004;94:400–5.
11 Programme Committee on Socio-Economic Inequalities in Health—Second Phase. Reducing socio-economic inequalities in health. Final report and policy recommendations from the Dutch Programme Committee on Socio-Economic Inequalities in Health—Second Phase. Den Haag: ZON MW, 2001.
12 Stronks K. Generating evidence on interventions to reduce inequalities in health: the Dutch case. Scan J Public Health 2002;30:20–5.
13 Rots-de Vries MC. Armoede en gezondheid van kinderen in de gemeente Breda. In Stronks K, editor. Sociaal-economische Gezondheidsverschillen: Van Verklaren naar Verkleinen, Deel 3. Den Haag: ZON, 2001:47–65.
14 Derriks M, Kat E de. Meer zorg, minder ziek? Evaluatie van een GG&GD interventie bij zieke leerlingen in het voortgezet onderwijs. Stronks K, editor. Sociaal-economische Gezondheidsverschillen: Van Verklaren naar Verkleinen, Deel 5. Den Haag: ZON, 2001:81–95.
15 van der Pal-de Bruin KM, de Walle HEK, et al. Effect van een landelijke en regionale interventie op de omvang van sociaal-economische verschillen in het periconceptioneel gebruik van foliumzuur. In: Stronks K, Gunning-Schepers LJ, editors. Sociaal-economische Gezondheidsverschillen: Van Verklaren naar Verkleinen, Deel 1. Den Haag: ZON, 1999:39–51.
16 de Walle HEK, van der Pal KM, de Jong LTW, et al. Effect of mass media campaign to reduce socioeconomic differences in women's awareness and behaviour concerning use of folic acid: cross-sectional study. BMJ 1999;319:291–2.
17 Abbema E, van Assema P, Kok GJ, et al. Effect evaluation of a comprehensive community intervention aimed at reducing socioeconomic health inequalities. Health Promot Int 2004;19:141–56.
18 Crone MR, Reijneveld SA, Willemsen MC, et al. Prevention of smoking in adolescents with lower education: a school based intervention study. J Epidemiol Community Health 2003;57:675–80.
19 Terstegge C, Schefman S, Mertens P, Thijs C. Tandenpoetsen op de basisschool. Effect op de verschillen in mondgezondheid tussen kinderen uit lage en hoge sociaal-economische groepen. Maastricht: University Maastricht, 2003.
20 Bongers PM, Luijsterburg PM, van den Heuvel FMM, et al. Evaluatie van nieuwe werkmethoden voor de metselploeg: opgehoogd metselen en mechanisch opperen. In: Stronks K, editor. Sociaal-economische Gezondheidsverschillen: Van Verklaren naar Verkleinen, Deel 5. Den Haag: ZON, 2001:21–38.
21 Kuijer PPFM. Effectiveness of interventions to reduce workload in refuse collectors [dissertation]. Amsterdam: Academic Medical Centre, 2002.
22 Clarenbeek J, van Sloten G. Working in sociotechnical production teams and occupational health. Experienced and conditional health effects of team working on team member and their leaders. In Stronks K, editor. Sociaal-economische Gezondheidsverschillen: Van Verklaren naar Verkleinen, Deel 5. Den Haag: ZON, 2001:63–80.
23 Sorgdrager J, Matthesius DM, Groothoff JW, et al. Een dokteres in de praktijk. Effecten van een praktijkverpleegkundige op de zorg voor patiënten met astma/COPD. Stronks K, editor. Sociaal-economische Gezondheidsverschillen: Van Verklaren naar Verkleinen, Deel 4. Den Haag: ZON, 2001:81–93.
24 Uitewaal P, Bruijnzeels M, de Hoop T, et al. Feasibility of diabetes peer education for Turkish type 2 diabetes patients in Dutch general practice. Patient Educ Couns 2004;53:359–63.[CrossRef][ISI][Medline]
25 Uitewaal PJM. Type 2 Diabetes mellitus and Turkish immigrants: an educational experiment in general practice [dissertation]. Rotterdam: Erasmus Medical Centre, 2003.
26 Poodt HD, Wierdsma AI. Verleiden tot vertrouwen. De resultaten van de inzet van lokale zorgnetwerken. In Stronks K, editor. Sociaal-economische Gezondheidsverschillen: Van Verklaren naar Verkleinen, Deel 2. Den Haag: ZON, 1999:31–47.
27 Mackenbach JP, Stronks K. A strategy for tackling health inequalities in the Netherlands. BMJ 2002;325:1029–32.
28 Rychetnik L, Frommer M, Hawe P, et al. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health 2002;56:119–27.
29 Deeks JJ, Dinnes J, D'Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7:27.
30 Erikson C. Learning and knowledge-production for public health: a review of approaches to evidence-based public health. Scand J Public Health 2000;28:298–308.[CrossRef][ISI][Medline]
31 Des Jarlais DC, Lyles C, Crepaz N, et al. Improving the reporting quality of nonrandomized evaluations of behavioural and public health interventions: the TREND statement. Am J Public Health 2004;94:361–6.
32 Rootman I, Goodstadt M, Hyndman B, et al. Evaluation in Health Promotion. Principles and Perspectives. Copenhagen: World Health Organization, 2001.
33 Whitehead M, Petticrew M, Macintyre SJ, et al. Evidence for public health policy on inequalities: 1: Assembling the evidence jigsaw. J Epidemiol Community Health 2004;58:817–21.
34 Campbell M, Fitzpatrick R, Haines A, et al. Framework for design and evaluation of complex interventions to improve health. BMJ 2000;321:694–6.
35 Kirkwood B. Making public health interventions more evidence based. BMJ 2004;328:966–7.
36 Kaufman JS, Kaufman S, Poole C. Causal inference from randomized trials in social epidemiology. Soc Sci Med 2003;57:2397–409.[CrossRef][ISI][Medline]
37 Mackenbach JP, Stronks K. The development of a strategy for tackling health inequalities in the Netherlands. Int J Equity Health 2004;3:11.[CrossRef][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||