Skip Navigation

The European Journal of Public Health 2007 17(Supplement 1):10-13; doi:10.1093/eurpub/ckm070
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow E-letters: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when E-letters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Delnoij, D. M.J.
Right arrow Articles by Groenewegen, P. P.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Delnoij, D. M.J.
Right arrow Articles by Groenewegen, P. P.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2007. Published by Oxford University Press on behalf of the European Public Health Association. All rights reserved.

Health services and systems research in Europe: overview of the literature 1995–2005

Diana M.J. Delnoij and Peter P. Groenewegen

NIVEL (Netherlands Institute for Health Services Research), The Netherlands

Correspondence: Diana Delnoij, PhD, NIVEL(Netherlands Institute for Health Services Reasearch), PO Box 1568, 3500 BN Utrecht, The Netherlands, tel: +31302729847, fax +31302729729, e-mail: d.delnoij{at}nivel.nl

Received June 7, 2007, accepted June 7, 2007


   Abstract
Top
 Abstract
Method
 Results
 Discussion
 Acknowledgements
 References
 
Introduction: Our objective, within the collaborative study SPHERE (Strengthening Public Health Research in Europe) is to give an overview of health services and health systems research in Europe, based on a search of the literature in PubMed and Embase. Method: The method used in this study consisted of: (i) A bibliometric analysis, and (ii) Classification of health services and systems research according to pre-defined criteria for a sample of 500 publications in the PubMed search. Results: Health services research is particularly strong in the Nordic countries. The number of publications on health services research has increased steadily between 1996 and 2004, 60% of the references found had a keyword related to ‘patient’. More than one-third of the references had a keyword related to ‘hospital’. The keyword ‘general practitioner’ occurred in 16% of the cases. The emphasis on this keyword was higher in those countries where the GP traditionally holds a strong position, but also in the new member states, Estonia and Slovenia. Of a smaller sample classified in depth; 57% addressed problems of efficiency and quality improvement; 27% focused on the organization of health care, cohesion and arrangement of supply according to needs and demands; only 10% addressed problems of inequalities and distribution of services. Conclusions: Health services research is a growing domain of research. As an applied discipline, health services research can be expected to closely follow political agendas. The majority of studies focus on improving the efficiency and quality of the system. Only 10% of the studies address inequalities in health utilization.

Keywords: bibliometry, Europe, health services research, literature review, public health research

Health care is one of the factors contributing to the health of individuals and populations, along with healthy lifestyles and a healthy environment. Therefore, the study of health care and the services it provides is complementary to, and partly overlaps, with the larger domain of public health research, even if ‘health’ itself is not always the topic of study or even among the set of variables taken into account.1 Health services research can be defined as ‘the multidisciplinary field of scientific investigation that studies how social factors, financing systems, organizational structures and processes, health technologies and personal behaviours affect access to health care, the quality and cost of health care and ultimately our health and well-being.’2 Health systems research is the sub-discipline of health services research that looks specifically at the health care system. A health care system can be defined as the collection of different health units that are organized and financed in order to provide a range of health services to a defined population or nation.3 The objective of this article is (i) to give an overview of health services and health systems research in Europe, based on a search of the literature in PubMed and Embase and (ii) to classify health services research according to the problems addressed, the dependent variables studied, the level of analysis and the type of data collected.


   Method
Top
 Abstract
 Method
Results
 Discussion
 Acknowledgements
 References
 
The objective of this study is to describe health services and systems research in Europe. Geographically, Europe was defined as the countries belonging to the European Economic Area (EEA), i.e. the 25 EU countries prior to January 2007 plus Iceland, Norway, Liechtenstein and Switzerland. European research was defined as research conducted by researchers affiliated with a European-based institute, and in the analyses aimed at classifying research we added the criterion that the studies should be about European health care systems.

In order to find health services research publications, we selected relevant keywords from the thesaurus of Medical Subject Headings (MeSH). However, preliminary searches conducted with that extensive list of keywords showed that this method was highly sensitive, but lacked specificity. Although ‘health services research’ is a MeSH-term, this keyword is not used consistently or in accordance with the definition of health services research.

Therefore, another strategy was designed, using a volume (2005) of the European Journal of Public Health (EJPH) which was manually scanned for articles on health services and health systems research. The MeSH terms of those articles were listed; and keywords that occurred at least twice were included. This strategy yielded the following key words: Primary health care; Physician's Practice Patterns; Physicians; Delivery of Health Care; Health Policy; Patient Satisfaction; Physician–Patient Relations; Health Services Accessibility; Health Services Needs and Demand; National Health Programs; Patient Education; Health Knowledge, Attitudes, Practice.

The sensitivity of the search strategy based on these keywords in the EJPH 2005 volume was satisfactory: 10% of the references were false positive, 10% were false negatives. The percentage of publications not describing health services research in this test-search was 31%. As a further preliminary check, the keywords were applied to all publications in PubMed between 2000 and 2005 from the Netherlands. The first 100 of the resulting references were manually scanned again and 76 were health services or health systems research. (This means that, a priori, we expected our search to overestimate the number of health services and systems research publications by 25–30%).

We selected publications with an abstract, in order to exclude letters, comments or other non-research publications. We ran our search strategy in both PubMed and Embase and combined the two, excluding duplicates. (Duplicate counting of references can also occur, e.g. when a reference has more than one country name in the address field, and such duplicates were also excluded).

Analyses
We conducted two types of analyses: a bibliometric analysis and a classification of a sample of references according to a pre-defined scheme. The bibliometric analysis took place on the combined PubMed and Embase reference database.

The reference database was built with Reference Manager. The bibliometric analysis was conducted with the Search References option in this programme. References were searched by country, by keyword and by year. Numbers of references were expressed as a rate per country in terms of population (number of inhabitants) and in terms of Gross Domestic Product (GDP).

Further classification of references was undertaken on a small (2%) random sample of the references. These references were reviewed independently by the two authors to answer the questions:

  • Does the abstract describe European research? Yes/No
  • Does the abstract describe empirical research? Yes/No
  • Does the abstract describe health services research? Yes/No

If it described health services research, the study was classified using a scoring form.

The scoring form was developed by the two authors of this article on the basis of a previous study of Dutch health services research.4 The main categories on the scoring form were:

Problems addressed:

  • Studies of health care organization, cohesion and arrangement of supply according to needs/demands (in short: organization, cohesion and arrangement);
  • Studies of inequalities and distribution;
  • Studies of efficiency and quality improvement.

Dependent variables studied:

  • Utilization and costs;
  • Health and quality of life;
  • Both;
  • Neither.

Level of analysis:

  • Micro-level;
  • Meso-level;
  • Macro-level;
  • Multi-level.

Type of data:

  • Quantitative data collected for research purposes (new or secondary analyses);
  • Qualitative data collected for research purposes (new or secondary analyses);
  • Administrative data;
  • Documents;
  • Literature review;
  • Clinical data;
  • Combinations/other.

Scoring took place in three rounds. In the first round abstracts of articles clearly not meeting the inclusion criteria were excluded. (e.g. author address New South Wales, not Wales; epidemiological studies; public health research focusing on determinants of health without a link to the health ‘care’ system; clinical research focusing on the development and testing of new diagnostic or treatment procedures; psychological research on individuals coping with disease.)

The two authors reviewed and scored the publications separately. Kappa scores were calculated and scores measuring level of agreement between the two reviewers were acceptable to high (ranging from {kappa} = 0.72 to {kappa} = 0.90), except for the classification of the main problems addressed ({kappa} = 0.24).

In the second round, the authors discussed the cases about which they disagreed in order to refine the classification criteria. Some cases of disagreement were the result of simple mistakes; however, the majority led to the definition of additional criteria for classification. Finally, in the third round reviewers reassessed abstracts about which they initially disagreed in order to come to a final classification. Results of this final classification are presented in this article.


   Results
Top
 Abstract
 Method
 Results
Discussion
 Acknowledgements
 References
 
Figure 1 shows the percentage of PubMed publications on health services research between 1995 and 2005 published in English or any other language for selected countries, namely those in which German, French, Spanish or Italian is (one of) the native languages.


Figure 1
View larger version (15K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Figure 1 Percentage of health services research publications in PubMed published in English between 1995 and2005 by a first author based in France, Spain, Germany, Austria, Italy, Switzerland or Belgium

 
From figure 1, it is clear that in Austria, Italy, Switzerland and Belgium more >90% of the publications found are in English. In Germany, the percentage of publications in English is 87%, in Spain it is 78% and in France 66%. For our further analysis, we looked at English-language publications.

The initial PubMed search prepared for the bibliometric analysis contained 21 051 English-language references. After initial exclusions and removal of address field duplicates, the final PubMed set consisted of 20 226 references. The Embase search resulted in 12 871 references. After combining the two sets, there were 32 126 references. There were 971 duplicates (4.8% of the PubMed search; 7.5% of the Embase search).

Bibliometric analysis
The number of publications on health services research increased steadily between 1996 and 2004. The average annual growth rate in this period is +34.9%.

In table 1, the number of references per country is presented. Data are given in crude numbers, and corrected for the number of inhabitants of a country as well as for GDP.


View this table:
[in this window]
[in a new window]

  		Table 1 Number of references per country: absolute numbers, and numbers per 1000 of the population and $1 000 000 GDP

 
There are large differences in the numbers of references per country, both in absolute as well as in relative figures. If the results are related to population size, Sweden, Finland, Iceland, Norway, Ireland and the Netherlands have high numbers of references. Relative to GDP Sweden and Finland have the highest production, but are followed by Estonia.

Of all the references, 60% had a keyword related to ‘patient’. This includes, for example, more specific keywords such as ‘patient advocacy’ or ‘patient compliance’. It also includes the keyword ‘patient satisfaction’, which by itself accounts for 15% of the total references. More than one-third of the references have a keyword related to ‘hospital’. The keyword ‘general practitioner’ occurs in 16% of the cases. The keywords ‘risk’ and ‘education’ occur in 14% of the references; ‘physician’ in 11% each. Over the years studied, the share of most of these keywords remains quite stable, with the exception of the keywords ‘General Practitioner’ and ‘patient satisfaction’(both showing a declining trend), and the keyword ‘risk’ (showing an increasing trend).

Countries differ profoundly in the emphasis on different keywords within their total set of publications. As an example of these differences, in figure 2 the percentage of publications per country with the keyword ‘general practitioner’ is presented.


Figure 2
View larger version (17K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Figure 2 ‘General practitioner’ as percentage of the total number of references per country

 
In figure 2, it can be observed that the emphasis on the keyword ‘General Practitioner’ is high in those countries where the GP traditionally holds a strong position in the health care system (the UK, Denmark and the Netherlands), but also in the new member states Estonia and Slovenia. ‘Patient satisfaction’ (data not shown in the figure) seems to be more important in the social health insurance systems of Austria, Germany, Belgium and France (but also in Slovenia), and less important in central and eastern European countries. ‘Education’ (data not shown in the figure) receives relatively more attention in many of the new EU member states, perhaps reflecting the fact that their health systems in transition have higher training needs than established systems.

Further classification of references
The first scoring round, using a 2% random sample yielding 500 abstracts revealed 42 abstracts not fitting the inclusion criteria and 36, which did not describe empirical research. The remaining abstracts (n = 422) were reviewed in order to identify studies belonging to the domain of health services research. Of the 422 publications reviewed, 53% (n = 225) were classified as health services research.

Of these 225 publications, 56.9% addressed problems of efficiency and quality improvement (table 2); 26.7% focused on the organization of health care, cohesion and arrangement of supply according to needs and demands; only 9.8% addressed problems of inequalities and distribution of services. About one-third of the studies looked at utilization and/or costs as the dependent variable. However, 56.9% of the publications could not be classified to either of the previously defined categories for dependent variables. In almost half of the studies, the level of analysis was the micro-level (the patient level); in almost a quarter the level of analysis is at the meso-level (providers, insurers or regions). In more than half (56.1%) of the studies, quantitative data were collected. Qualitative data were collected in 15.2% of the studies, and administrative or clinical data in 7.2% of the studies.


View this table:
[in this window]
[in a new window]

  		Table 2 Classification of health services research according to problems addressed, dependent variables studied, the level of analysis and method of data collection

 

   Discussion
Top
 Abstract
 Method
 Results
 Discussion
Acknowledgements
 References
 
This article has presented the results of a bibliometric analysis of health services research and a classification of a small sample of health services publications according to pre-defined criteria.

The results of the bibliometric analyses have to be interpreted with caution. Although we expected our search strategy to have a satisfactory specificity (preliminary searches indicated that about 30% of our references database would actually consist of publications that were ‘not’ health services research), in the final database about 47% of the references were not health services research, but instead were epidemiological, clinical or psychological research. We based our search strategy on the 2005 volume of the European Journal of Public Health. Apparently, the keywords that describe health services research in this journal are not fully representative of health services research published in other journals. Keeping this limitation in mind, health services research can be regarded as a growing domain with an annual average growth rate of references of 34.9% over the past 10 years.

There are large differences between countries. Health services research seems to be strong in most of the Nordic countries, and comparatively weak in small countries, in social health insurance countries and in transitional countries. Some countries, notably Germany, Italy and Poland, can be expected to catch up in the future, that is, if they continue their current pace of growth. It should be noted, however, that the number of references for some countries is underestimated, because we limited the search to English-language publications.

In comparing the ‘health services research output’ of countries, we adjusted the number of references for population size and for GDP. This was done in order to identify countries where health services research is comparatively strong or weak, assuming that the number of references in scientific journals is a good proxy for the strength of a research field. However, this assumption needs further testing in future studies.

The topics studied in health services research were classified by two independent reviewers, using a pre-defined scoring form. More than half of the studies focused on problems that deal with the efficiency and quality of health care. This could be caused by the fact that the issues of quality and safety have been high on the health policy agenda in the past decade. But, the large proportion of studies in the field of quality and efficiency is comparable with previous findings for health services research conducted in the Netherlands in 1991–92.4

As an applied discipline, health services research can be expected to follow political agendas closely.5,6 This is demonstrated by the fact that there are some clear relations between health system characteristics and the share of certain keywords in the total number of references per country. ‘General practitioner’ is an important keyword in those countries where GPs traditionally hold strong positions; ‘patient satisfaction’ is emphasized in social insurance systems, but less so in Central and Eastern European countries; and ‘education’ is an important topic in health systems in transition. The relation between health services and systems research and health policy is another area in which further research is warranted.

The topic of inequalities has been addressed in <10% of the studies. It is unclear to what extent this reflects a political reality or an implicit ‘division of tasks’ between public health research/epidemiology and health services research, with social inequalities in health and health care being addressed in the former domain. However, based on the classification of topics studied, we come to the tentative conclusion that health services and systems research tends to overemphasize the importance of doing things right (quality and efficiency), perhaps at the expense of doing the right things (arrangement/cohesion) for the right people (inequality). In this respect, health services research might benefit from a public health perspective.


   Acknowledgements
Top
 Abstract
 Method
 Results
 Discussion
 Acknowledgements
References
 
SPHERE (co-ordinator Prof. Mark McCarthy) was funded by the European Commission Sixth Framework Research programme, during the period 2005–07. Data from this article were presented at the 14th Conference of the European Association for Public Health (EUPHA), Montreux, Switzerland, 16 November 2006.

Conflict of interest: None declared.


   References
Top
 Abstract
 Method
 Results
 Discussion
 Acknowledgements
 References
 
1 Groenewegen PP, Delnoij DMJ. Health services and health systems research: domain and methodology. In: Working paper for the SPHERE meeting in Scarborough (2005) June.

2 Lohr KN, Steinwachs DM. Health services research: an evolving definition of the field. Health Serv Res (2002) 37:7–9.[Web of Science][Medline]

3 Øvretveit J. Comparative and Cross-cultural Health Research. A practical guide (1998) Radcliffe Medical Press.

4 Groenewegen PP, Bensing JM, Bosman J. Een inventarisatie van gezondheidszorgonderzoek in Nederland (An inventory of health services research in the Netherlands). In: NIVEL, oktober 1993, in: Raad voor Gezondheidsonderzoek (Health Research Council). (1994) RGO: 's-Gravenhage. Advies Gezondheidszorgonderzoek (Advies nr. 11).

5 Ten Asbroek AHA. Health Services Research at Work for National Health Policy (dissertation) (2006) Amsterdam: University of Amsterdam.

6 Bensing JM, Caris-Verhallen WM, Dekker J, et al. Doing the right thing and doing it right: toward a framework for assessing the policy relevance of health services research. Int J Technol Assess Health Care (2003) 19:604–12.[Web of Science][Medline]


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow E-letters: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when E-letters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Delnoij, D. M.J.
Right arrow Articles by Groenewegen, P. P.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Delnoij, D. M.J.
Right arrow Articles by Groenewegen, P. P.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?