European public health research literatures—measuring progress
Mark McCarthy1 and Aileen Clarke2
1 University College London, Gower Street, London WC1E 6BT, UK,
2 Health Sciences Research Institute, Warwick Medical School, University of Warwick, Coventry, CV4 7AL, UK
Correspondence: Aileen Clarke, Health Sciences Research Institute, Warwick Medical School, University of Warwick, Coventry, CV4 7AL, UK, Tel: +44 (0) 24 76 51089, Fax: +44 (0) 24 76 51089, e-mail: aileen.clarke{at}warwick.ac.uk
Received June 6, 2007, accepted June 7, 2007
Public health research makes an important contribution to Europe and its citizens. The collaborative study SPHERE (Strengthening Public Health Research in Europe) was developed in response to a call from the European Commission's Sixth Framework Research Programme. It includes 19 partners in 12 countries, structured around the European Public Health Association and led by the UK Faculty of Public Health. This article provides an overview to a Supplement to the European Journal of Public Health, including eight original papers making bibliometric studies on public health literatures, and two commentaries from European experts. European public health publications are similar in number to the US, if a range of databases are consulted. The majority are published in English and tend to be descriptive rather than evaluative. The integration of Europe as a political, social and economic unity will strengthen public health collaboration and funding across borders in Europe.
Public health research has a strong history in Europe. It has contributed knowledge to achieve our present levels of health, and also demonstrated challenges yet to be overcome. Public health research is aimed at the organizational rather than the individual level of health—the fields of health promotion, disease control and health services. Knowledge of how diseases are caused can make it possible to control them even without detailed understanding of biological mechanisms, and evaluative research can demonstrate whether policy choices and interventions are effective across differing social contexts. Public health surveillance provides information both about health status and health needs, while analysis of health trends can demonstrate both expected and unexpected impacts of broader social forces.
Public health policy and practice contribute at least as much now—and, historically, arguably more—to population disease control as medical treatment does.1 There are innovative methodological developments in public health research.2 Yet this is not reflected in the allocation of health research funds. The experimental laboratory continues to be the paradigm for medical research, in part due to a new dynamic from genetic and molecular research, with pharmaceutical ‘spin-off’ companies providing financial drive. Public health research rarely benefits from this ‘economic drive’,3 and its end product, of improved health through prevention, for example, is difficult to quantify in financial terms.4 An analysis of expenditure on ‘health’ research in the UK showed that only 2–3% of spending was on health services research,5 and a more recent report by a group of British research funding agencies also showed that most research funding goes to ‘underpinning’ and ‘aetiology’ research, while only 7% on ‘prevention’ and ‘health services’ together.6
Support for health research has been growing through the programmes of the European Commission's Directorate for Research, first established in the 1970s. However, in the Sixth Framework Programme, covering 2002–06, public health research was separated organizationally from medical research, with a constrained budget and limited research tasks.7,8 Within this programme, the Commission put forward a call for research ‘to examine the ways and means by which research priorities in the field of public health at the European level can be identified, taking into account the interaction and needs of stakeholders—policy-makers, the scientific community and end users—and how research outcomes can be best integrated into the policy process’. SPHERE (Strengthening Public Health Research in Europe) was proposed in response to this call.
SPHERE
SPHERE was developed in collaboration with the European Public Health Association (EUPHA), which has a structure of academic thematic Sections and representative national public health associations.9 The project was set up with multiple partners from a number of different countries. In all, there were 17 work packages, undertaken by 19 partners from 12 countries (including five EU accession countries). Information was collected across all European Economic Area (EEA) countries and also for some other countries that were members of EUPHA.
The project's formal ‘work packages’, undertaken by contributing members, were in three groups: analysis of European public health literature; mapping and discussion of European public health research by stakeholders; and management of the consortium. The papers in this Supplement represents findings from the analysis of public health literature. The six thematic review topic areas were selected in part because of their diversity within public health research, and in part because they reflected Sections within EUPHA. We chose not to address the literature according to discipline, e.g. epidemiology, statistics, sociology, psychology, etc. but rather to see how these disciplinary areas fitted within practical themes. The overall aim of the literature reviews was to make recommendations for future research policy and funding, based on an identification of the strengths and gaps in the public health literature produced in Europe over the past 10 years.
The six thematic papers are supplemented by a paper focusing on French databases, an overview across all themes, and two commentary papers. SPHERE is a 3-year project, covering the years 2005–07, and has used annual EUPHA conferences to present the work to a wider audience of researchers. The two commentary papers was commissioned to provide informed discussion: the authors were asked to give their views impartially.
 |
Procedure |
|---|
 Top Procedure LimitationsDiscussionReferences |
|---|
A common set of methods agreed for the papers in this Supplement is shown in the Appendix. A framework for each area of research was developed with definitions, key words and inclusion and exclusion criteria. Search strategies based on those frameworks were designed, piloted and refined. For most of the reviews and overviews, MedLine/PubMed was used as the main database, although for some Embase was also used. For each of the reviews, a further process of identification of relevant publications and further classification using a sample of the output was then undertaken. Rates of publications by country, by population and by Gross Domestic Product (GDP) were calculated. Each of the reviews faced particular technical issues in deciding how to define topic areas adequately, which data sets to interrogate and how to identify research papers to include (aiming to maximize specificity, and positive predictive value).
| Box 1 SPHERE Guidelines for public health literature overviews Reports should include:
Methods
Results Results should be presented to indicate:
Results should also be presented as a text-based overview which will provide:
Discussion, Conclusions and Recommendations This section should include:
|
|
Limitations |
|---|
|
TopProcedureLimitationsDiscussionReferences |
|---|
As is noted in the Public Health Management literature review in this Supplement,10 ‘Public health is not a well-defined discipline with clear boundaries in research terms since it includes contributions from a wide range of social and behavioural sciences. Consequently, this has made the task of searching the databases both especially complicated and perhaps a more subjective exercise than might have been desirable.’
There were considerations of which index to use. Medline puts emphasis on biological and clinical research, particularly in the USA, with the broader fields of health sciences and administration as supplementary areas. Embase has a greater European coverage, but emphasizes drugs and pharmacology research as well as psychiatry, addictions and health policy and management. The overlap between these databases appears to be stronger for randomized controlled studies than observational studies.11,12 There is no primary database for public health research.
There are also obvious drawbacks in the use of these large bibliographic databases to measure research outputs. Research must be published in a journal which is indexed by the database. Indexing must be accurate: however thorough a search strategy might be, some mis-coding and misallocation of publications within databases will inevitably occur. And language and publication bias might be operating, since apparent research production from certain countries, particularly Eastern European countries, for some topic areas appeared low.
|
Discussion |
|---|
|
TopProcedureLimitationsDiscussionReferences |
|---|
Bearing these limitations in mind, a number of findings of generalizable interest have emerged from this work. One finding is that different countries appear to be strong in different research areas reflecting more local interest, and historical and cultural patterns, (for example France is a strong publisher of research in genetic epidemiology). Another finding is that in some topic areas (for example in the health promotion and infectious disease topic areas) more observational than evaluative interventional research was undertaken. Perhaps it is easier, or more acceptable, to undertake research to document problems than to investigate interventions to ameliorate problems. Yet studies of interventions, either specific or policies and politics, are greatly needed to develop the evidence base for public health practice.
This is the first time that a broad approach has been taken to describing the public health research literature across European countries. The Alliance for Health Policy and Systems Research, supported by the World Health Organization, made a bibliometric study of the world health systems research literature.13 Citations were divided by world geography north/south, with European and American literature together. Of 19 health systems research topics defined, seven were frequent in the ‘north’ literature: pharmaceutical policy and management; quality of care; organization and delivery; programme evaluation; costing and cost-effectiveness; policy process; and information, education and communication. A more recent bibliometric study from the USA 14 investigated occupational health, epidemiology and public health literatures according to world regions. They used different databases and search strategy from our work, identified substantially fewer publications and reported a higher proportion—61%—of public health literature as emanating from the USA. They did not investigate specialties within public health or countries within Europe.
The diversity of languages in Europe is important. Findings from public health research, as with other social sciences, need to be assessed within their context of culture and policy setting. Most countries have developed their scientific disciplines through own-language journals and these remain important means of communication. 15 The SPHERE literature reviews generally included articles published in Europe in all languages, as the literature databases provide translations of titles. Our overview from the world public health literature16 would suggest that up to 25% of public health research publications in several European countries are not in English. Restrictions on journals included within the databases may further bias against non-English language journals, and it is likely that more ‘grey’ literature will be in a language other than English. Yet, public health scientists, perhaps more than their practitioner colleagues, are likely to be able to use the English language, and there would be considerable advantages for communication in Europe if own-language publications provided accompanying scientific abstracts in English.
Bibliometry is only one way of assessing public health research, and it has acknowledged limitations. In allocating by country, it is necessary to consider whether to take the country of a lead author, or all authors or authors as a fraction of one. Discussing the use of bibliometry, Wallin17 and Nederhof18 observe that citation indexes cover up to 90% of life science research, but much less coverage in the humanities and social sciences. For example, comparing social science publications from Dutch universities with citation listing found a range from only 2% for public administration to 62% for experimental psychology; and Australian social sciences and humanities research (but excluding behavioural sciences) showed a coverage range of 25–44%. In explanation, Nederhof19 recognizes that social and humanities research has greater national and regional orientation, greater publication in books, delays in indexing, a different pace of theoretical development and more often a single scholar rather than team research.
Bibliometry is also different in character from the more frequent use in public health research of databases for systematic reviews.16 Reviews use reproducible methods of identification of literature, but seek high sensitivity through interrogation across a wide range of literature sources, and high specificity by hand-searching and reduction down to a closely focused research question and methodology. In contrast, describing publication patterns by bibliometry uses a broader brush, a more summative approach.14 Yet, it is of interest that our studies reduced down to include 10–20% of the initial literature assessed, a proportion similar to systematic reviews.
Our SPHERE partners found differences in the distributions of the different literatures across Europe. Although guidelines were provided, the actual reviews each used were slightly different approaches, and will have underlying biases. The differences were evident in some studies where Embase and Medline were both used. We may assume there are some undefined but systematic differences in referencing public health research between these two databases, as well as difficulties in standardizing MeSH terms in the European context. The limitations of comparing laboratory sciences and social sciences may also contribute to these differences.
Our study has not sought to assess quality of product. Research quality is difficult to define, especially in comparing across disciplines. Journal impact factors and citation indices describe the extent to which research is used by other researchers but are open to manipulation. Quality may not be well assessed by the current peer review process. As Wallin17 says: ‘The quality of manuscripts can, therefore, never be completely assessed by a few experts. This is why it is not surprising that many articles even in the most prestigious journals remain uncited, just as, on the other hand, articles in more humble (low prestige) journals can attract many citations. It is only after the articles have been exposed to the global research society's "peer review" that the original reviewers’ evaluation of the manuscript's importance becomes irrelevant’.
This broad overview of research and research gaps in European public health allows one to ask some important questions, in order to focus a future research agenda:
- Is ‘enough’ research being done? Is ‘good enough’ research being done?
- How can research policy be used to direct research to areas where need is greatest and where most benefit can be achieved?
- Are there different ‘needs for research’ across Europe and how generalizable is public health research between different countries?
- Is there an ‘inverse research law’ by country (less research where more is needed) and how could this be dealt with between countries?
- How can research funders best direct research to the policy/fiscal governmental level where public health impact is often greatest?
- Can studies between countries be used more actively to investigate how national policies (for example in trade, environment, infectious disease and population mobility) contribute to international public health differences?
This study was undertaken at a time of considerable political change in Europe. The European Union has developed in size and character, with 12 new countries joining over the period 2004–07, while the European Economic Area (including non-aligned countries Iceland, Norway and Liechtenstein), and Switzerland are closely associated with the EU. More countries to the east and south east of Europe have candidate or associate status. This integration in Europe brings challenges to public health research— the different traditions in research priorities, in resources for academic work, in linking research priorities with national or local policy goals, and in improving communication between practitioners and with the public. SPHERE is contributing by mapping production of public health research at both national and European levels, and developing the knowledge base for more effective commissioning of health research in the future European Research Area.20
Conflict of interest: None declared.
|
References |
|---|
|
TopProcedureLimitationsDiscussionReferences |
|---|
1 Mackenbach JP. The contribution of medical care to mortality decline: McKeown revisited. (1996) 49:1207–13.
2 Clarke A, McCarthy M, Álvarez-Dardet C, et al. New directions in European public health research: report of a workshop. J Epidemiol Commun Health (2007) 61:194–7.
3 McCarthy M. Competitive markets for European public health research? Eur J Pub Health (2007) 17:235–6.
4 Leiner G, Schuppe M, editors. Health and wealth: economic and social dimensions of health. In: Bad Hofgastein: International Forum Gastein (2002).
5 Dash P, Gowman N, Traynor M. Increasing impact of health services research. Br Med J (2003) 327:1339–41.
6 UK Clinical Research Collaboration. UK health research analysis. (2006) London: UK Clinical Research Collaboration.
7 McCarthy M. Public-health research in Europe. Lancet (2002) 36:1439.
8 Saracci R, Olsen J, Hofman A. Health research policy in the European Union. Br Med J (2005) 330:1459–60.
9 McCarthy M. Strengthening public health research in Europe - SPHERE. In: Urbanisation and health—Tellnes G, ed. (2005) Oslo: Oslo Academic Press. 338–47.
10 Hunter DJ, Brown J. A review of health management research. Eur J Pub Health (2007) 17(Suppl 1):33–37.
11 Sampson M, Barrowmann NJ, Moher D, et al. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol (2003) 56:943–55.[CrossRef][ISI][Medline]
12 Lemeshow AR, Blum RE, Berlin JA, et al. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol (2005) 58:867–73.[CrossRef][ISI][Medline]
13 Soteriades ES, Falagas ME. A bibliometric analysis in the fields of preventive medicine, occupational and environmental medicine, epidemiology, and public health. BMC Public Health (2006) 6:301.[CrossRef][Medline]
14 Policy Alliance for Health, Research Systems. Trends in international publications on health systems research. (2004) Global Forum for Health Research: Geneva.
15 McCarthy M, Panà A. Public health journals: an endangered species? Eur J Public Health (2005) 15:222–3.
16 Clarke A, Gatineau ML, Grimaud OMR, et al. Bibliometric analysis of public health research in Europe. Eur J Pub Health (2007) 17(Suppl 1):43–49.
17 Wallin JA. Bibliometric methods: pitfalls and possibilities. Basic Clin Pharmocol Toxicol (2005) 97:261–75.[CrossRef]
18 Nederhof AJ. Bibliometric monitoring of research performance in the social sciences and the humanities: a review. Scientometrics (2006) 66:81–100.[CrossRef][ISI]
19 Kahn KS, ter Reit, G Glanville, J et al. Undertaking systematic reviews on research on effectiveness. (2001) York: NHS Centre for Reviews and Dissemination.
20 Commission. European. The European research area: new perspectives. Green Paper. Available at: http://ec.europa.eu/research/era/consultation-era_en.html#greenpaper.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||