The European Journal of Public Health Advance Access originally published online on March 8, 2006
The European Journal of Public Health 2006 16(6):652-659; doi:10.1093/eurpub/ckl023
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Health Services Research |
Made in the USA: the import of American Consumer Assessment of Health Plan Surveys (CAHPS®) into the Dutch social insurance system
Diana M. J. Delnoij1, Guus ten Asbroek2, Onyebuchi A. Arah2, Johan S. de Koning2, Piet Stam3, Aldien Poll4, Barbara Vriens4, Paul Schmidt4 and Niek S. Klazinga2
1 NIVEL (Netherlands institute for health services research), PO Box 1568, 3500 BN Utrecht, The Netherlands
2 Academic Medical Centre of the University of Amsterdam, Department of Social Medicine, PO Box 22660, 1100 DD Amsterdam, The Netherlands
3 Agis Zorgverzekeringen/Erasmus University Rotterdam, Institute of Health Policy and Management, PO Box 1738, 3000 DR Rotterdam, The Netherlands
4 Agis Zorgverzekeringen, PO Box 19, 3800 HA Amersfoort, The Netherlands
Correspondence: Diana M. J. Delnoij, NIVEL (Netherlands institute for health services research), PO Box 1568, 3500 BN Utrecht, The Netherlands, tel: +31 30 2729847, fax: +31 30 2729729, e-mail: d.delnoij{at}nivel.nl
Received March 1, 2005, accepted January 19, 2006
 |
Abstract |
|---|
 Top Abstract MethodsResultsDiscussionAPPENDIXReferences |
|---|
Background: In the Netherlands, managed competition between health plans has been introduced. For Dutch health plans this implies that they need to collect data about their own performance and that of the care providers they contract. To that end, Consumer Assessment of Health Plan Surveys (CAHPS) instruments have recently been adopted by a large Dutch health plan. Objectives: This paper presents the results of a validation study of the Dutch version of the CAHPS Adult Commercial questionnaire. The questions addressed are as follows: Can this questionnaire be adapted for use in the context of the Dutch insurance system? and Can it generate valid information about the quality of health care and the performance of Dutch health plans? Methods: The translated questionnaire has been mailed to a sample of 977 enrollees. The psychometric properties of the translated instrument have been studied, and the results have been compared with those of other Dutch and American studies. Results: The net response rate was 51% (n = 500). In general, the questionnaires were filled out completely and consistently. Principal component analyses revealed a factor that can be labelled as patient-centredness in the primary process. It contains the domains that in the CAHPS literature are described as ‘courteous/helpful staff’ and ‘doctors communicating well’. Conclusions: The translated version of the CAHPS Adult Commercial questionnaire is a promising tool for Dutch health plans. More research is needed on the external and the content validity of these questionnaires in the Dutch context.
Keywords: health services research, patient satisfaction, questionnaire development, social insurance
In a number of countries, e.g. Germany, Switzerland, and the Netherlands, managed competition between health plans has been introduced.1 In 1992, in the Netherlands free choice of health plans has been introduced along with a system of regulated price competition between health plans. It is expected that enrollees will choose the cheaper insurance with the better service, thus inducing health plans to increase their efficiency and service orientation.2 For Dutch health plans this implies that they need to collect data about their own performance and that of the care providers they contract, as perceived by their enrollees. They can use this information in their negotiations with contracted providers, and as consumer information on which enrollees can base their choice of health plan.
In their search for useful tools that measure providers' performance, health plans can choose from a variety of ‘families’ of questionnaires that measure the quality of care from the patient's perspective or the patient-centredness of health care.3–15 All these questionnaires measure patients' experiences rather than patient satisfaction.16 The domains covered are also comparable to a certain extent.17 All the instruments focus on different aspects of the patient-centredness of health care such as respect, dignity, prompt attention, autonomy, etc.
These domains are also addressed by CAHPS® (Consumer Assessment of Health Plan Surveys).18–21 CAHPS consists of a set of standardized questionnaires and reporting methods, aimed at providing consumers with the information they need when choosing a health plan.22 CAHPS focuses on the service of the health plan itself and the relation between health plan performance and access to and quality of care. CAHPS is therefore particularly suitable for use by Dutch health insurers that have to act as strategic purchasers of care.
CAHPS instruments have been adopted by a large Dutch health plan (Agis) that has a major market share in the city of Amsterdam and the central regions of the Netherlands. Agis has field-tested a translated version of the CAHPS Adult Commercial questionnaire, version 3.0. This paper presents the results of a validation study of that Dutch version of CAHPS. The research questions are as follows:
- Can the American CAHPS Adult Commercial questionnaire be adapted for use in the context of the Dutch insurance system?
- Can this questionnaire generate valid information about the quality of health care and the performance of Dutch health plans?
The second question could be addressed in the validation study only to a limited extent, through comparison of the test results with data from other Dutch studies and with the American benchmark data.
|
Methods |
|---|
|
TopAbstractMethodsResultsDiscussionAPPENDIXReferences |
|---|
Translation of the CAHPS Adult Commercial questionnaire
The 67 items in our questionnaire (see Appendix) stem mostly from the American CAHPS Adult Commercial questionnaire and the supplementary CAHPS questions. These supplementary questions also addressed the fluency of the respondent in—in our case—the Dutch language and the use of interpreters. The questions were added because
10% of the Dutch population, and close to 50% of the Amsterdam metropolitan population covered by Agis, consists of migrants. All the selected questions were translated forward and backward by a total of four interpreters, using the standard procedure for translating survey instruments (figure 1).
|
Fielding the questionnaire
From its general database of the insured population, Agis selected a representative sample of 1000 adult insured (aged 18 or older) who were insured on 1 April 2003, i.e. 6 months prior to the sample selection. A check was conducted whether these people had not recently deceased. After this first cleaning of the data, 977 insured remained in the pilot and received a first postal questionnaire in the beginning of November 2003. After three reminders, a total of 544 respondents returned their questionnaire. Fresh questionnaires were sent with the second reminder. Figure 2 shows the number of questionnaires returned by date. The peaks correspond with the reminders. Late responders (defined as respondents whose questionnaire was received after 15 December 2003) are significantly more often male (
2 = 6.71, P = 0.01) and tend to be younger, though this relationship is not statistically significant. Among the respondents, people over 55 were overrepresented (2 = 47.9, P = 0.00).
|
The respondents are also older than the general Dutch population (7% is over 80, compared to 4.5% of the adult Dutch population23) and on a 5-point scale more often report a fair or poor (as opposed to good, very good, and excellent) health status (32% compared to 19.4% in the Dutch population24). This should be kept in mind when comparing the results with those of other Dutch studies.
Validation of the questionnaire
In the validation process, the subsequent steps were followed that are described in the CAHPS guidelines T4: Preparing the Data for Analysis, Document Number 14 from the CAHPS® Kit (http://www.cahps-sun.org/Cahpskit/KitsLogin.asp). This means that respondents were excluded who did not confirm that they were insured with Agis (n = 17); who had not been insured with Agis for at least 12 months (n = 9); and who stated that someone else had filled out the questionnaire for them (n = 17). After that, we checked whether the questionnaire was filled out completely. This was done by looking at a set of 19 so-called key questions, of which at least 10 should be answered. Incomplete questionnaires (n = 1) were excluded from further analyses.
The database that had thus been cleaned consisted of 500 cases and was used to look at the following:
- The consistency of the data: We checked whether respondents followed skip-rules and whether they gave consistent answers.
- Frequencies and missing values: Questions with more than 5% missing values or extremely skewed items, with more than 90% of the respondents choosing the most positive answer, were considered to be candidates for exclusion from a next version of the questionnaire.
- Underlying factors, scales, and reliability: Principal component and reliability analyses were conducted on the questions with respect to patients' experiences with their health care in the past 12 months.
- Comparability of the findings with results from other Dutch patient surveys and the American National CAHPS Benchmarking Database: The results of the Dutch pilot were compared with the American data and it was tested whether relations that are reported in the American literature can be reproduced in the Dutch data. This was be done by looking at the relation between respondents' evaluations of care and their age, level of education, and general and mental health status. Multiple regression analyses were carried out on the general ratings of the personal doctor, the specialist, the health care in the past 12 months, and the health plan.
|
Results |
|---|
|
TopAbstractMethodsResultsDiscussionAPPENDIXReferences |
|---|
Consistency of the data
In general, the questionnaires were filled out completely and consistently. This became evident from examining the completeness with which the 19 so-called key questions were filled out (only one case had to be excluded because of incomplete data). But it is also the conclusion that was drawn after examining whether respondents followed skip-rules and whether they gave consistent answers. There was always a limited number of respondents who did answer questions that they were supposed to skip, but this number was generally between 20 and 30 respondents (so
5% of the respondents); with a minimum of 4 (respondents giving a score between 0 and 10 for their specialist, whereas they had previously answered that they had not visited a specialist) and a maximum of 49 (respondents stating to have had problems contacting the customer service of the insurance company, whereas they had previously stated not to have tried contacting the customer service). An inconsistency not related to skip instructions in the questionnaire, was found for eight respondents who said that their specialist was their personal doctor, whereas earlier they had answered that they had no personal doctor. Other major inconsistencies were not detected in the data.
Frequencies and missing values
Table 1 shows the frequencies of respondents using health care and services offered by the insurance company. In table 1, it can be seen that in the past 12 months only nine respondents said they had needed the help of an interpreter in their contact with health care providers. Some other services too were used by a relatively small percentage of respondents. This is the case for out-of-hours care (n = 81). Also, comparatively few people had sought information about the insurance company (n = 76) or had filed a complaint (n = 60).
|
In the past 12 months, about one-third of the respondents used the out-of-hours services of family physicians (n = 169, not in table), and about two-thirds had seen a family physician or specialist in the past 12 months not counting emergency or out-of-hours care (n = 313, not in table). Comparatively few people (14%, n = 70, not in table) visited an emergency department of a hospital.
The respondents, who had used care, were asked about their experiences with respect to access to services, prompt attention, communication, quality of information, etc. As expected, the distribution of scores on the frequency with which quality standards are met are skewed towards positive answers. But none of the items violated the practical rule set in advance, that no more than 90% of the answers should be in the most positive category.
Factors, scales, and reliability
The questionnaire contains a series of questions about the patient-centredness of care received in the past 12 months. Those were as follows:
In the past 12 months, how often
- Were you taken to the exam room within 15 min of your appointment?
- Did office staff at a doctor's office or clinic treat you with courtesy and respect?
- Was office staff at a doctor's office or clinic as helpful as you thought they should be?
- Did doctors or other health providers listen carefully to you?
- Did doctors or other health providers explain things in a way you could understand?
- Did doctors or other health providers show respect for what you had to say?
- Did doctors or other health providers spend enough time with you?
- Did you find it difficult to talk to a doctor or other care provider or to understand them, because you spoke different languages?
On these items, we performed a principal component analysis with varimax rotation. This resulted in a two-factor solution. Item ‘i’ (taken to the exam room within 15 min) had a factor loading of <0.40 on both factors. In the American data too, this item did not fit into the observed factor structure.25 The items that did have a factor loading of over 0.40 on factor 1 were items ‘(ii)’ through ‘(vii)’. Factor 2 consisted of items ‘(viii)’ (spoke different languages) and ‘(vii)’ (spend enough time with you). Because the latter item ‘(vii)’ also loaded high on factor 1, we decided to run another factor analysis only on items ‘(ii)’ through ‘(vii)’. This resulted in a one-factor solution. Table 2 shows the factor loadings of the different items on that factor. This factor can be labelled as patient-centredness in the primary process, and it contains the domains that in the CAHPS literature are described as ‘courteous/helpful staff’ and ‘doctors communicating well’. In American validation studies too, these two domains can be forced into one factor. But in the American data, the items ‘(ii)’ and ‘(iii)’ about office staff then have factor loadings of below 0.70.25 This is not the case in the Dutch data. Here, item ‘(v)’ (explain things in a way you could understand) has a comparatively low factor loading.
|
Subsequently, we calculated the reliability of the scale for patient-centredness of the primary process. Cronbach's alpha was 0.83. So the six items form a reliable scale. The scale score correlates significantly and positively with respondents' ratings as follows:
- General rating of the quality of health care on scale of 0–10 (Pearson's r = 0.51, P = 0.00).
- General rating of their personal doctor or nurse (Pearson's r = 0.47, P = 0.00).
- General rating of their specialist (Pearson's r = 0.38, P = 0.00).
- General rating of their health plan (Pearson's r = 0.30, P = 0.00).
This indicates that enrollees with better experiences in the primary process tend to give higher ratings. Significantly, better ratings for the personal doctor and for the health plan were also given by respondents who had been enlisted longer with Agis (the health plan under study).
Comparability with other Dutch and American data
The final step in our validation study consisted of a comparison of our findings with those of other Dutch studies and with the American benchmark data.
Since CAHPS questionnaires have not been used before in the Dutch health care system, comparison with other Dutch data is rather difficult. However, some observations can be made. In a large national study of general practice in the Netherlands, a series of generic items from QUOTE questionnaires were included in the face-to-face interviews conducted with 12 699 respondents from the general population.26 Some of these items are quite comparable with CAHPS items. For instance, questions referred to the degree with which people felt that their family physician spent enough time with them and the degree to which the family physician explained things well. The percentages of respondents who had positive and very positive experiences (measured on a 4-point scale) with these aspects of family doctors' care varied between 90% and 95%.27 Those scores are comparable with the percentage of respondents in our validation study who usually or always had positive experiences.
Other checks that were conducted relate to the utilization data that the Dutch translation of the CAHPS generated. We found that the number of people who said that they had visited a specialist was higher in our sample than in the general Dutch population (44.6% compared to 37.8%).28 Also, the percentage of respondents who had used the out-of-hours services of family physicians is higher than the percentage reported for the general Dutch population (34.3% compared to 26.4%).29 These differences may be caused by the fact that respondents in our study are older than the general Dutch population and more often report a bad health status.
A global comparison with the American CAHPS benchmark data is possible for scores such as the percentage of respondents experiencing problems or the percentage that usually or always has positive experiences and by testing whether relations that are reported in the American literature can be reproduced in the Dutch data. In the American Medicare population, the younger, sicker, and more educated patients give lower ratings.30 Therefore, we checked the relation between respondents' evaluations of care and their age, level of education and general health status via a multiple regression analysis. Controlling for education and health status, a higher age is associated with more positive ratings of the personal doctor (beta = 0.17, P < 0.05), health care in the past 12 months (beta = 0.30, P < 0.05), and the health plan (beta = 0.36, P < 0.05). Respondents who have finished no or primary school only give a higher rating to their health plan (beta = 0.13, P < 0.05). So, do respondents with a better general health status (beta = –0.16, P < 0.05; health status measured on a 5-point scale were 1 is ‘excellent’ and 5 is ‘poor’). To get an idea of the strength of the associations, in table 3 the mean ratings for the different subgroups are presented.
|
A comparison of the Dutch outcomes with the American benchmark data suggests that respondents in the US system experience more problems with the accessibility of care than do their Dutch counterparts, especially when it comes to finding a personal doctor or nurse and waiting for consent from the insurance company. However, there are no large differences in the extent to which the two populations get care quickly. Also, the frequency with which one experiences patient-centred care does not seem to differ profoundly between the two systems. The largest differences can be observed with respect to the general ratings of personal doctors, specialists, health care in the past 12 months, and health plans (figure 3). The percentage of people giving ratings between 0 and 6 is somewhat higher in the Netherlands than in the United States. But particularly the percentage of respondents giving ratings between 9 and 10 is much higher in the United States than in the Netherlands.
|
|
Discussion |
|---|
|
TopAbstractMethodsResultsDiscussionAPPENDIXReferences |
|---|
The purpose of this paper was to describe the results of a validation study of a Dutch version of the CAHPS Adult Commercial questionnaire, which was ‘imported’ from the United States. The comparison with American studies suggests that questions referring to the primary process in health care generate scores that are quite comparable between the American and the Dutch health system. American respondents have more problems getting access to care, but once they manage to get into the system the patient-centredness of the two health care systems is remarkably similar. Interestingly, the percentage of respondents giving ratings between 9 and 10 is much higher in the United States than in the Netherlands. Given the overall similarities in experiences, these differences probably reflect cultural differences rather than differences in the quality of care. A possible explanation could be that the Dutch respondents relate a rating between 0 and 10 to the grading systems that is used in the Dutch school system. The grade 10 reflects a flawless performance, which is why Dutch respondents may be more hesitant than Americans to give this rate.
Despite the similarities in experiences, we could not completely reproduce the factor structure of the American data and the relations between respondents' characteristics and ratings:
- In the Dutch data, items referring to courtesy and helpfulness of the staff in doctors' offices load high on the factor that also contains the items about doctor–patient communication. Whereas in the American data, experiences with office staff form a separate factor. Perhaps this can be explained from the relatively small scale of doctors' offices in the Dutch health care system. For example, almost half of the Dutch family physicians work in a single-handed practice with the help of one practice assistant.31 In such a setting, the distinction between courtesy of the staff (being one practice assistant, who also provides services) and the communicative skills of the doctor may be difficult to make.
- Zaslavsky et al.30 report that, in the Medicare Managed Care CAHPS data, health status and age are positively related to the ratings of the personal doctor, the specialist, the health care in the past months, and the health plan. And controlling for health status and age, respondents with a higher education rated health plans, health care, the personal doctor, and the specialist lower than those with less education. Except for the ratings of the health plan, we could not fully reproduce these findings in our study. We did find a positive relation between age and the ratings of personal doctors, specialists, and health care in the past 12 months. But controlling for age, in our study the level of education and the perceived general health status were not associated with the ratings of doctors and health care.
Our study also revealed some practical improvements that can be made in next versions of a Dutch questionnaire. At the time when the questionnaire was translated, it had seemed obvious that the questions with respect to the ‘personal doctor’ would be interpreted by a majority of respondents as referring to their family physician. In the Dutch health care system, family physicians have personal lists and almost every Dutch citizen is enlisted with his/her own (personal) family doctor. Apparently, for a substantial number of respondents this is an administrative reality only, because in our sample 36.2% stated that they do not have a personal doctor or nurse. In a future Dutch survey, we suggest to add two questions asking (i) if the respondent is enlisted with a family physician (the percentage stating ‘yes’ should be between 95% and 100%) and (ii) if their personal doctor is the same as their family physician (analogue to the CAHPS question whether the specialist is the same as the personal doctor).
Apart from this improvement, future surveys should either exclude certain questions or increase the sample size. Because in the sample that we studied some items were relevant to a limited number of respondents only. For example, a very low number of people had needed an interpreter. Perhaps this is not surprising, considering that the questionnaire was in Dutch. Those who are not fluent in Dutch at all will probably not have returned the questionnaire. Also, few people had sought information about the insurance company or had filed a complaint. If a sample size of 1000 or fewer is used, these questions can be excluded from the survey. If these topics would be of specific interest for health plans, we recommend increasing the sample size.
The crude response rate in this study was 56% (net response rate 51%). Compared to other consumer surveys that have recently been conducted in the Netherlands this is relatively high, considering the fact that the general population is addressed. The response rate in disease-specific surveys tends to be higher (around or over 70%). But previous surveys of consumer experiences with health insurance had a much lower response rate (25–30%).32 The fact that three reminders were sent in this study contributed to the relatively high response (cf. figure 2). The third reminder had a relatively small effect on the number of questionnaires returned. However, it does seem to result in getting somewhat more (young) men to participate in a study in which women are slightly overrepresented still. Without the third reminder this overrepresentation would have been larger.
In conclusion, the translated version of the CAHPS Adult Commercial questionnaire is a promising tool that can enable Dutch health plans to disclose transparent information about their own service and client orientation and about the quality of the care they contract. In the summer of 2005, NIVEL conducted a large survey aimed at comparing the performance of all the Dutch health insurance companies. In that survey, we used an improved version of the questionnaire that has been validated here. The data are currently analysed and will be presented on http://www.kiesbeter.nl, the consumer information portal of the Dutch Ministry of Health, Welfare, and Sports.
This more recent survey will provide ample opportunity to further develop the CAHPS questionnaire for the Dutch market. More research is needed on the external and the content validity of these questionnaires in the Dutch context. The CAHPS questionnaires have been developed on the basis of focus group discussions with American people.33 We have to study more deeply what it is that we are measuring with these questions when we pose them to Dutch respondents.
Apart from that the role of CAHPS in the Dutch health care reforms needs to be monitored. This role is potentially 2-fold: CAHPS information can be used by health plans in negotiations with contracting providers, and it can be used by consumers in choosing between health plans. Health plans increasingly have a business case to engage in strategic purchasing of care. Time will tell to what extent CAHPS information is useful for them.
As yet, it is unclear to what extent consumers in the future will choose between competing health plans on the basis of CAHPS information. Currently, Dutch consumers perceive few differences between the quality and services of health plans. For that reason, they see little benefits in going through the administrative turmoil of changing to another health plan.34 Reports based on CAHPS questionnaires may enable health plans to establish more distinctive profiles on the basis of which clients can choose. This can help to break the vicious circle in which insured do not switch to another health plan because they perceive no differences, and health plans do not bother to excel in performance because their clients do not change to a competitor anyway.
Key points
|
|
APPENDIX |
|---|
|
TopAbstractMethodsResultsDiscussionAPPENDIXReferences |
|---|
|
|
References |
|---|
|
TopAbstractMethodsResultsDiscussionAPPENDIXReferences |
|---|
1 Dixon A, Pfaff M, Hermesse J. Solidarity and competition in social health insurance countries. In Saltman RB, Busse R, Figueras J (eds), Social Health Insurance Systems in Western Europe. European Observatory on Health Systems and Policies Series. Berkshire: Open University Press, 2004:170–86.
2 Schut FT, Greß S, Wasem J. Consumer price sensitivity and social health insurer choice in Germany and the Netherlands. Int J Health Care Finance Econ 2003;3:117–38.[CrossRef][Medline]
3 Bullen N, Reeves R, Richards N. Development and pilot testing of the NHS Acute Trust Outpatient Survey 2003. Oxford: Picker Institute Europe, 2003.
4 Jenkinson C, Coulter A, Bruster S. The Picker Patient Experience Questionnaire: development and validation using data from in-patient surveys in five countries. Int J Qual Health Care 2002;14:353–8.
5 World Health Organization, Background Paper for the Technical Consultation on Responsiveness Concepts and Measurement; 13–14 September 2001, Geneva. Geneva: WHO,2001.
6 de Silva A, Valentine N. Measuring Responsiveness: Results of a Key Informants Survey in 35 Countries. Geneva: WHO, 2004.
7 Blendon RJ, Schoen C, DesRoches CM, et al. Inequities in health care: a five-country survey. Health Aff 2002;21:182–91.
8 Sixma HJ, Kerssens JJ, Campen CV, Peters L. Quality of care from the patients' perspective: from theoretical concept to a new measuring instrument. Health Expect 1998;1:82–95.[CrossRef][Medline]
9 van Campen C, Sixma HJ, Kerssens JJ, Peters L. Assessing noninstitutionalized asthma and COPD patients' priorities and perceptions of quality of health care: the development of the QUOTE-CNSLD instrument. J Asthma 1997;34:531–8.[ISI][Medline]
10 van Campen C, Sixma HJ, Kerssens JJ, et al. Assessing patients' priorities and perceptions of the quality of health care: the development of the QUOTE-Rheumatic-Patients instrument. Br J Rheumatol 1998;37:362–8.
11 van der Eijk I, Sixma H, Smeets T, et al. Quality of health care in inflammatory bowel disease: development of a reliable questionnaire (QUOTE-IBD) and first results. Am J Gastroenterol 2001;96:3329–36.[ISI][Medline]
12 Hekkink CF, Sixma HJ, Wigersma L, et al. QUOTE-HIV: an instrument for assessing quality of HIV care from the patients' perspective. Qual Saf Health Care 2003;12:188–93.
13 Nijkamp MD, Sixma HJ, Afman H, et al. Quality of care from the perspective of the cataract patient. QUOTE cataract questionnaire. J Cataract Refract Surg 2002;28:1924–31.[CrossRef][ISI][Medline]
14 Sixma H, Nijkamp MD, Nuijts RM. Quality of care from the perspective of cataract patients; a new measuring instrument. Gerontology 200147(Suppl 1):486.
15 Sixma HJ, van Campen C, Kerssens JJ, Peters L. Quality of care from the perspective of elderly people: the QUOTE-elderly instrument. Age Ageing 2000;29:173–8.
16 Cleary PD, Edgman-Levitan S. Health care quality. Incorporating consumer perspectives. JAMA 1997;278:1608–12.[Abstract]
17 Da Silva A, A framework for measuring responsiveness. GPE Discussion Paper Series: No. 32. World Health Organization. Available at: http://www3.who.int/whosis/discussion_papers/pdf/paper32.pdf (accessed on 5 October 2004).
18 Hays RD, Shaul JA, Williams VS, et al. Psychometric properties of the CAHPS 1.0 survey measures. Consumer Assessment of Health Plans Study. Med Care 1999;37 (3 Suppl):MS22–31.[CrossRef][ISI][Medline]
19 Guadagnoli E, Epstein AM, Zaslavsky A, et al. Providing consumers with information about the quality of health plans: the Consumer Assessment of Health Plans demonstration in Washington State. Jt Comm J Qual Improv 2000;26:410–20.[Medline]
20 Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS®) 2.0 Adult Core Survey. Health Services Research 38 2003;6, Part I:1509–27.
21 Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the Consumer Assessment of Health Plans Study survey. Med Care 2002;40:485–99.[CrossRef][ISI][Medline]
22 McGee J, Kanouse DE, Sofaer S, et al. Making survey results easy to report to consumers: how reporting needs guided survey design in CAHPS. Consumer Assessment of Health Plans Study. Med Care 1999;37 (3 Suppl):MS32–40.[CrossRef][ISI][Medline]
23 Percentage based on data for 2003, population of 20 years of age and older. Available at: http://statline.cbs.nl/StatWeb/Table.asp?HDR=T&LA=nl&DM=SLNL&PA=37296ned&D1=0-2,4-7,9-13&D2=0,5,10,15,20,25,30,35,40,45,50,(l-2)-l&STB=G1 (accessed on 29 September 2004).
24 Available at: http://statline.cbs.nl/StatWeb/table.asp?LYR=G2:0,G3:2&LA=nl&DM=SLNL&PA=03799&D1=91-93&D2=0-11&D3=0&HDR=T&STB=G1 (accessed on 29 September 2004).
25 Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS®) 2.0 Adult Core Survey. Health Serv Res 2003;38 (6, Part I):1509–27.
26 Schellevis FG, Westert GP, De Bakker DH, et al. De tweede Nationale Studie naar ziekten en verrichtingen in de huisartsenpraktijk: aanleiding en methoden. Huisarts en Wetenschap 2003;46:7–11.
27 Braspenning J, Schellevis FG. Grol RPTM, editors Tweede nationale studie naar ziekten en verrichtingen in de huisartspraktijk. Kwaliteit huisartsenzorg belicht. Utrecht-Nijmegen: NIVEL-WOK, 2004.
28 Available at: http://statline.cbs.nl/StatWeb/Table.asp?HDR=T&LA=nl&DM=SLNL&PA=7042mc&D1=0,121,154,260,293,390&D2=(l-5)-l&STB=G1 (last accessed on 17 November 2004).
29 van der Schee E, Delnoij D, Broerse A. Weinig reden tot klagen. Med Contact 2003;58:1953–6.
30 Zaslavsky AM, Zaborski LB, Ding L, et al. Adjusting performance measures to ensure equitable plan comparisons. Health Care Financ Rev 2001;22:109–26.
31 Kenens R, Hingstman L. Cijfers uit de registratie van huisartsen: peiling 2003. Utrecht:NIVEL, 2003.
32 Beukema A, Jakobs C, Sixma H. Keuzegids Zorgverzekeraars. Den Haag: Consumentenbond 2005.
33 Harris-Kojetin LD, Fowler FJ. Jr Brown JA, et al. The use of cognitive testing to develop and evaluate CAHPS 1.0 core survey items. Consumer Assessment of Health Plans Study. Med Care199937(3 Suppl): MS10–21.
34 Gress S, Delnoij D, Groenewegen P. Exit and Voice in Dutch Social Health Insurance. DICE Rep J Inst Compar 2003;1:47–53.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||