Email updates

Keep up to date with the latest news and content from BMC Health Services Research and BioMed Central.

Open Access Research article

The consumer quality index (CQ-index) in an accident and emergency department: development and first evaluation

Nanne Bos1*, Leontien M Sturms12, Augustinus JP Schrijvers1 and Henk F van Stel1

Author Affiliations

1 Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands

2 Dutch Network for Emergency Care, Tilburg, The Netherlands

For all author emails, please log on.

BMC Health Services Research 2012, 12:284  doi:10.1186/1472-6963-12-284


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6963/12/284


Received:8 August 2012
Accepted:16 August 2012
Published:28 August 2012

© 2012 Bos et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Assessment of patients’ views are essential to provide a patient-centred health service and to evaluating quality of care. As no standardized and validated system for measuring patients’ experiences in accident and emergency departments existed, we have developed the Consumer Quality index for the accident and emergency department (CQI A&E).

Methods

Qualitative research has been undertaken to determine the content validity of the CQI A&E. In order to assess psychometric characteristics an 84-item questionnaire was sent to 653 patients who had attended a large A&E in the Netherlands. Also, fifty importance questions were added to determine relevance of the questions and for future calculations of improvement scores. Exploratory factor analysis was applied to detect the domains of the questionnaire.

Results

Survey data of 304 (47%) patients were used for the analysis. The first exploratory factor analysis resulted in three domains based on 13 items: ‘Attitude of the healthcare professionals’, ‘Environment and impression of the A&E’ and ‘Respect for and explanation to the patient’. The first two had an acceptable internal consistency. The second analysis, included 24 items grouped into 5 domains: ‘Attitude of the healthcare professionals’, ‘Information and explanation’, ‘Environment of the A&E’,’Leaving the A&E’ and ‘General information and rapidity of care’. All factors were internal consistent. According to the patients, the three most important aspects in healthcare performance in the A&E were: trust in the competence of the healthcare professionals, hygiene in the A&E and patients’ health care expectations. In general, the highest improvement scores concerned patient information.

Conclusions

The Consumer Quality index for the accident and emergency department measures patients’ experiences of A&E healthcare performance. Preliminary psychometric characteristics are sufficient to justify further research into reliability and validity.

Keywords:
Factor analysis, statistical; Emergency medical services; Patient experiences; Patient satisfaction, statistics and numerical data; Questionnaires, standards; Health care surveys

Background

Healthcare services have shown an increasing interest in the quality of care they provide [1]. After clinical outcome evaluations, evaluations based on the patient’s perspective have become more prominent since the introduction of patient-centred care [2]. International organisations such as the Organization for Economic Cooperation and Development (OECD) and the World Health Organization (WHO), have emphasised the importance of the patient’s perspective in the evaluation of healthcare delivery. National and cross-national comparisons of patients’ experiences are important for identifying areas in need of improvement [3]. Patients’ experiences provide information on which healthcare professionals, patients, and health-insurance companies may base their decisions. Furthermore, it enables the government and the Health Care Inspectorate to monitor the quality of healthcare. Finally, the standardized measurement of patients’ experiences enables evaluations of research of research projects.

It is preferable to measure patients’ experiences rather than their satisfaction, as they have shown to be more objective and to yield more detailed information for quality improvement [4]. One theory is that satisfaction is a multi-dimensional concept, partly based on expectations and personal preferences. This complicates the objective measurement of the quality of care. When a product fails to match expectations, the quality will be judged as unsatisfactory [5].

In the Netherlands, the Consumer Quality Index (CQ-index), a standardized method for developing surveys and measuring healthcare quality from the patient’s perspective, was introduced in 2006 in order to promote patient-centred care. In order to obtain reliable and valid questionnaires, the development process has been prescribed in a manual and it is guided and controlled by a scientific advisory board. The content validity is ascertained during a qualitative phase which includes a literature search, interviews with experts, and patient focus groups. After this phase a pilot study on the CQ-index should be performed to determine internal consistency. The CQ-index is characterised by combining patients’ experiences with the relative importance of each experience item resulting in a list of priorities for improvement of quality of care. Several CQ-indices for a variety of community services, care settings and condition-specific patients’ groups have been developed, such as the rheumatoid arthritic questionnaire, the cataract questionnaire, the hip and knee questionnaire, and breast-cancer questionnaire [2,6,7]. Different patient groups turned out to have different priorities, which stressed the need for specific questionnaires [8]. In emergency medicine, two CQ-indices for general practitioners (office hours and out-of-hours), and a CQ-index for maternity services are available. The CQ-indices for ambulance services and dispatch centres are under development. This study completes the set of questionnaires for emergency services with the development of the CQ-index for the accident and emergency department.

In the Netherlands, general practitioners are positioned as gatekeepers, also in the case of emergency care. Emergency care by GPs is provided from local GP-practices during working hours, and out-of-hours in regional GP-cooperatives [9]. Patients need to consult their GP for referral to accident and emergency departments (A&Es) in hospitals. However, the number of self-referrals to A&Es is growing [10]. Patients transported by ambulance in need of emergency care are brought directly to the A&E [11,12]. A&Es are often the place where patients form their first impression of a hospital and a positive experience may influence decisions about future visits and personal patient recommendations [13]. Measuring the quality of care in the A&E as experienced by patients may provide valuable information, for instance for identifying areas in need of improvement.

The goal of this study is to develop and pilot test a CQ-index for the A&E department (CQI A&E). This questionnaire aims to measure healthcare performance in the A&E as experienced by the patient.

Methods

Qualitative and constructive phase

The prescribed CQ-index guidelines were applied during the development of the CQI A&E [14,15]. The first phase of the development is a qualitative phase. The aim of this phase is to detect all relevant quality aspects of healthcare performance in the A&E. We carried out a literature search in Pubmed, including a search for existing questionnaires, and interviews with three experts, in order to compose a topic list for focus group discussions with patients about healthcare performance in the A&E. For the focus groups, a consecutive sample of 177 patients treated in the A&E at the University Medical Center Utrecht, aged 18 and older, with known postal address and phone number, were sent an invitation by postal mail to participate, in the first week after their A&E attendance. In a subsequent step, all patients were called and invited a second time to participate in a patient focus group. Seventeen patients confirmed their participation. Two researchers acted as moderators during the focus group discussions. After the focus groups the first draft questionnaire was defined. This draft was sent to ten patients. Within one week, cognitive interviews were performed by telephone in order to ensure that the questions were relevant, unambiguous, understandable and useful to patients, and whether patients had experienced any problems during self-completion of the paper questionnaire. Unclear questions in the CQI A&E were rephrased. Afterwards the CQI A&E consisted of 84 questions divided into 9 categories: General; Before arrival in the A&E; Reception desk A&E; Health professionals in the A&E; Pain; Examination and treatment; Leaving the A&E; General A&E; About you. 52 questions out of the total of 84 questions were constructed as so called ‘experience questions’. The other questions included ‘skip or go to’ items, opinion questions and demographic questions (Additional file 1).

Additional file 1. CQI A&E.

Format: DOC Size: 97KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Importance study

An importance study was undertaken to determine the relative importance of the items in the questionnaire to patients visiting the A&E. Firstly, importance scores were used to decide whether a question should be retained or deleted prior to the factor analysis. Secondly, importance scores are necessary for calculating improvement scores. For each experience question a corresponding importance question was formulated. For example: ‘Was the signposting to the A&E of the hospital a problem?’ with the corresponding importance question ‘How important is the signposting to the A&E of the hospital to you?’. This resulted in a temporary set of 50 extra importance questions in the CQI A&E. Importance questions of two experience questions were unclearly phrased or difficult to understand, and therefore left out of the importance study.

Psychometric phase

The questionnaire was pilot tested in the psychometric phase to assess the psychometric properties. The three phases are presented in Table 1.

Table 1. Three development phases of the CQI A&E

Patients

For the pilot test all 653 patients who visited the A&E of a large urban hospital in the course of one week in January 2010, were included. The hospital was centrally located in the Netherlands. 38,000 patients visit the A&E annually. The A&E treats patients in need of urgent care, except for multiple trauma patients, who are referred to specialized trauma centres. Patients who attended to the A&E with a known postal address and no reported death were eligible.

The paper questionnaire and covering letter were sent by postal mail within one week after the visit to the A&E. Up to three reminders were sent to non-respondents: after 1, 4 and 6 weeks. The recipients were able to return the questionnaire in a postage paid envelope. The study protocol was approved by the Medical Ethical Committee of the University Medical Center, Utrecht.

Data analysis

The hospital registration system provided data on gender, age, referral to A&E (ambulance, general practitioner, self-referred, other), day and time of the visit, triage code, symptoms of which the patients complained (abdominal pain, traumatic injuries, shortness of breath, collapse, chest pain, arrhythmia, malaise/fever, stroke, infection, intoxication, other). The respondents’ gender and age profile was compared to the total sample and to non-respondents using a Chi square or t-test, in order to assess whether it was representative. Questionnaires which had been filled in by someone other than the respondent and questionnaires with more than fifty percent of the answers missing, including skip (or ‘go to’) instructions after the questions, were not used for analysis.

Data quality and exploratory factor analysis

The data set was first analysed in order to identify item response rates and frequency distributions. Questionnaire items were excluded from further analysis if they had an item non-response of >10% of expected responses or extreme skew of >90% of responses in the same category (i.e. a ceiling or floor effect). Spearman’s correlation coefficient was calculated to check for correlations between items (r > 0.70). Where items had a negative wording, their scales were reversed to ensure comparability in the analysis. Exploratory Factor Analysis (EFA) was used to group the experience questions. In the first EFA only the 13 experience items with a 4-point Likert scale were included. EFA was performed with oblique rotation [16]. In EFA several criteria need to be fulfilled. The Kaiser-Meyer-Olkin Measure of Adequacy (KMO) is a measure of sampling adequacy (threshold: KMO >0.60). Bartlett's test of sphericity is used to test the null hypothesis that the variables in the population correlation matrix are uncorrelated (threshold: p < 0.05). The Eigenvalue represents the amount of the total variance explained by the factor (threshold: Eigenvalue > 1, also known as the Kaiser criterion). A variety of analyses were performed, whereby options like ‘fixed number of factors yes or no’ and ‘replace missing values by mean yes or no’ were tested. The domains in the final EFA fulfilled the statistical criteria, explained the highest percentage of variance and had a clear interpretation. In a subsequent step, factor loadings were obtained (threshold: factor load > 0.40). We calculated a measure of internal consistency, Cronbach’s alpha (α), in order to estimate the reliability of the reported factors. Cronbach’s alpha coefficients above 0.70 were considered reliable. In this stage of development alpha coefficients between 0.60-0.70 were provisionally accepted. The α of the total factor should not increase by deleting one of the items. Item-total correlation (ITC) had to be higher than 0.40. When following the CQI guidelines, the majority of experience questions were omitted when constructing domains. Therefore, a second EFA was performed, including all 52 experience questions, with response categories on 2-, 3-, and 4-point Likert scales. This was done to prevent loss of content, thereby ignoring one criterion of the CQI guidelines.

Experience scores, importance scores and improvement scores

The experience scores and importance scores were calculated as means of response categories (i.e. no/a big problem/never/not important = 1, sometimes/of some importance = 2, a bit of a problem = 2.5, a great deal/important = 3, yes/not a problem/always/extremely important = 4). A domain score was computed as the mean of the experience scores of items contributing to the domain [17]. Quality improvement scores were calculated by multiplying the importance scores with the percentages of the negative response categories ‘never’, ‘sometimes’, ‘big problem’ or ‘no’ on the corresponding experience questions. The improvement scores were an estimate for the potential improvement of quality of care and are useful for internal monitoring, whereas domain scores are more relevant for external monitors. Scores above 0.5 may potentially improve quality of care (range: 0–4). All analyses were performed using the statistical software SPSS 17.0.

Results

Qualitative and constructive phase

A review of the literature was conducted, using the PubMed database. A search with Mesh major headings ‘Emergency Service, hospital’ AND ‘Consumer satisfaction’ resulted in 364 hits. All abstracts from 1993 until August 2010 were reviewed for quality dimensions and aspects of care in the A&E. In 53 articles quality aspects were described. The two most frequently used questionnaires were the Consumer Emergency Care Satisfaction Scale (CECSS) and the Emergency Department Patient Satisfaction Survey (EDPSS). Together with the most discussed topics in interviews with experts a topic-list was composed, which was used in patient focus groups discussions. Quality aspects for A&E healthcare delivery were: patient history, accessibility, empathy and attitude of healthcare professionals, autonomy, cooperation, waiting time, competence, triage, treatment, communication, information, pain management, discharge management, re-admittance, privacy, environment, global rating, safety, diagnostic tests, rapidity, refreshments, and accompaniment. The quality aspects were used to formulate the questions and compile the draft questionnaire. After cognitive interviews, the questionnaire was adjusted and questions were added or rephrased where necessary. Substantial adjustments were: the question ‘At what time did you visit the A&E?’ was added to the questionnaire; the question ‘Was the accessibility of the A&E a problem? was rephrased to ‘Was the travelling time to the A&E of the hospital a problem?; ‘Did you have to wait a second time after your first contact with a healthcare professional?’ was rephrased to ‘Was your health problem first briefly assessed by a nurse and did you then have to wait again in the waiting room?’; ‘What score would you give the healthcare professionals?’ was deleted; ‘Did you get the care you expected from the A&E?’ was added. Full detailed information of the qualitative and constructive phase were reported and approved by the advisory board of the CQI A&E [18].

Importance study

Fifty importance questions were used to calculate the most important aspects in healthcare performance in the A&E according to patients. The five most important aspects were: trust in the competence of healthcare professionals (3.66), hygiene in the A&E (3.65), patients’ healthcare expectations (3.65), patients’ healthcare needs (3.64) and being taken seriously by healthcare professionals (3.63) (Table 2). The five least important items were: information on the order in which patients were treated (2.54), availability of refreshments (2.53), information on an admission letter for the general practitioner (2.50), pleasant atmosphere in the waiting room (2.57) and having to tell the same story about the health problem (2.61). Importance score are ranged from 0–4.

Table 2. Importance scores (I) with corresponding quality improvement scores (Q) and corresponding experience scores (E)

Psychometric phase

Patients

368 out of 653 patients (56%) returned the questionnaire. Two uncompleted questionnaires were excluded, as were 52 questionnaires which had been filled in by someone other than the patient, for instance by the representatives of patients aged 0–11 years. The dataset for the analysis contained 304 questionnaires (47%). Patients’ characteristics are presented in Table 3. No differences between respondents and non-respondents were found for gender, age, referral, day and time of the visit or symptoms. A significant difference was found for the triage code (p = 0.01). Respondents were triaged in more urgent categories than non-respondents. 34% of the respondents were admitted to a hospital ward after their visit to the A&E.

Table 3. Characteristics of the study sample

Data quality

The item ‘access to results of previous visit’ and the item ‘pain control by healthcare professionals’ had a non-response >10%. Extremely skewed items (>90%) were the items ‘signposting to the A&E’, ‘travelling time’, ‘signage in the hospital’ and ‘talking about patients in the presence of the patient’. The five items were left out of the factor analyses. None of the importance items had a remarkably lower score than the average score, or was extremely skewed. Spearman correlations were calculated; none of the correlation coefficients of experience questions or importance questions were above the 0.70 threshold. All experience and importance questions with corresponding frequency distributions on response categories and experience or importance scores are presented in Additional file 2 and Additional file 3.

Additional file 2. Frequency distributions, mean experience scores, and 95% confidence interval of experience questions.

Format: DOC Size: 280KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Additional file 3. Frequency distributions, mean importance scores, and 95% confidence interval of importance questions.

Format: DOC Size: 277KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

First factor analysis and internal consistency

The first EFA, based on 13 items, showed a 3-factor solution with an explained variance of 56%, covering all items (KMO 0.883, Bartlett’s test p < 0.001, N = 298). The first domain measured the quality aspect ‘attitude of healthcare professionals’, the second domain ‘environment and impression of the A&E’ and the third domain ‘respect for and explanation to the patient’ (Table 4). Cronbach’s alpha coefficient of the first domain was 0.85. The alpha coeffiecient of the second domain was 0.60,,and the alpha coefficient of the third domain was 0.42. The item-total correlations of the third factor were below 0.40. The internal consistency did not increase by taking an item out of this third domain.

Table 4. Domains, items, and internal consistency of the first factor analysis

Second factor analysis and internal consistency

The second analysis was performed on 52 items. Reliability analysis showed that the questionnaire contained four domains with Cronbach’s alpha coefficients above 0.70, and one domain with an alpha coefficient of 0.67. The five domains had an explained variance of 51%. Like the first analysis, this second analysis fulfilled the predefined criteria; Eigenvalue > 1 and KMO = 0.837. However, Bartlett’s test of sphericity was not significant (p = 1.00; N = 298). Domains, items and Cronbach’s alpha coefficients are presented in Table 5. All item-total correlations were above 0.40, with the exception of the item ‘consistency of provided information’. The Cronbach’s alpha of the domain remained the same if the item was left out. The five domains covered 24 items.

Table 5. Domains, items, and internal consistency of the second factor analysis

Quality improvement

For every item the experience score and quality improvement score were computed. Within the top 20 of most important quality aspects, four aspects stood out as far as their improvement potential was concerned: ‘information on side-effects of the medication’, ‘information by healthcare professionals on danger signals to watch out for after leaving the A&E’, ‘information by healthcare professionals on readmission in case of health problems’, and ‘availability of parking space near the A&E’ (Table 2). Three out of four items were dealing with information needs at the end of the A&E visit, and belonged to the 14 items with the highest improvement scores, i.e. a quality improvement score >1; range 0–4 (Table 6).

Table 6. Items with quality improvement scores >1 (Q) with corresponding importance scores (I) and corresponding experience scores (E)

Discussion

The aim of this study has been to construct and test a questionnaire to measure patients’ experiences with the accident and emergency department, according to the guidelines of the Consumer Quality index [15]. This first version of the Consumer Quality index for the accident and emergency department (CQI A&E) consisted of 84 questions. 52 questions were phrased as ‘experience questions’. The response rate (47%) of the pilot test was comparable to other postal surveys involving A&E patients [14,19]. However, disease-specific CQ-indices showed better response rates (68%–84%) [2,6,7,20,21]. To determine the construct of the questionnaire, two exploratory factor analyses were performed. The first analysis was performed including thirteen items. Three domains were constituted (‘attitude of healthcare professionals’, ‘environment and impression of the A&E’, ‘respect for and explanation to the patient’). A second analysis comprising all 52 experience questions was performed with the aim of including more questions on experiences deemed important by patients. Five domains ‘attitude of healthcare professionals’, ‘information and explanation’, ‘environment of the A&E’, ‘leaving the A&E’, and ‘general information and rapidity of care’ were constructed, covering 24 items. Two out of three domains in the first EFA were internally consistent, whereas all domains in the second EFA were internally consistent. Internal consistency of domains increases by increasing the number of respondents. Despite of lower response numbers in the domains of the second EFA, the internal consistency exceeded the internal consistency of the first EFA. The percentage of explained variance of the five domains decreased five percent compared to the explained variance of the three domains of the first analysis.

Two main goals for CQI-data can be distinguished. The first goal is to compare quality of care between healthcare providers. A strict (following the CQI-guidelines), statistically correct EFA was performed in order to generate the information needed for making a valid comparison between A&Es. Researchers, the Health Care Inspectorate, health-insurance companies, hospital boards and the Ministry of Health are the intended users of these outcomes. However, the main customer in healthcare is the patient. Therefore, the second goal is to acquire the information needed for quality improvement within a healthcare institution. To this end, an alternative EFA was performed. To include more content of the questionnaire in the domains, more questions were added. We think this information may help A&E managers and others to start evaluating quality improvement projects. Both goals represent a different way of constructing domains in questionnaires. Following all criteria in the CQI guidelines, domains are constructed using the perspective of a reflective measurement model [22,23]. Only items with a 4-point Likert scale were included. Items that did not fit into any domain, and domains that did not fulfil the statistical criteria, were omitted [24]. The qualitative phase was carried out in order to detect all aspects related to healthcare performance in the A&E. Each aspect is a unique part of the provided care and together they form the construct ‘quality of care’. To end up with a few statistically related items neglects the broad range of the aspects. Therefore, a formative measurement model may be better suited to construct domains in experience questionnaires. The latter theory concerns the construction of domains based on content and not solely on strict statistical criteria. In the second analysis, we did not try to construct domains solely from a formative perspective, but we tried to include as many experience questions as possible, while still achieving internally consistent and interpretable domains. We included all domains that came up in the second EFA, thereby doubling the content of the questionnaire included in internally consistent domains. Although we only relaxed one of the criteria of CQI-guidelines, we think that these domains are better suited for quality improvement and that they are also suited for benchmarking. The improvement scores provide concrete tailor-made information, which can be helpful for management and staff.

In accordance with most CQ-indices, the domains on communication, information, attitude of healthcare providers (often within the communication domain) and the environment of the health service, are part of the CQI A&E. Domains regarding accessibility and leaving the organisation are also often found [6,7,20,25]. There are a lot of similarities to other CQI instruments, whereas the similarities with satisfaction questionnaires are few. The A&E satisfaction questionnaire Quality Patient Perspective discussed patient participation [19]. The Swedish A&E Patient Satisfaction Survey revealed 3 factors: caring, teaching and clinical competence [26].

The most important aspects in healthcare performance in the A&E from the patient’s perspective dealt with competence of professionals, hygiene and expectations. It has to be determined whether these importance scores are valid across study populations.

In an English importance study, with 16 participants who had visited the accident and emergency department the most important aspect was confidence and trust in the doctors and nurses. Secondly, ‘being treated with respect and dignity’, and thirdly ‘explanation about condition and treatment’ were important items. Interestingly, waiting time did not feature in the top 20 of most important aspects in our study. As regards the top three of least important aspects, our study concorded with the English study on aspects such as refreshments in the waiting room and not being asked details about the patient’s condition or illness too often. However, in the English study the number of respondents was limited, and inclusion criteria broader.

The study has same limitations. Firstly, we used the pilot study dataset of respondents of only one hospital. In the next phase of the development, the stability of the domain structure will be assessed in a dataset of 20 A&Es, and therefore the presented domains are preliminary outcomes. The discriminative capacity of the CQI A&E will be assessed in that phase as well. Secondly, in this study a consecutive sample was used for the psychometric analysis. All patients who had visited the A&E within an average week were included. The gender and age profile of the respondents was representative for the A&E population of the research hospital. Respondents and non-respondents were comparable as regards age, gender, day or time of attendance and symptoms. Therefore, it is unlikely that the low response rate has affected the outcomes. A significant difference between both groups was found for the triage code. The least severely injured patients (blue triage code) were underrepresented within the respondents group. These patients are often discharged without experiencing all aspects of healthcare performance and perhaps did not think of themselves as a ‘true’ A&E patient. This might have introduced selection bias among respondents.

The study protocol had advantages such as sending the questionnaires to the patient’s home, instead of administering them in person in the A&E, which prevents selection bias caused by healthcare professionals. Also, all patients received the questionnaire within one week after their A&E visit, which limits the recall bias. However, patients’ symptoms might have evoked recall bias.

Conclusions

The Consumer Quality index for the accident and emergency department measures patients’ experiences of A&E healthcare performance. Preliminary psychometric characteristics of the CQI A&E are good, but further research on reliability and validity is needed. Depending on the viewpoint, exploratory factor analysis results in two or five internally consistent domains. The 5-domain structure seems preferable, as this includes more content of the questionnaire while maintaining internal consistency. Furthermore, the improvement scores of each item provide information that makes it possible to identify aspects that require consideration in order to increase quality of care. The preliminary outcomes and the discriminative capacity have to be confirmed in future research by means of the CQI A&E.

Abbreviations

CQ-index, Consumer quality index; ED, Emergency Department; CQI ED, Consumer Quality Index for the Emergency Department; EFA, Exploratory factor analysis; KMO, Kaiser-Meyer-Olkin measure of adequacy; ITC, Item total correlation.

Competing interests

As indicated in the paper, none of the authors have conflicts of interest to disclose.

Authors’ contributions

NB has participated in the study concept, study design, acquisition of the data, data analysis, data interpretation, drafting the manuscript and critical revision of the manuscript. LS has participated in the study concept, study design, acquisition of the data, data analysis, data interpretation, drafting the manuscript and critical revision of the manuscript. AS has participated in the study concept and study design, critical revised the manuscript, and supervised the study. HS has participated in the study concept, study design, acquisition of the data, data analysis, data interpretation, drafting the manuscript and critical revision of the manuscript. All authors have read and approved the final manuscript.

Acknowledgements

R.K. Stellato, MSc. has contributed substantially to the analysis and interpretation of the data.

References

  1. Groenewold O, Klazinga N, Wagner C, Arah OA, Thompson A, Bruneau C, Sunol R: Investigating organizational quality improvement systems, patient empowerment, organizational culture, professional involvement and the quality of care in European hospitals: the ‘Deepening our Understanding of Quality Improvement in Europe (DUQuE)’ project.

    BMC Health Serv Res 2010, 10:281. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  2. Zuidgeest M, Sixma H, Rademakers J: Measuring patients’ experiences with rheumatic care: the consumer quality index rheumatoid arthritis.

    Rheumatol Int 2009, 30:159-167. PubMed Abstract | Publisher Full Text OpenURL

  3. Garrett A, Solheim E, Danielsen K: National and cross-national surveys of patient experiences: a structured review. Norwegian Knowledge Centre for the Health Services, Oslo; 2008. OpenURL

  4. Cleary PD, Edgman-Levitan S: Health care quality. Incorporating consumer perspectives.

    JAMA 1997, 278:1608-1612. PubMed Abstract | Publisher Full Text OpenURL

  5. Sixma HJ, Kerssens JJ, Campen CV, Peters L: Quality of care from the patients’ perspective: from theoretical concept to a new measuring instrument.

    Health Expect 1998, 1:82-95. PubMed Abstract | Publisher Full Text OpenURL

  6. Stubbe JH, Brouwer W, Delnoij DM: Patients’ experiences with quality of hospital care: the Consumer Quality Index Cataract Questionnaire.

    BMC Ophthalmol 2007, 7:14. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  7. Stubbe JH, Gelsema T, Delnoij DM: The Consumer Quality Index Hip Knee Questionnaire measuring patients’ experiences with quality of care after a total hip or knee arthroplasty.

    BMC Health Serv Res 2007, 7:60. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  8. de Boer D, Delnoij D, Rademakers J: Do patient experiences on priority aspects of health care predict their global rating of quality of care? A study in five patient groups.

    Health Expect 2010, 13:285-297. PubMed Abstract OpenURL

  9. Huibers L, Giesen P, Wensing M, Grol R: Out-of-hours care in western countries: assessment of different organizational models.

    BMC Health Serv Res 2009, 9:105. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  10. van Charante EP M, ter Riet G, Bindels P: Self-referrals to the A&E department during out-of-hours: patients’ motives and characteristics.

    Patient Educ Couns 2008, 70:256-265. PubMed Abstract | Publisher Full Text OpenURL

  11. Thijssen WA, Giesen PH, Wensing M: Emergency departments in The Netherlands.

    Emerg Med J 2012, 29:6-9. PubMed Abstract | Publisher Full Text OpenURL

  12. van Geloven AA, Luitse JS, Simons MP, Volker BS, Verbeek MJ, Obertop H: Emergency medicine in the Netherlands, the necessity for changing the system: results from two questionnaires.

    Eur J Emerg Med 2003, 10:318-322. PubMed Abstract | Publisher Full Text OpenURL

  13. Limbourn S, Celenza A: Patient perceptions of caring and association with emergency department activity and access block.

    Emerg Med Australas 2011, 23:169-180. PubMed Abstract | Publisher Full Text OpenURL

  14. Delnoij DM, ten Asbroek G, Arah OA, de Koning JS, Stam P, Poll A, Vriens B, Schmidt P, Klazinga NS: Made in the USA: the import of American Consumer Assessment of Health Plan Surveys (CAHPS) into the Dutch social insurance system.

    Eur J Public Health 2006, 16:652-659. PubMed Abstract | Publisher Full Text OpenURL

  15. Sixma H, Hendriks M, de Boer D, Delnoij D: Manual CQI instruments. An instruction for the development and use of Consumer Quality Index (CQI) questionnaires (in Dutch). The Dutch Centre for Consumer Experience in Healthcare, Utrecht; 2008. OpenURL

  16. Stevens J: Exploratory and confirmatory factor analysis. In Applied multivariate statistics for the social sciences. Lawrence Erlbaum Associates, Mahwah; 1996:362-428. OpenURL

  17. Norman G: Likert scales, levels of measurement and the “laws” of statistics.

    Adv Health Sci Educ Theory Pract 2010, 15:625-632. PubMed Abstract | Publisher Full Text OpenURL

  18. Bos N, Sturms LM, van HF S, Schrijvers AJP: Consumer Quality Index for the Accident and Emergency Department; measure in development (in Dutch). Julius Center for Health Sciences and Primary Care, Utrecht; 2010. PubMed Abstract | Publisher Full Text OpenURL

  19. Frank C, Asp M, Fridlund B, Baigi A: Questionnaire for patient participation in emergency departments: development and psychometric testing.

    J Adv Nurs 2011, 67:643-651. PubMed Abstract | Publisher Full Text OpenURL

  20. Damman OC, Hendriks M, Sixma HJ: Towards more patient centred healthcare: A new Consumer Quality Index instrument to assess patients’ experiences with breast care.

    Eur J Cancer 2009, 45:1569-1577. PubMed Abstract | Publisher Full Text OpenURL

  21. van Empel I, Aarts JW, Cohlen BJ, Huppelschoten DA, Laven JS, Nelen WL, Kremer JA: Measuring patient-centredness, the neglected outcome in fertility care: a random multicentre validation study.

    Hum Reprod 2010, 25:2516-2526. PubMed Abstract | Publisher Full Text OpenURL

  22. de Vet HC, Terwee CB, Bouter LM: Current challenges in clinimetrics.

    J Clin Epidemiol 2003, 56:1137-1141. PubMed Abstract | Publisher Full Text OpenURL

  23. Streiner DL: Clinimetrics vs. psychometrics: an unnecessary distinction.

    J Clin Epidemiol 2003, 56:1142-1145. PubMed Abstract | Publisher Full Text OpenURL

  24. Coltman T, Devinney TM, Midgley DF, Venaik S: Formative versus reflective measurement models: Two applications of formative measurement.

    Journal of Business Research 2008, 61:1250-1262. Publisher Full Text OpenURL

  25. Triemstra M, Winters S, Kool RB, Wiegers TA: Measuring client experiences in long-term care in the Netherlands: a pilot study with the Consumer Quality Index Long-term Care.

    BMC Health Serv Res 2010, 10:95. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  26. Kristensson J, Ekwall A: Psychometric properties of the consumer emergency care satisfaction scale: tested on persons accompanying patients in emergency department.

    J Nurs Care Qual 2008, 23:277-282. PubMed Abstract | Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6963/12/284/prepub