Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Be careful with triage in emergency departments: interobserver agreement on 1,578 patients in France

BMC Emergency Medicine volume 11, Article number: 19 (2011) Cite this article

Abstract

Background

For several decades, emergency departments (EDs) utilization has increased, inducing ED overcrowding in many countries. This phenomenon is related partly to an excessive number of nonurgent patients. To resolve ED overcrowding and to decrease nonurgent visits, the most common solution has been to triage the ED patients to identify potentially nonurgent patients, i.e. which could have been dealt with by general practitioner. The objective of this study was to measure agreement among ED health professionals on the urgency of an ED visit, and to determine if the level of agreement is consistent among different sub-groups based on following explicit criteria: age, medical status, type of referral to the ED, investigations performed in the ED, and the discharge from the ED.

Methods

We conducted a multicentric cross-sectional study to compare agreement between nurses and physicians on categorization of ED visits into urgent or nonurgent. Subgroups stratified by criteria characterizing the ED visit were analyzed in relation to the outcome of the visit.

Results

Of 1,928 ED patients, 350 were excluded because data were lacking. The overall nurse-physician agreement on categorization was moderate (kappa = 0.43). The levels of agreement within all subgroups were variable and low. The highest agreement concerned three subgroups of complaints: cranial injury (kappa = 0.61), gynaecological (kappa = 0.66) and toxicology complaints (kappa = 1.00). The lowest agreement concerned two subgroups: urinary-nephrology (kappa = 0.09) and hospitalization (kappa = 0.20). When categorization of ED visits into urgent or nonurgent cases was compared to hospitalization, ED physicians had higher sensitivity and specificity than nurses (respectively 94.9% versus 89.5%, and 43.1% versus 30.9%).

Conclusions

The lack of physician-nurse agreement and the inability to predict hospitalization have important implications for patient safety. When urgency screening is used to determine treatment priority, disagreement might not matter because all patients in the ED are seen and treated. But using assessments as the basis for refusal of care to potential nonurgent patients raises legal, ethical, and safety issues. Managed care organizations should be cautious when applying such criteria to restrict access to EDs.

Peer Review reports

Background

In the past 30 years, the number of visits to emergency departments (EDs) has increased, inducing overcrowding in many countries [1]. ED overcrowding is related to multiple complex problems: overburdened inpatient facilities, inadequate ED space, insufficient staffing, and inaccessibility to primary care services [2–6]. ED overcrowding has resulted in a longer stay in the ED and worse outcomes for persons who truly require emergency care [2, 7, 8].

Several review of the emergency medicine literature regarding EDs use and access to care over the past 30 years reveals significant evolution [9, 10]. Indeed, concerns have been raised in several countries about the increasing numbers of patients attending EDs [1, 11, 12] with particular attention given to "inappropriate" or "nonurgent" ED use [13–15].

Using ED, rather than primary care settings, for nonurgent care contributes to the phenomenon of ED overcrowding [10]. This can reduce the continuity of care and impair preventive care and appropriate therapy for chronic conditions [14–17].

To resolve ED overcrowding and decrease the number of nonurgent ED patients, many solutions have been proposed [18], such as educational interventions recommending people should seek other sources of care before considering ED [19–23] or implantation of "gatekeepers" who require patients to have authorization from their primary care provider before going to the ED [24, 25]. The most common solution has been for a nurse to triage the ED patients to identify potentially nonurgent patients, i.e. which could have been dealt with by general practitioner (GP) [20]. The main objective of triage is to assign a degree of urgency to patients depending on their complaint severity. In most of cases, the triage process is used to determine the priority of treatment in the ED. But many authors have proposed using the triage process to refer nonurgent patients to alternative sites of care [5, 22, 19, 26].

Refusing care to nonurgent ED patients or referring them to alternative sites for care raises legal, ethical, and safety issues. Because there is no consensual method of triage, it is impossible to reliably and reproducibly identify nonurgent ED patients, as evidenced by the variability of proportions of such patients in the literature (from 4.8% to 90%) [10, 19] and by the poor agreement between different methods of triage for the same patient group [10].

The objective of our study was to measure agreement on the urgency of an ED visit between the points of views of triage nurses and ED physicians. Second, we sought to determine if the level of agreement is consistent among different sub-groups based on following explicit criteria: age, medical status, and type of referral to the ED.

Methods

Study Design and Setting

A multicentric cross-sectional study was conducted over a 3-day period (a weekday and two weekend days), in April 2007, in a sample of EDs located in the Provence-Alpes-Côte d'Azur (PACA) region, in France. This region has a population of 4.8 million which represent 7.6% of the population of France, and covers 34,400 km2 with population densities from 153 persons per km2 [27]. The PACA region is served by a total of 53 EDs, which treated between 11,000 and 65,000 ED patients per year. The distribution of 53 EDs allows to 99% of the population access to an ED in less than 45 minutes (and 85% in less than 15 minutes). Private (17%) and public (83%) hospitals were represented.

The 53 EDs were classified according to the following two strata: the number of annual visits to these EDs (high attendance (25 000 or over visits per year) and medium or low attendance (less than 25 000 visits per year)) and, the geographical location of EDs (located urban area characterized by higher population density with at least 2 000 residents and by the urban-type land use, not allowing any gaps of typically more than 200 meters [28] or not). Finally, 17 EDs were randomly selected among the 53 EDs according the two strata. Table 1 describes the characteristics of these 17 EDs.

Table 1 Hospital Characteristics
Full size table

Population and Data collection

All patients aged 18 years and older who presented in one of participant EDs between the hours of 8 AM and 12 midnights were included. Study hours were limited because of few patients come after midnight [29]. Patients were excluded if they required immediate medical care and had communication difficulties.

Immediately after the admittance and nurse triage, all patients agreeing to participate were interviewed face to face in the ED by a trained research assistant who was not involved in care. The anonymous standardized questionnaire collected the following variables [Additional file 1]:

  • Patients' characteristics: demographic (age, sex) and socio-economic characteristics (employment status, health insurance status), utilization of health care services (having a primary care physician, Yes/No response from the patient), health status (suffering from chronic disease, Yes/No response from the patient).

  • ED visit characteristics: type of referral to the ED (self-referral, health care professional or other referral i.e. police, ambulance, employer, school, sports facility), chief complaint, duration of the presenting complaint, and mode of arrival. At the end of the ED consultation, the research assistant collected if the patient had diagnostic tests and treatments performed in the ED, and visit disposition (hospitalization).

Moreover, during their activity, trained triage nurses, after the admittance, and trained ED physicians, immediately at the end of the consultation, were asked to complete the questionnaire for each patient seen [Additional file 2]. They independently gave their expert opinion concerning the urgency of the admission of the patient. All ED health professionals had at least one year' experience of the ED.

Categorization of the urgency of the ED admission

According to the literature review, patients categorized as nonurgent are defined as those "who could have been dealt with by general practitioner" [10].

The categorization was conducted in two times and from two categories of ED health care professionals. Triage nurses, immediately after the admittance, and ED physicians, immediately at the end of the consultation, were asked to answer the rhetorical question, "Could this problem be taken care of by a primary care physician?" and, if the answer was yes, the ED visit was categorized as "nonurgent".

First, nurses conducted their triage interviews in the usual manner, i.e. without the use of written protocols or algorithms. The categorization was only done from a brief interview of the patient and included patient complaint(s). Second, as triage nurses, the ED physicians' categorization was done without the use of written protocols or algorithms. But the categorization was done from clinical examination, medical record, results of diagnostic tests, and from treatment performed in the ED. This categorization was performed in blind; ED physician raters did not have access to the triage nurses' notes.

For each patient, categorization was performed in usual conditions without disturbing the activity of ED health professionals. Triage nurses had not attended training session specifically for this study; however categorization of urgency is part of their qualifications [30].

Data analysis

Data were analyzed on Spss 16.0 by using proportions or means, and standard deviations of all variables. The main outcome variable was whether the ED visit was urgent or not.

To evaluate the level of agreement on triage categories between nurses and ED physicians, we calculated the chance-adjusted measure of agreement (Kappa-value) from 4 × 4 tables. Qualitative descriptions of agreement were as follows: 0.81-1.0 = "almost perfect", 0.61-0.80 = "substantial", 0.41-0.60 = "moderate", 0.21-0.40 = "fair", 0.0-0.20 = "slight" [31]. Kappa-values are reported with 95% confidence intervals (CIs). Sensitivity, specificity, and positive and negative predictive values (PPV and NPV) of accuracy of categorization into urgent and nonurgent case between triage nurses and ED physicians who were the reference. To assess the discrimination power of this model, a receiver operating characteristic (ROC) curve was constructed. The ROC curve is a graphic method for indicating the trade-off between the true-positive rate (sensitivity) and the false-positive rate (1 - the specificity) of a test or diagnostic manoeuvre. Generally, the most discriminating tests have the largest area under the ROC curve, the maximum being 1.0 [32]. Moreover, sensitivity, specificity, PPV and NPV of decision for hospitalization were calculated. For these analyses, ED patients hospitalized at the end of the consultation were compared with patients categorized into urgent or nonurgent cases by triage nurses and by ED physicians.

Subgroup analysis

Analyses of agreement were performed within subgroups stratified by explicit criteria. Subgroups were defined according to the following criteria:

  • Age: 75 years or older versus ager younger than 75 years,

  • Chief complaints recorded in 22 subgroups of case mix based on the "French Emergency Nurses Classification" [33],

  • Suffering from chronic disease versus none,

  • Duration of presenting complaints: 24 hours or less versus more than 24 hours,

  • Mode of arrival: own transport versus ambulance transport,

  • Type of referral to the ED recorded in 3 subgroups: self-referral, health care professional, other referral.

Kappa-values with 95% CI were analyzed within all these subgroups.

Ethical Considerations

Our study is a non interventional research and does not need to be approved by an ethics committee under the criteria of the bioethics law. So, our study does not require the authorization of the National Commission for Informatics and Freedom due respect for patient anonymity [34].

Results

During the study period, 1,949 adult patients visiting the 17 emergency departments were eligible for the study and 1,928 were included (98.9%). EDs received a mean of 113.4 adult patients ± 48.1 (median = 103, minimum = 31 and maximum = 172).

Of the 1,928 patients included, 350 were excluded from the analysis because data were not available from both triage nurses and ED physicians. The final study sample comprised 1,578 patients for whom two assessments were obtained.

Demographic characteristics and insurance status of ED patients [Table 2]

Table 2 Characteristics of the study population
Full size table

Of the 1,578 patients included in the study, 52.4% were males and the mean age of ED patients (± standard deviation (SD)) was 45.2 years ± 21.4 (from 18 to 100 years); 14.3% of patients were 75 years old and over. Most patients had primary health insurance with supplementary coverage (86.0%); 10.4% of them were covered by French health insurance specifically for individuals and families with low incomes and resources (named "CMUC"). The majority of included patients were followed by a general practitioner (92.9%). More than one third suffered from chronic disease (36.7%).

Characteristics of ED visits [Table 3]

Table 3 Characteristics of the ED visits
Full size table

Presenting complaints had lasted less than 24 hours for 77.7% of patients. Only 17% had been referred to the ED by a primary care physician. The others were self-referred (63.4%) or referred for medico-legal reasons (19.4%) (Employer, school, police...).

More than half of patients were consulting the ED for non-trauma complaints. Nearly two thirds of ED patients received diagnostic tests; 59.2% received treatment in the ED, and 22.7% were hospitalized.

Variability in the proportions of nonurgent ED visits and overall agreement between triage nurses and ED physicians

Of the 1,578 ED visits, the proportion of nonurgent ED patients was 26% according to triage nurses upon the entry, and 34.3% according to ED physicians at the end of the consultation (p < 0.001, Table 4). Overall level of agreement was moderate (kappa = 0.43 ± 0.02; 95% CI, 0.39% to 0.48%). The model showed a high sensitivity of 88.0% (Table 5). The area under the ROC curve was 0.70 with 95% CI 0.68 to 0.73 (Figure 1).

Table 4 Triage agreement between nurses and ED physicians
Full size table
Table 5 Sensitivity, specificity, and predictive value in prediction of urgent or nonurgent cases
Full size table
Figure 1
figure 1

Receiver operating curve (ROC) and area under curve (AUC) values for categorization into urgent and nonurgent case between triage nurses and ED physicians who were the reference.

Full size image

Table 4 shows the distribution between triage categories determined by triage nurses upon the entry to the ED and by ED physicians at the end of the consultation. Of the 1,036 patients categorized as urgent by ED physicians, 124 (12%) of them were categorized as nonurgent by triage nurses. These 124 patients were, for the majority, women (54%), self-referred (68.0%) and suffering from a medical problem for more than 24 hours (29.0%).

Variability in agreement between triage nurses and ED physicians within subgroups from explicit criteria characterizing the ED visit

Within the 17 EDs, the levels of agreement were variable, ranging from 0.21 to 0.71. The highest kappa value concerned an ED with the smallest number of patients (n = 31).

Table 6 shows results of analyses in subgroups. The levels of agreement within all subgroups based on explicit criteria were low (from moderate to slight) except in 3 subgroups of case mix.

Table 6 Subanalyses of agreement of explicit criteria
Full size table

The levels of agreement within the 22 subgroups of complaints were variable, ranging from 0.09 to 1.00. Among the 22 subgroups, 10 showed fair inter-observer agreement (k = 0.21-0.40) and 7 moderate agreement (k = 0.41-0.60). The lowest level of agreement concerned the subgroup of urinary-nephrology (k = 0.09, slight). The highest kappa-values concerned three subgroups of complaints: cranial injury (k = 0.61, substantial), gynecological complaints (k = 0.66, substantial) and toxicology complaints (k = 1.00, almost perfect).

For the other subgroups, levels of agreement were also low (from 0.20 to 0.47) and showed considerable variability. The lowest level of agreement concerned the subgroup of hospitalization (k = 0.20, slight) and the highest concerned the three following subgroups: duration of the presenting complaint (> 24 hours, k = 0.47), suffering from chronic disease (k = 0.47) and self-referral (k = 0.46). These three levels of agreement were moderate.

Is that hospital admission is a relevant indicator to categorize patients into urgent or nonurgent cases?

Hospital admission is not a relevant indicator. The distribution of categorization of urgency relative to hospitalization status is shown in Table 7. Whatever the professional who conducted the categorization (triage nurse or ED physician), most urgent patients were not hospitalized. Among the 409 nonurgent patients identified by triage nurses, 9% were hospitalized. These patients had no specific characteristics. Similarly, among the 536 nonurgent patients identified by ED physicians, 18 were hospitalized (3.4%). The majority of these 18 patients were older (70%, mean age 69.2 years ± 4.7; median 79.5 years), and reported neuropsychological problems (20%) and alteration of clinical status (20%).

Table 7 Relationship between categorization of ED visits and hospitalization (n = 1,553)
Full size table

When categorization of ED visits into urgent or nourgent cases was compared to hospitalization, ED physicians had higher sensitivity and specificity than nurses (respectively 94.9% versus 89.5%, and 43.1% versus 30.9%). Overall, for ED physicians and triage nurses, positive predictive values were low (32.8% versus 27.5%) and negative predictive values were higher (96.6% versus 90.9%) [Table 8].

Table 8 Sensitivity, specificity, and predictive value in prediction of hospitalization
Full size table

Discussion

Our study shows a moderate level of agreement between triage nurses and ED physicians in decisions to categorize patients in urgent or nonurgent cases. This finding corroborates the results of the previous studies of Brillman et al., Caterino et al., Frey et al., O'Brien et al., and Lowe et al., who used the same method and also found poor kappa levels of agreement [35–39]. Kelly et al. are the only ones who found a high level of agreement between nurses and ED physicians (kappa = 0.74), probably because the categorization performed by the nurses and physicians was conducted at the same time (after patients' discharge from the ED) and was based on chart review [40]. In our study, like in the others, categorization was performed at two times: upon the entry to the ED by triage nurses, and at the end of visit by ED physicians. Moreover, our data was collected from a representative sample, indeed the socio-demographic and ED visit characteristics were similar to those reported in the literature [6, 10, 29].

Whatever the subgroups stratified by explicit criteria, the level of agreement remained moderate, except for three subgroups of complaint: toxicology, gynecological and cranial injury subgroups. The high levels of agreement for these three subgroups can be explained by the homogeneneity of case mix. For example, the subgroup of toxicology concerned only two kinds of diagnoses: carbon monoxide poisoning and alcoholism.

We also found a low level of agreement for the sub-group of patients older than 75 years. Relative to younger ED patients, elderly patients have a complex mix of medical and social needs which increases the difficulty to categorize patients into urgent or nonurgent cases.

Our study shows a slight level of agreement between triage nurse and ED physicians within the subgroup of hospitalization. This finding corroborates previous studies [34, 41] which have shown limitations in using the criterion of hospitalization as an outcome variable to categorize patients into nonurgent cases [2, 34, 41]. However, this variable is often chosen by authors because it is the only concrete outcome variable recognized as the surrogate indicator of the need for prompt care. The low predictive positive value found in our study corroborates that hospitalization is not a consistent outcome variable to categorize patients into urgent or nonurgent cases.

It is not defined that all urgent patients need hospitalization after ED consultation and/or that all nonurgent patients should be discharged to home. However, urgents patients with potentially serious complaints (chest or abdominal pain, asthma...) or serious clinical signs (hypoglycemia, persistent fever, alteration in blood pressure) may be investigated, treated, and discharged from the ED. Moreover, the decision to hospitalize a patient categorized as nonurgent may be somewhat subjective and at times based largely on multiple social, economic factors or because of deficiencies in downstream interventions that are specific to a particular patient population. In this case, hospitalized patients categorized as nonurgent by ED physicians could be described as inappropriate. The results showed that physicians were not influenced by the final disposition of hospitalization. Indeed, hospitalized patients categorized as nonurgent (n = 18) were elderly and cognitively impaired.

The finding of low agreement between triage nurses and ED physicians is due partly to the two times of categorization. Indeed, the categorization conducted by ED physicians at the end of the consultation have the benefit of information based on supplementary explicit criteria, like the results of diagnostic tests performed during the ED visit and/or a consultation with a specialist physician. Our objective is not to reconsider the role of the triage nurse; we recognize that a brief triage performed by a nurse cannot always predict whether the patient has an urgent problem or not. However, this finding highlights the potential unsafe of triage, especially if the objective of the triage is to redirect nonurgent patients outside the ED. Indeed, the risk is to inadvertently refuse care to patients who truly in need of emergency interventions.

Limitations

Several potential limitations should be addressed. Firstly, while we examined in great detail the different sub-groups based on following explicit criteria: age, medical status, and type of referral to the ED, we did not analyze the impact of the trained ED health professionals themselves. We conducted the study with ED health care professionals present during the inclusion period, in the usual manner, i.e. without the use of written protocols or algorithms. However, in previous studies measuring level of agreement, training, experience, knowledge, and skill of ED health professionals did not influence kappa values [35, 42, 43]. The authors found substantial disagreement even among health care professionals with the same training. Secondly, when designing our test study, sample size calculation should have been performed in order to guarantee the design accuracy. But, we performed a sample size calculation retrospectively based on the methodology of Flack VF et al [44]. Data were analysed on PASS 2008. In a test for agreement between two raters using Kappa statistic, a sample size of 1,986 subjects achieves 80% power to detect a true Kappa value of 0.43 in test of null hypothesis: Kappa = 0.50 versus alternative hypothesis: Kappa <> 0.50 when there are two categories with frequencies equal to 0.70 and 0.30. This power calculation is based on a significance level of 0.050. Thus, we included 1,578 patients in our study. Moreover, we found six similar studies which compared different methods of categorization in the same population [35, 40]. These articles showed considerable variability in levels of agreement between the different methods to categorize ED visits into nonurgent or urgent cases, ranging in κ value from 0.20 to 0.74. These studies did not perform a sample size calculation.

Conclusions

This multicentric study of 1,578 adults on triage to identify nonurgent patients demonstrates triage conducted by nurses is not consistent. The lack of physician-nurse agreement and the inability to predict hospitalization have important implications for patient safety. When categorization of urgency is used to determine the priority of treatment into the ED, disagreement might not matter because all patients in the ED are seen and treated. When urgency assessments are used as the basis for refusal of care to potential ED patients, the uncertainly is a matter of greater concern. Therefore, considerable caution should be used when managed care organizations apply such criteria to restrict access to EDs.

References

  1. Hoot NR, Aronsky D: Systematic review of emergency department crowding: causes, effects, and solutions. Ann Emerg Med. 2008, 52: 126-36. 10.1016/j.annemergmed.2008.03.014.

    Article  PubMed  Google Scholar 

  2. Lowe RA, Bindman AB, Ulrich SK, Norman G, Scaletta TA, Keane D, Washington D, Grumbach K: Refusing care to emergency department of patients: evaluation of published triage guidelines. Ann Emerg Med. 1994, 23: 286-93. 10.1016/S0196-0644(94)70042-7.

    Article  CAS  PubMed  Google Scholar 

  3. Sempere-Selva T, Peiró S, Sendra-Pina P, Martínez-Espín C, López-Aguilera I: Inappropriate use of an accident and emergency department: magnitude, associated factors, and reasons-an approach with explicit criteria. Ann Emerg Med. 2001, 37: 568-79. 10.1067/mem.2001.113464.

    Article  CAS  PubMed  Google Scholar 

  4. Derlet RW, Richards JR: Overcrowding in the nation's emergency departments: complex causes and disturbing effects. Ann Emerg Med. 2000, 35: 63-8. 10.1016/S0196-0644(00)70105-3.

    Article  CAS  PubMed  Google Scholar 

  5. Derlet RW, Nishio D, Cole LM, Silva J: Triage of patients out of the emergency department: three-year experience. Am J Emerg Med. 1992, 10: 195-9. 10.1016/0735-6757(92)90207-E.

    Article  CAS  PubMed  Google Scholar 

  6. Gentile S, Vignally P, Durand AC, Gainotti S, Sambuc R, Gerbeaux P: Nonurgent patients in the emergency department? A French formula to prevent misuse. BMC Health Serv Res. 2010, 10: 66-10.1186/1472-6963-10-66.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Schull MJ, Kiss A, Szalai JP: The effect of low-complexity patients on emergency department waiting times. Ann Emerg Med. 2007, 49: 257-64. 10.1016/j.annemergmed.2006.06.027.

    Article  PubMed  Google Scholar 

  8. American College of Emergency Physicians: Hospital and emergency department overcrowding. Ann Emerg Med. 1990, 19: 336-10.1016/S0196-0644(05)82059-1.

    Article  Google Scholar 

  9. Richardson LD, Hwang U: Access to care: a review of the emergency medicine literature. Acad Emerg Med. 2001, 8: 1030-6. 10.1111/j.1553-2712.2001.tb01111.x.

    Article  CAS  PubMed  Google Scholar 

  10. Durand AC, Gentile S, Devictor B, Palazzolo S, Vignally P, Gerbeaux P, Sambuc R: ED patients: how nonurgent are they? Systematic review of the emergency medicine literature. Am J Emerg Med. 2011, 29: 333-45. 10.1016/j.ajem.2010.01.003.

    Article  PubMed  Google Scholar 

  11. Baubeau D, Deville A, Joubert M, Fivaz C, Girard I, Le Laidier S: Les passages aux urgences de 1990 à 1998: une demande croissante de soins non programmés. Direction de la recherche, des études, de l'évaluation et des statistiques: Etudes et Résultats. 2000, 72:

    Google Scholar 

  12. Derlet RW, Richards JR: Overcrowding in the nation's emergency departments: complex causes and disturbing effects. Ann Emerg Med. 2000, 35: 63-8. 10.1016/S0196-0644(00)70105-3.

    Article  CAS  PubMed  Google Scholar 

  13. Cunningham PJ, Clancy CM, Cohen JW, Wilets M: The use of hospital emergency department for non urgent health problems: a national perspective. Med Care Res Rev. 1995, 52: 453-74. 10.1177/107755879505200402.

    Article  CAS  PubMed  Google Scholar 

  14. Bezzina AJ, Smith PB, Cromwell D, Eagar K: Primary care patients in the emergency department: who are they? A review of the definition of the 'primary care patient' in the emergency department. Emerg Med Australas. 2005, 17: 472-9. 10.1111/j.1742-6723.2005.00779.x.

    Article  PubMed  Google Scholar 

  15. Padgett DK, Brodsky B: Psychosocial factors influencing non-urgent use of the emergency room: a review of the literature and recommendations for research and improved service delivery. Soc Sci Med. 1992, 35: 1189-97. 10.1016/0277-9536(92)90231-E.

    Article  CAS  PubMed  Google Scholar 

  16. Peneff J: Les malades des urgences. Edited by: Métaillé. 2000

    Google Scholar 

  17. Baker DW, Stevens CD, Brook RH: Regular source of ambulatory care and medical care utilization by patients presenting to a public hospital emergency department. JAMA. 1994, 271: 1909-12. 10.1001/jama.271.24.1909.

    Article  CAS  PubMed  Google Scholar 

  18. Derlet RW, Richards JR: Ten solutions for emergency department crowding. West J Emerg Med. 2008, 9: 24-7.

    PubMed  PubMed Central  Google Scholar 

  19. Gill JM: Use of hospital emergency departments for nonurgent care: a persistent problem with no easy solutions. Am J Manag Care. 1999, 5: 1565-8.

    CAS  PubMed  Google Scholar 

  20. Murphy AW: 'Inappropriate' attenders at accident and emergency departments I: definition, incidence and reasons for attendance. Fam Pract. 1998, 15: 23-32. 10.1093/fampra/15.1.23.

    Article  CAS  PubMed  Google Scholar 

  21. Liggins K: Inappropriate attendance at accident and emergency departments: a literature review. J Adv Nurs. 1993, 18: 1141-5. 10.1046/j.1365-2648.1993.18071141.x.

    Article  CAS  PubMed  Google Scholar 

  22. Murphy AW: 'Inappropriate' attenders at accident and emergency departments II: health service responses. Fam Pract. 1998, 15: 33-7. 10.1093/fampra/15.1.33.

    Article  CAS  PubMed  Google Scholar 

  23. Afilalo J, Marinovich A, Afilalo M, Colacone A, Léger R, Unger B, Giguère C: Nonurgent emergency department patient characteristics and barriers to primary care. Acad Emerg Med. 2004, 11: 1302-10. 10.1111/j.1553-2712.2004.tb01918.x.

    Article  PubMed  Google Scholar 

  24. Young GP, Lowe RA: Adverse outcomes of managed care gatekeeping. Acad Emerg Med. 1997, 4: 1129-36. 10.1111/j.1553-2712.1997.tb03695.x.

    Article  CAS  PubMed  Google Scholar 

  25. Simonet D: Cost reduction strategies for emergency services: insurance role, practice changes and patients accountability. Health Care Anal. 2009, 17: 1-19. 10.1007/s10728-008-0081-0.

    Article  PubMed  Google Scholar 

  26. Waldrop RD, Harper DE, Mandry C: Prospective assessment of triage in an urban emergency department. South Med J. 1997, 90: 1208-12. 10.1097/00007611-199712000-00009.

    Article  CAS  PubMed  Google Scholar 

  27. Institut National de la Statistique et des Etudes Economiques: Provence Alpes Côtes d'Azur: présentation de la région. [http://www.insee.fr/fr/regions/provence/default.asp?page=faitsetchiffres/presentation/presentation.htm]

  28. Institut National de la Statistique et des Etudes Economiques: Unité urbaine (Définition). [http://www.insee.fr/fr/methodes/default.asp?page=definitions/unite-urbaine.htm]

  29. Carrasco V, Baubeau D: Les usagers des urgences. Premiers résultats d'une enquête nationale. Direction de la recherche, des études, de l'évaluation et des statistiques: Etudes et Résultats. 2003, 212:

    Google Scholar 

  30. Société Francophone de Médecine d'Urgence: Référentiel Infirmière Organisateur de l'Accueil. 2004, [http://www.sfmu.org/documents/ressources/referentiels/ioa2004.pdf]

    Google Scholar 

  31. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-74. 10.2307/2529310.

    Article  CAS  PubMed  Google Scholar 

  32. Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982, 143: 29-36.

    Article  CAS  PubMed  Google Scholar 

  33. Taboulet P, Moreira V, Haas L, Porcher R, Braganca A, Fontaine JP, Poncet MC: Triage with the French Emergency Nurses Classification in Hospital scale: reliability and validity. Eur J Emerg Med. 2009, 16: 61-7. 10.1097/MEJ.0b013e328304ae57.

    Article  PubMed  Google Scholar 

  34. Loi n°2004-806 du 9 août 2004: relative à la politique de santé publique. Protection des personnes en matière de santé.

  35. Brillman JC, Doezema D, Tandberg D, Sklar DP, Davis KD, Simms S, Skipper BJ: Triage: limitations in predicting need for emergent care and hospital admission. Ann Emerg Med. 1996, 27: 493-500. 10.1016/S0196-0644(96)70240-8.

    Article  CAS  PubMed  Google Scholar 

  36. Caterino JM, Holliman CJ, Kunselman AR: Underestimation of case severity by emergency department patients: implications for managed care. Am J Emerg Med. 2000, 18: 254-6. 10.1016/S0735-6757(00)90115-3.

    Article  CAS  PubMed  Google Scholar 

  37. Frey L, Schmidt J, Derksen DJ, Skipper B: A rural emergency department. West J Med. 1994, 160: 38-42.

    CAS  PubMed  PubMed Central  Google Scholar 

  38. O'Brien GM, Shapiro MJ, Woolard RW, O'Sullivan PS, Stein MD: "Inappropriate" emergency department use: a comparison of three methodologies for identification. Acad Emerg Med. 1996, 3: 252-7. 10.1111/j.1553-2712.1996.tb03429.x.

    Article  PubMed  Google Scholar 

  39. Lowe RA, Bindman AB: Judging who needs emergency department care: a prerequisite for policy-making. Am J Emerg Med. 1997, 15: 133-6.

    Article  CAS  PubMed  Google Scholar 

  40. Kelly LJ, Birtwhistle R: Is this problem urgent? Attitudes in a community hospital emergency room. Can Fam Physician. 1993, 39: 1345-52.

    CAS  PubMed  PubMed Central  Google Scholar 

  41. Beveridge R, Ducharme J, Janes L, Beaulieu S, Walter S: Reliability of the Canadian emergency department triage and acuity scale: interrater agreement. Ann Emerg Med. 1999, 34: 155-9. 10.1016/S0196-0644(99)70223-4.

    Article  CAS  PubMed  Google Scholar 

  42. Gill JM, Reese CL, Diamond JJ: Disagreement among health care professionals about the urgent care needs of emergency department patients. Ann Emerg Med. 1996, 28: 474-9. 10.1016/S0196-0644(96)70108-7.

    Article  CAS  PubMed  Google Scholar 

  43. Sim J, Wright CC: The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005, 85: 257-68.

    PubMed  Google Scholar 

  44. Flack VF, Afifi AA, Lachenbruch PA, Schouten HJA: Sample size determinations for the two rater kappa statistic. Psychometrika. 1987, 53: 321-325.

    Article  Google Scholar 

Pre-publication history

Download references

Acknowledgements

We thank all 17 emergency departments who participated. The study would not have been possible without the kind and efficient support of all the ED staffs.

Chief ED physician (Emergency Department, Town):

Gerbeaux P, MD, PhD (La Conception University Hospital, Marseille); Jean P, MD (Nord University Hospital, Marseille); Alazia M, MD (Sainte Marguerite University Hospital, Marseille); Kiegel P, MD (Pays d'Aix Hospital, Aix En Provence); Lindenmeyer E, MD (Saint Joseph Hospital, Marseille); Hamon M, MD (General Hospital, Martigues); Valeri L, MD (General Hospital, La Ciotat); Minguet JM (General Hospital, Draguignan); Pedrant G, MD (General Hospital, Toulon and La Seyne); Vuides G, MD (Genral Hospital, Hyères); Hidoux MA, MD (General Hospital, Gap); Burckel S, MD (General Hospital, Dignes Les Bains); Olivier P, MD (Henri Duffaut Hospital, Avignon); Liauthaud H, MD (General Hospital, Apt); Vanneyre J, MD (General Hospital, Pertuis); Mattei V, MD (La Palmosa Hospital, Menton).

Author information

Authors and Affiliations

  1. Laboratoire de Santé Publique, Faculté de Médecine, Equipe de recherche EA 3279 "Evaluation hospitalière-Mesure de la santé perçue", Marseille, France

    Anne-Claire Durand, Stéphanie Gentile & Roland Sambuc

  2. Service d'Accueil des Urgences, Hôpital de La Conception, Marseille, France

    Patrick Gerbeaux

  3. Service d'Accueil des Urgences, Hôpital Sainte Marguerite, Marseille, France

    Marc Alazia

  4. Service d'Accueil des Urgences, Hôpital du Pays d'Aix, Aix en Provence, France

    Pierre Kiegel

  5. Service d'Accueil des Urgences, Centre Hospitalier Général, Martigues, France

    Stephane Luigi

  6. Service d'Accueil des Urgences, Hôpital Saint Joseph, Marseille, France

    Eric Lindenmeyer

  7. Service d'Accueil des Urgences, Hôpital Henri Duffaut, Avignon, France

    Philippe Olivier

  8. Service d'Accueil des Urgences, Centre Hospitalier Général, Gap, France

    Marie-Annick Hidoux

Authors
  1. Anne-Claire Durand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stéphanie Gentile
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick Gerbeaux
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marc Alazia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pierre Kiegel
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stephane Luigi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Eric Lindenmeyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Philippe Olivier
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Marie-Annick Hidoux
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Roland Sambuc
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anne-Claire Durand.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

ACD and SG participated in the design and the coordination of the study, performed the statistical analysis and helped to draft the manuscript. SG, ACD, PG, MA, PK, SL, PO and MAH participated in the design of the study, interpreted the results and helped to draft the manuscript. RS participated in the statistical analysis and helped to draft the results. All authors read and approved the final manuscript.

Electronic supplementary material

12873_2011_131_MOESM1_ESM.DOC

Additional file 1: Patient questionnaire. Questionnaire used to assess the urgency of an ED visit and to explore factors associated or not with this assessment. (DOC 49 KB)

Additional file 2: ED physician questionnaire. Questionnaire used to assess the ED visit. (DOC 36 KB)

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Durand, AC., Gentile, S., Gerbeaux, P. et al. Be careful with triage in emergency departments: interobserver agreement on 1,578 patients in France. BMC Emerg Med 11, 19 (2011). https://doi.org/10.1186/1471-227X-11-19

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-227X-11-19

Keywords

BMC Emergency Medicine

ISSN: 1471-227X

Contact us