Cataract is very prevalent in developed countries, and the incidence is expected to increase as the result of the aging of the population and increased life expectancies 1. In addition, cataract extraction is the most common surgical intervention in developed countries 2. Due to the high prevalence of this condition, clinicians, managers, and health payers must decide who should undergo cataract extraction. Variations in the use of cataract extraction also have been reported, which led to the need to study if overuse or underuse of the procedure was occurring 345. The final goals are to increase the quality of care and the cost-effectiveness in our medical system, which depends on the development of appropriateness criteria to make appropriate decisions.

A way to develop explicit criteria is the methodology developed in the 1980s by the RAND-UCLA group 6. This is a very popular methodology that has been used in many different diagnostic and therapeutic procedures since then, including cataract extraction. RAND chose cataract extraction as one of the first procedures for which to develop appropriateness criteria 7. In the1990's, Tobacman et al. applied explicit criteria for cataract extraction following this methodology 8. Although phacoemulsification has been done since the 1970s and some of the cataract extractions that were reported previously in assessment of appropriateness were performed by phacoemulsification, at that time, phacoemulsification was just beginning to be performed, but now it is the primary surgical technique used for uncomplicated cataract extraction in developed countries 9. As happens when a new treatment is introduced, the criteria to perform the intervention may change, because new variables should be considered in the decision making process, and some variables that previously were important are no longer so.

The purpose of this study was to update the appropriateness criteria, using the RAND method, for patients undergoing cataract extraction by phacoemulsification exclusively.

The same 765 indications were evaluated in the two rounds because no new variables or categories were introduced by the panel of experts in the second round. Agreement among panelists reached 40.1% in the first round and 53.3% in the second round, being then a 0.9% of disagreement at 2^nd round. Finally, 32.9% indications were rated as appropriate, 30.1% as uncertain, and 37% as inappropriate (Table 1). Of the three main categories analyzed (simple cataract, cataract with diabetic retinopathy, and cataract with other ocular pathologies), higher agreement was found for simple cataract (57.5%), with a higher appropriateness rate (54.9%). For cataract with diabetic retinopathy, 29.7% indications were rated as appropriate, and 25.2% for cataract with other ocular pathologies (Table 2). Intraclass correlation coefficient among the 12 panelists scores at 2^nd round was 0.69.

Table 3 shows the changes in scoring by panelists. Panelists 1 and 6 scored more extremely in round 1 and tended to regress to the mean in round 2, as also can be seen by the higher conformity score. Panelists 7 and 10 did not modify their scores between rounds.

The influence of each variable included in the algorithm was analyzed by linear and logistic regression by category (Table 4). For simple cataract, visual acuity in the operated eye and visual function were the most influential variables. For the other two categories, visual function was the most influential variable followed by the anticipated visual acuity after surgery and the visual acuity in the cataractous eye. Both linear and logistic regression models provided similar results. The type of cataract was not statistically significant in any model, and the contralateral visual acuity was relevant in some cases.

Finally, the criteria developed by the panel of experts were summarized by CART analysis. Figure 1 (Additional file 2) shows the decision tree for simple cataract. The indications considered appropriate were: 1) visual acuity in the cataractous eye lower than 0.5 and difficulty with activities of daily living, 2) visual acuity of 0.1 or less and glare or difficulty with recreational activities, 3) visual acuity between 0.2 to 0.4 and difficulty with recreational activities (if low or medium surgical complexity), or in cases in which glare is the patient complaint, low surgical complexity, or if medium surgical complexity a contralateral visual acuity between 0.2–0.4, and 4) visual acuity higher than 0.4 if there is difficulty with activities of daily living and low or medium surgical complexity. Globally, cataract extraction was considered inappropriate in patients without impaired visual function with a visual acuity higher than 0.1, or if in the presence of glare or difficulty with recreational activities the visual acuity was higher than 0.4, depending on surgical complexity.

In patients with diabetic retinopathy (Figure 2, Additional file 2), the indications considered appropriate were: 1) visual acuity in the cataractous eye of 0.1 or less and difficulty with activities of daily living and an anticipated postoperative visual acuity higher than 0.1, or unilateral cataract with low-to-medium surgical complexity and difficulty with recreational activities or glare and an anticipated postoperative visual acuity between 0.2 to 0.4, or difficulty with recreational activities or glare and an anticipated postoperative visual acuity higher than 0.4. In addition, patients with a visual acuity between 0.2 and 0.4 and an anticipated postoperative visual acuity higher than 0.4 with difficulty with activities of daily living, or with difficulty with recreational activities but with low or medium surgical complexity.

In patients with other ocular pathologies (Figure 3, Additional file 2), the indications considered appropriate were: 1) visual acuity in the cataractous eye of 0.1 or lower with glare, with low surgical complexity, and with an anticipated postoperative visual acuity higher than 0.4, 2) difficulty with recreational activities and an anticipated visual acuity higher than 0.1, 3) difficulty with activities of daily living and an anticipated postoperative visual acuity higher than 0.1, 4) visual acuity between 0.2 and 0.4 and an anticipated postoperative visual acuity higher than 0.5 with difficulty with activities of daily living or recreational activities but low surgical complexity.

The classification of the previous decision trees was compared with the original panel scores (Table 5). No indication scored as appropriate by the panel was classified as inappropriate and vice versa. Globally, misclassification error was 0.053. By category, six indications were classified erroneously into the simple cataract group (misclassification error, 0.039), 22 into cataract with diabetic retinopathy (misclassification error, 0.072), and 13 into cataract with other ocular pathologies (misclassification error, 0.042). The numbers of misclassified indications are also indicated in the three figures for each final appropriateness node.

We collected information on 1,053 patients to identify how many of the theoretical indications were used in clinical practice. Globally, 205 of 765 theoretical (26.8%) indications were assigned to 1,053 patients. By category, for simple cataract, 98 of 153 theoretical (64%) indications were assigned to 815 patients with that diagnosis. Of those with diabetic retinopathy, 22 of 306 theoretical (7.2%) indications were assigned to 27 patients. Of those with other ocular pathologies, 85 of 306 theoretical (27.8%) indications were assigned to 211 patients within that category.

RAND appropriateness methodology had been used to create explicit criteria to evaluate the appropriateness of cataract extraction 781314. Nevertheless, as the developers of the RAND methodology pointed out, this methodology cannot capture changes over time with the advent of new diagnostic or therapeutic techniques 15. This also is the case with cataract extraction. Phacoemulsification is now the most frequently performed technique in developed countries9 for cataract extraction. Our study incorporates new explicit criteria that can be used with patients undergoing only phacoemulsification.

The work of our panel of experts showed a low disagreement rate among panelists in the second round, compared with other studies in which a similar methodology was used 16. Also, the intraclass correlation coefficient showed acceptable results. This partially support the reliability of the tool. This can be due to the fact that our panel of experts included only ophthalmologists. The agreement rate was slightly higher for simple cataract than for the other two categories and the proportion of indications scored as appropriate also was slightly higher. This is important because this was the most frequently encountered group in the pilot field study. As shown, previous panels comprised of doctors of one specialty tended to consider more scenarios appropriate than multidisciplinary panels 17. However, we assembled a mono-specialists panel of expert ophthalmologists because in our health system ophthalmologists alone make the decision to perform cataract surgery. In addition to their interest in our conclusions, our criteria should be available to other physician who may play a role in the referral of these patients to the ophthalmologist so that those patients can be considered for surgery if appropriate.

Our results indicated that among the variables included in the criteria, a few play an important role, as reflected by the linear and logistic regression of all the scores. In addition, the decision trees based on CART analysis used only a few variables. The three important variables were: with simple cataract, the preoperative visual acuity in the cataractous eye, as evaluated by the ophthalmologist; the subjective visual function reported by the patient; and in some cases the surgical complexity. In the other two diagnostic groups (cataract with diabetic retinopathy or with other ocular pathologies), the most important variables were the preoperative visual acuity in the cataractous eye, the subjective visual function and the anticipated postoperative visual acuity in the cataractous eye. Here, again, the preoperative visual acuity in the operated eye was the most important variable, while the visual function was third. These results partially support the face and construct validity of our criteria, since those variables have been identified in different studies as the ones that should play an important role when deciding the appropriateness of cataract extraction8. However, studies that did not use the RAND methodology identified the same variables as relevant. Therefore, visual function has been reported by different studies as a key variable from the patient perspective 18. From the ophthalmologist perspective, the visual acuity in the cataractous eye is the primary variable, followed by others such as the presence of ocular comorbidities or the visual acuity in the contralateral eye 5.

The first study to create explicit criteria following the RAND method developed 1,953 indications during the first round that increased to 2,905 in the second round 8. The number of diagnoses (ocular comorbidities) included was greater than in our study. In our case, we collected information about the prevalence in clinical practice of all possible ocular pathologies in patients presenting with the need for cataract extraction before we created the indications. We gathered the information from the literature, our administrative databases, and the ophthalmologists of our research team. Since some diagnoses were extremely uncommon, we focused on those most frequently encountered: simple cataract, cataract with diabetic retinopathy, and a new global category called cataract with other ocular pathologies. The last group included some diagnoses that also had been included in the previous RAND studies 8 as separate groups, which led those authors to have a high number of indications. We included them in a single group due to the criteria of our ophthalmologists who considered first, from the standpoint of surgical intervention, that all the diagnostic groups included in this category have in common that affect the anticipated postoperative visual acuity. Second, they considered that there was no other additional criteria-variable- for each of those diagnostic groups who may force us to have them separately.

RAND methodology has some limitations. Among them, the excessive number of indications that were developed19. In our case 765 were far fewer than those in previous studies of cataract surgery appropriateness or even in other RAND studies of other non-ophthalmic procedures 1620. We considered other diagnostic groups, such as Fuchs' corneal dystrophy or uveitis, but decided not to included Fuchs' corneal dystrophy in this study due to the low prevalence; however, uveitis was included as a surgical complexity. A large number of indications compromised the work of the panel of experts because it forced them to spend considerable time scoring all indications. An additional problem related to the previous one is that the number of theoretical indications used in clinical practice were very few. We found that only 27% of all indications were used, and the percentages varied considerably among the three diagnosis groups, from 73% in simple cataract to 7.2% in cataract with diabetic retinopathy. This means that we asked the panel of experts to score an large number of indications that were unlikely to be present in clinical practice. This can bias the scoring of such indications because no evidence can be found in the literature about the efficacy of phacoemulsification for those indications due to the low prevalence, and because the panelists probably have no experience with them 19. For this reason, we recommend to future developers of explicit criteria using this methodology to, first, study the prevalence of the different diagnostic groups that lead to the intervention and then develop the theoretical indications that are more likely to be found in clinical practice. Additional criteria also should be the presence of important variability in the use of the procedure or the absence of evidence.

There are different ways to use these explicit criteria. As other authors have done with other procedures, and even with cataract extraction, the explicit criteria can be used for utilization review and health services research studies14 and can be converted to practice guidelines for use by clinicians, managers, or health payers21. However, the criteria should be used cautiously, because as the developers of the RAND method pointed out, other patient circumstances not included in these criteria may play an important role in clinician decision making 22.

In conclusion, this study updates previous work that created explicit criteria for cataract extraction following the RAND methodology. The criteria created are based on variables identified by most studies as relevant, which partially support their validity. Finally, we summarized the results in accurate decision trees, which allow use of the criteria in clinical practice or in the development of practice guidelines.

The author(s) declare that they have no competing interests.

JMQ conceived of the study, coordinated and participated in the design of the study, and drafted the manuscript. AE participated in the design and helped to draft the manuscript. IA performed the statistical analyses and drafted the manuscript. The IRYSS-Appropriateness Cataract Group participated in the design of the study and helped to draft the manuscript. All authors read and approved the final manuscript.