Skip to main content
Download PDF

Effectiveness of implementation strategies for increasing clinicians’ use of five validated imaging decision rules for musculoskeletal injuries: a systematic review

BMC Emergency Medicine volume 24, Article number: 84 (2024) Cite this article

Abstract

Background

Strategies to enhance clinicians’ adherence to validated imaging decision rules and increase the appropriateness of imaging remain unclear.

Objective

To evaluate the effectiveness of various implementation strategies for increasing clinicians’ use of five validated imaging decision rules (Ottawa Ankle Rules, Ottawa Knee Rule, Canadian C-Spine Rule, National Emergency X-Radiography Utilization Study and Canadian Computed Tomography Head Rule).

Design

Systematic review.

Methods

The inclusion criteria were experimental, quasi-experimental study designs comprising randomised controlled trials (RCTs), non-randomised controlled trials, and single-arm trials (i.e. prospective observational studies) of implementation interventions in any care setting. The search encompassed electronic databases up to March 11, 2024, including MEDLINE (via Ovid), CINAHL (via EBSCO), EMBASE (via Ovid), Cochrane CENTRAL, Web of Science, and Scopus. Two reviewers assessed the risk of bias of studies independently using the Cochrane Effective Practice and Organization of Care Group (EPOC) risk of bias tool. The primary outcome was clinicians’ use of decision rules. Secondary outcomes included imaging use (indicated, non-indicated and overall) and knowledge of the rules.

Results

We included 22 studies (5-RCTs, 1-non-RCT and 16-single-arm trials), conducted in emergency care settings in six countries (USA, Canada, UK, Australia, Ireland and France). One RCT suggested that reminders may be effective at increasing clinicians’ use of Ottawa Ankle Rules but may also increase the use of ankle radiography. Two RCTs that combined multiple intervention strategies showed mixed results for ankle imaging and head CT use. One combining educational meetings and materials on Ottawa Ankle Rules reduced ankle injury imaging among ED physicians, while another, with similar efforts plus clinical practice guidelines and reminders for the Canadian CT Head Rule, increased CT imaging for head injuries. For knowledge, one RCT suggested that distributing guidelines had a limited short-term impact but improved clinicians’ long-term knowledge of the Ottawa Ankle Rules.

Conclusion

Interventions such as pop-up reminders, educational meetings, and posters may improve adherence to the Ottawa Ankle Rules, Ottawa Knee Rule, and Canadian CT Head Rule. Reminders may reduce non-indicated imaging for knee and ankle injuries. The uncertain quality of evidence indicates the need for well-conducted RCTs to establish effectiveness of implementation strategies.

Peer Review reports

Introduction

Imaging decision rules are tools that can increase the appropriateness of imaging requests [1] by guiding clinicians on when imaging studies, such as X-rays or CT scans, are indicated (e.g. for suspected serious conditions or injuries) and when they are unnecessary (e.g. for less concerning conditions or injuries). Several validated imaging decision rules can help clinicians differentiate patients at high or low risk of having a serious pathology (e.g., fracture) following a musculoskeletal injury [2]. These include the Canadian CT Head Rule, Canadian C-spine Rule, National Emergency X-Radiography Utilization Study (NEXUS) guideline, Ottawa Ankle Rules and Ottawa Knee Rule [3,4,5,6]. The Canadian CT Head Rule can help clinicians identify patients at low risk of brain injury and who do not require a CT scan (99–100% sensitivity) [2]. The Canadian C-spine Rule (99–100% sensitivity) [3] and the NEXUS guideline (83–100% sensitivity) [5] assess potential cervical spine injuries using criteria including cervical spine tenderness, level of alertness, neurological deficits, evidence of intoxication, painful distracting injuries, patient’s age, mechanism of injury, neck mobility and numbness in arms or legs. The Ottawa Ankle Rules (99.4% sensitivity) [3] and Ottawa Knee Rule (98.5% sensitivity) [4] determine the necessity for ankle and knee X-rays, respectively, through criteria such as weight-bearing ability and tenderness. Use of these rules can help ensure clinicians do not miss serious injuries while avoiding unnecessary or non-indicated imaging [7,8,9].

Several studies have investigated the effectiveness of strategies to increase clinicians’ use of validated imaging decision rules. However, results are conflicting possibly due to variations in the target population. For example, one single-arm trial found that educational meetings, reminders, and audit and feedback increased emergency department (ED) clinicians’ (Triage nurses, Emergency nurse practitioners, and Medical staff) use of the Ottawa Ankle Rules [1] and reduced use of ankle X-rays. However, another study found educational meetings and materials about the Ottawa Ankle Rules increased clinicians’ (physician assistants, residents, and attending physicians) use of the rules but did not reduce use of imaging [10].

Due to differing results in the literature and no previous review on this topic, a systematic review is needed to identify strategies that can increase clinicians’ use of validated imaging decision rules for musculoskeletal injuries and improve the appropriateness of imaging. The primary aim of this systematic review was to evaluate the effect of various implementation strategies on clinicians’ use of five validated imaging decision rules (Ottawa Ankle Rules, Ottawa Knee Rule, Canadian C-Spine Rule, National Emergency X-Radiography Utilization Study and Canadian Computed Tomography Head Rule). The secondary aims were to evaluate the effect of implementation strategies on imaging use (indicated, non-indicated and overall) and clinician knowledge.

Methods

We conducted the systematic review in accordance with the “Preferred reporting items for systematic reviews and meta-analyses” (PRISMA) statement [11], and the protocol was registered prospectively on PROSPERO (CRD42020150131). We also followed guidelines from the Effective Practice and Organization of Care (EPOC) group for the conduct of a systematic review of implementation strategies [12].

Search strategy

The following electronic databases were searched from the earliest record until March 11, 2024: MEDLINE (via Ovid), CINAHL (via EBSCO), EMBASE (via Ovid), Cochrane Central Register of Controlled Trials (CENTRAL), Web of Science and Scopus. We consulted a librarian to develop the search strategy and used a combination of keywords related to the five decision rules (Ottawa Ankle Rules, Ottawa Knee Rule, Canadian C-Spine Rule, National Emergency X-Radiography Utilization Study and Canadian Computed Tomography Head Rule) (Supplementary File 1). We also performed citation tracking and hand-searching the reference lists of included studies to identify studies missed by the primary electronic database search.

Two reviewers (PK and JRZ) independently familiarised themselves with the inclusion/exclusion criteria and performed the selection of studies by subsequently screening the title, abstract, and full text of studies retrieved through our electronic database searches. Discrepancies were resolved through discussion or consultation with a third reviewer (CGM).

Inclusion and exclusion criteria

Study design

We included experimental and quasi-experimental study designs (e.g., randomised controlled trials, non-randomised controlled trials) and single-arm trials (i.e. prospective observational studies) of implementation interventions in any care setting. Retrospective and cross-sectional observational studies, case series and case studies were excluded. There were no language or geographic restrictions.

Participants

Participants were healthcare professionals involved in the management of people with musculoskeletal injuries in any care setting (e.g., general practitioners, ED physicians, physiotherapists, ED nurses).

Interventions and comparators

We included studies that investigated the effectiveness of any intervention that aimed to increase clinicians’ use of the Canadian CT Head Rule, Canadian C-spine Rule, NEXUS guidelines, Ottawa Ankle Rules or Ottawa Knee Rule. The EPOC Intervention Taxonomy was used to classify the types of implementation strategies used in each study [13]. Examples of implementation strategies included the distribution of clinical practice guidelines, reminders, interactive educational meetings, audit and feedback, distribution of educational materials, patient-mediated interventions (e.g. any intervention aimed at changing the performance of healthcare professionals through interactions with patients, or information provided by or to patients) [13], and monitoring the performance of the delivery of healthcare. Both single and multi-component interventions were included. No restriction was placed on the comparison intervention (e.g., another implementation strategy, usual care, no intervention).

Outcomes

The primary outcome was clinicians’ use of decision rules. Study investigators could have assessed use of the rules either by clinician self-report (via clinician surveys) or by audits of the clinical notes (documented use of the rules (yes/no) or documented clinical features suggesting use of the rules (yes/no)). Secondary outcomes included use of imaging (e.g., X-ray, CT) as assessed by audits of clinical notes and treatment recording forms, and knowledge of the rules as assessed by surveys. Use of imaging was categorised as indicated, non-indicated and overall. Documentation of clinical features consistent with imaging decision rules was used to determine whether imaging was indicated or not. Table 1 explains the outcomes in more detail.

Table 1 Definitions of variables for data extraction
Full size table

Data extraction and quality assessment

Two reviewers (GW and KR) independently used a standardised form, developed collaboratively by three of the authors: PK, JZ, and CM (Supplementary Table 1) to extract data on country, study design and setting, sample size, participant characteristics, implementation strategy (and comparison), and outcomes. Disagreements were resolved by discussion and re-checking the study report. Four authors were contacted to obtain full text or additional data, but they did not respond.

Two reviewers (GW and KR) independently assessed the risk of bias of included studies using the Cochrane EPOC risk of bias tool [12]. This tool was specifically developed to assess the risk of bias in studies investigating strategies to change the practice of healthcare providers. The reviewers judged a study to be at ‘low-risk’, ‘high-risk’ or ‘unclear risk’ of bias for the following domains: random sequence generation, allocation concealment, baseline outcome measurements similar, baseline characteristics similar, incomplete outcome data, knowledge of the allocated interventions adequately prevented during the study, protection against contamination, selective outcome reporting, and other risk of bias. Judgments were based on how the identified bias would influence the results of the study. Disagreements in ratings were resolved by a third reviewer (PK).

Data analysis

Due to the heterogeneity of interventions and outcome measures, findings were not pooled across studies. Instead, a narrative synthesis of published results was performed. We did not apply the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) [15] approach to assess the overall quality of evidence and strength of recommendation as we could not provide a summary measure for any intervention effect.

Patient and public involvement

We did not involve patients and members of the public in the design of this study.

Results

Study characteristics

After removing duplicates and screening 3517 titles and abstracts and 99 full-text reports, 22 studies were included (Fig. 1). This included five RCTs, one non-randomised controlled trial, and 16 single-arm trials. Of the included studies, 10 focus on the Ottawa Ankle Rules, five on the Canadian CT Head Rule, three on the Ottawa Knee Rule, three on the NEXUS guidelines and two on the Canadian C-spine Rule. Ten studies reported data on use of rules, 14 on overall imaging use, six on indicated or non-indicated imaging and two on knowledge of the rules. The studies provided data from the following countries: 10 from the USA, six from Canada, two from the UK, two from Australia, one from France and one from Ireland. The study settings included 17 community/tertiary/teaching hospital EDs, three acute care centres and two major trauma centres. The review involved 1,271 clinicians and 35,010 patients. The included types of clinicians were ED physicians in 13 studies, junior doctors in 10 studies, ED nurse practitioners in 6 studies, physician assistants in 4 studies, ED nurses in 4 studies, physiotherapists and trauma team leaders in 1 study each (Table 2). Some studies included multiple clinician types. Detailed characteristics of the included studies is shown in Supplementary Tables 2 and a summary of main findings are shown in Table 3.

Fig. 1
figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram

Full size image
Table 2 Study characteristics summary
Full size table
Table 3 Summary of main findings
Full size table

Risk of bias

Supplementary Table 3 presents the risk of bias scores for each study according to the Cochrane EPOC tool. Three studies were at low risk of bias for all but one domain (random sequence generation [16], protection against contamination [17] and selective outcome reporting [18]). The key findings from our risk of bias assessment are that 16 studies were at high risk of bias for random sequence generation and allocation concealment, 15 for ‘knowledge of allocated interventions,’ and 13 for protection against contamination. On the other hand, 18 studies were at low risk of bias for selective outcome reporting, 13 for protection against contamination, 12 for ‘other bias,’ and 8 for ‘baseline characteristics similar.’ The biases identified could potentially affect the validity and reliability of the study results. The table also shows high variability across studies and highlights areas that require caution when interpreting the findings.

Use of rules (primary outcome)

Ottawa Ankle Rules

One RCT found pop-up reminders in electronic medical records increased medical doctors’ and physician assistants’ use of the Ottawa Ankle Rules compared to no intervention (93% vs. 62%, p = 0.02) [19]. Another RCT compared the effectiveness of educational materials related to Ottawa Ankle Rules combined with educational meetings against no intervention and the use of rules in the intervention group was reported as 93% [16]. The study, however, was missing data on the control group’s use of rules (Table 4).

Table 4 Summary of results from included studies
Full size table

One single-arm trial found pop-up reminders in electronic medical records and reminder posters increased use of the Ottawa Ankle Rules (pre-to-post intervention: 56–96%, p < 0.001) among ED physicians [20]. One single-arm trial found educational meetings combined with educational materials increased use of the Ottawa Ankle Rules both before triage (pre-to-post intervention: 3–7%, p < 0.001) and after triage (20–83%, p < 0.001) among triage nurses, other nursing staff, residents, physician assistants, nurse practitioners, and physicians [10]. Another single-arm trial testing the same implementation strategy plus patient-mediated interventions reported that 95% of ED physicians used the Ottawa Ankle Rules during the post-intervention phase (no pre-intervention data was reported) [21]. One single-arm trial found educational meetings combined local opinion leaders, local consensus processes and reminder posters increased ED clinicians’ use of the Ottawa Ankle Rules in both tertiary (request forms: 41–83%, p < 0.001; case notes: 58–95%, p < 0.001) and community hospitals (request forms: 34–65%, p < 0.001; case notes: 52–81%, p < 0.001) [1].

Ottawa Knee Rule

One single-arm trial found pop-up reminders in electronic medical records and reminder posters had unclear effects on the use of the Ottawa Knee Rule (76–93%, p-value not reported) [22] among ED junior doctors. However, another single-arm trial found educational meetings combined with reminder posters increased use of the Ottawa Knee Rule (36–61%, p = 0.05) among ED non-consultant hospital doctors [23] (Table 4).

NEXUS

A single-arm trial had an unclear effect of distribution of NEXUS guidelines among paediatric ED physicians (85–80%, p-value not reported) [24] (Table 4).

Canadian CT Head Rule

An RCT compared the effectiveness of educational materials related to the Canadian CT Head Rule combined with pop-up reminders in electronic medical records and clinical practice guidelines against no intervention [25]. Following the intervention, the use of rules in the intervention group was reported as 78%. However, the study lacked data on the control group’s use of rules (Table 4). A single-arm trial found educational meetings combined with reminder posters and clinical practice guidelines related to the Canadian CT Head Rule reported a significant increase in the use of the rules by clinicians in the emergency department (pre-to-post intervention: 64.6–74.3%, p = 0.01) [26].

Indicated and non-indicated imaging (secondary outcome)

Ottawa Ankle Rules

One RCT comparing educational meetings combined with educational materials related to the Ottawa Ankle Rules to no intervention found the indicated radiographs to be 98.5% and non-indicated radiographs to be 20.5% in the intervention group [16]. This study, however, was missing control group data for both indicated and non-indicated radiographs.

Ottawa Knee Rule

A single-arm trial found that use of reminder posters of Ottawa Knee rule had an unclear effect on use of indicated knee radiography (86–97%, p-value not provided) however, decreased use of non-indicated knee radiography (33–11%, p = 0.016) among ED junior doctors’ [22]. A single-arm trial studied the effect of educational meetings combined with reminders on Ottawa Knee rule use and found unclear effects on indicated imaging (91–93%, p-value not provided) and decreased non-indicated imaging (59–43%, p = 0.05) among the non-consultant hospital doctors [23] (Table 4).

NEXUS

One single-arm trial found that distribution of clinical practice guidelines related to NEXUS decreased paediatric ED physicians’ use of indicated imaging (pre-to-post intervention: 61–38%, p = 0.01) and however, had an unclear effect on non-indicated (18–15%, p-value not provided) cervical spine CT scans [24]. Another single-arm trial combining educational meetings, materials, and reminders had unclear effects on indicated cervical imaging (55–78%, p-value not provided) but decreased non-indicated imaging (45–22%, p = 0.002) [27]. Another single-arm trial of ED clinicians showed that interactive educational meetings combined with audit and feedback, continuous quality improvement and reminder posters had unclear effects on indicated radiographs (84–87%, p-value not reported) and no effect on non-indicated imaging (16–13%, p = 0.2) [28].

Canadian C-spine Rule

The same single-arm trial of ED clinicians involving the above multicomponent interventions about the Canadian C-spine Rule had unclear effects on both indicated (71–78%) and non-indicated imaging (29–23%) [28].

Overall imaging use (secondary outcome)

Ottawa Ankle Rules

One RCT found that pop-up reminders about the Ottawa Ankle Rules in electronic medical records increased medical doctors’ and physician assistants’ use of radiography for ankle injuries compared to no intervention (64% vs. 49%, p < 0.01) [19]. Another RCT found educational meetings combined with educational materials related to the Ottawa Ankle Rules reduced ED physicians’ use of ankle radiography (pre-to-post intervention: 98% vs. 76%, p = 0.03, relative reduction = 22.4%, 95% confidence interval [CI], 19.8-24.9%) whereas there was no decrease in the no intervention control group (99% vs. 99%, relative increase = 0.5%, 95% CI: 0-1.4%, p-value not reported) [16]. A quasi-experimental controlled trial found a combination of three interventions at different time points (e.g., educational materials and meetings initially, then guidelines – Ottawa Ankle Rules, then audit and feedback) did not affect ED physicians’ use of ankle radiography compared to no intervention (92% vs. 93%, p = 0.54) [29].

One single-arm trial found that reminders about the Ottawa Ankle Rules did not affect clinicians’ use of ankle/foot radiography (ankle - pre vs. post: 78% vs. 77%, p = 0.839; foot − 50% vs. 46%, p = 0.352; ankle and foot − 97% vs. 95%, p = 0.379) [20] (Table 4). Five single-arm trials studied the effect of educational meetings combined with other interventions [10] such as educational materials [10, 30,31,32] local consensus processes [1], reminders [1] and patient-mediated interventions [21] on clinicians’ use of ankle imaging. Some studies found a decrease in clinicians’ (ranging from 9% [1] to 22% [21]) use of ankle radiography whereas, one study did not find a significant change in ankle radiography referrals (73% v. 78%, p = 0.11) [31].

Ottawa Knee Rule

A single-arm trial found that reminders about the Ottawa Knee rule did not affect clinicians’ use of knee radiography (59% vs. 56%, p = 0.726) [22] (Table 4). Two other single-arm trials found that educational meetings when combined with reminders [23], audit and feedback [33] and educational materials [33] related to Ottawa Knee Rule found a reduction in non-consultant hospital doctors’ (67% vs. 64%, p = 0.05) [23] use of knee radiography whereas there was an unclear effect on ED physicians’ (92% vs. 71%, p-value not provided) [33] use of knee radiography.

NEXUS

One single-arm trial found that the distribution of NEXUS guidelines decreased paediatric ED physicians’ use of cervical spine CT scans (pre-to-post intervention: 43–29%, p = 0.01) [24] (Table 4).

Canadian C-spine Rule

A single-arm trial investigated the impact of combining educational meetings, and reminders with clinical practice guidelines on the Canadian C-Spine Rule [34] and observed a non-statistical reduction in c-spine imaging (pre vs. post: 67.3% vs. 50.4%, p = 0.16) [34].

Canadian CT Head Rule

An RCT found clinical practice guidelines combined with educational meetings, educational materials and pop-up reminders about the Canadian CT Head rule in electronic medical records increased ED physicians’ use of CT imaging for head injuries in both the intervention (63% vs. 76%, difference = 13.3%, 95% CI: 9.7-17%) and the control groups (68% vs. 74%, difference = 6.7%, 95% CI: 2.6%-0.8%) but the between-group difference was not significant (p = 0.16) [25]. Another RCT that combined reminders about the Canadian CT Head Rule with audit and feedback and continuous quality improvement showed no decrease in the use of head CT among clinicians, in both the intervention (pre vs. post use: 41.5% vs. 39.8%, p = 0.31) and the control group (42.9% vs. 42.7%, p-value not provided) [17] when compared with no intervention. A single-arm trial found that a combination of educational meetings and reminders with clinical practice guidelines on Canadian CT Head Rule had unclear effects on head CT (83.4% to 73.4, p-value not provided) [35].

Mean knowledge scores (secondary outcome)

Ottawa Ankle Rules

One RCT found the distribution of the Ottawa Ankle Rules using a mnemonic and the standard description of the rules was not superior to a standard description of the rules alone for increasing residents’ and medical students’ knowledge of the ankle component (mean = 10.9 vs. 10.2 on a 0–13 scale, difference = 0.7, 95% CI: -0.3-1.7, p = 0.16) and foot component of the rules (7.6 vs. 7.5, difference = 0.1, 95% CI: -0.7-0.9, p = 0.80) at three weeks post-intervention [18]. At 5–9 months, there was no effect on knowledge of the ankle component (10.1 vs. 8.9; difference = 1.18; 95% CI: 0.57–1.81, p = 0.039) but an increase in mean knowledge of the foot component of the rules (6.5 vs. 7.8; mean difference = 1.32; 95% CI: 0.78–1.87, p = 0.004) [18] (Table 4).

Canadian CT Head Rule

One single-arm trial found interactive educational meetings and materials of the Canadian CT Head rule had unclear effects on clinicians’ knowledge score (scale range 0-100%) of the rule (mean of 49% to mean of 89%, no p-value reported) [36]. Clinicians included nurse practitioners, physician assistants, attending physicians, postgraduate students and registered nurses.

Discussion

Results from the four RCTs that provided appropriate data for some outcomes in this review are conflicting. One trial suggested reminders may increase medical doctors’ and physician assistants’ use of the Ottawa Ankle Rules but could also lead to an increase in ankle imaging [19]. Another trial suggested educational meetings and materials on Ottawa Ankle Rules may decrease ED physicians’ use of imaging for ankle injuries [16]. Regarding the Canadian CT Head Rule, the effectiveness of reminders when combined with audit and feedback, did not significantly reduce head CT use among clinicians [17]. However, another trial found that educational meetings and materials on the Canadian CT Head rule, when combined with the distribution of clinical practice guidelines and reminders, may increase ED physicians’ use of CT imaging for head injuries [25]. Unfortunately, none of these RCTs provided relevant data on use of indicated and non-indicated imaging.

For knowledge of the rules, one RCT found clinical practice guidelines did not improve clinicians’ short-term knowledge of the Ottawa Ankle Rules but may increase long-term knowledge [18]. Results from 16 single-arm trials suggested that multi-component interventions may increase clinicians’ use of decision rules and knowledge but their impact on imaging use (overall, indicated and non-indicated) is mixed.

Strengths and weaknesses of the study

This review has several strengths including a comprehensive search strategy across multiple databases to identify eligible studies, a large sample size (n = 1271 clinicians and n = 35,010 patients) and several methodological steps performed in duplicate (e.g., selection of studies, data extraction, risk of bias assessment) to improve validity and accuracy. There are also some limitations. All the included studies were conducted in developed countries (USA, Canada, UK, Australia, France and Ireland), which may limit the generalisability of the findings to developing countries. We were unable to perform a meta-analysis due to heterogeneity in interventions and outcomes. Most studies were at high risk of bias as they did not have a control group, did not use randomisation, did not blind participants which may lead to Hawthorne effect, and did not report data appropriately. These limitations may introduce bias, affecting result reliability. Hence, care should be taken when interpreting findings.

Meaning of the study

The effectiveness of implementation strategies varied across studies, particularly when certain strategies were combined with others. For example, while educational meetings and materials related to the Ottawa Ankle Rules seem to decrease the use of imaging (overall) for ankle injuries [16], the combination of educational meetings, materials, clinical practice guidelines, and reminders regarding the Canadian CT Head rule increased the utilisation of CT (overall) for head injuries [25]. Variations in the apparent effectiveness of some implementation strategies could be due to differences in the acceptability of rules among clinicians. Our recent systematic review (34 studies) explored awareness and use of the same five validated imaging decision rules among clinicians (Canadian CT Head Rule, Canadian C-spine Rule, NEXUS guidelines, Ottawa Ankle Rule, and Ottawa Knee Rule) [14]. We found varying levels of usage for different rules and that clinicians’ attitudes towards these rules may contribute to the differences in their utilisation. For example, some clinicians easily accept some of the rules, while others don’t use the rules or don’t plan on using them in the future.

Variation in the effectiveness of some implementation strategies could also be attributed to barriers to implementation experienced by clinicians, which our previous review provided insights on [14]. Our review highlighted that a substantial number of clinicians lack awareness of validated imaging decision rules with percentages ranging from 31 to 99% across different regions and rules. The review also found that even among those who are aware of these rules, there appears to be a gap in their implementation. Some of the most commonly reported barriers to using imaging decision rules included lack of research to support their use (64%) [37], the complexity of the rules (63%) [38], lack of time at triage to use the rules or EDs being too busy (39%), and heavy workload making it difficult to apply the rules (37%) [39]. This suggests there is a need to identify and tailor implementation strategies to address these barriers to achieve the desired change in clinicians’ use of decision rules [40].

Categorising barriers into factors related to individual clinicians, social context, and organisation could be one way to guide the development of implementation strategies that increase clinicians’ adoption of decision rules [1]. Grimshaw (2001) summarised 41 systematic reviews focusing on professional education and quality assurance interventions to improve quality of care. They found quantitative data suggesting that using a variety of interventions targeting multiple barriers to change is more likely to result in behaviour change compared to relying on a single intervention [41]. However, it should also be noted that although multi-component interventions may be more comprehensive and have the potential to address multiple barriers to implementation, it is important to consider the potential costs and resources required for implementing multi-component interventions [42].

Comparison to existing research

Reminders were seen to decrease the use of non-indicated imaging for ankle and knee injuries when used as a single-component intervention [22, 23] or in combination with educational meetings [22, 23]. These findings appear to be consistent with the effectiveness of reminders in the broader literature. For example, an overview of 41 systematic reviews that aimed to synthesise interventions to improve the quality of care provided by clinicians found that reminders were more effective than other interventions in changing clinicians’ behaviour [41]. Reminders prompt adherence to clinical guidelines by providing simple, timely information aimed at improving professional practices and patient outcomes [43], and may be particularly useful for busy clinicians who treat patients with a range of conditions (e.g. general practitioners, ED physicians).

A study among paediatric patients showed a non-significant decrease in ED physicians’ use of NEXUS guidelines after the distribution of clinical practice guidelines [24]. However, there was also a notable reduction in neck CT scans, representing a positive outcome. The findings align with the existing literature. For example, a systematic review of 4 studies involving 4502 paediatric patients highlighted the low sensitivity, reliability, and clinical acceptability of the NEXUS criteria in pediatric trauma patients [44]. Furthermore, uncertainties in evaluating mental status in children under 3 years old [45] might also contribute to the observed reduction in NEXUS guideline adherence among paediatric ED physicians.

Our review demonstrated that the effectiveness of educational interventions for reducing clinicians’ use of imaging (overall) may depend on the specific context and type of imaging. One RCT found that educational meetings and educational materials related to Ottawa Ankle Rules may decrease clinicians’ use of ankle radiography (overall), whereas, another RCT found that the addition of reminders of Canadian CT Head rule use and clinical practice guidelines to these interventions may increase ED physicians’ use of CT (overall) for head injuries [25]. This appears to be consistent with the broader literature. For example, a systematic review (n = 11 RCTs) evaluated the effectiveness of knowledge translation interventions in enhancing the adoption and implementation of clinical practice guidelines for musculoskeletal conditions [46]. The review found that there were inconsistent effects of the interventions on professional practice (such as a change in practice or behaviour, knowledge, skills, and self-efficacy) [46]. The study found that while educational meetings had a positive effect in enhancing the appropriate use of diagnostic imaging for spinal disorders, combining interactive educational meetings with local opinion leaders did not significantly impact physiotherapists’ clinical practice for non-specific low back pain. Another systematic review (n = 5 studies) studied the effect of multi-component interventions such as interactive educational meetings (which included interactive sessions, practical sessions, problem-solving, feedback, and reminders) on clinicians’ implementation of certain guideline recommendations from low back pain and whiplash [47]. The study highlighted the importance of considering the quality and quantity of strategies when implementing any physiotherapy guidelines, as studies with higher guideline adherence at baseline, or those that used multiple educational meetings achieved greater adherence compared to others.

Unanswered questions and future research

While we found some evidence on the effectiveness of implementation strategies for increasing clinicians’ use of imaging decision rules, there is still a need for high-quality RCTs in this area. For example, of the 22 included studies, only five were RCTs and only one of these reported appropriate data for our primary outcome. Similarly, of the four RCTs that reported data on imaging use, only one reported on whether the imaging was indicated or not, but the study did not report data from the control group, making it hard to draw conclusions about the effectiveness of implementation strategies for these outcomes. Another area for future research is to compare different combinations of implementation strategies to see if using multiple strategies together is more effective than using a single strategy. Additionally, there should be a focus on understanding whether different types of imaging decision rules require different implementation strategies for effective uptake. This could be investigated by exploring the effectiveness of different types of multi-component interventions such as a combination of reminders, educational meetings, and educational materials, tailored to the specific needs of different healthcare professionals, as well as evaluating the sustainability of implementing these strategies over time.

Conclusion

Reminders whether as a single-component intervention or as part of a multi-component approach when combined with educational meetings have demonstrated effectiveness in increasing clinicians’ use of the Ottawa Ankle Rules and reducing ankle imaging. Pop-up reminders, meetings, and posters improve adherence to the Ottawa Knee Rule and Canadian CT Head Rule. However, the dissemination of the NEXUS guidelines slightly reduced the use of the rule but notably also reduced the use of neck CT among paediatric ED physicians representing a positive outcome. These varying effects of interventions highlight the need for future research to compare different combinations of implementation strategies as this will help explore whether specific imaging decision rules benefit from tailored implementation approaches. A lack of appropriately conducted and reported RCTs makes it difficult to draw firm conclusions.

Availability of data and materials

All data generated or analysed during this study are included in this published article (and its supplementary information files).

References

  1. Bessen T, Clark R, Shakib S, Hughes G. A multifaceted strategy for implementation of the Ottawa ankle rules in two emergency departments. BMJ. 2009;339:b3056.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Graham ID, Stiell IG, Laupacis A, O’Connor AM, Wells GA. Emergency physicians’ attitudes toward and use of clinical decision rules for radiography. Acad Emerg Med. 1998;5(2):134–40.

    Article  CAS  PubMed  Google Scholar 

  3. Beckenkamp PR, Lin C-WC, Macaskill P, Michaleff ZA, Maher CG, Moseley AM. Diagnostic accuracy of the Ottawa ankle and midfoot rules: a systematic review with meta-analysis. Br J Sports Med. 2017;51(6):504–10.

    Article  PubMed  Google Scholar 

  4. Bachmann LM, Haberzeth S, Steurer J, ter Riet G. The accuracy of the Ottawa knee rule to rule out knee fractures - a systematic review. Ann Intern Med. 2004;140(2):121–4.

    Article  PubMed  Google Scholar 

  5. Michaleff ZA, Maher CG, Verhagen A, Rebbeck T, Lin CWC. Accuracy of the Canadian C-spine rule and NEXUS to screen for clinically important cervical spine injury in patients following blunt trauma: a systematic review. Can Med Association J (CMAJ). 2012;184(16):E867-876.

    Article  Google Scholar 

  6. Harnan SE, Pickering A, Pandor A, Goodacre SW. Clinical decision rules for adults with minor head injury: a systematic review. J Trauma. 2011;71(1):245–51.

    PubMed  Google Scholar 

  7. Stiell IG, Wells GA, Vandemheen KL, Clement CM, Lesiuk H, De Maio VJ, et al. The Canadian C-spine rule for radiography in alert and stable trauma patients. JAMA. 2001;286(15):1841–8.

    Article  CAS  PubMed  Google Scholar 

  8. Graham ID, Stiell IG, Laupacis A, McAuley L, Howell M, Clancy M, et al. Awareness and use of the Ottawa ankle and knee rules in 5 countries: can publication alone be enough to change practice? Ann Emerg Med. 2001;37(3):259–66.

    Article  CAS  PubMed  Google Scholar 

  9. Stiell IG, Clement CM, Grimshaw J, Brison RJ, Rowe BH, Schull MJ, et al. Implementation of the Canadian C-Spine rule: prospective 12 centre cluster randomised trial. BMJ. 2009;339(7729):1071.

    Google Scholar 

  10. Ashurst JV, Nappe T, Digiambattista S, Kambhampati A, Alam S, Ortiz M, et al. Effect of triage-based use of the Ottawa foot and ankle rules on the number of orders for radiographic imaging. J Am Osteopath Assoc. 2014;114(12):890–7.

    Article  PubMed  Google Scholar 

  11. Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Cochrane Effective Practice and Organisation of Care. EPOC resources for review authors. 2017. Available from: https://epoc.cochrane.org/resources/epoc-resources-review-authors. Cited 2023 Feb 3.

  13. Cochrane Effective Practice and Organisation of Care. EPOC Taxonomy. 2015. Available from: https://epoc.cochrane.org/epoc-taxonomy. Cited 2019 Feb 20.

  14. Kharel P, Zadro JR, Chen Z, Himbury MA, Traeger AC, Linklater J, Maher CG. Awareness and use of five imaging decision rules for musculoskeletal injuries: a systematic review under review. 2023.

  15. Brozek JL, Akl EA, Alonso-Coello P, Lang D, Jaeschke R, Williams JW, et al. Grading quality of evidence and strength of recommendations in clinical practice guidelines: part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions. Allergy (Copenhagen). 2009;64(5):669–77.

    CAS  Google Scholar 

  16. Auleley GR, Ravaud P, Giraudeau B, Kerboull L, Nizard R, Massin P, et al. Implementation of the Ottawa ankle rules in France - A multicenter randomized controlled trial. JAMA. 1997;277(24):1935–9.

    Article  CAS  PubMed  Google Scholar 

  17. Andruchow JE, Grigat D, McRae AD, Innes G, Vatanpour S, Wang D, et al. Decision support for computed tomography in the emergency department: a multicenter cluster-randomized controlled trial. CJEM. 2021;23(5):631–40.

    Article  PubMed  Google Scholar 

  18. Gravel J, Roy M, Carriere B. 44-55-66-PM, a mnemonic that improves retention of the Ottawa ankle and foot rules: a randomized controlled trial. Acad Emerg Med. 2010;17(8):859–64.

    Article  PubMed  Google Scholar 

  19. Tajmir S, Raja AS, Ip IK, Andruchow J, Silveira P, Smith S, Khorasani R. Impact of clinical decision support on radiography for acute ankle injuries: a randomized trial. Western J Emerg Med. 2017;18(3):487–95.

    Article  Google Scholar 

  20. Silveira PC, Ip IK, Sumption S, Raja AS, Tajmir S, Khorasani R. Impact of a clinical decision support tool on adherence to the Ottawa Ankle rules. Am J Emerg Med. 2016;34(3):412–8.

    Article  PubMed  Google Scholar 

  21. Stiell I, Wells G, Laupacis A, Brison R, Verbeek R, Vandemheen K, Naylor CD. Multicenter trial to introduce the Ottawa Ankle rules for use of radiography in acute ankle injuries. BMJ. 1995;311(7005):594–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Atkinson P, Boyle A, Chisholm E. X-ray requesting patterns before and after introduction of the Ottawa knee rules in a UK emergency department. Eur J Emerg Med. 2004;11(4):204–7.

    Article  PubMed  Google Scholar 

  23. O’Sullivan MJ, O’Sullivan I. Application of the Ottawa knee rules in assessing acute knee injuries. Ir Med J. 2006;99(7):199–200.

    PubMed  Google Scholar 

  24. Rostas J, Cason B, Simmons J, Frotan MA, Brevard SB, Gonzalez RP. Implementation of pediatric cervical spine clearance guidelines at a combined trauma center: twelve-month impact. J Trauma Acute Care Surg. 2015;78(6):1095–100.

    Article  PubMed  Google Scholar 

  25. Stiell IG, Clement CM, Grimshaw JM, Brison RJ, Rowe BH, Lee JS, et al. A prospective cluster-randomized trial to implement the Canadian CT head rule in emergency departments. Can Med Assoc J. 2010;182(14):1527–32.

    Article  Google Scholar 

  26. Helms R. Improving the management of adults with mild traumatic brain Injury: an Initiative to reduce unnecessary computed tomographic scans in the emergency department. Adv Emerg Nurs J. 2023;45(4):327–40.

    Article  PubMed  Google Scholar 

  27. Baker M, Jaeger C, Hafley C, Waymack J. Appropriate CT cervical spine utilisation in the emergency department. BMJ Open Qual. 2020;9(4):e000844.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Griffith B, Vallee P, Krupp S, Jung M, Slezak M, Nagarwala J, et al. Screening cervical spine CT in the emergency department, phase 3: increasing effectiveness of imaging. J Am Coll Radiol. 2014;11(2):139–44.

    Article  PubMed  Google Scholar 

  29. Holroyd BR, Wilson D, Rowe BH, Mayes DC, Noseworthy T. Uptake of validated clinical practice guidelines: experience with implementing the Ottawa Ankle rules. Am J Emerg Med. 2004;22(3):149–55.

    Article  PubMed  Google Scholar 

  30. Gwilym SE, Aslam N, Ribbans WJ, Holloway V. The impact of implementing the Ottawa ankle rules on ankle radiography requests in A&E. Int J Clin Pract. 2003;57(7):625–7.

    Article  CAS  PubMed  Google Scholar 

  31. Cameron C, Naylor CD. No impact from active dissemination of the Ottawa Ankle rules: further evidence of the need for local implementation of practice guidelines. Can Med Assoc J. 1999;160(8):1165–8.

    CAS  Google Scholar 

  32. Stiell IG, Greenberg GH, Wells GA, McKnight RD, Cwinn AA, Cacciotti T, et al. Derivation of a decision rule for the use of radiography in Acute knee injuries. Ann Emerg Med. 1995;26(4):405–13.

    Article  CAS  PubMed  Google Scholar 

  33. Wigder HN, Ballis SFC, Lazar L, Urgo R, Dunn BH. Successful implementation of a guideline by peer comparisons, education, and positive physician feedback. J Emerg Med. 1999;17(5):807–10.

    Article  CAS  PubMed  Google Scholar 

  34. Kerr D, Bradshaw L, Kelly AM. Implementation of the Canadian C-spine rule reduces cervical spine x-ray rate for alert patients with potential neck injury. J Emerg Med. 2005;28(2):127–31.

    Article  PubMed  Google Scholar 

  35. Kim JS, Carlucci M. Reducing unnecessary head computed tomographic scans in an adult emergency department. J Nurse Practitioners. 2021;17(6):749–52.

    Article  Google Scholar 

  36. Zakhari R, Sterrett SE. Attitudes toward evidence-based clinical decision support tools to reduce exposure to ionizing radiation: the Canadian CT head rule. J Am Assoc Nurs Pract. 2016;28(12):659–67.

    Article  Google Scholar 

  37. Burns EC, Yanchar NL. Using cervical spine clearance guidelines in a pediatric population: a survey of physician practices and opinions. Can J Emerg Med. 2011;13(1):1–6.

    Article  Google Scholar 

  38. Brehaut JC, Stiell IG, Graham ID. Will a new clinical decision rule be widely used? The case of the Canadian C-Spine rule. Acad Emerg Med. 2006;13(4):413–20.

    PubMed  Google Scholar 

  39. Clement CM, Stiell IG, Davies B, O’Connor A, Brehaut JC, Sheehan P, et al. Perceived facilitators and barriers to clinical clearance of the cervical spine by emergency department nurses: a major step towards changing practice in the emergency department. Int Emerg Nurs. 2011;19(1):44–52.

    Article  PubMed  Google Scholar 

  40. Kharel P, Zadro JR, Sundaram CS, McCaffery K, Dodd RH, McLennan C, Maher CG. Physiotherapists’ attitudes, views, and beliefs about choosing wisely recommendations: a qualitative study. Musculoskelet Sci Pract. 2022;61:102610.

    Article  PubMed  Google Scholar 

  41. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39(8):II2-45.

    CAS  PubMed  Google Scholar 

  42. Grimshaw J, Thomas R, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess (Winchester Eng). 2004;8(6):iii–iv.

    CAS  Google Scholar 

  43. Pantoja T, Grimshaw JM, Colomer N, Castañon C, Leniz Martelli J. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019;12(12):Cd001174.

    PubMed  Google Scholar 

  44. Ekhator C, Nwankwo I, Nicol A. Implementation of national emergency X-Radiography utilization study (NEXUS) criteria in Pediatrics: a systematic review. Cureus. 2022;14(10):e30065.

    PubMed  PubMed Central  Google Scholar 

  45. Pannu GS, Shah MP, Herman MJ. Cervical spine clearance in pediatric trauma centers: the need for standardization and an evidence-based protocol. J Pediatr Orthop. 2017;37(3):e145-149.

    Article  PubMed  Google Scholar 

  46. Al Zoubi FM, Menon A, Mayo NE, Bussières AE. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18(1):435.

    Article  PubMed  PubMed Central  Google Scholar 

  47. van der Wees PJ, Jamtvedt G, Rebbeck T, de Bie RA, Dekker J, Hendriks EJM. Multifaceted strategies may increase implementation of physiotherapy clinical guidelines: a systematic review. Aust J Physiother. 2008;54(4):233–41.

    Article  PubMed  Google Scholar 

Download references

Funding

None received.

Author information

Authors and Affiliations

  1. The University of Sydney, Sydney Musculoskeletal Health, Sydney, NSW, Australia, PO Box M179, Priti Kharel - Level 10 North, King George V Building, Royal Prince Alfred Hospital, Missenden Road, 2050

    Priti Kharel, Joshua R. Zadro, Adrian Traeger & Christopher G. Maher

  2. Sydney School of Public Health, The University of Sydney, Sydney, NSW, Australia

    Grace Wong & Kittirut Rojanabenjawong

  3. Castlereagh Imaging, Sydney, NSW, Australia

    James Linklater

Authors
  1. Priti Kharel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joshua R. Zadro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Grace Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kittirut Rojanabenjawong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adrian Traeger
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. James Linklater
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Christopher G. Maher
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PK and JRZ contributed to the study design, data collection, data analysis, data interpretation, and manuscript writing. GW and KR assisted in data extraction and quality assessment. CGM provided supervision and contributed to manuscript writing. The final manuscript was approved by AT and JL, and all authors reviewed and agreed to it.

Corresponding author

Correspondence to Priti Kharel.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kharel, P., Zadro, J.R., Wong, G. et al. Effectiveness of implementation strategies for increasing clinicians’ use of five validated imaging decision rules for musculoskeletal injuries: a systematic review. BMC Emerg Med 24, 84 (2024). https://doi.org/10.1186/s12873-024-00996-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12873-024-00996-x

Keywords

BMC Emergency Medicine

ISSN: 1471-227X

Contact us