<?xml version='1.0'?>
<!DOCTYPE art SYSTEM 'http://www.biomedcentral.com/xml/article.dtd'>
<art>
   <ui>1472-6963-8-75</ui>
   <ji>1472-6963</ji>
   <fm>
      <dochead>Research article</dochead>
      <bibl>
         <title>
            <p>Interventions designed to improve the quality and efficiency of medication use in managed care: A critical review of the literature &#8211; 2001&#8211;2007</p>
         </title>
         <aug>
            <au id="A1">
               <snm>Lu</snm>
               <mi>Y</mi>
               <fnm>Christine</fnm>
               <insr iid="I1"/>
               <email>Christine_lu@hms.harvard.edu</email>
            </au>
            <au id="A2">
               <snm>Ross-Degnan</snm>
               <fnm>Dennis</fnm>
               <insr iid="I1"/>
               <email>Dennis_Ross-Degnan@hms.harvard.edu</email>
            </au>
            <au id="A3">
               <snm>Soumerai</snm>
               <mi>B</mi>
               <fnm>Stephen</fnm>
               <insr iid="I1"/>
               <email>Stephen_Soumerai@hms.harvard.edu</email>
            </au>
            <au id="A4" ca="yes">
               <snm>Pearson</snm>
               <fnm>Sallie-Anne</fnm>
               <insr iid="I2"/>
               <email>sallie.pearson@unsw.edu.au</email>
            </au>
         </aug>
         <insg>
            <ins id="I1">
               <p>Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, MA USA</p>
            </ins>
            <ins id="I2">
               <p>Faculty of Medicine, University of New South Wales and Clinical School, Prince of Wales Hospital, Sydney, Australia</p>
            </ins>
         </insg>
         <source>BMC Health Services Research</source>
         <issn>1472-6963</issn>
         <pubdate>2008</pubdate>
         <volume>8</volume>
         <issue>1</issue>
         <fpage>75</fpage>
         <url>http://www.biomedcentral.com/1472-6963/8/75</url>
         <xrefbib>
            <pubidlist>
               <pubid idtype="pmpid">18394200</pubid>
               <pubid idtype="doi">10.1186/1472-6963-8-75</pubid>
            </pubidlist>
         </xrefbib>
      </bibl>
      <history>
         <rec>
            <date>
               <day>20</day>
               <month>9</month>
               <year>2007</year>
            </date>
         </rec>
         <acc>
            <date>
               <day>07</day>
               <month>4</month>
               <year>2008</year>
            </date>
         </acc>
         <pub>
            <date>
               <day>07</day>
               <month>4</month>
               <year>2008</year>
            </date>
         </pub>
      </history>
      <cpyrt>
         <year>2008</year>
         <collab>Lu et al; licensee BioMed Central Ltd.</collab>
         <note>This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<url>http://creativecommons.org/licenses/by/2.0</url>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</note>
      </cpyrt>
      <abs>
         <sec>
            <st>
               <p>Abstract</p>
            </st>
            <sec>
               <st>
                  <p>Background</p>
               </st>
               <p>Managed care organizations use a variety of strategies to reduce the cost and improve the quality of medication use. The effectiveness of such policies is not well understood. The objective of this research was to update a previous systematic review of interventions, published between 1966 and 2001, to improve the quality and efficiency of medication use in the US managed care setting.</p>
            </sec>
            <sec>
               <st>
                  <p>Methods</p>
               </st>
               <p>We searched MEDLINE and EMBASE for publications from July 2001 to January 2007 describing interventions targeting drug use conducted in the US managed care setting. We categorized studies by intervention type and adequacy of research design using commonly accepted criteria. We summarized the outcomes of well-controlled strategies and documented the significance and magnitude of effects for key study outcomes.</p>
            </sec>
            <sec>
               <st>
                  <p>Results</p>
               </st>
               <p>We identified 164 papers published during the six-year period. Predominant strategies were: educational interventions (n = 20, including dissemination of educational materials, and group or one-to-one educational outreach); monitoring and feedback (n = 22, including audit/feedback and computerized monitoring); formulary interventions (n = 66, including tiered formulary and patient copayment); collaborative care involving pharmacists (n = 15); and disease management with pharmacotherapy as a primary focus (n = 41, including care for depression, asthma, and peptic ulcer disease). Overall, 51 studies met minimum criteria for methodological adequacy. Effective interventions included one-to-one academic detailing, computerized alerts and reminders, pharmacist-led collaborative care, and multifaceted disease management. Further, changes in formulary tier-design and related increases in copayments were associated with reductions in medication use and increased out-of-pocket spending by patients. The dissemination of educational materials alone had little or no impact, while the impact of group education was inconclusive.</p>
            </sec>
            <sec>
               <st>
                  <p>Conclusion</p>
               </st>
               <p>There is good evidence for the effectiveness of several strategies in changing drug use in the managed care environment. However, little is known about the cost-effectiveness of these interventions. Computerized alerts showed promise in improving short-term outcomes but little is known about longer-term outcomes. Few well-designed, published studies have assessed the potential negative clinical effects of formulary-related interventions despite their widespread use. However, some evidence suggests increases in cost sharing reduce access to essential medicines for chronic illness.</p>
            </sec>
         </sec>
      </abs>
   </fm>
   <meta>
      <classifications>
         <classification type="bmc" subtype="user_supplied_xml" id="refman"/>
      </classifications>
   </meta>
   <bdy>
      <sec>
         <st>
            <p>Background</p>
         </st>
         <p>Managed care organizations (MCOs) are the predominant form of health insurance coverage in the United States. MCOs provide health care to over 160 million enrollees and almost 90% of physicians have at least one managed care contract <abbrgrp><abbr bid="B1">1</abbr></abbrgrp>. By enrollment numbers in 2005, preferred provider organizations (PPOs) are the most common form of managed care (61%) followed by health maintenance organizations (HMOs; 21%), and point-of-service and conventional plans cover a small portion of the insured (15% and 3%, respectively) <abbrgrp><abbr bid="B1">1</abbr></abbrgrp>.</p>
         <p>Healthcare payers, including MCOs, grapple with the challenge of providing access to essential care that improves health outcomes in the face of increasing need to control healthcare costs. Growth in pharmaceutical spending over the past decade is partly due to increased ingredient costs per prescription, higher levels of utilization, and changes in the drugs being prescribed <abbrgrp><abbr bid="B2">2</abbr></abbrgrp>. A variety of strategies have been used by MCOs to contain escalating drug expenditures and to improve the quality of medication use.</p>
         <p>Drug formularies are a common cost containment strategy used by healthcare payers. Formularies are lists of preferred pharmaceutical products covered by an institution within various therapeutic categories <abbrgrp><abbr bid="B3">3</abbr></abbrgrp>. Health plans continue to switch from 1-tier plans (same copayment for all medications under coverage) and 2-tier plans (a lower copayment for generic drugs and a higher copayment for brand-name drugs) to 3-tier plans that include a third, higher copayment(s) for non-preferred brand-name medications. Such 3-tier plans are now the dominant managed care formulary structure <abbrgrp><abbr bid="B4">4</abbr></abbrgrp>. Since 2004, a number of plans have created a fourth tier of cost-sharing for specific types of drugs such as lifestyle medications and biologics <abbrgrp><abbr bid="B5">5</abbr></abbrgrp>. Patient copayments for prescription drugs are another strategy which aims to sensitize patients to the costs of medications so as to discourage use of non-essential medications. Copayments in health plans have risen substantially over the past decade <abbrgrp><abbr bid="B5">5</abbr></abbrgrp>. From 2000 to 2006, the average copayment for generic drugs increased 57% (from $7 to $11), while copayments for preferred brand-name drugs increased 85% (from $13 to $24) and copayments for non-preferred brand-name drugs increased 123% (from $17 to $38) <abbrgrp><abbr bid="B5">5</abbr></abbrgrp>. In addition, MCOs attempt to influence medicines use and improve quality of care through educational programs, prescribing feedback, and computer-based information system <abbrgrp><abbr bid="B6">6</abbr></abbrgrp>. Disease management programs are also used to improve care delivery and health outcomes for patients with specific chronic illnesses <abbrgrp><abbr bid="B6">6</abbr></abbrgrp>.</p>
         <p>The validity and reliability of information about the effectiveness and unintended consequences of drug-related interventions depends heavily on the strength of the research design <abbrgrp><abbr bid="B7">7</abbr><abbr bid="B8">8</abbr></abbrgrp>. By critically appraising studies, systematic reviews can provide useful and more objective information to clinicians, patients, and policy makers. In our previous systematic review of studies evaluating drug-related interventions in MCOs published prior to June 2001 <abbrgrp><abbr bid="B6">6</abbr></abbrgrp>, we found a number of consistently effective interventions. These included participatory clinical guideline development, one-to-one and group educational outreach, and enhanced patient-specific feedback. Dissemination of educational materials alone and aggregate feedback to clinicians about their patients were shown to be ineffective. These findings are consistent with other reviews evaluating interventions outside the managed care setting <abbrgrp><abbr bid="B7">7</abbr><abbr bid="B9">9</abbr><abbr bid="B10">10</abbr></abbrgrp>. We also found that dissemination of educational materials with drug samples &#8211; a technique used widely by the pharmaceutical industry to influence prescribing and patient demand &#8211; was effective in changing medication use, while disease management showed promise in improving short-term outcomes in the management of diabetes and depression <abbrgrp><abbr bid="B6">6</abbr></abbrgrp>. Importantly, methodologically acceptable studies of the impact of formulary changes and financial strategies as drug cost-containment policies were too few to be included in our previous review. This represented a significant gap in knowledge because such policies are widely used to control pharmaceutical expenditures and it is likely they have negative as well as positive impacts on clinical outcomes. At that time there was also a paucity of evidence of the effectiveness of interventions in more 'lightly' managed care.</p>
         <p>The purpose of this review is to update our previous systematic review of interventions to improve medication use in MCOs <abbrgrp><abbr bid="B6">6</abbr></abbrgrp>. Specifically, we aim to describe interventions to improve the quality and efficiency of medication use in the US managed care setting published between July 2001 and January 2007; detail key features of methodologically adequate studies; summarize intervention effects; and identify the most successful types of interventions.</p>
      </sec>
      <sec>
         <st>
            <p>Methods</p>
         </st>
         <sec>
            <st>
               <p>Search Strategy</p>
            </st>
            <p>We performed a systematic review of published interventions conducted in MCOs to reduce pharmaceutical costs or improve use of medicines. We searched MEDLINE and EMBASE from 2001 through January 2007 using a combination of search terms describing the study setting (e.g. managed care programs, health maintenance organizations, preferred provider organizations), methodology (e.g. randomized controlled trials, intervention studies, program evaluation, health services research, comparative studies), intervention types (e.g. formulary, education, practice guidelines, cost containment, quality assurance, risk sharing, reimbursement mechanisms), drug use (e.g. prescription and non-prescription drugs, drug therapy, drug utilization, drug monitoring, drug substitution), and clinician practice patterns (e.g. physician, clinicians, provider, practitioner, physician practice patterns, prescribing practice). The reference lists of studies identified in this way were reviewed to identify studies that our search strategy may have missed. The search retrieved all studies published since January 2001 including some of those analyzed in our previous review.</p>
            <p>A total of 2899 articles were identified by the search. Two investigators (CYL, SAP) each independently reviewed half of the study abstracts. Studies were included in the review if they met the following criteria: were published between July 2001 and January 2007 (inclusive), conducted in the US managed care setting including all forms of HMOs, PPOs, and independent practice associations (IPAs); described intervention(s) targeting medication use, including over-the-counter medications, herbals, or vitamins; included a clear description of methods; and measured drug-related outcomes. Disease management interventions focusing on pharmacotherapy were also considered. We have also included in the current review the four methodologically adequate formulary interventions not reported in our previous review. Due to the rapid growth in research on this topic since 2001, there are now sufficient studies to draw more valid conclusions. Clinical effectiveness trials and cost effectiveness studies of medications, descriptive studies, and those examining vaccinations were excluded.</p>
         </sec>
         <sec>
            <st>
               <p>Rating Study Quality</p>
            </st>
            <p>We reviewed the methodological adequacy of studies identified by our search. Based on the ability of specific research designs to control adequately for common threats to internal validity <abbrgrp><abbr bid="B8">8</abbr><abbr bid="B11">11</abbr><abbr bid="B12">12</abbr></abbrgrp>, our definition for methodologically adequate studies included: randomized controlled trials (RCTs); pre-post studies with non-randomized comparison group(s); and interrupted time series analysis with or without comparison group. Interrupted time series designs examine changes in outcomes of interest using multiple observations (at least 4 by our definition) before and after an intervention <abbrgrp><abbr bid="B12">12</abbr></abbrgrp>. Research designs deemed methodologically inadequate were pre-post studies without a comparison group, non-randomized post-only studies, and cross-sectional studies. Such designs have been shown to produce invalid results in previous research <abbrgrp><abbr bid="B7">7</abbr><abbr bid="B8">8</abbr></abbrgrp>. In addition, studies were only included if each comparison group included at least 20 subjects to ensure the reliability of results.</p>
         </sec>
         <sec>
            <st>
               <p>Data extraction</p>
            </st>
            <p>We extracted key features of the characteristics, methods, and outcomes of methodologically adequate studies using a standardized data extraction tool. Key features included: setting, objective/s, health condition/s addressed, intervention components, predominant intervention strategy, intervention target/s, and up to three primary outcome measures reported by the authors. To check reliability of data extraction, two investigators (CYL, SAP) independently extracted data from the included studies and calculated effect sizes of outcome measures for one-third of the studies in this review.</p>
         </sec>
      </sec>
      <sec>
         <st>
            <p>Results</p>
         </st>
         <p>One hundred and fifty one published studies met our initial inclusion criteria, of which 31% met our criteria for minimally acceptable methodology. The majority of these studies (45%) were conducted in staff, group and mixed model settings and 6 studies were conducted in IPAs, network model organizations, and Medicare or Medicaid MCOs; the remaining studies (40%) did not provide detailed information on the type of managed care setting.</p>
         <p>We classified studies according to predominant strategy and key intervention components [Table <tblr tid="T1">1</tblr>]. In order of frequency, predominant strategies were formulary-related interventions (40% of studies), disease management (25%), monitoring and feedback (13%), educational interventions (12%), and collaborative care involving pharmacists (10%).</p>
         <tbl id="T1">
            <title>
               <p>Table 1</p>
            </title>
            <caption>
               <p>Studies identified according to key features and predominant intervention strategy</p>
            </caption>
            <tblbdy cols="7">
               <r>
                  <c>
                     <p/>
                  </c>
                  <c ca="center">
                     <p>
                        <b>Educational</b>
                     </p>
                  </c>
                  <c ca="center">
                     <p>
                        <b>Monitoring &amp; Feedback</b>
                     </p>
                  </c>
                  <c ca="center">
                     <p>
                        <b>Formulary*</b>
                     </p>
                  </c>
                  <c ca="center">
                     <p>
                        <b>Collaborative Care Involving Pharmacists</b>
                     </p>
                  </c>
                  <c ca="center">
                     <p><b>Disease Management</b>&#8224;</p>
                  </c>
                  <c ca="center">
                     <p>
                        <b>TOTAL</b>
                     </p>
                  </c>
               </r>
               <r>
                  <c cspan="7">
                     <hr/>
                  </c>
               </r>
               <r>
                  <c ca="left">
                     <p>
                        <b>All studies</b>
                     </p>
                  </c>
                  <c ca="center">
                     <p>20</p>
                  </c>
                  <c ca="center">
                     <p>22</p>
                  </c>
                  <c ca="center">
                     <p>66</p>
                  </c>
                  <c ca="center">
                     <p>15</p>
                  </c>
                  <c ca="center">
                     <p>41</p>
                  </c>
                  <c ca="center">
                     <p>164</p>
                  </c>
               </r>
               <r>
                  <c ca="left">
                     <p>
                        <b>Methodologically acceptable studies</b>
                     </p>
                  </c>
                  <c ca="center">
                     <p>7</p>
                  </c>
                  <c ca="center">
                     <p>16</p>
                  </c>
                  <c ca="center">
                     <p>15</p>
                  </c>
                  <c ca="center">
                     <p>5</p>
                  </c>
                  <c ca="center">
                     <p>8</p>
                  </c>
                  <c ca="center">
                     <p>51</p>
                  </c>
               </r>
               <r>
                  <c indent="1" ca="left">
                     <p>RCT</p>
                  </c>
                  <c ca="center">
                     <p>5</p>
                  </c>
                  <c ca="center">
                     <p>10</p>
                  </c>
                  <c ca="center">
                     <p>2</p>
                  </c>
                  <c ca="center">
                     <p>3</p>
                  </c>
                  <c ca="center">
                     <p>7</p>
                  </c>
                  <c ca="center">
                     <p>27</p>
                  </c>
               </r>
               <r>
                  <c indent="1" ca="left">
                     <p>Interrupted time series</p>
                  </c>
                  <c ca="center">
                     <p>0</p>
                  </c>
                  <c ca="center">
                     <p>5</p>
                  </c>
                  <c ca="center">
                     <p>5</p>
                  </c>
                  <c ca="center">
                     <p>0</p>
                  </c>
                  <c ca="center">
                     <p>0</p>
                  </c>
                  <c ca="center">
                     <p>10</p>
                  </c>
               </r>
               <r>
                  <c indent="1" ca="left">
                     <p>Pre/post studies with comparison</p>
                  </c>
                  <c ca="center">
                     <p>2</p>
                  </c>
                  <c ca="center">
                     <p>1</p>
                  </c>
                  <c ca="center">
                     <p>8</p>
                  </c>
                  <c ca="center">
                     <p>2</p>
                  </c>
                  <c ca="center">
                     <p>1</p>
                  </c>
                  <c ca="center">
                     <p>14</p>
                  </c>
               </r>
               <r>
                  <c ca="left">
                     <p><b>Interventions in methodologically acceptable studies</b>&#8225;</p>
                  </c>
                  <c ca="center">
                     <p>17</p>
                  </c>
                  <c ca="center">
                     <p>18</p>
                  </c>
                  <c ca="center">
                     <p>21</p>
                  </c>
                  <c ca="center">
                     <p>5</p>
                  </c>
                  <c ca="center">
                     <p>8</p>
                  </c>
                  <c ca="center">
                     <p>69</p>
                  </c>
               </r>
            </tblbdy>
            <tblfn>
               <p>* published studies from 1966 through January 2007.</p>
               <p>&#8224; Only studies with pharmacotherapy as a primary focus.</p>
               <p>&#8225; Studies may have examined more than one intervention and multiple studies may have examined the same intervention</p>
            </tblfn>
         </tbl>
         <p>The primary interventions tested in studies that met criteria for methodological adequacy were: educational interventions (7), monitoring and feedback (16), formulary-related interventions (15 including 4 published prior to July 2001), collaborative care involving pharmacists (5), and disease management (8). We categorized methodologically adequate interventions involving financial incentives as formulary-related interventions because the predominant focus of these studies was to assess the effects of differential patient copayment associated with tiered formularies. Additional file <supplr sid="S1">1</supplr> summarizes the key aspects of the 51 methodologically acceptable studies.</p>
         <suppl id="S1">
            <title>
               <p>Additional file 1</p>
            </title>
            <text>
               <p>Key features of 51 methodologically acceptable studies</p>
            </text>
            <file name="1472-6963-8-75-S1.doc">
               <p>Click here for file</p>
            </file>
         </suppl>
         <sec>
            <st>
               <p>Educational interventions</p>
            </st>
            <sec>
               <st>
                  <p>i. Dissemination of Educational Materials Alone</p>
               </st>
               <p>The dissemination of educational materials alone (e.g. newsletters, clinical guidelines, and audiovisual materials) is used commonly as a behavior change strategy, although this approach has been shown repeatedly to be ineffective as a stand-alone intervention <abbrgrp><abbr bid="B6">6</abbr><abbr bid="B9">9</abbr></abbrgrp>. This strategy was one arm of an RCT conducted by Majumdar et al <abbrgrp><abbr bid="B13">13</abbr></abbrgrp> in the form of a clinical guideline and 'toolkit' (a list of eligible patients and patient educational materials); another arm of this trial evaluated the impact of group education (discussed below). Not surprisingly, the materials only intervention failed to increase the testing and treatment of Helicobacter pylori (HP) infection or to reduce proton pump inhibitor and histamine-2 blocker prescribing.</p>
            </sec>
            <sec>
               <st>
                  <p>ii. One-to-One Education</p>
               </st>
               <p>One-to-one educational outreach (often referred to as academic detailing) has been shown to be an effective technique to change many aspects of physician behavior, particularly medication use <abbrgrp><abbr bid="B14">14</abbr><abbr bid="B15">15</abbr></abbrgrp>. Simon et al <abbrgrp><abbr bid="B16">16</abbr></abbrgrp> demonstrated that one-to-one education sessions delivered by peer leaders increased guideline adherence and the use of diuretics and beta-blockers for hypertension.</p>
               <p>Two studies evaluated the impact of one-to-one patient counseling by pharmacists. One intervention successfully decreased mortality rates in patients using drugs with a narrow therapeutic index; hospitalizations were also reduced by both brief and detailed pharmacist counseling, although the effects were not significant <abbrgrp><abbr bid="B17">17</abbr></abbrgrp>. Results from another study <abbrgrp><abbr bid="B18">18</abbr></abbrgrp> showed that detailed pharmacist counseling on adherence to medications with follow-up telephone calls did not increase the rate of eradication of HP infection over pharmaceutical treatment without counseling, but it did increase patient satisfaction.</p>
            </sec>
            <sec>
               <st>
                  <p>iii. Group Education</p>
               </st>
               <p>Group education relies on either didactic or problem-based approaches to influence behavior <abbrgrp><abbr bid="B7">7</abbr></abbrgrp>. In our review peer leader education in small practice-based groups was shown to increase testing and treatment for HP infection; however, it was unsuccessful in reducing the use of proton pump inhibitors or histamine-2 blockers <abbrgrp><abbr bid="B13">13</abbr></abbrgrp>. Simon et al <abbrgrp><abbr bid="B16">16</abbr></abbrgrp> demonstrated that an intervention involving group academic detailing improved antihypertensive prescribing; individual academic detailing was more successful than group visits in increasing the use of diuretics and beta-blockers as recommended by clinical guidelines. There is some evidence that both group and individual detailing interventions are cost-saving <abbrgrp><abbr bid="B16">16</abbr><abbr bid="B19">19</abbr></abbrgrp>. Another study using group academic detailing by psychiatrists and interactive discussions to improve depression management by primary care physicians demonstrated no significant increase in antidepressant prescribing or treatment duration at 12 months post-intervention <abbrgrp><abbr bid="B20">20</abbr></abbrgrp>. A significant reduction in antibiotic prescription rates was achieved over a 3-year period by using a multifaceted intervention comprised of peer leader education sessions to practitioners, provision of mailed educational materials (based on materials from the Centers for Disease Control and Prevention) to parents of young children, and dissemination of key educational messages via radio, newspaper, television and pamphlets in hospitals, clinics, and pharmacies directed at the general public <abbrgrp><abbr bid="B21">21</abbr></abbrgrp>.</p>
            </sec>
         </sec>
         <sec>
            <st>
               <p>Monitoring and feedback</p>
            </st>
            <sec>
               <st>
                  <p>i. Prescribing audit and feedback</p>
               </st>
               <p>Audit and feedback interventions report physicians' past or current clinical practice, usually in relation to the practice of peers or accepted standards. Audit and feedback can achieve small to moderate improvements in physicians' practice <abbrgrp><abbr bid="B22">22</abbr><abbr bid="B23">23</abbr></abbrgrp>. Four studies in this review used prescriber feedback combined with educational materials to change medication use. Providing clinical recommendations to physicians with annual mailed peer-comparison feedback over a four-year period achieved small increases in the use of statins, beta-blockers, and angiotensin-converting enzyme inhibitors among patients with coronary heart disease <abbrgrp><abbr bid="B24">24</abbr></abbrgrp>. Simon et al <abbrgrp><abbr bid="B25">25</abbr></abbrgrp> demonstrated that internet-based feedback to residents on their prescribing practices complemented by educational materials did not increase the proportion of diabetic patients receiving glycemic monitoring or change antihypertensive prescribing. However, residents were required to actively access a password-protected personalized website to be exposed to the intervention <abbrgrp><abbr bid="B25">25</abbr></abbrgrp>.</p>
               <p>Two studies targeted physicians and patients using educational materials and feedback. The first achieved a small reduction in antibiotic prescribing for acute respiratory tract infections by delivering practice guidelines and performance feedback to physicians, combined with mailing educational materials to patient households and placing information in waiting and examination rooms <abbrgrp><abbr bid="B26">26</abbr></abbrgrp>. In the second study, a significant increase in antidepressant adherence was achieved by providing physicians with a list of non-adherent patients, guidelines, and feedback on their prescribing, along with mailing educational information to patients initiating therapy and reminder letters to non-adherent patients <abbrgrp><abbr bid="B27">27</abbr></abbrgrp>.</p>
            </sec>
            <sec>
               <st>
                  <p>ii. Computerized real-time alerts</p>
               </st>
               <p>The common types of information technology used in clinical care include electronic medical records, computerized physician order entry, and real-time decision support (such as reminders and prompts). Computerized order entry with clinical decision support has been demonstrated to improve prescribing and reduce medication error rates <abbrgrp><abbr bid="B28">28</abbr></abbrgrp>. Smith et al <abbrgrp><abbr bid="B29">29</abbr></abbrgrp> used a computerized provider order entry system to provide prescribing alerts regarding medicines with potential contraindications and therapeutic alternatives. They found a reduction in dispensing rates of potentially contraindicated medicines (including tricyclic-antidepressants) among the elderly (aged > 65 years). Another intervention of electronic reminders to providers successfully increased the proportion of female patients receiving osteoporosis medications or bone mineral density measurements; total calcium intake, as a measure for quality of care, also increased <abbrgrp><abbr bid="B30">30</abbr></abbrgrp>. The addition of patient reminder letters did not increase the impact of alerts alone on the target outcomes <abbrgrp><abbr bid="B30">30</abbr></abbrgrp>.</p>
               <p>Two studies in this review combined computerized alerts and group academic detailing <abbrgrp><abbr bid="B31">31</abbr><abbr bid="B32">32</abbr></abbrgrp>. Interrupted time series analyses demonstrated that computer-based drug-specific and patient age-specific alerts were effective in reducing prescribing of medications that interact with warfarin <abbrgrp><abbr bid="B31">31</abbr></abbrgrp> and the use of medications contraindicated in the elderly <abbrgrp><abbr bid="B32">32</abbr></abbrgrp>. In both studies, group education failed to achieve any additional change in medication use over the alerts alone.</p>
            </sec>
            <sec>
               <st>
                  <p>iii. Reminders and telephone outreach</p>
               </st>
               <p>Bambauer et al <abbrgrp><abbr bid="B33">33</abbr></abbrgrp> evaluated the impact of alerting prescribers via faxed letters about patients who had gaps of more than 10 days in refilling anti-depressant prescriptions during the first six months of therapy. The faxed alerts to prescribers had no discernable effect on the proportion of non-adherent patients or the number of days without antidepressant treatment during the 12-month follow-up period.</p>
               <p>The effectiveness of computerized reminders has also been demonstrated to improve rates of recommended laboratory drug monitoring. In two studies by Raebel et al <abbrgrp><abbr bid="B34">34</abbr><abbr bid="B35">35</abbr></abbrgrp> daily computerized alerts identifying incomplete laboratory tests for a range of drugs (such as amiodarone, carbamazepine, and statins) were sent to pharmacists who telephoned patients about the missing tests. This intervention successfully increased the completion of laboratory monitoring both at the initiation of and during the course of drug therapy <abbrgrp><abbr bid="B34">34</abbr><abbr bid="B35">35</abbr></abbrgrp>. In contrast, neither non-intrusive computerized alerts to physicians, nor presenting information on computer screens without requiring additional actions, were ineffective in changing laboratory monitoring rates at the initiation of drug therapy <abbrgrp><abbr bid="B36">36</abbr></abbrgrp>.</p>
               <p>A study by Feldstein et al <abbrgrp><abbr bid="B37">37</abbr></abbrgrp> compared three interventions to increase laboratory test completion for drugs prescribed frequently in primary care. Electronic reminders to providers about missing laboratory tests, with accompanying sample letters for distribution to patients, were successful in increasing the proportion of patients completing tests; however, the impact was greater when patients were contacted directly. Two other interventions &#8211; reminding patients about incomplete tests via automated telephone voice message or telephone calls by nurses &#8211; were both more effective than patient-specific alerts to providers in increasing laboratory test completion <abbrgrp><abbr bid="B37">37</abbr></abbrgrp>.</p>
               <p>Telephone outreach to patients by pharmacists was evaluated in three papers. In one study <abbrgrp><abbr bid="B38">38</abbr><abbr bid="B39">39</abbr></abbrgrp> three monthly telephone calls by pharmacists to monitor patients taking antidepressants increased medication adherence and improved depression symptoms, and not surprisingly, increased patient feedback to pharmacists about medication use <abbrgrp><abbr bid="B38">38</abbr><abbr bid="B39">39</abbr></abbrgrp>. However, in the management of asthma, three outreach telephone calls about five months apart with tailored feedback to patients did not change medication or healthcare utilization <abbrgrp><abbr bid="B40">40</abbr></abbrgrp>.</p>
            </sec>
         </sec>
         <sec>
            <st>
               <p>Formulary interventions</p>
            </st>
            <sec>
               <st>
                  <p>i. Tiered formularies</p>
               </st>
               <p>Tiered formularies with different levels of patient copayment to encourage use of preferred drugs are used increasingly to enhance cost-effective use of medicines <abbrgrp><abbr bid="B41">41</abbr></abbrgrp>. Six papers in this review evaluated the impact of tiered formulary changes.</p>
               <p>Changing from a single copayment to a three-tier structure reduced continuation of the original medications, increased switching to agents in a lower tier, discontinued use of all drugs in the affected class in some patients, and decreased plan spending while increasing enrollee spending <abbrgrp><abbr bid="B42">42</abbr><abbr bid="B43">43</abbr></abbrgrp>. Similar but smaller effects were observed when changing from a two-tier to a three-tier formulary <abbrgrp><abbr bid="B42">42</abbr></abbrgrp>. These changes in the pattern of medication use were seen in a number of prescription drug classes examined, including angiotensin-converting enzyme inhibitors, statins, proton pump inhibitors, and medications for attention-deficit hyperactivity disorder <abbrgrp><abbr bid="B42">42</abbr><abbr bid="B43">43</abbr></abbrgrp>. A change from two to three tiers decreased claims for third tier medications and net insurer costs, while patient out-of-pocket copayments increased significantly in both the first year <abbrgrp><abbr bid="B44">44</abbr></abbrgrp> and second year <abbrgrp><abbr bid="B45">45</abbr></abbrgrp> of follow-up. In a study by Nair et al <abbrgrp><abbr bid="B46">46</abbr></abbrgrp> change from a two-tier to a three-tier design increased compliance of prescribers to preferred medications and increased overall medication use (primarily due to increases in use of medications in the two lower tiers).</p>
               <p>Patient cost-sharing (either in the form of fixed, percent, or tiered copayments) is an important factor influencing medication use. Roblin et al <abbrgrp><abbr bid="B47">47</abbr></abbrgrp> showed that large (> $10 per 30-day supply) and moderate ($7&#8211;$10) increases in cost-sharing attenuated an increasing trend of oral hypoglycemic medication use by 18.5% and 9.2% respectively over a 6-month period, while small ($1&#8211;$6) increases in copayments had no impact.</p>
            </sec>
            <sec>
               <st>
                  <p>ii. Prior authorization</p>
               </st>
               <p>Prior authorization (PA) policies are commonly used to control drug utilization and cost <abbrgrp><abbr bid="B48">48</abbr></abbrgrp>. In a health plan that instituted a PA policy for COX-2 inhibitors, Hartung et al <abbrgrp><abbr bid="B49">49</abbr></abbrgrp> showed a significant reduction in COX-2 inhibitor use as well as a small decrease in use of gastrointestinal protectants (e.g. proton pump inhibitors and histamine-2 blockers) at one year post-intervention. Another study demonstrated a reduction in pharmacy utilization but an increase in medical utilization (particularly hospitalization and emergency department visits) after a COX-2 inhibitor PA policy was implemented <abbrgrp><abbr bid="B50">50</abbr></abbrgrp>.</p>
            </sec>
            <sec>
               <st>
                  <p>iii. Formulary/coverage change</p>
               </st>
               <p>After vaginal anti-fungal products became available over-the-counter, Gurwitz et al <abbrgrp><abbr bid="B51">51</abbr></abbrgrp> showed a significant reduction in anti-fungal prescriptions. Studies by McDonough et al <abbrgrp><abbr bid="B52">52</abbr></abbrgrp> and Andrade et al <abbrgrp><abbr bid="B53">53</abbr></abbrgrp> both evaluated the impact of formulary switches between agents in the same therapeutic class and found increases in the prescribing of the preferred agents (lisinopril and esterified estrogen tablets, respectively). After loratidine became available as an over-the-counter medication, Sullivan et al <abbrgrp><abbr bid="B54">54</abbr></abbrgrp> found that prescriptions decreased for both antihistamines and the total category of allergic rhinitis medications (including inhaled corticosteroids and leukotriene receptor antagonists) with a substantial reduction in pharmacy costs for these medications. Antihistamine prescriptions were further decreased by switching antihistamines to the third formulary tier or by imposing a PA policy for these medications <abbrgrp><abbr bid="B54">54</abbr></abbrgrp>.</p>
               <p>Two studies by Delate et al <abbrgrp><abbr bid="B55">55</abbr><abbr bid="B56">56</abbr></abbrgrp> evaluated the effects of a formulary change with communication to physicians and/or patients informing them of the change. The first study evaluated the impact of notifying patients via mail of impending formulary changes <abbrgrp><abbr bid="B56">56</abbr></abbrgrp> and showed an increase in subsequent formulary adherence. The second study used letters to notify physicians and patients of formulary changes and showed a small increase in conversion of inefficient dosage regimens to a recommended daily dosage of the same medication <abbrgrp><abbr bid="B55">55</abbr></abbrgrp>. Similarly, DeZearn et al <abbrgrp><abbr bid="B57">57</abbr></abbrgrp> evaluated the effect of formulary change with letters to prescribers encouraging the use of generic cimetidine and found a reduction in prescriptions for the brand name product and an increase in the prescribing of generic cimetidine.</p>
            </sec>
         </sec>
         <sec>
            <st>
               <p>Collaborative care involving pharmacists</p>
            </st>
            <p>Collaboration of pharmacists in the care process has emerged as a promising approach to improve quality of care and patient outcomes <abbrgrp><abbr bid="B58">58</abbr></abbrgrp>. Five studies in this review that evaluated collaborative care interventions supported evidence of effectiveness.</p>
            <p>Finley et al <abbrgrp><abbr bid="B59">59</abbr></abbrgrp> showed increases in antidepressant adherence and small reductions in resource utilization when pharmacists were used to titrate antidepressant doses, under the supervision of psychiatrists. Okamoto et al <abbrgrp><abbr bid="B60">60</abbr></abbrgrp> demonstrated reduction in blood pressure and improvement in patient quality of life (as measured by SF-36) without increasing total costs per patient when pharmacists were responsible for educating patients, altering therapy by administering more appropriate or less expensive drugs to achieve similar or improved blood pressure control, and ordering laboratory tests as required. Another study showed a significant increase in the proportion of patients receiving screening and a reduction in total cholesterol and low-density lipoprotein cholesterol levels when pharmacists were responsible for monitoring and titrating medications, and monitoring laboratory testing <abbrgrp><abbr bid="B61">61</abbr></abbrgrp>. Straka et al <abbrgrp><abbr bid="B62">62</abbr></abbrgrp> evaluated the impact of collaboration between pharmacists and providers in developing patient-specific care plans which were followed by pharmacists monitoring and educating patients. The intervention was successful in reducing low-density lipoprotein levels over the short-term and the effects persisted 18 months after discontinuation of the intervention; the proportion of patients achieving the low-density lipoprotein cholesterol goal also increased significantly <abbrgrp><abbr bid="B62">62</abbr></abbrgrp>. Borenstein et al <abbrgrp><abbr bid="B63">63</abbr></abbrgrp> evaluated the impact of a multifaceted program involving individual and group education sessions for physicians run by pharmacists, provision of a list of patients with uncontrolled hypertension, individualized patient education, monitoring by pharmacists, and hypertension clinics to assess blood pressure and adherence to antihypertensive medications. Treatment decisions by physicians considered assessment results and blood pressure measures provided by pharmacists. This intervention effectively reduced systolic blood pressure and increased the proportion of patients achieving blood pressure goal level <abbrgrp><abbr bid="B63">63</abbr></abbrgrp>. A non-significant increase in total provider visits (physician and pharmacist visits) was also observed <abbrgrp><abbr bid="B63">63</abbr></abbrgrp>.</p>
         </sec>
         <sec>
            <st>
               <p>Disease management interventions</p>
            </st>
            <p>Disease management programs are structured, population-based approaches for identifying, treating, and monitoring persons with chronic illness. Generally, these are multidisciplinary, integrated approaches aimed to improve processes of care and patient outcomes <abbrgrp><abbr bid="B64">64</abbr></abbrgrp>. Seven papers describing disease management interventions that had pharmacotherapy as a primary focus were included in this review.</p>
            <sec>
               <st>
                  <p>i. Depression</p>
               </st>
               <p>Depression is a common medical condition that is associated with significant social and functional impairment as well as high direct and indirect health care costs <abbrgrp><abbr bid="B65">65</abbr></abbrgrp>. Research suggests depression can cause greater functional disability than other chronic conditions such as diabetes, chronic lung disease, and hypertension <abbrgrp><abbr bid="B66">66</abbr></abbrgrp>.</p>
               <p>In one study, psychiatrists reviewed patient psychosocial history and current depressive episodes over multiple sessions. The intervention, which also included books and videotapes for patient education and pharmacologic treatment where necessary, effectively increased antidepressant adherence at 12 months with modest improvements in depression symptoms among severely depressed patients <abbrgrp><abbr bid="B67">67</abbr></abbrgrp>. Katon et al <abbrgrp><abbr bid="B68">68</abbr></abbrgrp> evaluated an individualized stepped-care depression treatment program provided by specialist nurses working collaboratively with primary care physicians for diabetic patients diagnosed with depression. The intervention increased the use of adequate antidepressant doses and drug adherence at 12 months; there were also modest improvements in depression symptoms and no change in diabetes outcomes <abbrgrp><abbr bid="B68">68</abbr></abbrgrp>. Unutzer et al <abbrgrp><abbr bid="B69">69</abbr></abbrgrp> evaluated a quality improvement intervention targeting treatment of depression with follow-up by specialist nurses. The intervention increased the proportion of patients using antidepressants and antidepressant use according to guideline-recommended doses among patients at high risk for relapse. In the same study, an intervention with a focus on quality of psychotherapy rather than medication use was less successful than the intervention targeting improved medication therapy <abbrgrp><abbr bid="B69">69</abbr></abbrgrp>.</p>
               <p>Ray et al <abbrgrp><abbr bid="B70">70</abbr></abbrgrp> examined the impact of transitioning patients from traditional Medicare and MCOs to specialty behavioral health organizations ('carve-out' programs). They found a significant reduction in the proportion of patients who continued antipsychotic therapy; this negative effect was greatest in high-risk patients.</p>
            </sec>
            <sec>
               <st>
                  <p>ii. Asthma</p>
               </st>
               <p>Lozano et al <abbrgrp><abbr bid="B71">71</abbr></abbrgrp> evaluated the impact of two interventions: physician peer leader education within practice groups and peer leader education coupled with nurse care planning, telephone monitoring and follow-up of medication use, asthma symptoms and support for self-management. Both interventions effectively reduced asthma symptoms and decreased the use of oral steroids; the planned care intervention (multifaceted) had a greater effect on these target outcomes than the group education intervention <abbrgrp><abbr bid="B71">71</abbr></abbrgrp>. However, at the practice-level, the two intervention approaches did not differ in changes in the proportion of patients using controller medications or oral steroids, or in frequency of medical visits <abbrgrp><abbr bid="B72">72</abbr></abbrgrp>.</p>
            </sec>
            <sec>
               <st>
                  <p>iii. Helicobacter pylori infection</p>
               </st>
               <p>Published guidelines recommend testing for HP in patients receiving long-term acid suppression therapy for acid-related disorders, and antibiotic treatment for those with HP-positive peptic ulcer disease <abbrgrp><abbr bid="B73">73</abbr><abbr bid="B74">74</abbr></abbrgrp>. A test and treat intervention for HP infection in patients receiving long-term acid-reducing therapy successfully improved gastrointestinal symptoms and reduced the use of acid-reducing medications and associated costs at 12 months <abbrgrp><abbr bid="B75">75</abbr></abbrgrp>. Ofman et al <abbrgrp><abbr bid="B76">76</abbr></abbrgrp> evaluated another test-and-treat program for HP infection provided collaboratively by physicians, pharmacists and nurses. This multifaceted intervention involved local development and implementation of test-and-treat guidelines, individual and group academic detailing, patient education by pharmacists, and HP testing and medication monitoring by nurses. The intervention successfully increased compliance with HP testing and the proportion of patients prescribed appropriate anti-HP therapy at six months with no significant change in total costs of related health services <abbrgrp><abbr bid="B76">76</abbr></abbrgrp>.</p>
            </sec>
         </sec>
      </sec>
      <sec>
         <st>
            <p>Discussion</p>
         </st>
         <p>There has been a substantial increase in published research evaluating interventions targeting medication use in managed care settings since our previous review <abbrgrp><abbr bid="B6">6</abbr></abbrgrp>. This review identified 151 intervention studies in a six-year publication period using a narrower inclusion criteria (that is, disease management studies only included where pharmacotherapy was a primary focus) compared with 105 studies over 35 years in our previous review (70 of which were reported since 1996). Despite the increase in research on this topic, a smaller proportion of published studies met our criteria for methodological adequacy (31% versus 46%). This failure to increase the degree of methodological rigor in published research is of concern because significant resources are used conducting studies of poor methodological quality and such research may produce scientifically invalid conclusions that influence future research and policy. Although not a focus of this review, we noted additional methodological issues apart from research design. For example, the proxy measures of adherence varied between studies (n = 10). Three studies used the medication possession ratios <abbrgrp><abbr bid="B20">20</abbr><abbr bid="B27">27</abbr><abbr bid="B59">59</abbr></abbrgrp>, while others measured the proportion of patients continuing treatment <abbrgrp><abbr bid="B33">33</abbr><abbr bid="B38">38</abbr><abbr bid="B42">42</abbr><abbr bid="B43">43</abbr><abbr bid="B67">67</abbr><abbr bid="B68">68</abbr><abbr bid="B70">70</abbr></abbrgrp>. The variability in measures of adherence and infrequent reporting of this important outcome in part reflects the well-known methodological challenges in the measurement of adherence <abbrgrp><abbr bid="B77">77</abbr></abbrgrp>.</p>
         <p>Several findings from this report are consistent with our previous review and the results of other systematic reviews on the impact of interventions designed to change medication use. We once again confirmed that dissemination of educational materials alone is ineffective and one-to-one educational outreach visits are effective in increasing adherence to prescribing guidelines <abbrgrp><abbr bid="B6">6</abbr><abbr bid="B9">9</abbr><abbr bid="B10">10</abbr><abbr bid="B15">15</abbr><abbr bid="B78">78</abbr></abbrgrp>. Further, this review confirms that multifaceted interventions are more likely to be successful in changing medication use than those using single strategies <abbrgrp><abbr bid="B6">6</abbr><abbr bid="B10">10</abbr><abbr bid="B78">78</abbr></abbrgrp>.</p>
         <p>The current analysis, like other reviews, suggests group education interventions may be beneficial but the findings are inconclusive <abbrgrp><abbr bid="B78">78</abbr><abbr bid="B79">79</abbr></abbrgrp>. We found that peer leader group education was effective in changing medication use but was less effective than one-to-one academic detailing <abbrgrp><abbr bid="B16">16</abbr><abbr bid="B31">31</abbr><abbr bid="B32">32</abbr></abbrgrp>. The absence of incremental impact of group education when combined with computer-based alerts, as reported by Simon et al <abbrgrp><abbr bid="B32">32</abbr></abbrgrp> and Feldstein et al <abbrgrp><abbr bid="B31">31</abbr></abbrgrp>, may reflect the weakness of the approach or the superior effectiveness of computerized alerts in changing the particular outcomes targeted in these interventions.</p>
         <p>Intervention approaches involving monitoring and feedback have been shown previously to produce small to moderate effects on medication use <abbrgrp><abbr bid="B6">6</abbr><abbr bid="B23">23</abbr><abbr bid="B80">80</abbr></abbrgrp>, and this review confirms these findings. Further, our review also supports the existing literature regarding the short-term effectiveness of real-time computer-based alerts in changing prescribing and test ordering <abbrgrp><abbr bid="B28">28</abbr><abbr bid="B81">81</abbr></abbrgrp>. However, most studies in our review evaluated electronic medical record systems with particular characteristics in a single HMO, thus limiting the generalizability of findings. Only one study <abbrgrp><abbr bid="B30">30</abbr></abbrgrp> reported patient-relevant outcomes (namely, quality of life measures).</p>
         <p>Our review provides evidence that tiered formulary and patient copayment interventions decrease non-preferred drug use, reduce overall insurer costs, and increase patient out-of-pocket expenses as intended. The use of prior authorization policies resulted in similar effects. However, our findings concur with previous reviews in managed care and other settings that these interventions may also be associated with undesirable effects such as increased rates of switching to other medications or discontinuation of essential and cost-effective medications <abbrgrp><abbr bid="B3">3</abbr><abbr bid="B48">48</abbr><abbr bid="B82">82</abbr><abbr bid="B83">83</abbr></abbrgrp>. In addition, some studies reported changes in measures such as hospitalization or medical resource utilization (e.g. office visits) resulting from formulary-related interventions <abbrgrp><abbr bid="B44">44</abbr><abbr bid="B45">45</abbr><abbr bid="B49">49</abbr><abbr bid="B50">50</abbr><abbr bid="B70">70</abbr></abbrgrp>.</p>
         <p>Evidence from the literature suggests that coordinating pharmacist services as a component of patient care improves quality of care <abbrgrp><abbr bid="B84">84</abbr><abbr bid="B85">85</abbr><abbr bid="B86">86</abbr></abbrgrp>. This review documents positive results for collaborative care led by pharmacists, including improved drug adherence and clinical outcomes. Only some data on costs and resource use (e.g. cost per mmHg of blood pressure decrease, average provider visit cost per patient) have been reported <abbrgrp><abbr bid="B59">59</abbr><abbr bid="B60">60</abbr><abbr bid="B63">63</abbr></abbrgrp>.</p>
         <p>Finally, our findings confirm that disease management interventions are associated with improvements in the process and quality of care, both short and longer term <abbrgrp><abbr bid="B87">87</abbr></abbrgrp>. However, these disease management interventions are by their nature multifaceted, and the type and number of features within each disease management program vary widely. Therefore, it is difficult to draw firm conclusions about the contribution of individual components to the overall impact.</p>
         <p>There are several limitations to this review which need to be acknowledged. Despite our intensive efforts to identify all of the published literature on interventions to improve the quality and efficiency of medication use in US managed care settings, our search strategy may have missed published articles. The full range of interventions currently used by managed care programs to influence use of medicines is not covered by the studies in our review. For example, no studies evaluated the impact of physician incentives on prescribing, despite that fact that this strategy is commonly used by MCOs. Further, our findings are likely to be subject to under-reporting bias with interventions showing no or negative effects, which are less likely to be reported in the peer-reviewed literature. Even when studies are published, many are methodologically inadequate; 69% of the studies identified in this review were not included in the detailed analyses because of design flaws. There are also fundamental difficulties in comparing interventions with diverse objectives, intervention targets, measurement methods, and outcomes. Finally, it is difficult to reach definitive conclusions about the contribution of individual components to the overall effect of multifaceted interventions.</p>
         <sec>
            <st>
               <p>Future Directions</p>
            </st>
            <p>Despite the substantial number of interventions to improve drug use in managed care, our understanding of the impacts of these interventions is still limited.</p>
            <p>First, a large proportion (40%) of the reviewed studies did not detail the specifics of the managed care setting within which the study was conducted. We found few studies conducted in PPOs despite their prominent role in the US managed care industry or in other "lightly" managed care settings. The majority of studies in our review were conducted in staff, group, or mixed-model HMOs, which represent more heavily managed settings. Future research should establish whether intervention approaches that are successful in such settings are also effective in more lightly managed settings. Second, most studies in this review used common surrogate outcome measures, such as prescribing rates or proportions of patients achieving specific medication use practices. Few studies reported medication adherence or patient-relevant outcomes such as clinical status, hospitalization rates, or quality of life measures. Furthermore, evidence about the cost-effectiveness of most interventions is still quite limited. Cost-effectiveness data are needed to select among many possible strategies for improving medication use and associated costs. Third, there is still a lack of publicly available, high quality evidence concerning the effects of interventions involving formulary changes or financial incentives, which are commonly used to influence medication use. Although the increased reporting of formulary-related interventions is encouraging, the high prevalence of inadequate designs (e.g. pre-post without comparison group, post-only) among studies of formulary-related interventions (77%) is striking. An important conclusion to be drawn from this review is that improvements in research methodology will be essential in order to produce valid and reliable study results about cost, quality of care, and patient outcomes to inform managed care policy decisions. It is highly likely these interventions do reduce costs to the payer as they continue to be dominant in the managed care environment. Evaluation of the patient-relevant outcomes of formulary-related interventions is a research priority. Finally, the durability of most of the intervention effects reported in this review is uncertain because few studies extend beyond two years follow-up and many cover even shorter periods. In particular, further research is needed to determine if computerized clinical support systems are associated with improved patient outcomes over the long-term.</p>
         </sec>
      </sec>
      <sec>
         <st>
            <p>Conclusion</p>
         </st>
         <p>There is good evidence for the effectiveness of several strategies in changing drug use in the US managed care setting. Policy makers have a solid and expanding knowledge base to employ in the design of educational, monitoring and feedback, and collaborative care interventions to improve medication use. Computerized alerts also show promise in improving short-term outcomes. Despite the wide use of formulary-related approaches to managing pharmaceutical use, there are still few well-designed studies assessing their effects on patient outcomes. In order to maximize health care cost-effectiveness while containing costs, it is crucial to continue to promote rigorous testing of interventions to improve medication use in managed care settings in order to expand available high quality evidence about their clinical and economic impacts.</p>
      </sec>
      <sec>
         <st>
            <p>Competing interests</p>
         </st>
         <p>The author(s) declare that they have no competing interests.</p>
      </sec>
      <sec>
         <st>
            <p>Authors' contributions</p>
         </st>
         <p>Conception and design: CYL, SAP, DRD, SBS. Administrative support: CYL, SAP. Collection and assembly of data: CYL, SAP. Data analysis and interpretation: CYL, SAP. Manuscript writing: CYL, SAP, DRD, SBS. Final approval of manuscript: CYL, SAP, DRD, SBS.</p>
      </sec>
   </bdy>
   <bm>
      <ack>
         <sec>
            <st>
               <p>Acknowledgements</p>
            </st>
            <p>This study was supported by the HMO Research Network Center for Education and Research on Therapeutics (Agency for Healthcare and Research &amp; Quality grant number: 1U18 HS/CA1039-01) and the Harvard Pilgrim Health Care Foundation.</p>
         </sec>
      </ack>
      <refgrp>
         <bibl id="B1">
            <title>
               <p>Trends and Indicators in the changing health care marketplace</p>
            </title>
            <aug>
               <au>
                  <cnm>Health Care Marketplace Project</cnm>
               </au>
            </aug>
            <publisher>Washington, DC, Henry J. Kaiser Family Foundation</publisher>
            <pubdate>2005</pubdate>
            <url>http://www.kff.org/insurance/7031/index.cfm</url>
         </bibl>
         <bibl id="B2">
            <title>
               <p>Prescription drug trends</p>
            </title>
            <aug>
               <au>
                  <cnm>Kaiser Family Foundation</cnm>
               </au>
            </aug>
            <publisher>Washington, DC, Henry J. Kaiser Family Foundation</publisher>
            <pubdate>2007</pubdate>
            <url>http://www.kff.org/rxdrugs/upload/3057_06.pdf</url>
         </bibl>
         <bibl id="B3">
            <title>
               <p>Effects of restrictive formularies in the ambulatory care setting</p>
            </title>
            <aug>
               <au>
                  <snm>Lexchin</snm>
                  <fnm>J</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care</source>
            <pubdate>2002</pubdate>
            <volume>8</volume>
            <fpage>69</fpage>
            <lpage>76</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">11814174</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B4">
            <title>
               <p>The effects of prescription drug cost sharing: a review of the evidence</p>
            </title>
            <aug>
               <au>
                  <snm>Gibson</snm>
                  <fnm>TB</fnm>
               </au>
               <au>
                  <snm>Ozminkowski</snm>
                  <fnm>RJ</fnm>
               </au>
               <au>
                  <snm>Goetzel</snm>
                  <fnm>RZ</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care 11:730-40,</source>
            <pubdate>2005</pubdate>
         </bibl>
         <bibl id="B5">
            <title>
               <p>Employer Health Benefits: 2006 Annual Survey</p>
            </title>
            <aug>
               <au>
                  <cnm>Kaiser Family Foundation and the Health Research and Education Trust</cnm>
               </au>
            </aug>
            <publisher>Washington, DC, Henry J. Kaiser Family Foundation</publisher>
            <pubdate>2006</pubdate>
            <url>http://www.kff.org/insurance/7527/index.cfm</url>
         </bibl>
         <bibl id="B6">
            <title>
               <p>Changing medication use in managed care: a critical review of the available evidence.</p>
            </title>
            <aug>
               <au>
                  <snm>Pearson</snm>
                  <fnm>SA</fnm>
               </au>
               <au>
                  <snm>Ross-Degnan</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Payson</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care</source>
            <pubdate>2003</pubdate>
            <volume>9</volume>
            <fpage>715</fpage>
            <lpage>731</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">14626470</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B7">
            <title>
               <p>Improving drug prescribing in primary care: A critical analysis of the experimental literature</p>
            </title>
            <aug>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
               <au>
                  <snm>McLaughlin</snm>
                  <fnm>TS</fnm>
               </au>
               <au>
                  <snm>Avorn</snm>
                  <fnm>J</fnm>
               </au>
            </aug>
            <source>Milbank Q</source>
            <pubdate>1989</pubdate>
            <volume>67</volume>
            <fpage>268</fpage>
            <lpage>317</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.2307/3350142</pubid>
                  <pubid idtype="pmpid">2698446</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B8">
            <title>
               <p>A critical analysis of studies of state drug reimbursement policies: research in need of discipline</p>
            </title>
            <aug>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
               <au>
                  <snm>Ross-Degnan</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Fortess</snm>
                  <fnm>EE</fnm>
               </au>
               <au>
                  <snm>Abelson</snm>
                  <fnm>J</fnm>
               </au>
            </aug>
            <source>Milbank Q</source>
            <pubdate>1993</pubdate>
            <volume>71</volume>
            <fpage>217</fpage>
            <lpage>252</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.2307/3350399</pubid>
                  <pubid idtype="pmpid">8510601</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B9">
            <title>
               <p>Printed educational materials: effects on professional practice and health care outcomes</p>
            </title>
            <aug>
               <au>
                  <snm>Freemantle</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Harvey</snm>
                  <fnm>EL</fnm>
               </au>
               <au>
                  <snm>Wolf</snm>
                  <fnm>F</fnm>
               </au>
               <au>
                  <snm>Grimshaw</snm>
                  <fnm>JM</fnm>
               </au>
               <au>
                  <snm>Grilli</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Bero</snm>
                  <fnm>LA</fnm>
               </au>
            </aug>
            <source>Cochrane Database of Systematic Reviews (2):CD000172,</source>
            <pubdate>2000</pubdate>
         </bibl>
         <bibl id="B10">
            <title>
               <p>Changing provider behavior: an overview of systematic reviews of interventions</p>
            </title>
            <aug>
               <au>
                  <snm>Grimshaw</snm>
                  <fnm>JM</fnm>
               </au>
               <au>
                  <snm>Shirran</snm>
                  <fnm>L</fnm>
               </au>
               <au>
                  <snm>Thomas</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Mowatt</snm>
                  <fnm>G</fnm>
               </au>
               <au>
                  <snm>Fraser</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Bero</snm>
                  <fnm>L</fnm>
               </au>
               <au>
                  <snm>Grilli</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Harvey</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Oxman</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>O'Brien</snm>
                  <fnm>MA</fnm>
               </au>
            </aug>
            <source>Med Care</source>
            <pubdate>2001</pubdate>
            <volume>8</volume>
            <fpage>II2</fpage>
            <lpage>45</lpage>
         </bibl>
         <bibl id="B11">
            <title>
               <p>Pharmaceutical policies: effects on rational drug use, an overview of 13 reviews (Protocol)</p>
            </title>
            <aug>
               <au>
                  <snm>Aaserud</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Dahlgren</snm>
                  <fnm>AT</fnm>
               </au>
               <au>
                  <snm>Sturm</snm>
                  <fnm>H</fnm>
               </au>
               <au>
                  <snm>K&#246;sters</snm>
                  <fnm>JP</fnm>
               </au>
               <au>
                  <snm>Hill</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Furberg</snm>
                  <fnm>CD</fnm>
               </au>
               <au>
                  <snm>Grilli</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Henry</snm>
                  <fnm>DA</fnm>
               </au>
               <au>
                  <snm>Oxman</snm>
                  <fnm>AD</fnm>
               </au>
               <au>
                  <snm>Ramsay</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Ross-Degnan</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>Cochrane Database of Systematic Reviews</source>
            <pubdate>2006</pubdate>
            <fpage>No.: CD004397</fpage>
         </bibl>
         <bibl id="B12">
            <aug>
               <au>
                  <snm>Cook</snm>
                  <fnm>TD</fnm>
               </au>
               <au>
                  <snm>Campbell</snm>
                  <fnm>DT</fnm>
               </au>
            </aug>
            <source>Quasi-Experimentation: Design &amp; Analysis Issues for Field Settings</source>
            <publisher>Boston, Houghton Mifflin Company</publisher>
            <pubdate>1979</pubdate>
         </bibl>
         <bibl id="B13">
            <title>
               <p>Controlled trial of interventions to increase testing and treatment for Helicobacter pylori and reduce medication use in patients with chronic acid-related symptoms</p>
            </title>
            <aug>
               <au>
                  <snm>Majumdar</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Ross-Degnan</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Farraye</snm>
                  <fnm>FA</fnm>
               </au>
               <au>
                  <snm>Lee</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Kemp</snm>
                  <fnm>JA</fnm>
               </au>
               <au>
                  <snm>LeCates</snm>
                  <fnm>RF</fnm>
               </au>
               <au>
                  <snm>Henning</snm>
                  <fnm>JM</fnm>
               </au>
               <au>
                  <snm>Tunis</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Schrammel</snm>
                  <fnm>P</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>Alimentary Pharmacology &amp; Therapeutics</source>
            <pubdate>2005</pubdate>
            <volume>21</volume>
            <fpage>1029</fpage>
            <lpage>1039</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1111/j.1365-2036.2005.02431.x</pubid>
                  <pubid idtype="pmpid" link="fulltext">15813839</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B14">
            <title>
               <p>Principles of educational outreach ('academic detailing') to improve clinical decision making</p>
            </title>
            <aug>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
               <au>
                  <snm>Avorn</snm>
                  <fnm>J</fnm>
               </au>
            </aug>
            <source>JAMA</source>
            <pubdate>1990</pubdate>
            <volume>263</volume>
            <fpage>549</fpage>
            <lpage>556</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/jama.263.4.549</pubid>
                  <pubid idtype="pmpid">2104640</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B15">
            <title>
               <p>Educational outreach visits: effects on professional practice and health care outcomes</p>
            </title>
            <aug>
               <au>
                  <snm>Thomson O'Brien</snm>
                  <fnm>MA</fnm>
               </au>
               <au>
                  <snm>Oxman</snm>
                  <fnm>AD</fnm>
               </au>
               <au>
                  <snm>Davis</snm>
                  <fnm>DA</fnm>
               </au>
               <au>
                  <snm>Haynes</snm>
                  <fnm>RB</fnm>
               </au>
               <au>
                  <snm>Freemantle</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Harvey</snm>
                  <fnm>EL</fnm>
               </au>
            </aug>
            <source>Cochrane Database of Systematic Reviews (2):CD000409,</source>
            <pubdate>2000</pubdate>
         </bibl>
         <bibl id="B16">
            <title>
               <p>Group versus individual academic detailing to improve the use of antihypertensive medications in primary care: a cluster-randomized controlled trial</p>
            </title>
            <aug>
               <au>
                  <snm>Simon</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Majumdar</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Prosser</snm>
                  <fnm>LA</fnm>
               </au>
               <au>
                  <snm>Salem-Schatz</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Warner</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Kleinman</snm>
                  <fnm>K</fnm>
               </au>
               <au>
                  <snm>Miroshnik</snm>
                  <fnm>I</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>American Journal of Medicine</source>
            <pubdate>2005</pubdate>
            <volume>118</volume>
            <fpage>521</fpage>
            <lpage>528</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1016/j.amjmed.2004.12.023</pubid>
                  <pubid idtype="pmpid" link="fulltext">15866255</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B17">
            <title>
               <p>Effects of ambulatory-care pharmacist consultation on mortality and hospitalization</p>
            </title>
            <aug>
               <au>
                  <snm>Yuan</snm>
                  <fnm>Y</fnm>
               </au>
               <au>
                  <snm>Hay</snm>
                  <fnm>JW</fnm>
               </au>
               <au>
                  <snm>McCombs</snm>
                  <fnm>JS</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care</source>
            <pubdate>2003</pubdate>
            <volume>9</volume>
            <fpage>45</fpage>
            <lpage>56</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">12549814</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B18">
            <title>
               <p>Helicobacter pylori eradication in dyspeptic primary care patients: a randomized controlled trial of a pharmacy intervention</p>
            </title>
            <aug>
               <au>
                  <snm>Stevens</snm>
                  <fnm>VJ</fnm>
               </au>
               <au>
                  <snm>Shneidman</snm>
                  <fnm>RJ</fnm>
               </au>
               <au>
                  <snm>Johnson</snm>
                  <fnm>RE</fnm>
               </au>
               <au>
                  <snm>Boles</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Steele</snm>
                  <fnm>PE</fnm>
               </au>
               <au>
                  <snm>Lee</snm>
                  <fnm>NL</fnm>
               </au>
            </aug>
            <source>Western Journal of Medicine</source>
            <pubdate>2002</pubdate>
            <volume>176</volume>
            <fpage>92</fpage>
            <lpage>96</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1071673</pubid>
                  <pubid idtype="pmpid">11897728</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B19">
            <title>
               <p>Economic analysis of a randomized trial of academic detailing interventions to improve use of antihypertensive medications</p>
            </title>
            <aug>
               <au>
                  <snm>Simon</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Rodriguez</snm>
                  <fnm>HP</fnm>
               </au>
               <au>
                  <snm>Majumdar</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Kleinman</snm>
                  <fnm>K</fnm>
               </au>
               <au>
                  <snm>Warner</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Salem-Schatz</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Miroshnik</snm>
                  <fnm>I</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
               <au>
                  <snm>Prosser</snm>
                  <fnm>LA</fnm>
               </au>
            </aug>
            <source>Journal of Clinical Hypertension</source>
            <pubdate>2007</pubdate>
            <volume>9</volume>
            <fpage>15</fpage>
            <lpage>20</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1111/j.1524-6175.2007.07808.x</pubid>
                  <pubid idtype="pmpid">17215654</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B20">
            <title>
               <p>Does physician education on depression management improve treatment in primary care?</p>
            </title>
            <aug>
               <au>
                  <snm>Lin</snm>
                  <fnm>EH</fnm>
               </au>
               <au>
                  <snm>Simon</snm>
                  <fnm>GE</fnm>
               </au>
               <au>
                  <snm>Katzelnick</snm>
                  <fnm>DJ</fnm>
               </au>
               <au>
                  <snm>Pearson</snm>
                  <fnm>SD</fnm>
               </au>
            </aug>
            <source>Journal of General Internal Medicine</source>
            <pubdate>2001</pubdate>
            <volume>16</volume>
            <fpage>617</fpage>
            <lpage>619</lpage>
            <xrefbib>
               <pubid idtype="doi">10.1046/j.1525-1497.2001.016009614.x</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B21">
            <title>
               <p>Changes in antibiotic prescribing for children after a community-wide campaign</p>
            </title>
            <aug>
               <au>
                  <snm>Perz</snm>
                  <fnm>JF</fnm>
               </au>
               <au>
                  <snm>Craig</snm>
                  <fnm>AS</fnm>
               </au>
               <au>
                  <snm>Coffey</snm>
                  <fnm>CS</fnm>
               </au>
               <au>
                  <snm>Jorgensen</snm>
                  <fnm>DM</fnm>
               </au>
               <au>
                  <snm>Mitchel</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Hall</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Schaffner</snm>
                  <fnm>W</fnm>
               </au>
               <au>
                  <snm>Griffin</snm>
                  <fnm>MR</fnm>
               </au>
            </aug>
            <source>JAMA</source>
            <pubdate>2002</pubdate>
            <volume>287</volume>
            <fpage>3103</fpage>
            <lpage>3109</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/jama.287.23.3103</pubid>
                  <pubid idtype="pmpid" link="fulltext">12069673</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B22">
            <title>
               <p>Audit and feedback: effects on professional practice and health care outcomes</p>
            </title>
            <aug>
               <au>
                  <snm>Jamtvedt</snm>
                  <fnm>G</fnm>
               </au>
               <au>
                  <snm>Young</snm>
                  <fnm>JM</fnm>
               </au>
               <au>
                  <snm>Kristoffersen</snm>
                  <fnm>DT</fnm>
               </au>
               <au>
                  <snm>O'Brien</snm>
                  <fnm>MA</fnm>
               </au>
               <au>
                  <snm>Oxman</snm>
                  <fnm>AD</fnm>
               </au>
            </aug>
            <source>Cochrane Database of Systematic Reviews (2):CD000259,</source>
            <pubdate>2006</pubdate>
         </bibl>
         <bibl id="B23">
            <title>
               <p>Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback</p>
            </title>
            <aug>
               <au>
                  <snm>Jamtvedt</snm>
                  <fnm>G</fnm>
               </au>
               <au>
                  <snm>Young</snm>
                  <fnm>JM</fnm>
               </au>
               <au>
                  <snm>Kristoffersen</snm>
                  <fnm>DT</fnm>
               </au>
               <au>
                  <snm>O'Brien</snm>
                  <fnm>MA</fnm>
               </au>
               <au>
                  <snm>Oxman</snm>
                  <fnm>AD</fnm>
               </au>
            </aug>
            <source>Quality &amp; Safety in Health Care</source>
            <pubdate>2006</pubdate>
            <volume>15</volume>
            <fpage>433</fpage>
            <lpage>436</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1136/qshc.2006.018549</pubid>
                  <pubid idtype="pmpid" link="fulltext">17142594</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B24">
            <title>
               <p>Quality of care for secondary prevention for patients with coronary heart disease: results of the Hastening the Effective Application of Research through Technology (HEART) trial</p>
            </title>
            <aug>
               <au>
                  <snm>Goff</snm>
                  <fnm>DC</fnm>
                  <suf>Jr.</suf>
               </au>
               <au>
                  <snm>Gu</snm>
                  <fnm>L</fnm>
               </au>
               <au>
                  <snm>Cantley</snm>
                  <fnm>LK</fnm>
               </au>
               <au>
                  <snm>Sheedy</snm>
                  <fnm>DJ</fnm>
               </au>
               <au>
                  <snm>Cohen</snm>
                  <fnm>SJ</fnm>
               </au>
            </aug>
            <source>American Heart Journal</source>
            <pubdate>2003</pubdate>
            <volume>146</volume>
            <fpage>1045</fpage>
            <lpage>1051</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1016/S0002-8703(03)00522-2</pubid>
                  <pubid idtype="pmpid" link="fulltext">14660997</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B25">
            <title>
               <p>Failure of Internet-based audit and feedback to improve quality of care delivered by primary care residents</p>
            </title>
            <aug>
               <au>
                  <snm>Simon</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>International Journal for Quality in Health Care</source>
            <pubdate>2005</pubdate>
            <volume>17</volume>
            <fpage>427</fpage>
            <lpage>431</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1093/intqhc/mzi044</pubid>
                  <pubid idtype="pmpid" link="fulltext">15831542</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B26">
            <title>
               <p>Antibiotic treatment of acute respiratory tract infections in the elderly: effect of a multidimensional educational intervention</p>
            </title>
            <aug>
               <au>
                  <snm>Gonzales</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Sauaia</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Corbett</snm>
                  <fnm>KK</fnm>
               </au>
               <au>
                  <snm>Maselli</snm>
                  <fnm>JH</fnm>
               </au>
               <au>
                  <snm>Erbacher</snm>
                  <fnm>K</fnm>
               </au>
               <au>
                  <snm>Leeman-Castillo</snm>
                  <fnm>BA</fnm>
               </au>
               <au>
                  <snm>Darr</snm>
                  <fnm>CA</fnm>
               </au>
               <au>
                  <snm>Houck</snm>
                  <fnm>PM</fnm>
               </au>
            </aug>
            <source>Journal of the American Geriatrics Society</source>
            <pubdate>2004</pubdate>
            <volume>52</volume>
            <fpage>39</fpage>
            <lpage>45</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1111/j.1532-5415.2004.52008.x</pubid>
                  <pubid idtype="pmpid" link="fulltext">14687313</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B27">
            <title>
               <p>Impact of an antidepressant management program on medication adherence</p>
            </title>
            <aug>
               <au>
                  <snm>Hoffman</snm>
                  <fnm>L</fnm>
               </au>
               <au>
                  <snm>Enders</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Luo</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Segal</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Pippins</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Kimberlin</snm>
                  <fnm>C</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care</source>
            <pubdate>2003</pubdate>
            <volume>9</volume>
            <fpage>70</fpage>
            <lpage>80</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">12549816</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B28">
            <title>
               <p>Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. [Review]</p>
            </title>
            <aug>
               <au>
                  <snm>Kaushal</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Shojania</snm>
                  <fnm>KG</fnm>
               </au>
               <au>
                  <snm>Bates</snm>
                  <fnm>DW</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2003</pubdate>
            <volume>163</volume>
            <fpage>1409</fpage>
            <lpage>1416</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.163.12.1409</pubid>
                  <pubid idtype="pmpid" link="fulltext">12824090</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B29">
            <title>
               <p>The impact of prescribing safety alerts for elderly persons in an electronic medical record: an interrupted time series evaluation</p>
            </title>
            <aug>
               <au>
                  <snm>Smith</snm>
                  <fnm>DH</fnm>
               </au>
               <au>
                  <snm>Perrin</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Feldstein</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Yang</snm>
                  <fnm>X</fnm>
               </au>
               <au>
                  <snm>Kuang</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Simon</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Sittig</snm>
                  <fnm>DF</fnm>
               </au>
               <au>
                  <snm>Platt</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2006</pubdate>
            <volume>166</volume>
            <fpage>1098</fpage>
            <lpage>1104</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.166.10.1098</pubid>
                  <pubid idtype="pmpid" link="fulltext">16717172</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B30">
            <title>
               <p>Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial</p>
            </title>
            <aug>
               <au>
                  <snm>Feldstein</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Elmer</snm>
                  <fnm>PJ</fnm>
               </au>
               <au>
                  <snm>Smith</snm>
                  <fnm>DH</fnm>
               </au>
               <au>
                  <snm>Herson</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Orwoll</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Chen</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Aickin</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Swain</snm>
                  <fnm>MC</fnm>
               </au>
            </aug>
            <source>Journal of the American Geriatrics Society</source>
            <pubdate>2006</pubdate>
            <volume>54</volume>
            <fpage>450</fpage>
            <lpage>457</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1111/j.1532-5415.2005.00618.x</pubid>
                  <pubid idtype="pmpid" link="fulltext">16551312</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B31">
            <title>
               <p>Reducing warfarin medication interactions: an interrupted time series evaluation</p>
            </title>
            <aug>
               <au>
                  <snm>Feldstein</snm>
                  <fnm>AC</fnm>
               </au>
               <au>
                  <snm>Smith</snm>
                  <fnm>DH</fnm>
               </au>
               <au>
                  <snm>Perrin</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Yang</snm>
                  <fnm>X</fnm>
               </au>
               <au>
                  <snm>Simon</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Krall</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Sittig</snm>
                  <fnm>DF</fnm>
               </au>
               <au>
                  <snm>Ditmer</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Platt</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2006</pubdate>
            <volume>166</volume>
            <fpage>1009</fpage>
            <lpage>1015</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.166.9.1009</pubid>
                  <pubid idtype="pmpid" link="fulltext">16682575</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B32">
            <title>
               <p>Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people</p>
            </title>
            <aug>
               <au>
                  <snm>Simon</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Smith</snm>
                  <fnm>DH</fnm>
               </au>
               <au>
                  <snm>Feldstein</snm>
                  <fnm>AC</fnm>
               </au>
               <au>
                  <snm>Perrin</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Yang</snm>
                  <fnm>X</fnm>
               </au>
               <au>
                  <snm>Zhou</snm>
                  <fnm>Y</fnm>
               </au>
               <au>
                  <snm>Platt</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>Journal of the American Geriatrics Society</source>
            <pubdate>2006</pubdate>
            <volume>54</volume>
            <fpage>963</fpage>
            <lpage>968</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1111/j.1532-5415.2006.00734.x</pubid>
                  <pubid idtype="pmpid" link="fulltext">16776793</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B33">
            <title>
               <p>Physician alerts to increase antidepressant adherence: fax or fiction?</p>
            </title>
            <aug>
               <au>
                  <snm>Bambauer</snm>
                  <fnm>KZ</fnm>
               </au>
               <au>
                  <snm>Adams</snm>
                  <fnm>AS</fnm>
               </au>
               <au>
                  <snm>Zhang</snm>
                  <fnm>F</fnm>
               </au>
               <au>
                  <snm>Minkoff</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Grande</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Weisblatt</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
               <au>
                  <snm>Ross-Degnan</snm>
                  <fnm>D</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2006</pubdate>
            <volume>166</volume>
            <fpage>498</fpage>
            <lpage>504</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.166.5.498</pubid>
                  <pubid idtype="pmpid" link="fulltext">16534035</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B34">
            <title>
               <p>Improving laboratory monitoring at initiation of drug therapy in ambulatory care: a randomized trial</p>
            </title>
            <aug>
               <au>
                  <snm>Raebel</snm>
                  <fnm>MA</fnm>
               </au>
               <au>
                  <snm>Lyons</snm>
                  <fnm>EE</fnm>
               </au>
               <au>
                  <snm>Chester</snm>
                  <fnm>EA</fnm>
               </au>
               <au>
                  <snm>Bodily</snm>
                  <fnm>MA</fnm>
               </au>
               <au>
                  <snm>Kelleher</snm>
                  <fnm>JA</fnm>
               </au>
               <au>
                  <snm>Long</snm>
                  <fnm>CL</fnm>
               </au>
               <au>
                  <snm>Miller</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Magid</snm>
                  <fnm>DJ</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2005</pubdate>
            <volume>165</volume>
            <fpage>2395</fpage>
            <lpage>2401</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.165.20.2395</pubid>
                  <pubid idtype="pmpid" link="fulltext">16287769</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B35">
            <title>
               <p>Randomized trial to improve laboratory safety monitoring of ongoing drug therapy in ambulatory patients</p>
            </title>
            <aug>
               <au>
                  <snm>Raebel</snm>
                  <fnm>MA</fnm>
               </au>
               <au>
                  <snm>Chester</snm>
                  <fnm>EA</fnm>
               </au>
               <au>
                  <snm>Newsom</snm>
                  <fnm>EE</fnm>
               </au>
               <au>
                  <snm>Lyons</snm>
                  <fnm>EE</fnm>
               </au>
               <au>
                  <snm>Kelleher</snm>
                  <fnm>JA</fnm>
               </au>
               <au>
                  <snm>Long</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Miller</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Magid</snm>
                  <fnm>DJ</fnm>
               </au>
            </aug>
            <source>Pharmacotherapy</source>
            <pubdate>2006</pubdate>
            <volume>26</volume>
            <fpage>619</fpage>
            <lpage>626</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1592/phco.26.5.619</pubid>
                  <pubid idtype="pmpid" link="fulltext">16637791</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B36">
            <title>
               <p>Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders</p>
            </title>
            <aug>
               <au>
                  <snm>Palen</snm>
                  <fnm>TE</fnm>
               </au>
               <au>
                  <snm>Raebel</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Lyons</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Magid</snm>
                  <fnm>DM</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care</source>
            <pubdate>2006</pubdate>
            <volume>12</volume>
            <fpage>389</fpage>
            <lpage>395</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">16834525</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B37">
            <title>
               <p>Improved therapeutic monitoring with several interventions: a randomized trial</p>
            </title>
            <aug>
               <au>
                  <snm>Feldstein</snm>
                  <fnm>AC</fnm>
               </au>
               <au>
                  <snm>Smith</snm>
                  <fnm>DH</fnm>
               </au>
               <au>
                  <snm>Perrin</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Yang</snm>
                  <fnm>X</fnm>
               </au>
               <au>
                  <snm>Rix</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Raebel</snm>
                  <fnm>MA</fnm>
               </au>
               <au>
                  <snm>Magid</snm>
                  <fnm>DJ</fnm>
               </au>
               <au>
                  <snm>Simon</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2006</pubdate>
            <volume>166</volume>
            <fpage>1848</fpage>
            <lpage>1854</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.166.17.1848</pubid>
                  <pubid idtype="pmpid" link="fulltext">17000941</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B38">
            <title>
               <p>Pharmacist telemonitoring of antidepressant use: effects on pharmacist-patient collaboration</p>
            </title>
            <aug>
               <au>
                  <snm>Rickles</snm>
                  <fnm>NM</fnm>
               </au>
               <au>
                  <snm>Svarstad</snm>
                  <fnm>BL</fnm>
               </au>
               <au>
                  <snm>Statz-Paynter</snm>
                  <fnm>JL</fnm>
               </au>
               <au>
                  <snm>Taylor</snm>
                  <fnm>LV</fnm>
               </au>
               <au>
                  <snm>Kobak</snm>
                  <fnm>KA</fnm>
               </au>
            </aug>
            <source>Journal of the American Pharmacists Association: JAPhA 45(3):344-53,</source>
            <pubdate>2005</pubdate>
            <lpage>Jun</lpage>
         </bibl>
         <bibl id="B39">
            <title>
               <p>Improving patient feedback about and outcomes with antidepressant treatment: a study in eight community pharmacies</p>
            </title>
            <aug>
               <au>
                  <snm>Rickles</snm>
                  <fnm>NM</fnm>
               </au>
               <au>
                  <snm>Svarstad</snm>
                  <fnm>BL</fnm>
               </au>
               <au>
                  <snm>Statz-Paynter</snm>
                  <fnm>JL</fnm>
               </au>
               <au>
                  <snm>Taylor</snm>
                  <fnm>LV</fnm>
               </au>
               <au>
                  <snm>Kobak</snm>
                  <fnm>KA</fnm>
               </au>
            </aug>
            <source>Journal of the American Pharmacists Association: JAPhA 46(1):25-32,</source>
            <pubdate>2006</pubdate>
            <lpage>Feb</lpage>
         </bibl>
         <bibl id="B40">
            <title>
               <p>Use and impact of an automated telephone outreach system for asthma in a managed care setting</p>
            </title>
            <aug>
               <au>
                  <snm>Vollmer</snm>
                  <fnm>WM</fnm>
               </au>
               <au>
                  <snm>Kirshner</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Peters</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Drane</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Stibolt</snm>
                  <fnm>T</fnm>
               </au>
               <au>
                  <snm>Hickey</snm>
                  <fnm>T</fnm>
               </au>
               <au>
                  <snm>Tom</snm>
                  <fnm>GI</fnm>
               </au>
               <au>
                  <snm>Buist</snm>
                  <fnm>AS</fnm>
               </au>
               <au>
                  <snm>O'Connor</snm>
                  <fnm>EA</fnm>
               </au>
               <au>
                  <snm>Frazier</snm>
                  <fnm>EA</fnm>
               </au>
               <au>
                  <snm>Mosen</snm>
                  <fnm>D</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care</source>
            <pubdate>2006</pubdate>
            <volume>12</volume>
            <fpage>725</fpage>
            <lpage>733</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">17149995</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B41">
            <title>
               <p>A benefit-based copay for prescription drugs: patient contribution based on total benefits, not drug acquisition cost</p>
            </title>
            <aug>
               <au>
                  <snm>Fendrick</snm>
                  <fnm>AM</fnm>
               </au>
               <au>
                  <snm>Smith</snm>
                  <fnm>DG</fnm>
               </au>
               <au>
                  <snm>Chernew</snm>
                  <fnm>ME</fnm>
               </au>
               <au>
                  <snm>Shah</snm>
                  <fnm>SN</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care</source>
            <pubdate>2001</pubdate>
            <volume>7</volume>
            <fpage>861</fpage>
            <lpage>867</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">11570020</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B42">
            <title>
               <p>The effect of incentive-based formularies on prescription-drug utilization and spending</p>
            </title>
            <aug>
               <au>
                  <snm>Huskamp</snm>
                  <fnm>HA</fnm>
               </au>
               <au>
                  <snm>Deverka</snm>
                  <fnm>PA</fnm>
               </au>
               <au>
                  <snm>Epstein</snm>
                  <fnm>AM</fnm>
               </au>
               <au>
                  <snm>Epstein</snm>
                  <fnm>RS</fnm>
               </au>
               <au>
                  <snm>McGuigan</snm>
                  <fnm>KA</fnm>
               </au>
               <au>
                  <snm>Frank</snm>
                  <fnm>RG</fnm>
               </au>
            </aug>
            <source>N Eng J Med</source>
            <pubdate>2003</pubdate>
            <volume>349</volume>
            <fpage>2224</fpage>
            <lpage>2232</lpage>
            <xrefbib>
               <pubid idtype="doi">10.1056/NEJMsa030954</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B43">
            <title>
               <p>Impact of 3-tier formularies on drug treatment of attention-deficit/hyperactivity disorder in children</p>
            </title>
            <aug>
               <au>
                  <snm>Huskamp</snm>
                  <fnm>HA</fnm>
               </au>
               <au>
                  <snm>Deverka</snm>
                  <fnm>PA</fnm>
               </au>
               <au>
                  <snm>Epstein</snm>
                  <fnm>AM</fnm>
               </au>
               <au>
                  <snm>Epstein</snm>
                  <fnm>RS</fnm>
               </au>
               <au>
                  <snm>McGuigan</snm>
                  <fnm>KA</fnm>
               </au>
               <au>
                  <snm>Muriel</snm>
                  <fnm>AC</fnm>
               </au>
               <au>
                  <snm>Frank</snm>
                  <fnm>RG</fnm>
               </au>
            </aug>
            <source>Archives of General Psychiatry</source>
            <pubdate>2005</pubdate>
            <volume>62</volume>
            <fpage>435</fpage>
            <lpage>441</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1403292</pubid>
                  <pubid idtype="pmpid" link="fulltext">15809411</pubid>
                  <pubid idtype="doi">10.1001/archpsyc.62.4.435</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B44">
            <title>
               <p>Effect of a three-tier prescription copay on pharmaceutical and other medical utilization</p>
            </title>
            <aug>
               <au>
                  <snm>Motheral</snm>
                  <fnm>B</fnm>
               </au>
               <au>
                  <snm>Fairman</snm>
                  <fnm>KA</fnm>
               </au>
            </aug>
            <source>Med Care</source>
            <pubdate>2001</pubdate>
            <volume>39</volume>
            <fpage>1293</fpage>
            <lpage>1304</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1097/00005650-200112000-00005</pubid>
                  <pubid idtype="pmpid">11717571</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B45">
            <title>
               <p>Retrospective, long-term follow-up study of the effect of a three-tier prescription drug copayment system on pharmaceutical and other medical utilization and costs</p>
            </title>
            <aug>
               <au>
                  <snm>Fairman</snm>
                  <fnm>KA</fnm>
               </au>
               <au>
                  <snm>Motheral</snm>
                  <fnm>BR</fnm>
               </au>
               <au>
                  <snm>Henderson</snm>
                  <fnm>RR</fnm>
               </au>
            </aug>
            <source>Clinical Therapeutics</source>
            <pubdate>2003</pubdate>
            <volume>25</volume>
            <fpage>3147</fpage>
            <lpage>3161</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1016/S0149-2918(03)90099-3</pubid>
                  <pubid idtype="pmpid" link="fulltext">14749153</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B46">
            <title>
               <p>Effects of a 3-tier pharmacy benefit design on the prescription purchasing behavior of individuals with chronic disease</p>
            </title>
            <aug>
               <au>
                  <snm>Nair</snm>
                  <fnm>KV</fnm>
               </au>
               <au>
                  <snm>Wolfe</snm>
                  <fnm>P</fnm>
               </au>
               <au>
                  <snm>Valuck</snm>
                  <fnm>RJ</fnm>
               </au>
               <au>
                  <snm>McCollum</snm>
                  <fnm>MM</fnm>
               </au>
               <au>
                  <snm>Ganther</snm>
                  <fnm>JM</fnm>
               </au>
               <au>
                  <snm>Lewis</snm>
                  <fnm>SJ</fnm>
               </au>
            </aug>
            <source>Journal of Managed Care Pharmacy</source>
            <pubdate>2003</pubdate>
            <volume>9</volume>
            <fpage>123</fpage>
            <lpage>133</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">14613341</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B47">
            <title>
               <p>Effect of increased cost-sharing on oral hypoglycemic use in five managed care organizations: how much is too much?</p>
            </title>
            <aug>
               <au>
                  <snm>Roblin</snm>
                  <fnm>DW</fnm>
               </au>
               <au>
                  <snm>Platt</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Goodman</snm>
                  <fnm>MJ</fnm>
               </au>
               <au>
                  <snm>Hsu</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Nelson</snm>
                  <fnm>WW</fnm>
               </au>
               <au>
                  <snm>Smith</snm>
                  <fnm>DH</fnm>
               </au>
               <au>
                  <snm>Andrade</snm>
                  <fnm>SE</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>Med Care</source>
            <pubdate>2005</pubdate>
            <volume>43</volume>
            <fpage>951</fpage>
            <lpage>959</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1097/01.mlr.0000178216.23514.b7</pubid>
                  <pubid idtype="pmpid">16166864</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B48">
            <title>
               <p>Prior authorization programs: a critical review of the literature</p>
            </title>
            <aug>
               <au>
                  <snm>MacKinnon</snm>
                  <fnm>NJ</fnm>
               </au>
               <au>
                  <snm>Kumar</snm>
                  <fnm>R</fnm>
               </au>
            </aug>
            <source>Journal of Managed Care Pharm</source>
            <pubdate>2001</pubdate>
            <volume>7</volume>
            <fpage>297</fpage>
            <lpage>302</lpage>
         </bibl>
         <bibl id="B49">
            <title>
               <p>Effects of a prior-authorization policy for celecoxib on medical service and prescription drug use in a managed care Medicaid population</p>
            </title>
            <aug>
               <au>
                  <snm>Hartung</snm>
                  <fnm>DM</fnm>
               </au>
               <au>
                  <snm>Touchette</snm>
                  <fnm>DR</fnm>
               </au>
               <au>
                  <snm>Ketchum</snm>
                  <fnm>KL</fnm>
               </au>
               <au>
                  <snm>Haxby</snm>
                  <fnm>DG</fnm>
               </au>
               <au>
                  <snm>Goldberg</snm>
                  <fnm>BW</fnm>
               </au>
            </aug>
            <source>Clinical Therapeutics</source>
            <pubdate>2004</pubdate>
            <volume>26</volume>
            <fpage>1518</fpage>
            <lpage>1532</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1016/j.clinthera.2004.09.013</pubid>
                  <pubid idtype="pmpid" link="fulltext">15531015</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B50">
            <title>
               <p>Medical and pharmacy expenditures after implementation of a cyclooxygenase-2 inhibitor prior authorization program</p>
            </title>
            <aug>
               <au>
                  <snm>Gleason</snm>
                  <fnm>PP</fnm>
               </au>
               <au>
                  <snm>Williams</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Hrdy</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Hartwig</snm>
                  <fnm>SC</fnm>
               </au>
               <au>
                  <snm>Lassen</snm>
                  <fnm>D</fnm>
               </au>
            </aug>
            <source>Pharmacotherapy</source>
            <pubdate>2005</pubdate>
            <volume>25</volume>
            <fpage>924</fpage>
            <lpage>934</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1592/phco.2005.25.7.924</pubid>
                  <pubid idtype="pmpid">16006271</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B51">
            <title>
               <p>The effect of an Rx-to-OTC switch on medication prescribing patterns and utilization of physician services: the case of vaginal antifungal products</p>
            </title>
            <aug>
               <au>
                  <snm>Gurwitz</snm>
                  <fnm>JH</fnm>
               </au>
               <au>
                  <snm>McLaughlin</snm>
                  <fnm>TJ</fnm>
               </au>
               <au>
                  <snm>Fish</snm>
                  <fnm>LS</fnm>
               </au>
            </aug>
            <source>Health Serv Res</source>
            <pubdate>1995</pubdate>
            <volume>30</volume>
            <fpage>672</fpage>
            <lpage>685</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1070084</pubid>
                  <pubid idtype="pmpid">8537226</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B52">
            <title>
               <p>Enalapril to lisinopril: economic impact of a voluntary angiotensin-converting enzyme-inhibitor substitution program in a staff-model health maintenance organization</p>
            </title>
            <aug>
               <au>
                  <snm>McDonough</snm>
                  <fnm>KP</fnm>
               </au>
               <au>
                  <snm>Weaver</snm>
                  <fnm>RH</fnm>
               </au>
               <au>
                  <snm>Viall</snm>
                  <fnm>GD</fnm>
               </au>
            </aug>
            <source>Ann Pharmacother</source>
            <pubdate>1992</pubdate>
            <volume>26</volume>
            <fpage>399</fpage>
            <lpage>404</lpage>
            <xrefbib>
               <pubid idtype="pmpid">1313320</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B53">
            <title>
               <p>Evaluation of a formulary switch from conjugated to esterified estrogens in a managed care setting</p>
            </title>
            <aug>
               <au>
                  <snm>Andrade</snm>
                  <fnm>SE</fnm>
               </au>
               <au>
                  <snm>Gurwitz</snm>
                  <fnm>JH</fnm>
               </au>
               <au>
                  <snm>Cernieux</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Fish</snm>
                  <fnm>LS</fnm>
               </au>
            </aug>
            <source>Med Care</source>
            <pubdate>2000</pubdate>
            <volume>38</volume>
            <fpage>970</fpage>
            <lpage>975</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1097/00005650-200009000-00010</pubid>
                  <pubid idtype="pmpid">10982118</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B54">
            <title>
               <p>The effect of the Rx-to-OTC switch of loratadine and changes in prescription drug benefits on utilization and cost of therapy</p>
            </title>
            <aug>
               <au>
                  <snm>Sullivan</snm>
                  <fnm>PW</fnm>
               </au>
               <au>
                  <snm>Nair</snm>
                  <fnm>KV</fnm>
               </au>
               <au>
                  <snm>Patel</snm>
                  <fnm>BV</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care</source>
            <pubdate>2005</pubdate>
            <volume>11</volume>
            <fpage>374</fpage>
            <lpage>382</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">15974556</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B55">
            <title>
               <p>Randomized controlled trial of a dose consolidation program</p>
            </title>
            <aug>
               <au>
                  <snm>Delate</snm>
                  <fnm>T</fnm>
               </au>
               <au>
                  <snm>Fairman</snm>
                  <fnm>KA</fnm>
               </au>
               <au>
                  <snm>Carey</snm>
                  <fnm>SM</fnm>
               </au>
               <au>
                  <snm>Motheral</snm>
                  <fnm>BR</fnm>
               </au>
            </aug>
            <source>Journal of Managed Care Pharmacy</source>
            <pubdate>2004</pubdate>
            <volume>10</volume>
            <fpage>396</fpage>
            <lpage>403</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">15369422</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B56">
            <title>
               <p>Effect of patient notification of formulary change on formulary adherence</p>
            </title>
            <aug>
               <au>
                  <snm>Delate</snm>
                  <fnm>T</fnm>
               </au>
               <au>
                  <snm>Henderson</snm>
                  <fnm>R</fnm>
               </au>
            </aug>
            <source>Journal of Managed Care Pharmacy</source>
            <pubdate>2005</pubdate>
            <volume>11</volume>
            <fpage>493</fpage>
            <lpage>498</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">15998166</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B57">
            <title>
               <p>Promoting use of a preferred histamine H2-receptor antagonist in managed care organizations</p>
            </title>
            <aug>
               <au>
                  <snm>DeZearn</snm>
                  <fnm>K</fnm>
               </au>
               <au>
                  <snm>Springer</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Benson</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Stephens</snm>
                  <fnm>C</fnm>
               </au>
            </aug>
            <source>Am J Health Syst Pharm</source>
            <pubdate>1996</pubdate>
            <volume>53</volume>
            <fpage>865</fpage>
            <lpage>867</lpage>
            <xrefbib>
               <pubid idtype="pmpid">8728383</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B58">
            <title>
               <p>Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes</p>
            </title>
            <aug>
               <au>
                  <snm>Gilbody</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Bower</snm>
                  <fnm>P</fnm>
               </au>
               <au>
                  <snm>Fletcher</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Richards</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Sutton</snm>
                  <fnm>AJ</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2006</pubdate>
            <volume>166</volume>
            <fpage>2314</fpage>
            <lpage>2321</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.166.21.2314</pubid>
                  <pubid idtype="pmpid" link="fulltext">17130383</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B59">
            <title>
               <p>Impact of a collaborative care model on depression in a primary care setting: a randomized controlled trial</p>
            </title>
            <aug>
               <au>
                  <snm>Finley</snm>
                  <fnm>PR</fnm>
               </au>
               <au>
                  <snm>Rens</snm>
                  <fnm>HR</fnm>
               </au>
               <au>
                  <snm>Pont</snm>
                  <fnm>JT</fnm>
               </au>
               <au>
                  <snm>Gess</snm>
                  <fnm>SL</fnm>
               </au>
               <au>
                  <snm>Louie</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Bull</snm>
                  <fnm>SA</fnm>
               </au>
               <au>
                  <snm>Lee</snm>
                  <fnm>JY</fnm>
               </au>
               <au>
                  <snm>Bero</snm>
                  <fnm>LA</fnm>
               </au>
            </aug>
            <source>Pharmacotherapy</source>
            <pubdate>2003</pubdate>
            <volume>23</volume>
            <fpage>1175</fpage>
            <lpage>1185</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1592/phco.23.10.1175.32760</pubid>
                  <pubid idtype="pmpid">14524649</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B60">
            <title>
               <p>Pharmacoeconomic evaluation of a pharmacist-managed hypertension clinic</p>
            </title>
            <aug>
               <au>
                  <snm>Okamoto</snm>
                  <fnm>MP</fnm>
               </au>
               <au>
                  <snm>Nakahiro</snm>
                  <fnm>RK</fnm>
               </au>
            </aug>
            <source>Pharmacotherapy</source>
            <pubdate>2001</pubdate>
            <volume>21</volume>
            <fpage>1337</fpage>
            <lpage>1344</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1592/phco.21.17.1337.34424</pubid>
                  <pubid idtype="pmpid">11714206</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B61">
            <title>
               <p>Effect of a clinical pharmacy service on lipid control in patients with peripheral arterial disease</p>
            </title>
            <aug>
               <au>
                  <snm>Rehring</snm>
                  <fnm>TF</fnm>
               </au>
               <au>
                  <snm>Stolcpart</snm>
                  <fnm>RS</fnm>
               </au>
               <au>
                  <snm>Sandhoff</snm>
                  <fnm>BG</fnm>
               </au>
               <au>
                  <snm>Merenich</snm>
                  <fnm>JA</fnm>
               </au>
               <au>
                  <snm>Hollis</snm>
                  <fnm>HW</fnm>
                  <suf>Jr.</suf>
               </au>
            </aug>
            <source>Journal of Vascular Surgery</source>
            <pubdate>2006</pubdate>
            <volume>43</volume>
            <fpage>1205</fpage>
            <lpage>1210</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1016/j.jvs.2006.02.019</pubid>
                  <pubid idtype="pmpid" link="fulltext">16765240</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B62">
            <title>
               <p>Achieving cholesterol target in a managed care organization (ACTION) trial</p>
            </title>
            <aug>
               <au>
                  <snm>Straka</snm>
                  <fnm>RJ</fnm>
               </au>
               <au>
                  <snm>Taheri</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Cooper</snm>
                  <fnm>SL</fnm>
               </au>
               <au>
                  <snm>Smith</snm>
                  <fnm>JC</fnm>
               </au>
            </aug>
            <source>Pharmacotherapy</source>
            <pubdate>2005</pubdate>
            <volume>25</volume>
            <fpage>360</fpage>
            <lpage>371</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1592/phco.25.3.360.61601</pubid>
                  <pubid idtype="pmpid" link="fulltext">15843283</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B63">
            <title>
               <p>Physician-pharmacist comanagement of hypertension: a randomized, comparative trial</p>
            </title>
            <aug>
               <au>
                  <snm>Borenstein</snm>
                  <fnm>JE</fnm>
               </au>
               <au>
                  <snm>Graber</snm>
                  <fnm>G</fnm>
               </au>
               <au>
                  <snm>Saltiel</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Wallace</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Ryu</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Archi</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Deutsch</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Weingarten</snm>
                  <fnm>SR</fnm>
               </au>
            </aug>
            <source>Pharmacotherapy</source>
            <pubdate>2003</pubdate>
            <volume>23</volume>
            <fpage>209</fpage>
            <lpage>216</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1592/phco.23.2.209.32096</pubid>
                  <pubid idtype="pmpid">12587810</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B64">
            <title>
               <p>Disease management in the American market</p>
            </title>
            <aug>
               <au>
                  <snm>Bodenheimer</snm>
                  <fnm>T</fnm>
               </au>
            </aug>
            <source>BMJ</source>
            <pubdate>2000</pubdate>
            <volume>320</volume>
            <fpage>563</fpage>
            <lpage>566</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1117603</pubid>
                  <pubid idtype="pmpid" link="fulltext">10688566</pubid>
                  <pubid idtype="doi">10.1136/bmj.320.7234.563</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B65">
            <title>
               <p>Effectiveness of disease management programs in depression: a systematic review</p>
            </title>
            <aug>
               <au>
                  <snm>Badamgarav</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Weingarten</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Henning</snm>
                  <fnm>JM</fnm>
               </au>
               <au>
                  <snm>Knight</snm>
                  <fnm>K</fnm>
               </au>
               <au>
                  <snm>Hasselblad</snm>
                  <fnm>V</fnm>
               </au>
               <au>
                  <snm>Gano</snm>
                  <fnm>A</fnm>
                  <suf>Jr.</suf>
               </au>
               <au>
                  <snm>Ofman</snm>
                  <fnm>JJ</fnm>
               </au>
            </aug>
            <source>American Journal of Psychiatry</source>
            <pubdate>2003</pubdate>
            <volume>160</volume>
            <fpage>2080</fpage>
            <lpage>2090</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1176/appi.ajp.160.12.2080</pubid>
                  <pubid idtype="pmpid" link="fulltext">14638573</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B66">
            <title>
               <p>The costs of depression: direct and indirect; treatment versus nontreatment. [Review]</p>
            </title>
            <aug>
               <au>
                  <snm>Panzarino</snm>
                  <fnm>PJ</fnm>
                  <suf>Jr.</suf>
               </au>
            </aug>
            <source>Journal of Clinical Psychiatry</source>
            <pubdate>1920</pubdate>
            <volume>59</volume>
            <fpage>11</fpage>
            <lpage>14</lpage>
         </bibl>
         <bibl id="B67">
            <title>
               <p>Long-term effects of a collaborative care intervention in persistently depressed primary care patients</p>
            </title>
            <aug>
               <au>
                  <snm>Katon</snm>
                  <fnm>W</fnm>
               </au>
               <au>
                  <snm>Russo</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Von Korff</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Lin</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Simon</snm>
                  <fnm>G</fnm>
               </au>
               <au>
                  <snm>Bush</snm>
                  <fnm>T</fnm>
               </au>
               <au>
                  <snm>Ludman</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Walker</snm>
                  <fnm>E</fnm>
               </au>
            </aug>
            <source>Journal of General Internal Medicine</source>
            <pubdate>2002</pubdate>
            <volume>17</volume>
            <fpage>741</fpage>
            <lpage>748</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1495114</pubid>
                  <pubid idtype="pmpid" link="fulltext">12390549</pubid>
                  <pubid idtype="doi">10.1046/j.1525-1497.2002.11051.x</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B68">
            <title>
               <p>The Pathways Study: a randomized trial of collaborative care in patients with diabetes and depression</p>
            </title>
            <aug>
               <au>
                  <snm>Katon</snm>
                  <fnm>WJ</fnm>
               </au>
               <au>
                  <snm>Von Korff</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Lin</snm>
                  <fnm>EH</fnm>
               </au>
               <au>
                  <snm>Simon</snm>
                  <fnm>G</fnm>
               </au>
               <au>
                  <snm>Ludman</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Russo</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Ciechanowski</snm>
                  <fnm>P</fnm>
               </au>
               <au>
                  <snm>Walker</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Bush</snm>
                  <fnm>T</fnm>
               </au>
            </aug>
            <source>Archives of General Psychiatry</source>
            <pubdate>2004</pubdate>
            <volume>61</volume>
            <fpage>1042</fpage>
            <lpage>1049</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archpsyc.61.10.1042</pubid>
                  <pubid idtype="pmpid" link="fulltext">15466678</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B69">
            <title>
               <p>Two-year effects of quality improvement programs on medication management for depression</p>
            </title>
            <aug>
               <au>
                  <snm>Unutzer</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Rubenstein</snm>
                  <fnm>L</fnm>
               </au>
               <au>
                  <snm>Katon</snm>
                  <fnm>WJ</fnm>
               </au>
               <au>
                  <snm>Tang</snm>
                  <fnm>L</fnm>
               </au>
               <au>
                  <snm>Duan</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Lagomasino</snm>
                  <fnm>IT</fnm>
               </au>
               <au>
                  <snm>Wells</snm>
                  <fnm>KB</fnm>
               </au>
            </aug>
            <source>Archives of General Psychiatry</source>
            <pubdate>2001</pubdate>
            <volume>58</volume>
            <fpage>935</fpage>
            <lpage>942</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archpsyc.58.10.935</pubid>
                  <pubid idtype="pmpid" link="fulltext">11576031</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B70">
            <title>
               <p>Effect of a mental health "carve-out" program on the continuity of antipsychotic therapy</p>
            </title>
            <aug>
               <au>
                  <snm>Ray</snm>
                  <fnm>WA</fnm>
               </au>
               <au>
                  <snm>Daugherty</snm>
                  <fnm>JR</fnm>
               </au>
               <au>
                  <snm>Meador</snm>
                  <fnm>KG</fnm>
               </au>
            </aug>
            <source>N Eng J Med</source>
            <pubdate>2003</pubdate>
            <volume>348</volume>
            <fpage>1885</fpage>
            <lpage>1894</lpage>
            <xrefbib>
               <pubid idtype="doi">10.1056/NEJMsa020584</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B71">
            <title>
               <p>A multisite randomized trial of the effects of physician education and organizational change in chronic-asthma care: health outcomes of the Pediatric Asthma Care Patient Outcomes Research Team II Study</p>
            </title>
            <aug>
               <au>
                  <snm>Lozano</snm>
                  <fnm>P</fnm>
               </au>
               <au>
                  <snm>Finkelstein</snm>
                  <fnm>JA</fnm>
               </au>
               <au>
                  <snm>Carey</snm>
                  <fnm>VJ</fnm>
               </au>
               <au>
                  <snm>Wagner</snm>
                  <fnm>EH</fnm>
               </au>
               <au>
                  <snm>Inui</snm>
                  <fnm>TS</fnm>
               </au>
               <au>
                  <snm>Fuhlbrigge</snm>
                  <fnm>AL</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
               <au>
                  <snm>Sullivan</snm>
                  <fnm>SD</fnm>
               </au>
               <au>
                  <snm>Weiss</snm>
                  <fnm>ST</fnm>
               </au>
               <au>
                  <snm>Weiss</snm>
                  <fnm>KB</fnm>
               </au>
            </aug>
            <source>Archives of Pediatrics &amp; Adolescent Medicine</source>
            <pubdate>2004</pubdate>
            <volume>158</volume>
            <fpage>875</fpage>
            <lpage>883</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archpedi.158.9.875</pubid>
                  <pubid idtype="pmpid" link="fulltext">15351753</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B72">
            <title>
               <p>Practice-level effects of interventions to improve asthma care in primary care settings: the Pediatric Asthma Care Patient Outcomes Research Team</p>
            </title>
            <aug>
               <au>
                  <snm>Finkelstein</snm>
                  <fnm>JA</fnm>
               </au>
               <au>
                  <snm>Lozano</snm>
                  <fnm>P</fnm>
               </au>
               <au>
                  <snm>Fuhlbrigge</snm>
                  <fnm>AL</fnm>
               </au>
               <au>
                  <snm>Carey</snm>
                  <fnm>VJ</fnm>
               </au>
               <au>
                  <snm>Inui</snm>
                  <fnm>TS</fnm>
               </au>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
               <au>
                  <snm>Sullivan</snm>
                  <fnm>SD</fnm>
               </au>
               <au>
                  <snm>Wagner</snm>
                  <fnm>EH</fnm>
               </au>
               <au>
                  <snm>Weiss</snm>
                  <fnm>ST</fnm>
               </au>
               <au>
                  <snm>Weiss</snm>
                  <fnm>KB</fnm>
               </au>
               <au>
                  <cnm>Pediatric Asthma Care Patient Outcomes Research Team</cnm>
               </au>
            </aug>
            <source>Health Serv Res</source>
            <pubdate>2005</pubdate>
            <volume>40</volume>
            <fpage>1737</fpage>
            <lpage>1757</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1361234</pubid>
                  <pubid idtype="pmpid" link="fulltext">16336546</pubid>
                  <pubid idtype="doi">10.1111/j.1475-6773.2005.00451.x</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B73">
            <title>
               <p>Helicobacter pylori-related disease: guidelines for testing and treatment</p>
            </title>
            <aug>
               <au>
                  <snm>Peterson</snm>
                  <fnm>WL</fnm>
               </au>
               <au>
                  <snm>Fendrick</snm>
                  <fnm>AM</fnm>
               </au>
               <au>
                  <snm>Cave</snm>
                  <fnm>DR</fnm>
               </au>
               <au>
                  <snm>Peura</snm>
                  <fnm>DA</fnm>
               </au>
               <au>
                  <snm>Garabedian-Ruffalo</snm>
                  <fnm>SM</fnm>
               </au>
               <au>
                  <snm>Laine</snm>
                  <fnm>L</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2000</pubdate>
            <volume>160</volume>
            <fpage>1285</fpage>
            <lpage>1291</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.160.9.1285</pubid>
                  <pubid idtype="pmpid" link="fulltext">10809031</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B74">
            <title>
               <p>Consensus conference. Medical treatment of peptic ulcer disease. Practice guidelines. Practice Parameters Committee of the American College of Gastroenterology.[erratum appears in JAMA 1996 May 1;275(17):1314]</p>
            </title>
            <aug>
               <au>
                  <snm>Soll</snm>
                  <fnm>AH</fnm>
               </au>
            </aug>
            <source>JAMA</source>
            <pubdate>1996</pubdate>
            <volume>275</volume>
            <fpage>622</fpage>
            <lpage>629</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/jama.275.8.622</pubid>
                  <pubid idtype="pmpid">8594244</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B75">
            <title>
               <p>A randomized controlled trial of test-and-treat strategy for Helicobacter pylori: clinical outcomes and health care costs in a managed care population receiving long-term acid suppression therapy for physician-diagnosed peptic ulcer disease</p>
            </title>
            <aug>
               <au>
                  <snm>Allison</snm>
                  <fnm>JE</fnm>
               </au>
               <au>
                  <snm>Hurley</snm>
                  <fnm>LB</fnm>
               </au>
               <au>
                  <snm>Hiatt</snm>
                  <fnm>RA</fnm>
               </au>
               <au>
                  <snm>Levin</snm>
                  <fnm>TR</fnm>
               </au>
               <au>
                  <snm>Ackerson</snm>
                  <fnm>LM</fnm>
               </au>
               <au>
                  <snm>Lieu</snm>
                  <fnm>TA</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2003</pubdate>
            <volume>163</volume>
            <fpage>1165</fpage>
            <lpage>1171</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.163.10.1165</pubid>
                  <pubid idtype="pmpid" link="fulltext">12767952</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B76">
            <title>
               <p>A randomized trial of an acid-peptic disease management program in a managed care environment</p>
            </title>
            <aug>
               <au>
                  <snm>Ofman</snm>
                  <fnm>JJ</fnm>
               </au>
               <au>
                  <snm>Segal</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Russell</snm>
                  <fnm>WL</fnm>
               </au>
               <au>
                  <snm>Cook</snm>
                  <fnm>DJ</fnm>
               </au>
               <au>
                  <snm>Sandhu</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Maue</snm>
                  <fnm>SK</fnm>
               </au>
               <au>
                  <snm>Lowenstein</snm>
                  <fnm>EH</fnm>
               </au>
               <au>
                  <snm>Pourfarzib</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Blanchette</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Ellrodt</snm>
                  <fnm>G</fnm>
               </au>
               <au>
                  <snm>Weingarten</snm>
                  <fnm>SR</fnm>
               </au>
            </aug>
            <source>American Journal of Managed Care 9(6):425-33,</source>
            <pubdate>2003</pubdate>
         </bibl>
         <bibl id="B77">
            <title>
               <p>Adherence to medication</p>
            </title>
            <aug>
               <au>
                  <snm>Osterberg</snm>
                  <fnm>L</fnm>
               </au>
               <au>
                  <snm>Blaschke</snm>
                  <fnm>T</fnm>
               </au>
            </aug>
            <source>New England Journal of Medicine 353(5):487-97,</source>
            <pubdate>2005</pubdate>
         </bibl>
         <bibl id="B78">
            <title>
               <p>Interventions to improve antibiotic prescribing practices in ambulatory care</p>
            </title>
            <aug>
               <au>
                  <snm>Arnold</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Straus</snm>
                  <fnm>SE</fnm>
               </au>
            </aug>
            <source>Cochrane Database of Systematic Reviews (4):CD003539,</source>
            <pubdate>2005</pubdate>
         </bibl>
         <bibl id="B79">
            <title>
               <p>Local opinion leaders: effects on professional practice and health care outcomes. [Review]</p>
            </title>
            <aug>
               <au>
                  <snm>Thomson O'Brien</snm>
                  <fnm>MA</fnm>
               </au>
               <au>
                  <snm>Oxman</snm>
                  <fnm>AD</fnm>
               </au>
               <au>
                  <snm>Haynes</snm>
                  <fnm>RB</fnm>
               </au>
               <au>
                  <snm>Davis</snm>
                  <fnm>DA</fnm>
               </au>
               <au>
                  <snm>Freemantle</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>Harvey</snm>
                  <fnm>EL</fnm>
               </au>
            </aug>
            <source>Cochrane Database of Systematic Reviews (2):CD000125,</source>
            <pubdate>2000</pubdate>
         </bibl>
         <bibl id="B80">
            <title>
               <p>Randomised trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care</p>
            </title>
            <aug>
               <au>
                  <snm>Simon</snm>
                  <fnm>GE</fnm>
               </au>
               <au>
                  <snm>VonKorff</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Rutter</snm>
                  <fnm>C</fnm>
               </au>
               <au>
                  <snm>Wagner</snm>
                  <fnm>E</fnm>
               </au>
            </aug>
            <source>BMJ</source>
            <pubdate>2000</pubdate>
            <volume>320</volume>
            <fpage>550</fpage>
            <lpage>554</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">27299</pubid>
                  <pubid idtype="pmpid" link="fulltext">10688563</pubid>
                  <pubid idtype="doi">10.1136/bmj.320.7234.550</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B81">
            <title>
               <p>Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. [Review]</p>
            </title>
            <aug>
               <au>
                  <snm>Kawamoto</snm>
                  <fnm>K</fnm>
               </au>
               <au>
                  <snm>Houlihan</snm>
                  <fnm>CA</fnm>
               </au>
               <au>
                  <snm>Balas</snm>
                  <fnm>EA</fnm>
               </au>
               <au>
                  <snm>Lobach</snm>
                  <fnm>DF</fnm>
               </au>
            </aug>
            <source>BMJ</source>
            <pubdate>2005</pubdate>
            <volume>330</volume>
            <fpage>765</fpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">555881</pubid>
                  <pubid idtype="pmpid" link="fulltext">15767266</pubid>
                  <pubid idtype="doi">10.1136/bmj.38398.500764.8F</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B82">
            <title>
               <p>Are incentive-based formularies inversely associated with drug utilization in managed care?</p>
            </title>
            <aug>
               <au>
                  <snm>Gleason</snm>
                  <fnm>PP</fnm>
               </au>
               <au>
                  <snm>Gunderson</snm>
                  <fnm>BW</fnm>
               </au>
               <au>
                  <snm>Gericke</snm>
                  <fnm>KR</fnm>
               </au>
            </aug>
            <source>Ann Pharmacother</source>
            <pubdate>2005</pubdate>
            <volume>39</volume>
            <fpage>339</fpage>
            <lpage>345</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1345/aph.1E380</pubid>
                  <pubid idtype="pmpid" link="fulltext">15644478</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B83">
            <title>
               <p>Benefits and risks of increasing restrictions on access to costly drugs in Medicaid</p>
            </title>
            <aug>
               <au>
                  <snm>Soumerai</snm>
                  <fnm>SB</fnm>
               </au>
            </aug>
            <source>Health Affairs</source>
            <pubdate>2004</pubdate>
            <volume>23</volume>
            <fpage>135</fpage>
            <lpage>146</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1377/hlthaff.23.1.135</pubid>
                  <pubid idtype="pmpid" link="fulltext">15002636</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B84">
            <title>
               <p>Evaluating the impact of pharmacists in mental health: a systematic review</p>
            </title>
            <aug>
               <au>
                  <snm>Finley</snm>
                  <fnm>PR</fnm>
               </au>
               <au>
                  <snm>Crismon</snm>
                  <fnm>ML</fnm>
               </au>
               <au>
                  <snm>Rush</snm>
                  <fnm>AJ</fnm>
               </au>
            </aug>
            <source>Pharmacotherapy</source>
            <pubdate>2003</pubdate>
            <volume>23</volume>
            <fpage>1634</fpage>
            <lpage>1644</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1592/phco.23.15.1634.31952</pubid>
                  <pubid idtype="pmpid">14695043</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B85">
            <title>
               <p>Clinical pharmacists and inpatient medical care: a systematic review. [Review]</p>
            </title>
            <aug>
               <au>
                  <snm>Kaboli</snm>
                  <fnm>PJ</fnm>
               </au>
               <au>
                  <snm>Hoth</snm>
                  <fnm>AB</fnm>
               </au>
               <au>
                  <snm>McClimon</snm>
                  <fnm>BJ</fnm>
               </au>
               <au>
                  <snm>Schnipper</snm>
                  <fnm>JL</fnm>
               </au>
            </aug>
            <source>Archives of Internal Medicine</source>
            <pubdate>2006</pubdate>
            <volume>166</volume>
            <fpage>955</fpage>
            <lpage>964</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1001/archinte.166.9.955</pubid>
                  <pubid idtype="pmpid" link="fulltext">16682568</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B86">
            <title>
               <p>Improving adherence to antidepressants: a systematic review of interventions. [Review]</p>
            </title>
            <aug>
               <au>
                  <snm>Vergouwen</snm>
                  <fnm>AC</fnm>
               </au>
               <au>
                  <snm>Bakker</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Katon</snm>
                  <fnm>WJ</fnm>
               </au>
               <au>
                  <snm>Verheij</snm>
                  <fnm>TJ</fnm>
               </au>
               <au>
                  <snm>Koerselman</snm>
                  <fnm>F</fnm>
               </au>
            </aug>
            <source>Journal of Clinical Psychiatry</source>
            <pubdate>2003</pubdate>
            <volume>64</volume>
            <fpage>1415</fpage>
            <lpage>1420</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">14728101</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B87">
            <title>
               <p>Interventions used in disease management programmes for patients with chronic illness-which ones work? Meta-analysis of published reports.[Review]</p>
            </title>
            <aug>
               <au>
                  <snm>Weingarten</snm>
                  <fnm>SR</fnm>
               </au>
               <au>
                  <snm>Henning</snm>
                  <fnm>JM</fnm>
               </au>
               <au>
                  <snm>Badamgarav</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Knight</snm>
                  <fnm>K</fnm>
               </au>
               <au>
                  <snm>Hasselblad</snm>
                  <fnm>V</fnm>
               </au>
               <au>
                  <snm>Gano</snm>
                  <fnm>A</fnm>
                  <suf>Jr.</suf>
               </au>
               <au>
                  <snm>Ofman</snm>
                  <fnm>JJ</fnm>
               </au>
            </aug>
            <source>BMJ</source>
            <pubdate>2002</pubdate>
            <volume>325</volume>
            <fpage>925</fpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">130055</pubid>
                  <pubid idtype="pmpid" link="fulltext">12399340</pubid>
                  <pubid idtype="doi">10.1136/bmj.325.7370.925</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
      </refgrp>
      <sec>
         <st>
            <p>Pre-publication history</p>
         </st>
         <p>The pre-publication history for this paper can be accessed here:</p>
         <p>
            <url>http://www.biomedcentral.com/1472-6963/8/75/prepub</url>
         </p>
      </sec>
   </bm>
</art>
