You are viewing the site in preview mode

Skip to main content

Advertisement

Decision support during electronic prescription to stem antibiotic overuse for acute respiratory infections: a long-term, quasi-experimental study

Abstract

Background

Interventions to support decision-making can reduce inappropriate antibiotic use for acute respiratory infections (ARI), but they may not be sustainable. The objective of the study is to evaluate the long-term effectiveness of a clinical decision-support system (CDSS) interposed at the time of electronic (e-) prescriptions for selected antibiotics.

Methods

This is a retrospective, observational intervention study, conducted within a large, statewide Veterans Affairs health system. Participants are outpatients with an initial visit for ARI. A CDSS was deployed upon e-prescription of selected antibiotics during the study period. From 01/2004 to 05/2006 (pre-withdrawal period), the CDSS targeted azithromycin and the fluoroquinolone gatifloxacin. From 05/2006 to 12/2011 (post-withdrawal period), the CDSS was retained for azithromycin but withdrawn for the fluoroquinolone. A manual record review was conducted to determine concordance of antibiotic prescription with ARI treatment guidelines.

Results

Of 1131 included ARI visits, 380 (33.6%) were guideline-concordant. For azithromycin, concordance did not change between the pre- and post-withdrawal periods, and adjusted odds of concordance was 8.8 for the full study period, compared to unrestricted antibiotics. For fluoroquinolones, guideline concordance decreased from 88.6% (39 of 44 visits) to 51.3% (59 of 115 visits), pre- vs. post-withdrawal periods (p < 0.005). The adjusted odds of concordance compared to “All Other Antibiotics” visits decreased from 24.4 (95% CI 9.0–66.3) pre-withdrawal to 5.5 (95% CI 3.5–8.8) post-withdrawal (p = .008). Concordance did not change between those same time periods for antibiotics that were never subjected to the intervention (“All Other Antibiotics”).

Conclusions

A CDSS interposed at the time of e-prescription of selected antibiotics can shift their use toward ARI treatment guidelines, and this effect can be maintained over the long term as long as the CDSS remains in place. Removal of the CDSS after 3.5 years of implementation resulted in a rise in guideline-discordant antibiotic use.

Background

Antibiotic overuse promotes the emergence of resistant microorganisms [1] and contributes to preventable morbidity and costs [2]. In outpatient settings, most antibiotic prescriptions are directed at acute respiratory infections (ARI) [3]. Misutilization of antibiotics for uncomplicated ARI remains common, years after publication of widely endorsed prescribing guidelines [4]. The rationale for the persistence of the problem is not fully understood, and may include lack of knowledge of the treatment guidelines, [5] lack of appreciation for the development of resistance, [6] and pressure to expediently satisfy perceived patient expectations [6]. Provider education can decrease needless antibiotic prescriptions for ARI, but no information delivery method has proven consistently effective, scalable and sustainable [7, 8].

The broad deployment of electronic medical records (EMR) is opening new educational avenues to medical providers. EMR-based clinical decision support systems (CDSS) can be integrated into the workflow and change clinical practice by providing timely, guideline-based, patient-specific recommendations [9]. A CDSS with a light footprint could conceivably remain in place indefinitely [10]. However, our knowledge on how to assure that such a tool remains useful over the long-term is still maturing.

In an effort to improve outpatient antibiotic stewardship at a large, statewide Veterans Affairs health system, a CDSS was interposed at the time of electronic order entry for two antibiotics commonly used for ARI: gatifloxacin, the mainstream formulary respiratory fluoroquinolone when it was available, and azithromycin. This intervention nearly eliminated unwarranted ARI prescriptions for the two agents for a period of at least 3.5 years [11]. When gatifloxacin was withdrawn from the market in May 2006 due to reports of dysglycemias [12], we considered the possibility that, after 3.5 years, the guideline information delivered by the CDSS with each attempt to prescribe an antibiotic had become assimilated into our standard of practice, thereby rendering the CDSS unnecessary. To gain insight into this possibility, we did not reapply the ARI CDSS to moxifloxacin, the formulary replacement for gatifloxacin. Meanwhile, the CDSS for azithromycin was left in place for a period of more than 4.5 years. In the current work, we take advantage of these conditions to simultaneously ask if the CDSS retained its effectiveness over the long-term and if its effect on decreasing antibiotic overuse for ARI could persist despite withdrawing the intervention.

Methods

This is a retrospective, observational study designed to assess the long-term effects of CDSS use and the consequence of CDSS withdrawal on concordance of antibiotic usage with widely-endorsed ARI treatment guidelines. The study was performed using medical record data from the Veterans Affairs Maryland Health Care System (VAMHCS) during a period extending from January 1, 2004, to December 31, 2011. The ARI CDSS was prompted each time an attempt was made to electronically prescribe azithromycin for the entire study period, or gatifloxacin from the beginning of the study period until May 2006 (pre-withdrawal period), when gatifloxacin was removed from the market. The CDSS did not target other antibiotics, including moxifloxacin which replaced gatifloxacin on the formulary after May 2006 (post-withdrawal period). Only one oral formulation of a respiratory fluoroquinolone was listed on the national VA formulary at any given time, with minimal overlap of listing during transitions. No non-formulary respiratory fluoroquinolones, such as levofloxacin, were prescribed for any of the ARI visits included in our study sample.

Description of the intervention

The ARI CDSS has been previously described [11]. For targeted antibiotics, the CDSS was deployed when electronic (e-) prescription was attempted. For azithromycin and gatifloxacin, the CDSS included treatment paths for the following diseases: community-acquired pneumonia, acute bronchitis, acute sinusitis, non-specific acute respiratory infection (ARI), and exacerbations of chronic obstructive pulmonary disease (COPD). An “Other” option was included for provider-supplied rationales. The CDSS used data mined from the EMR and information entered by the provider to assess whether an antibiotic prescription would be consistent with published ARI guidelines. Diagnostic criteria and antibiotic prescribing criteria have been previously defined [11]. For appropriate rationales, the CDSS led to order entry for the antibiotic and generated documentation describing why the drug was being used. If antibiotics could be safely withheld, the CDSS did not lead to a prescription. Instead, it provided guidance on how to maintain patient satisfaction when withholding antibiotics. For both safety and tolerability, providers could override the system by providing their own rationale in the “Other” pathway or by prescribing antibiotics not subject to the CDSS.

Study participants

An automated case-detection algorithm (CDA) previously found to identify 73% of patients with ARI was applied to EMR-derived relational databases [13]. The algorithm flagged outpatient visits if providers assigned an ARI-related diagnostic code OR prescribed a cough suppressant AND if the clinical notes documented at least two ARI symptoms, as assessed by automated text analysis [13].

Record review

A record review was completed for 1) all flagged visits associated with azithromycin, gatifloxacin, or moxifloxacin prescription, 2) a random sample of all flagged visits associated with “All Other Antibiotics”, limited to 50% of flagged visits to retain feasibility of the manual record review, and 3) a random sample of 200 flagged visits where no antibiotics were given, as such visits had rarely been found to be discordant in the past [11].

Sampled flagged visits were manually abstracted for data elements needed to assign diagnosis and treatment for pneumonia, acute bronchitis, or sinusitis [11]. Three independent reviewers, including a pulmonary medicine specialist, examined all free-text EMR entries on the day of index for the flagged visits, with the exception of notes issued by the CDSS itself. Presence or absence of individual ARI symptoms and related time courses were abstracted. A stated diagnosis of pneumonia or inclusion of pneumonia in the list of differential diagnoses was coded as pneumonia. Chest imaging was not reviewed. Reviewers cross-validated 10% of each other’s work. Inter-rater reliability was determined for: a) the application of inclusion/exclusion criteria (kappa = 81.0%); and b) for the final assessment of concordance of antibiotic prescribing with guidelines (kappa = 85.5%).

Exclusion, diagnostic and concordance criteria

After the automated CDA identified possible ARI visits, pre-determined exclusion criteria were applied using manually abstracted information. These criteria included: 1) Stated diagnosis of COPD; 2) Did not meet case definition for outpatient sinusitis, acute bronchitis, or pneumonia; 3) Not an initial visit for a given ARI episode; 4) No provider documentation; 5) Antibiotic unequivocally prescribed for a non-ARI diagnosis; 6) Records could not be accessed for review. Cases of pharyngitis were excluded from our study according to criteria 2 because it was not an available pathway for the CDSS and because the CDSS-targeted antibiotics were found not to be misused in patients whose only ARI diagnosis was pharyngitis [11].

Abstracted information for each visit was reassembled into ARI diagnoses and visit concordance was determined according to the previously described case definitions and treatment guidelines [11]. Except for pneumonia, ARI diagnoses were not mutually exclusive. A visit was considered “concordant” if antibiotic use followed guidelines for at least one ARI diagnosis.

Antibiotic usage

The Veterans Affairs Pharmacy Benefits Management Services database was queried to compare national to VAMHCS usage of azithromycin and fluoroquinolones (ProClarity software, Microsoft Corp, Redmond CA).

Statistical analysis

Based on the accuracy of the screening case-detection algorithm [13] and allowing for attrition due to misclassification, we calculated that abstracting 1800 flagged outpatient encounters would result in at least an 80% power to detect a 10% change in concordance after CDSS withdrawal.

The potential confounders for the relationship between study time periods and guideline concordance were identified using T-tests for continuous variables and Chi-square tests for categorical variables. Patient-specific covariates were sex, self-reported race, age, and documented ARI symptoms. Covariates were included in the multivariable logistic regression if they were found to have an interaction, according to standard practice. Multivariable logistic regression and odds ratios were calculated to determine the association between CDSS withdrawal and guideline concordant antibiotic prescribing (SAS software v. 9.2, SAS Institute Inc., Cary, NC). A chi-square test was used to compare guideline concordance between pre-withdrawal and post-withdrawal periods. A p-value of .05 or less was considered statistically significant.

Results

Study population

Over the 8 year study period, the ARI CDA flagged 7745 visits. A total of 1759 visits were reviewed, including all visits during which azithromycin, gatifloxacin, or moxifloxacin was prescribed (n = 455). Of the 1759 visits manually reviewed, 628 were excluded according to the pre-defined criteria. The study included 1131 visits for 1118 patients with an initial visit for ARI. These results are summarized in Fig. 1.

Fig. 1
figure1

Study flow diagram. The population was primarily male (82.5%) and older. Cough was the most common ARI symptom, followed by sputum production and fever/chills/night sweats. ARI symptoms were similar whether or not antibiotic usage was concordant with guidelines (Table 1)

Concordance by diagnosis

Of the 1131 total ARI visits, 380 (33.6%) were guideline concordant, including 285 visits that resulted in an antibiotic prescription for ARI, as well as 95 of the 96 ARI visits where no antibiotics were given. Providers diagnosed pneumonia in 227 visits (20% of all ARI visits). All pneumonia patients received antibiotics, which constituted the majority of the study’s concordant visits (n = 227/380 or 59.7%). Of the 751 discordant ARI visits, 750 (99.9%) visits were discordant because antibiotics were prescribed when they were not warranted. In only one visit were antibiotics not prescribed when the guidelines suggested that they should be considered, a patient with acute sinusitis who had experienced sinus pain and/or tenderness and purulent nasal drainage for 7 or more days. Acute bronchitis was the sole diagnosis in 467 (62.2%) discordant visits, and sinusitis was the sole diagnosis in 31 (4.1%) discordant visits; 252 discordant visits (33.6%) met criteria for both acute bronchitis and sinusitis (Table 1).

Table 1 Characteristics of study visits. Characteristics include patient demographics, symptoms present during the index visit, and ARI conditions for which diagnostic criteria were met

CDSS exposure is associated with guideline concordance

The CDSS was applied in 130 visits leading to azithromycin prescriptions during the whole study period, 81 (62.3%) of which were concordant, and in 44 visits leading to fluoroquinolone (gatifloxacin) prescriptions during the pre-withdrawal period, 39 (88.6%) of which were concordant (Table 2). By comparison, when visits resulted in the prescription of antibiotics that were never subjected to the CDSS during the study period (“All Other Antibiotics”), 106 of 746 (14.2%) were concordant. The adjusted odds of concordance were 8.8 (95% CI 5.7–13.6) for ARI visits leading to azithromycin prescriptions and 24.4 (95% CI 9.0–66.3) for ARI visits leading to gatifloxacin prescriptions when compared to visits for which “All Other Antibiotics” were prescribed. The CDSS was discontinued for 115 visits leading to fluoroquinolone prescriptions during the post-withdrawal period, and adjusted odds of guideline concordance was 5.5 times higher (95% CI 3.5–8.8) than for all visits from the group “All Other Antibiotics”.

Table 2 Odds of guideline concordant prescription for ARI and CDSS exposure

CDSS withdrawal decreases concordance

The CDSS targeted the fluoroquinolone gatifloxacin during the 3.5 year pre-withdrawal period. The CDSS was not implemented for the fluoroquinolone moxifloxacin for the 4.5 year post-withdrawal period. For those fluoroquinolones, guideline concordance decreased from 88.6% (39 of 44 visits) during the pre-withdrawal period to 51.3% (59 of 115 visits) post-withdrawal (p = .002). The corresponding adjusted odds of concordance compared to “All Other Antibiotics” visits decreased from 24.4 (95% CI 9.0–66.3) pre-withdrawal to 5.5 (95% CI 3.5–8.8) post-withdrawal (p = .008). For azithromycin, the CDSS remained in place for the entire 8 year study period, and there was no significant difference in guideline concordance between the pre-withdrawal period (49/69 or 71.0%) and the post-withdrawal period (32/61 or 52.5%). There was also no significant difference in concordance for the other control group, “All Other Antibiotics”, between the pre-withdrawal period (44/268 or 16.4%) and the post-withdrawal period (62/478 or 13.0%).

CDSS withdrawal increases fluoroquinolone use

VA pharmacy antibiotic usage is reported as a ratio comparing the total number of prescriptions for a single antibiotic and the total number of prescriptions for all drugs per quarter. During the pre-withdrawal period, VAMHCS usage was lower than national VA usage for both azithromycin (Fig. 2, solid black versus dashed black lines) and gatifloxacin (solid gray versus dashed gray lines). In the post-withdrawal period, average use of azithromycin remained lower at VAMHCS than nationwide (solid vs. dashed black line). Nationally, fluoroquinolone use decreased from the pre- to the post-withdrawal period (dashed gray line). Within VAMHCS, the trend was in the opposite direction, with usage of moxifloxacin rising (solid gray line) to ultimately match national usage.

Fig. 2
figure2

Comparison of national and Maryland VA usage of CDSS targeted antibiotics. The ratio (Number of prescriptions / All drug prescriptions) is multiplied by 100 and the percentage is reported on the y-axis. The vertical dotted line indicates when moxifloxacin replaces gatifloxacin in the VA formulary and defines the pre- and post-withdrawal periods. Note the marked seasonality of usage for azithromycin and the fluoroquinolones

Out of 381 visits during the pre-withdrawal period, 39 (10.2%) were associated with guideline concordant fluoroquinolone use, and 5 (1.3%) were associated with non-guideline concordant fluoroquinolone use. Had this relative utilization been maintained with the 654 visits in the post-withdrawal period, we would have expected 67 (10.2% of 654) visits with guideline concordant and 9 (1.3% of 654) visits with non-guideline concordant fluoroquinolone prescriptions. In actuality, there were fewer than expected guideline concordant prescriptions (59 out of 654 or 9.0%) and much more than expected guideline discordant prescriptions (56 out of 654 or 8.6%). These data suggest that the observed increase in fluoroquinolone usage shown in Fig. 2 could be accounted for by an increase in discordant prescriptions.

Discussion and conclusions

We studied the long-term effect of a CDSS interposed at the time of e-prescription on concordance with ARI treatment guidelines. The sustained effectiveness of the CDSS was demonstrated in two ways. First, over an implementation period of 3.5 years (for gatifloxacin) to 8 years (for azithromycin), the proportion of concordant visits was higher with the CDSS than without. Second, concordance decreased when the CDSS was withdrawn for the fluoroquinolone, without a corresponding change for antibiotics either continuously (azithromycin) or never subjected to the intervention. Together, these data indicate that a CDSS at the time of prescription can be an effective, long-term strategy for curtailing misuse of the drugs that it targets. This effect may not persist after the CDSS is removed, but CDSS can be designed to be a durable part of provider workflow.

We took several steps to strengthen this study’s quasi-experimental design: a) to minimize sampling bias, we obtained a population-based sample of ARI visits through a previously validated ARI CDA; b) we used a manual reference review and explicit criteria to identify ARI diagnoses and concordance with treatment guidelines; c) we compared the change in guideline concordance when the CDSS was withdrawn against two control groups, one continuously exposed to the CDSS during the whole study period (azithromycin), and one never exposed to the intervention (“All Other Antibiotics”); d) we did not implement complementary educational interventions, thereby increasing the likelihood that the observed effects on prescribing practices could be attributed to the CDSS itself; and e) we compared VAMHCS to national drug utilization to show that the local increase in fluoroquinolone use following CDSS withdrawal ran contrary to national trends, and thus was unlikely to simply be the result of perceived advantages of moxifloxacin over gatifloxacin. Our findings of stable azithromycin use and increased fluoroquinolone use during the post-withdrawal period also ran contrary to those observed in a national study of one million VA ARI visits, where azithromycin usage increased and fluoroquinolone usage decreased in 2012 compared to 2005 [14].

The sustainability of the CDSS intervention distinguishes it from other attempts to address antibiotic stewardship for ARI. Financial incentives, clinician workshops, or audits with providers can all decrease unnecessary antibiotics for ARI [15,16,17,18,19]. Because these interventions are resource intensive, they usually cannot be continued indefinitely. The durability of their effect therefore depends upon changing the antibiotic prescribing culture. Of the three published examples of outpatient interventions whose effect was maintained for multiple years, two featured voluntary participation and may thus have selected for particularly motivated providers [20, 21]. Another was conducted within small, stable practices, i.e. in environments expected to be amenable to cultural changes [22]. Our results suggest that it is possible to foster the judicious use of antibiotics in health care settings that experience a continuous influx of new trainees and part-time staff, i.e. where it may be difficult to provide timely information about prescribing guidelines through conventional means [16]. Our results suggests that the effectiveness of the CDSS required its continued implementation. This is nevertheless possible because as the CDSS becomes an integral part of the workflow and providers become familiar with its informational content, it ultimately consumes minimal resources,

We do not know what propelled the rate of discordant antibiotic prescriptions once the CDSS was discontinued. Some providers may not have assimilated the ARI treatment guidelines, either because of insufficient CDSS exposures, or because they limited their interaction with the CDSS to what was strictly needed to generate a prescription thus missing the scientific arguments that would lead to a permanent change in behavior. Other providers may have internalized the content of the CDSS, but this knowledge may not have been sufficient to maintain concordant practices in the presence of external pressures such as time expediency, patient or subspecialist demands, or ongoing performance measures of patient satisfaction [23]. Given these realities, the process-of-care into which the CDSS operated included non-informational elements that may indeed prove crucial to antibiotic management, such as providing implicit institutional support for the decision to withhold antibiotics, or requiring providers to commit a written justification for the use of antibiotics to the EMR, where it is potentially vulnerable to subsequent peer review.

Our study has some limitations that warrant consideration. The biases associated with a pre−/post- intervention study design have been described [24]. The intervention was deployed in a veteran population that is mostly male, is older, and has more co-morbidities than the general population [25]. Providers within the VA system have long used the EMR for virtually all aspects of patient care, and this familiarity may have an impact on the acceptance and effectiveness of the CDSS. We did not formally reassess the performance characteristics of the ARI CDA. However, the CDA yielded a validated ARI case density commensurate with past performance measurements. Moreover, the structured components of the algorithm (cough suppressant prescriptions, ARI diagnostic codes) are not expected to be susceptible to changes in language patterns. In the unlikely possibility that the free-text describing common respiratory symptoms changed over the years, there is no reason to believe that these changes would systematically create a bias for or against concordance for any specific antibiotic group. The retrospective data collection meant that symptoms and signs were assumed to be absent if they were not documented in the medical record. ARI episodes could therefore have been missed or mislabeled, along with the rationale justifying the use of antibiotics. Our study focused mostly on ARI visits for which antibiotics were prescribed, and so could not assess possible shunting of utilization between different antibiotics. However, a prior assessment that included all ARI visits indicated that the CDSS intervention did not transfer inappropriate use from the CDSS-targeted agents to alternative antibiotics, nor did it promote the diagnosis of pneumonia to justify prescribing antibiotics [11]. If we excluded visits where the antibiotic was clearly prescribed for a non-ARI diagnosis, we did not evaluate how frequently providers supplied rationales other than those provided by ARI guidelines to prescribe CDSS-targeted agents. We limited CDSS targeting to azithromycin and fluoroquinolones, because they were the two most commonly prescribed antibiotics for ARI and were not commonly used for diseases other than ARI [11]. Different effect sizes may therefore have been observed if different or additional antibiotics had been targeted, or if the CDSS was implemented in health systems with different prescribing patterns [14, 16]. Our study did not collect information about emergence of resistant organisms, clinical efficacy or patient safety. However, no patient with provider-assigned diagnosis of pneumonia failed to receive antibiotics, a finding consistent with our past work with 2125 patients [11]. Other studies have demonstrated no increases in patient follow-up visits, need for antibiotic prescription after the index visit, or mortality in cohorts where antibiotic stewardship programs successfully decreased over-prescribing for ARI [26,27,28,29]. Taken together, these limitations represent design opportunities for future studies aimed at extending e-prescription-based approaches to other antibiotics, drugs, diseases or settings.

There is a continued need for better methods to promote responsible antibiotic use, and any intervention must match the scale and historical resilience of the problem. A CDSS systematically interposed at the time of e-prescription provided significant and durable improvements in the use of antibiotics for outpatients with ARI. The approach could be adapted to optimize the use of any medication and take advantage of the increasing integration of the EMR in medical practices.

Abbreviations

ARI:

Acute respiratory infection

CDA:

Case-detection algorithm

CDSS:

Clinical decision-support system

CI:

Confidence interval

COPD:

Chronic obstructive pulmonary disease

EMR:

Electronic medical record

VAMHCS:

Veterans Affairs Maryland Health Care System

References

  1. 1.

    Costelloe C, Metcalfe C, Lovering A, Mant D, Hay AD. Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis. BMJ. 2010;340:c2096.

  2. 2.

    MacDougall C, Polk RE. Antimicrobial stewardship programs in health care systems. Clin Microbiol Rev. 2005;18(4):638–56.

  3. 3.

    Lindbaek M. Prescribing antibiotics to patients with acute cough and otitis media. Brit J Gen Prac. 2006;56(524):164–6.

  4. 4.

    Barnett ML, Linder JA. Antibiotic prescribing to adults with sore throat in the United States, 1997-2010. JAMA Intern Med. 2014;174(1):138–40.

  5. 5.

    Linder JA, Schnipper JL, Tsurikova R, Volk LA, Middleton B. Self-reported familiarity with acute respiratory infection guidelines and antibiotic prescribing in primary care. Int J Qual Health C. 2010;22(6):469–75.

  6. 6.

    Lopez-Vazquez P, Vazquez-Lago JM, Figueiras A. Misprescription of antibiotics in primary care: a critical systematic review of its determinants. J Eval Clin Pract. 2012;18(2):473–84.

  7. 7.

    Steinman MA, Gonzales R, Linder JA, Landefeld CS. Changing use of antibiotics in community-based outpatient practice, 1991-1999. Ann Intern Med. 2003;138(7):525–33.

  8. 8.

    Vanderweil SG, Pelletier AJ, Hamedani AG, Gonzales R, Metlay JP, Camargo CA Jr. Declining antibiotic prescriptions for upper respiratory infections, 1993-2004. Acad Emerg Med. 2007;14(4):366–9.

  9. 9.

    Ebell M. AHRQ white paper: use of clinical decision rules for point-of-care decision support. Med Decis Mak. 2010;30(6):712–21.

  10. 10.

    Jenders RA, Osheroff JA, Sittig DF, Pifer EA, Teich JM. Recommendations for clinical decision support deployment: synthesis of a roundtable of medical directors of information systems. AMIA Annual Symposium Proceedings 2007:359-63.

  11. 11.

    Rattinger GB, Mullins CD, Zuckerman IH, Onukwugha E, Walker LD, Gundlapalli A, et al. A sustainable strategy to prevent misuse of antibiotics for acute respiratory infections. PLoS One. 2012;7(12):e51147.

  12. 12.

    Park-Wyllie LY, Juurlink DN, Kopp A, Shah BR, Stukel TA, Stumpo C, et al. Outpatient gatifloxacin therapy and dysglycemia in older adults. N Engl J Med. 2006;354(13):1352–61.

  13. 13.

    DeLisle S, South B, Anthony JA, Kalp E, Gundlapallli A, Curriero FC, et al. Combining free text and structured electronic medical record entries to detect acute respiratory infections. PLoS One. 2010;5(10):e13377.

  14. 14.

    Jones BE, Sauer B, Jones MM, Campo J, Damal K, He T, et al. Variation in outpatient antibiotic prescribing for acute respiratory infections in the veteran population: a cross-sectional study. Ann Intern Med. 2015;163(2):73–80.

  15. 15.

    Naughton C, Feely J, Bennett K. A RCT evaluating the effectiveness and cost-effectiveness of academic detailing versus postal prescribing feedback in changing GP antibiotic prescribing. J Eval Clin Pract. 2009;15(5):807–12.

  16. 16.

    Gerber JS, Prasad PA, Fiks AG, Localio AR, Grundmeier RW, Bell LM, et al. Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial. JAMA. 2013;309(22):2345–52.

  17. 17.

    Martens JD, Werkhoven MJ, Severens JL, Winkens RA. Effects of a behaviour independent financial incentive on prescribing behaviour of general practitioners. J Eval Clin Pract. 2007;13(3):369–73.

  18. 18.

    Gjelstad S, Hoye S, Straand J, Brekke M, Dalen I, Lindbaek M. Improving antibiotic prescribing in acute respiratory tract infections: cluster randomised trial from Norwegian general practice (prescription peer academic detailing (Rx-PAD) study). BMJ. 2013;347:f4403.

  19. 19.

    Grover ML, Nordrum JT, Mookadam M, Engle RL, Moats CC, Noble BN. Addressing antibiotic use for acute respiratory tract infections in an academic family medicine practice. Am J Med Qual. 2013;28(6):485–91.

  20. 20.

    Regev-Yochay G, Raz M, Dagan R, Roizin H, Morag B, Hetman S, et al. Reduction in antibiotic use following a cluster randomized controlled multifaceted intervention: the Israeli judicious antibiotic prescription study. Clin Infect Dis. 2011;53(1):33–41.

  21. 21.

    Litvin CB, Ornstein SM, Wessell AM, Nemeth LS, Nietert PJ. Use of an electronic health record clinical decision support tool to improve antibiotic prescribing for acute respiratory infections: the ABX-TRIP study. J Gen Intern Med. 2013;28(6):810–6.

  22. 22.

    Cals JW, de Bock L, Beckers PJ, Francis NA, Hopstaken RM, Hood K, et al. Enhanced communication skills and C-reactive protein point-of-care testing for respiratory tract infection: 3.5-year follow-up of a cluster randomized trial. Ann Fam Med. 2013;11(2):157–64.

  23. 23.

    Aujesky D, McCausland JB, Whittle J, Obrosky DS, Yealy DM, Fine MJ. Reasons why emergency department providers do not rely on the pneumonia severity index to determine the initial site of treatment for patients with pneumonia. Clin Infect Dis. 2009;49(10):e100–8.

  24. 24.

    Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu J, Peterson DE, et al. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc. 2006;13(1):16–23.

  25. 25.

    Nelson KM, Starkebaum GA, Reiber GE. Veterans using and uninsured veterans not using veterans affairs (VA) health care. Public Health Rep. 2007;122(1):93–100.

  26. 26.

    Jenkins TC, Irwin A, Coombs L, Dealleaume L, Ross SE, Rozwadowski J, et al. Effects of clinical pathways for common outpatient infections on antibiotic prescribing. Am J Med. 2013;126(4):327–35. e12

  27. 27.

    McGinn TG, McCullagh L, Kannry J, Knaus M, Sofianou A, Wisnivesky JP, et al. Efficacy of an evidence-based clinical decision support in primary care practices: a randomized clinical trial. JAMA Intern Med. 2013;173(17):1584–91.

  28. 28.

    Gonzales R, Anderer T, McCulloch CE, Maselli JH, Bloom FJ Jr, Graf TR, et al. A cluster randomized trial of decision support strategies for reducing antibiotic use in acute bronchitis. JAMA Intern Med. 2013;173(4):267–73.

  29. 29.

    Samore MH, Bateman K, Alder SC, Hannah E, Donnelly S, Stoddard GJ, et al. Clinical decision support and appropriateness of antimicrobial prescribing: a randomized trial. JAMA. 2005;294(18):2305–14.

Download references

Acknowledgements

The authors declare financial support from the Department of Veterans Affairs (TQ,CF, and SD) and the Centers for Disease Control and Prevention (EV,KR,LW, and SD). The funding bodies had no role in the collection, analysis, and interpretation of data; in the writing of the manuscript; and in the decision to submit the manuscript for publication.

Funding

This material is the result of work supported with resources and the use of facilities at the VA Maryland Health Care System. This work was also supported in part by grants SHP-08-162 and IIR 06–119-1 from the Office of Research and Development, Health Services Research and Development Service, Department of Veterans Affairs. Components of the study involving evaluation of the effects of the CDSS and its removal were also supported by funding through the Maryland Department of Health and Mental Hygiene’s work with the Get Smart: Know When Antibiotics Work program, part of the Centers for Disease Control and Prevention’s Epidemiology and Laboratory Capacity for Infectious Diseases Cooperative Agreement.

Availability of data and materials

The dataset supporting the conclusions of this article is included within the article as an Additional file 1. The dataset headings are explained in Additional file 2.

Author information

SD had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. JG, EV, KR, TS, and SD conceived and designed the experiments. JG, EV, and KR performed the data abstraction. JG, EV, KR, CF, LW, and SD analyzed the data. JG, EV, KR, CF, LW, and SD wrote the paper. LW and SD obtained funding. JG, EV, TS, and SD provided administrative, technical, and/or material support. JG, EV, KR, LW, and SD provided study supervision. All authors read and approved the final manuscript.

Correspondence to Sylvain DeLisle.

Ethics declarations

Ethics approval and consent to participate

This project was approved by the VAMHCS Research & Development Committee, the Maryland Department of Health and Mental Hygiene (DHMH) Institutional Review Board (IRB), and the IRB for the University of Maryland, Baltimore (Protocol HP-00052745). Patient records and information were anonymized and de-identified prior to analysis. The IRB for the University of Maryland, Baltimore, specifically granted a waiver of consent as the research-related risks were limited to information confidentiality and because the study would not otherwise have been feasible given the large number of participants.

Consent for publication

Not applicable.

Competing interests

There were no financial relationships with any organizations that might have an interest in the submitted work in the previous 5 years, and no other relationships or activities that could appear to have influenced the submitted work.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Coded dataset for the study. (XLS 263 kb)

Additional file 2:

Legend for supplementary data file. (PDF 155 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark