- Research article
- Open Access
- Open Peer Review
Derivation and validation of a simple clinical bedside score (ATLAS) for Clostridium difficile infection which predicts response to therapy
BMC Infectious Diseasesvolume 13, Article number: 148 (2013)
Clostridium difficile infection (CDI) continues to be a frequent and potentially severe infection. There is currently no validated clinical tool for use at the time of CDI diagnosis to categorize patients in order to predict response to therapy.
Six clinical and laboratory variables, measured at the time of CDI diagnosis, were combined in order to assess their correlation with treatment response in a large CDI clinical trial database (derivation cohort). The final categorization scheme was chosen in order to maximize the number of categories (discrimination) while maintaining a high correlation with clinical cure assessed two days after the end of therapy. Validation of the derived scoring scheme was done on a second large CDI clinical trial database (validation cohort). A third comparison was done on the two pooled databases (pooled cohort).
In the derivation cohort, the best discrimination and correlation with cure was seen with a five-component ATLAS score (age, treatment with systemic antibiotics, leukocyte count, albumin and serum creatinine as a measure of renal function), which divided CDI patients into 11 groups (scores of 0 to 10 inclusive) and was highly correlated with treatment outcome (R2=0.95; P<0.001). This scheme showed excellent prediction of cure in the validation cohort (overall Kappa=95.2%; P<0.0001), as well as in the pooled cohort, regardless of treatment (fidaxomicin or vancomycin).
A combination of five simple and commonly available clinical and laboratory variables measured at the time of CDI diagnosis, combined into a scoring system (ATLAS), are able to accurately predict treatment response to CDI therapy. The ATLAS scoring system may be useful in stratifying CDI patients so that appropriate therapies can be chosen to maximize cure rates, as well as for categorization of patients in CDI therapeutic studies in order allow comparisons of patient groups.
Clostridium difficile infection (CDI) has emerged during the last decade as a serious and increasingly common healthcare-associated infection . The emergence of hypervirulent strains has resulted in elevated rates of CDI-related complications (e.g. colectomy, need for intensive care) and increased mortality, especially among the elderly [2, 3]. Newer treatment options, such as novel antibacterials , immune modulators  and immunotherapeutics , have led to a recent expansion in the number of clinical trials involving subjects with this serious infection. Despite more than 3 decades of research involving this condition, a validated “severity scale” has yet to be developed which correlates with treatment response, which is predictive of severe outcomes (i.e. colectomy, need for intensive care, or attributable mortality) or which predicts CDI recurrence. Although several predictive scoring systems and clinical variables have been described in limited CDI populations or case series, none have been validated on large CDI databases [7–16]. While the choice of therapy for malignant neoplasms and subjects with sepsis is often based on validated criteria which consist of both patient and disease-related variables in order to maximize treatment response [17, 18], no such validated scheme exists for CDI. The absence of such a categorization system means that the choice of therapies for a particular patient is often not evidence-based, and clinical trials investigating CDI outcomes may be comparing dissimilar patient populations. Recent guidelines which have put forward “severity categories” for CDI have not validated these categorizations and their correlation with treatment outcome, disease outcome and CDI recurrence are unknown [19, 20]. We have used 2 large clinical therapeutic trials for treating CDI in order to derive and then validate a categorization system to discriminate among CDI patients and correlate the grouping with treatment response.
Two large databases, which were derived while conducting therapeutic trials that compared fidaxomicin and vancomycin for the treatment of CDI, were used for the present analyses [4, 21]. The clinical and trial details for the two identical CDI therapy studies are described elsewhere . Briefly, 10 days of therapy with either vancomycin or fidaxomicin was administered to CDI patients. The first trial (“003”) enrolled patients in the United States and Canada; the second trial (“004”) enrolled patients in those two countries as well as in Europe. The response to treatment was assessed two days following the last day of therapy. Patients considered as a “cure” were then followed for an additional 28 days to evaluate them for a CDI recurrence. This present analyses used all patients included in each of the respective trials if they had a confirmed diagnosis of CDI and received at least 1 dose of study medication (“modified intent to treat” group; mITT). Since the vancomycin and fidaxomicin arms had nearly identical cure rates , all patients in each study were combined into a single group regardless of the therapy they were randomized to receive. The mITT group of patients consisted of 596 individuals in the 003 study, 509 individuals in the 004 study, and a total of 1105 subjects in both combined studies. All subjects in both studies gave informed consent which also allowed for secondary analyses of the databases such as in the present investigation. No additional form of ethical approval was required to do this subgroup risk analysis, as the original ethical approval for the trial covered such analyses. The study sponsor permitted the authors to access the trial data for this analysis; the dataset used was preexisting, de-identified and required no further collection of data from patients.
The six clinical and laboratory parameters used in the analyses were chosen for their ready availability, their ease of calculation, prior correlation with CDI outcome in case series [7–16], and the fact that they had been collected and were available in the two CDI clinical trials of interest. The six parameters, measured on the day of entry into the study (i.e. within 48 hours of a positive C. difficile toxin assay) were: age (in years) [Ag], treatment with systemic antibiotics (which occurred on one or more days of CDI therapy) [Tr], temperature (in degrees Celsius) [Te], total leukocyte count [L], serum albumin [Al], and serum creatinine as a measure of renal function [S]. A logistic regression model was created using “cure” as the dependant variable and the six clinical/lab parameters as the independent variables. Due to high co-correlation of some of the six independent variables (i.e. age, albumin and serum creatinine showing high Pearson correlation coefficients), it was decided instead to use these indices in a clinical score. Scores of 0, 1 or 2 were assigned to each parameter value, based on their relative importance as seen in previously published analyses (Table 1). All possible combinations of these six parameters were assessed for their correlation with cure at end of therapy. Correlation was assessed using linear regression, with a forced value of 100 on the y axis for a score of 0 on the x axis, in order to make the ensuing regression formula clinically meaningful. Statistical analyses were performed with SPSS software.
During the first step of this analysis, the derivation process, the 003 trial database was used to test the combinations for their correlation with treatment response.
The optimal combination in this derivation analysis, to be used for the validation analysis, was considered to be the combination which met the following criteria:
Correlation with cure having an R2 value ≥ 0.9
Correlation with cure having a P value ≤ 0.01
If multiple combinations met the above 2 conditions, the combination consisting of the largest number of variables was considered as the optimal combination, since this scheme would be able to categorize the patients into the largest number of distinct groups (i.e. highest discriminative ability and category utility) .
During the second step of this analysis, the validation process, the predicted cure rate from the optimum combination chosen in step one (using the derived regression formula) was compared to the actual CDI cure rate as seen in the 004 clinical trial database. This was done by means of Kappa statistics of the cure rate for each score category.
During a final step of this analysis, the two clinical studies (003 and 004) were pooled and the optimum combination which had been derived in the first step was used to compare the predicted and actual cure rates for this entire cohort, again using a Chi-Square analysis for each score group.
Step 1. Derivation of the optimal categorization scoring system.
All of the possible combinations and permutations of the six chosen variables are listed in Table 2, along with their respective correlations with treatment response in the first (003) clinical trial, as demonstrated by the respective R2 and P values. The 12 combinations which each demonstrated an R2 value of ≥ 0.9 with a P value of ≤ 0.01 were:
Age, leukocyte count
Age, treatment with systemic antibiotics, leukocyte count
Age, temperature, serum creatinine
Age, temperature, leukocyte count
Temperature, albumin, leukocyte count
Age, treatment with systemic antibiotics, temperature, serum creatinine
Age, temperature, albumin, serum creatinine
Age, temperature, leukocyte count, albumin
Temperature, leukocyte count, albumin, serum creatinine
Age, treatment with systemic antibiotics, leukocyte count, albumin, serum creatinine
Of these 12 candidate combinations, the most discriminating (containing the largest number of variables and thereby separating the patients into the largest number of unique groups) was the combination of age (Ag), treatment with systemic antibiotics (Tr), leukocyte count (L), serum albumin (Al) and serum creatinine as a measure of renal function (S) (abbreviated herewith as ATLAS). The ATLAS combination produced a scoring system which was able to place the CDI patients into 11 unique categories (scores 0 to 10, inclusive) and this correlated with treatment cure with an R-squared value of 0.95 and a highly significant P value of < 0.001. The regression equation for this correlation was shown to be: cure rate = 100 – [5.08 × (ATLAS score)]. The receiver operating characteristics of the ATLAS score for predicting treatment cure are shown in Figure 1 (area under the curve was calculated to be 0.71).
Step 2. Validation of the categorization system from step 1.
The actual cure rates and the predicted cure rates (derived from the regression equation calculated in step 1) for each of the ATLAS categories of the CDI patients in the mITT population of the 004 clinical trial are seen in Table 3.
The ATLAS categorization scheme was able to very closely (overall Kappa=95.2%; P<0.0001) predict the actual cure rate for these patients in every score category.
Step 3. Capacity of the ATLAS score to predict cure for the pooled CDI patient databases, and by treatment allocation.
In order to assess how the ATLAS scoring system would perform when all patients in both trials were placed into a single database, this categorization scheme was used to compare the predicted cure rates and the actual cure rates for all mITT patients, as well as for the two sub-groups of patients as determined by treatment allocation (i.e. fidaxomicin or vancomycin). The results of these analyses can be seen in Table 4. Again, excellent predictive ability of the ATLAS system is seen for the entire database and for each of the two assigned treatment groups.
The current classifications of “mild”, “moderate”, “severe”, and “fulminant” CDI are unvalidated, subjective, and have not yet been shown to be clinically useful. An easy to use, objective, clinically relevant and validated system of categorizing CDI patients is needed in order to choose among the growing number of available therapies, to decide which patients might benefit from adjunctive therapies, to facilitate communication among healthcare providers, to prognosticate the outcome of therapy, and to categorize patients for CDI intervention trials.
Two large CDI clinical trial databases, the fidaxomicin/vancomycin comparative trials, were used to derive and then to validate a CDI patient categorization scheme which could predict cure at the end of therapy. This derived scoring system, the ATLAS score, was able to categorize CDI patients into 11 distinct categories and was able to predict clinical cure with a high degree of accuracy.
Several issues concerning these analyses should be mentioned. Firstly, the two databases used for the analyses were phase 3 clinical trials, which excluded extremely ill CDI patients. As per the study protocol, patients were excluded if they had “life-threatening or fulminant CDAD” (WBC >30 × 109/L; temperature >40°C or evidence of hypotension and septic shock, peritoneal signs or significant dehydration), toxic megacolon, or were likely to die within 72 hours of study enrollment . Therefore, there is an under-representation of CDI patients in the upper extremes of the score values. The distribution of patients, by ATLAS score, in the two clinical trial databases (Figure 2) may not represent all CDI patient populations in all healthcare systems. For instance, CDI patients in a specialty medical unit (e.g. transplant unit) or who manifest CDI while in an intensive care unit (ICU) may not conform to the prediction scheme, even though immunocompromised patients and ICU patients were enrolled into the two analyzed studies, if they were otherwise eligible. Secondly, other clinical or laboratory variables measured at the time of CDI diagnosis but not included in this analysis, may also aid in the predictive model of clinical cure. However, we chose clinical variables which have been repeatedly demonstrated to be correlated with disease outcome. Inclusion of the infecting strain type (i.e. NAP1/027/BI) into the categorization scheme might increase the predictive ability, since infections with this strain have been shown to increase the chance of poor outcomes from CDI . However, at the present time, typing of the infecting strain is not widely available, and other non-NAP1/027/BI hyper-virulent strain types have already been described and may continue to emerge in the future .
Thirdly, a different use of the clinical variables with other weighting schemes, might give a more accurate predictive model. The variations in factor weighting and combinations are virtually limitless, and multivariate regression analysis was not helpful in determining the weighting scheme, since many of the factors highly predictive of outcome (i.e. albumin, age, serum creatinine) are very highly correlated with each other and negated each other in the multivariate models. Fourthly, it should be noted that only clinical cure was assessed in these analyses. We did a preliminary analysis investigating the capacity of this scoring system to predict CDI recurrence at 28 days post-therapy, and the five-component ATLAS score performed poorly in this regard. Other combinations performed better for predicting CDI recurrence, and those analyses will be undertaken at a later time. Lastly, this scoring system worked well in predicting response to both fidaxomicin and to vancomycin. Since no patients received metronidazole in these 2 clinical studies, it remains unknown how the ATLAS score would perform in predicting cure rates with this agent. However, the ATLAS categorization of CDI patients may now allow post-hoc comparisons of CDI patients treated with vancomycin and metronidazole in other databases in order to examine if there is truly a difference in outcome with these two agents in sub-groups of patients with specific scores.
In conclusion, a combination of five simple clinical and laboratory variables measured at the time of CDI diagnosis, combined into an 11-category scoring system (ATLAS), seems to be able to accurately predict treatment response to CDI therapy by either vancomycin or fidaxomicin. The ATLAS scoring system may be useful to stratify CDI patients in order to prospectively evaluate and compare CDI therapies among patient categories, to choose therapies for patient sub-groups to maximize cure rates, to categorize patients in different CDI therapeutic studies in order to allow between-study comparisons of patient groups, and to stratify patients upon entry into CDI therapeutic trials.
Jarvis WR, Schlosser J, Jarvis AA, Chinn RY: National point prevalence of clostridium difficile in US health care facility inpatients, 2008. Am J Infect Control. 2009, 37: 263-270. 10.1016/j.ajic.2009.01.001.
Loo VG, Poirier L, Miller MA: A predominantly clonal multi-insitutional outbreak of clostridium difficile-associated diarrhea with high morbidity and mortality. New Engl J Med. 2005, 353: 2442-2449. 10.1056/NEJMoa051639.
Goorhuis A, Bakker D, Corver J: Emergence of clostridium difficile infection due to a new hypervirulent strain, polymerase chain reaction ribotype 078. Clin Infect Dis. 2008, 47: 1162-1170. 10.1086/592257.
Louie TG, Miller MA, Mullane KM: Fidaxomicin versus vancomycin for clostridium difficile infection. New Engl J Med. 2011, 364: 422-431. 10.1056/NEJMoa0910812.
Sougioultzis S, Kyne L, Drudy D, Keates S, Maroo S, Pothoulakis C, Giannasca PJ, Lee CK, Warny M, Monath TP, Kelly CP, Clostridium difficile toxoid vaccine in recurrent C. difficile-associated diarrhea: Clostridium difficile toxoid vaccine in recurrent C. Difficile-associated diarrhea. Gastroenterology. 2005, 128: 764-770. 10.1053/j.gastro.2004.11.004.
Lowy I, Molrine DC, Leav BA: Treatment with monoclonal antibodies against clostridium difficile toxins. N Engl J Med. 2010, 362: 197-205. 10.1056/NEJMoa0907635.
Rubin MS, Bodenstein LE, Kent KC: Severe clostridium difficile colitis. Dis Colon Rectum. 1995, 38: 350-354. 10.1007/BF02054220.
Belmares J, Gerding DN, Parada JP, Miskevics S, Weaver F, Johnson S: Outcome of metronidazole therapy for clostridium difficile disease and correlation with a scoring system. J Infect. 2007, 55: 495-501. 10.1016/j.jinf.2007.09.015.
McEllistrem MC, Carman RJ, Gerding DN, Genheimer CW, Zheng L: A hospital outbreak of clostridium difficile disease associated with isolates carrying binary toxin genes. Clin Infect Dis. 2005, 40: 265-272. 10.1086/427113.
Zar FA, Bakkanagari SR, Moorthi KM, Davis MB: A comparison of vancomycin and metronidazole for the treatment of clostridium difficile-associated diarrhea, stratified by disease severity. Clin Infect Dis. 2007, 45: 302-307. 10.1086/519265.
Henrich TJ, Krakower D, Bitton A, Yokoe DS: Clinical risk factors for severe clostridium difficile-associated disease. Emerg Infect Dis. 2009, 15: 415-422. 10.3201/eid1503.080312.
Louie T, Gerson M, Grimard D: Results of a phase III trial comparing tolevamer, vancomycin and metronidazole in patients with clostridium difficile-associated diarrhea (CDAD). program and abstracts of the 47th annual interscience conference on antimicrobial agents and chemotherapy. 2007, Chicago, Illinois: Abstract K-425a, September 17–20
Gujja D, Friedenberg FK: Predictors of serious complications due to clostridium difficile infection. Aliment Pharmacol Ther. 2009, 29: 635-642. 10.1111/j.1365-2036.2008.03914.x.
UPHS guidelines for antimicrobial therapy. 2011, http://www.uphs.upenn.edu/bugdrug/. Last accessed November 7
Fujitani S, George WL, Murthy AR: Comparison of clinical severity score indices for clostridium difficile infection. Infection Control & Hospital Epidemiology. 2011, 32: 220-228. 10.1086/658336.
Toro DH, Amaral-Mojica KM, Rocha-Rodriguez R, Gutierrez-Nunez J: An innovative severity score index for C. Difficile infection: a prospective study. Infect Dis in Clin Practice. 2011, 19: 336-339. 10.1097/IPC.0b013e31821895a8.
Brouwers MC, Rawski E, Spithoff K, Oliver TK: Inventory of cancer guidelines: a tool to advance the guideline enterprise and improve the uptake of evidence. Expert Review of Pharmacoeconomics & Outcomes Research. 2011, 11: 151-161. 10.1586/erp.11.11.
Levy MM, Dellinger RP, Townsend SR: The surviving sepsis campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Intensive Care Med. 2010, 36: 222-231. 10.1007/s00134-009-1738-3.
Cohen SH, Gerding DN, Johnson S: Clinical practice guidelines for clostridium difficile infection in adults: 2010 update by the society for healthcare epidemiology of America (SHEA) and the infectious diseases society of America (IDSA). Infection Control & Hospital Epidemiology. 2010, 31: 431-455. 10.1086/651706.
Bauer MP, Kuijper EJ, van Dissel JT: European society of clinical microbiology and infectious diseases (ESCMID): treatment guidance document for clostridium difficile infection (CDI). Clin Microbiol Infect. 2009, 15: 1067-1079. 10.1111/j.1469-0691.2009.03099.x.
Crook DW, Walker AS, Kean Y: Fidaxomicin versus vancomycin for Clostridium difficile infection: meta-analysis of pivotal randomized controlled trials. Clin Infect Dis. 2012, 55 (Supp 2): S93-S103.
Corter JE, Gluck MA: Explaining basic categories: feature predictability and information. Psychol Bull. 1992, 111: 291-303.
Miller M, Gravel D, Mulvey M: Health care-associated clostridium difficile infection in Canada: patient age and infecting strain type are highly predictive of severe outcome and mortality. Clin Infect Dis. 2010, 50: 194-201. 10.1086/649213.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2334/13/148/prepub
We would like to acknowledge and thank all of the site investigators, the site research personnel, and the patients themselves who participated in the two large clinical trials comparing fidaxomicin and vancomycin, from which this extensive and rich database has arisen.
Various portions of this analysis were presented at the Interscience Conference on Antimicrobial Agents and Chemotherapy in September 2010 and at the Annual Meeting of the Infectious Diseases Society of America in October 2010.
The statistical programming was performed and paid for by Optimer Pharmaceuticals.
Dr. M. Miller was an advisory board member for Optimer Pharmaceuticals, Novartis, Sanofi and Merck and received research support from Optimer Pharmaceuticals. Dr. T. Louie received research support from Cubist, Cempra, Actelion and Optimer Pharmaceuticals and was a paid speaker for Optimer Pharmaceuticals. Dr. K. Mullane received research support and travel support for scientific presentations from Optimer Pharmaceuticals. Dr. K. Weiss received research support from Novartis, Merck and Optimer Pharmaceuticals and was a paid speaker for Optimer Pharmaceuticals. Dr. A. Lentnek and Dr. Y. Golan received research support from Optimer Pharmaceuticals. Y. Kean was an employee of Optimer Pharmaceuticals from 2010 to 2012, and Dr. P. Sears is currently an employee of Optimer Pharmaceuticals.
All authors have been substantially involved in all stages of the conception and design of this study and in the analysis and interpretation of the data. All authors have been involved in drafting the manuscript, revising it critically for its content and all have seen the submitted manuscript and have given final approval of the version to be published.