- Research article
- Open Access
- Open Peer Review
Evaluation of the diagnostic accuracy of nonstructural protein 1 Ag-based tests for dengue virus in Asian population: a meta-analysis
BMC Infectious Diseasesvolume 15, Article number: 360 (2015)
Nonstructural protein 1 (NS1) Ag-based tests are useful for detecting dengue virus (DENV), but there is lack of evidence on the diagnostic accuracy of NS1 Ag-based tests in Asian population. Thus, we conducted this meta-analysis to obtain the overall estimated and summarized performance of the NS1 Ag-based tests in the detection of DENV in Asia.
PubMed, Embase and Medline were searched for studies that evaluated the diagnostic validity of NS1 Ag-based tests between January 1990 and November 2014. Data were analyzed by Meta-Disc and STATA software.
A total of 18 studies including 3342 dengue cases and 1904 control cases which fulfilled the inclusion criteria were considered for analysis. The pooled sensitivity and specificity for NS1 Ag-based tests was 66 % (95 % CI 64.5–67.5) and 97.9 % (95 % CI 97.3–100), respectively. STRIP has the overall highest sensitivity (72.9 %, 95 % CI 70.1–75.5). According to viral serotype, the test with the highest sensitivity for DENV1, DENV2 and DENV3 were Platelia (83.7 %, 95 % CI 79.7–87.1), Panbio (71.8 %, 95 % CI 65.5–80.9) and STRIP (81.9 %, 95 % CI 75.5–87.2) respectively. The highest sensitivity for primary infection was Platelia (95.1 %, 95 % CI 92.6–96.9) and for secondary infection was STRIP (64 %, 95 % CI 53.2–73.9).
Our meta-analysis suggests that NS1 Ag-based test is a good diagnostic method for DENV with a high specificity. However, viral serotype, serological status, clinical severity and the duration of illness are the main factors influencing the diagnostic accuracy.
Dengue, a vector-borne disease, has become one of the most serious public problems due to the increasing morbidity, with over one billion people at risk in tropical and subtropical areas . Dengue virus (DENV), genus flavivirus, is antigenically classified into four serotypes (DENV1-4). DENV is an arbovirus, mainly transmitted by aedesaegypti mosquitoes. The virus is thought to be responsible for close to 400 million infections per year worldwide .
The World Health Organization (WHO) 2009 guidelines identified three diagnostic tests as golden standards for dengue diagnosis: viral isolation and identification, nucleotide detection, and serological tests for IgM or IgGseroconversion . However, these methods have certain limitations. For example, viral isolation is difficult to perform and time-consuming. Nucleotide detection such as RT-PCR requires specialized laboratory equipments. And serological tests for IgM or IgG cannot be used for early onset diagnosis. Therefore, a more efficient and accurate detection method for DENV is warranted. Nonstructural protein 1 (NS1) is a glycoprotein that is abundantly produced by DENV in the early stage of infection, and can be detected in the serum [4, 5]. In contrast to IgM, which can only be detected at the beginning of the fifth day of the disease, NS1 antigen capture could be performed at the onset of symptoms.
Currently, several laboratory methods based on the capture of DENV NS1 antigen are available. The two main methods for detecting DENV infection are enzyme-linked immunosorbent assay (ELISA) (Platelia (Platelia Dengue NS1 Ag-ELISA Kit) and Panbio (Panbio Dengue Early ELISA Kit)) and immunochromatography (IC) (STRIP, Dengue NS1 Ag STRIP Kit). Previous studies have performed meta-analysis of the diagnostic accuracy of NS1 Ag-based tests including all published articles, however, there is still lack of evidence on the diagnostic accuracy of NS1 Ag-based tests in Asian population. Therefore, we conducted a meta-analysis to obtain the overall estimated and summarized performance of NS1 Ag-based tests in the detection of DENV in Asia.
Search strategy and inclusion criteria
A literature search was performed to screen studies in human that focused on the diagnostic performance of NS1 for the detection of Dengue. We searched PubMed, Medline and Embase database for relevant citations published in English from January 1990 to November 2014. The keywords “NS1” and “Dengue” were used. Two reviewers independently reviewed each publication. The abstracts were read to identify potentially eligible articles and then the full texts of these articles were examined to determine whether they should be included in our study. Any disagreement was discussed and solved by a third reviewer. The inclusion criteria were: (1) samples or patients with dengue confirmed by the standard methods including viral isolation and identification, RNA detection, or serological test for IgM and/or IgG; (2) samples or patients with dengue investigated by NS1 Ag-based capture methods; (3) have reported sufficient data to allow us to calculate the true positive (TP), false negative (FN), false positive (FP) and true negative (TN) values; (4) at least 20 samples from patients and control group, respectively; (5) the participants were Asian. Studies with an overlapping patients sample were excluded.
Data extraction and quality assessment
The following information from each study was extracted independently by two reviewers: (1) first author name; (2) year of publication; (3) location of studies; (4) number of patients; (5) detection methods; (6) event numbers in TP, FN, TN and FP arms. QUADAS questionnaire was used to assess the quality of the studies included in this meta-analysis.
The pooled sensitivity, specificity, diagnostic odds ratio (DOR) and the likelihood ratios (positive likelihood ratio (PLR) and negative likelihood ratio (NLR)) for single test were calculated by Meta-Disc version 1.4. The DOR was calculated as PLR/NLR. A hierarchical summarized receiver operating characteristic (HSROC) curve was plotted by the software. We also constructed the area under curve (AUC) that serves a global measure of the test performance.
The I-squared value (I2) was used to assess the statistical heterogeneity among the studies. The estimate below 25 % was regarded as low heterogeneity, while above 75 % was labeled as high heterogeneity . If heterogeneity existed, a random effect model was used for meta-analysis; otherwise, a fixed effect model was chosen. Subgroup analyses were performed according to the detection methods (ELISA vs. IC), the manufacturers of the test kit, DENV serotypes, serological status, clinical severity and days after onset of fever to assess potential sources of variation in the study results.
In addition, a meta-regression was used according to the following pre-defined characteristics to explore the source of heterogeneity in the studies: study design, publication year, origin of sample, and sample size. A p-value of < 0.05 was considered to be statistically significant. To assess potential publication bias, the Deeks funnel plot asymmetry test was used, with p < 0.05 indicating the presence of publication bias . Fagan nomograms, a two-dimensional graphical tool for estimating how much the result of a diagnostic test changes the probability that a patient has a disease, was used to estimate the clinical value of the index test.
A total of 143 citations were obtained via database searches, and eighteen met the inclusion criteria for this study (Fig. 1). These studies included 3342 dengue cases and 1904 control cases. Among the dengue cases, 439 patients had dengue hemorrhagic fever (DHF). 3129 samples were tested by Platelia Dengue NS1 Ag-ELISA Kit (Bio-Rad), 1081 samples by Early Dengue NS1 ELISA Kit (Pan-bio), and 1896 samples by Dengue NS1 Ag STRIP (Bio-Rad) (Table 1).
Overall accuracy of the tests
The overall sensitivity and specificity of NS1 Ag-based test kits were 66 % (95 % CI 64.5–67.5) and 97.9 % (95 % CI 97.3–100), respectively. The sensitivity of NS1 Ag-based test kits in primary infection patients was 88 % (95 % CI 85.8–89.9) and in secondary infection patients was 60.8 % (95 % IC 57.8–63.8). According to the DENV serotypes, DENV1 had the highest sensitivity of 79.5 % (95 % CI 76.6–82.3) and DENV4 had the lowest sensitivity of 46 % (95 % CI 36–59.3). The AUC (Area Under ROC Curve) was 0.96. According to clinical severity and test assays, NS1 Ag-based test kits were more effective in diagnosing dengue fever (DF) than DHF, and IC had a higher sensitivity than ELISA (Table 2).
Accuracy of the tests on the viral serotype, serological status and clinical severity
Among the included studies, eleven used Platelia, five usedPanbio and seven used Strip. For Platelia, the sensitivity and specificity were 66.2 % (95 % CI 64.1–68.2) and 97.3 % (95 % CI 96.1–98.2), respectively. The values for Panbio were 54 % (95 % CI 50.1–57.9) and 97.2 % (95 % CI 95.2–98.6), and the values for Strip were 72.9 % (95 % CI 70.1–75.5) and 99.1 % (95 % CI 98.2–99.6), respectively. The AUCs of Platelia, Panbio and Strip were 0.97, 0.96 and 0.98, respectively (Table 3).
When evaluating the accuracy of these tests for DENV1, the sensitivity of Platelia was 83.7 % (95 % CI 79.7–87.1), while the value for Panbio was 73.7 % (95 % CI 65.5–80.9) and the value for Strip was 72.9 % (95 % CI 70.1–75.5). For DENV2, the pooled sensitivity was 56.9 % (95 % CI 51.9–61.8) for Platelia, 71.8 % (95 % CI 65.5–80.9) for Panbio and 69 % (95 % CI 63–74.5) for Strip. For DENV3, the pooled sensitivity was 69.9 % (95 % CI 64.8–74.6) for Platelia, 63.3 % (95 % CI 48.3–76.6) for Panbio and 81.9 % (95 % CI 75.5–87.2) for Strip. For DENV4, the pooled sensitivity was 42.9 (95 % CI 32.1–54.1) for Platelia (Table 3).
Regarding the classification of primary or secondary dengue infection, the sensitivities of Platelia, Panbio and Strip were 95.1 % (95 % CI 92.6–96.9), 72.6 % (95 % CI 65.6–78.9) and 87.3 % (95 % CI 83.6–90.5) for primary infection, and 63 % (95 % CI 59.5–66.4), 50.8 % (95 % CI 43.5–58) and 64 % (95%CI 53.2–73.9) for secondary infection, respectively (Table 3).
To verify whether such tests had significant variations in the performance in patients with DF or DHF, we performed subgroup analysis according to clinical severity, and the results showed that the pooled sensitivity of Platelia was 58.7 % (95 % CI 52.7–64.5) for DF, and 72.1 % (95 % CI 67.6–76.3) for DHF (Table 3).
Accuracy of the tests on illness duration
We also evaluated the influence of the illness duration on the accuracy of the NS1 Ag-based tests. It was found that the sensitivity of the tests was higher in the first 3 days after the illness onset than in the following 4–7 days (Table 4).
Post-test probability and publication bias
To obtain the post-test probability, we used Fagan’s nomogram to perform a simulation of an environment that had a prevalence of 42.9 % for dengue disease, an estimate that was generated based on the studies selected. Thus, the probability for someone that had the disease but was not detected by Platelia was 18 % in this model (Fig. 2a). Similarly, the value for Panbio was 25 %, and the value for Strip was 15 % (Fig. 2b-c). In contrast, the post-test probability of sick patient with a positive test result was 99 % for Platelia, 100 % for Panbio, and 98 % for Strip (Fig. 2a-c).
Additionally, the Deeks funnel plot did not show any potential publication bias for the subgroup studies (PPlatelia = 0.95, PPanbio = 0.17 and PStrip = 0.51).
The dramatic increase in Asian dengue burden has promoted social interest in improving dengue diagnosis. Although many dengue vaccines are under development, none has been licensed to date. Therefore, early diagnosis of dengue leads to appropriate clinical management and better outcome. Soluble NS1 detection in the serum or plasma of a DENV-infected patient is a novel method for early dengue diagnosis. To evaluate the accuracy of NS1 Ag-based test in dengue diagnosis, we performed a meta-analysis using published studies.
Our results indicated that NS1 Ag-based test has a high accuracy for dengue diagnosis. The AUC of NS1 Ag-based test was 0.96. The pooled specificity (97.9 %, 95 % CI 97.3–100) was extremely high, but the pooled sensitivity (66 %, 95 % CI 64.5–67.5) was relatively low. In this regard, NS1 Ag-based test may not be suitable to detect dengue as low sensitivity may result in misdiagnosis. Considering that the sensitivity of NS1 Ag-based test may be influenced by test method, viral serotype, serological status and clinical severity, we also performed subgroup analysis. According to the analysis of the different kits (Platebia, Panbioand STRIP), it appeared that Dengue NS1 Ag STRIP Kit had the higher sensitivity, followed by Platebia Dengue NS1 Ag-ELISA Kit.
Our meta-analysis showed that NS1 Ag-based test had a lower sensitivity for DENV4 and a higher sensitivity for DENV1 and DENV3. According to the analysis of the different kits, Platebia had the highest sensitivity for DENV1, Panbio had the highest sensitivity for DENV2, and STRIP had the highest sensitivity for DENV3. Unfortunately, the sensitivity data for DENV4 was not available for Panbio and STRIP, thus we were unable to compare the sensitivity of the three kits for DENV4. The reason for the wide range of total accuracy of serotyping for dengue may be due to the quantitative differences in the secreted NS1 by the different viral serotypes. However, this hypothesis needs to be confirmed by additional studies.
It was shown that the sensitivity of NS1 Ag-based test was higher for primary dengue infection than secondary infection. For patients with primary dengue infection, Platelia had the highest sensitivity. As for secondary infection, STRIP had the highest sensitivity. Detection of NS1 antigen in secondary infection may be hampered by a rapid rise in antibody levels due to the anamnestic antibody response , which leads to the formation of immune complexes and thus preventing the binding of capture or detection antibodies to NS1 antigen. However, simple and repeat hit of secondary DENV infected patients’ serums was shown to be very useful for increasing the sensitivity of the DENV NS1 Ag-based test. In addition, our results also revealed a small decrease of sensitivity of the NS1 Ag-based test for patients with DF compared to those with DHF. Moreover, the sensitivity of NS1 Ag-based test was higher in the first 3 days after illness onset than in the following 4–7 days, and again, the increased antibodies generated by the host’s immune response could be the reason.
There was significant heterogeneity in this meta-analysis and caution must be taken when interpreting the results. In addition, we have used meta-regression analysis to explore the potential factors that may be the source of the heterogeneity. Unfortunately, none of the covariates examined was the source of heterogeneity. Further studies were needed to identify the potential source for the heterogeneity.
There were also some limitations in our study. Firstly, the standard diagnosis methods of NS1 were different in the included studies, e.g., some studies used viral culture while others used serological test as standard method. Secondly, only a few authors used other types of fever as control samples, while most used samples from healthy persons as control. Thirdly, data were not divided into groups based on gender, age or other variables, due to the limited studies. Despite these limitations, we believe that our analysis could contribute to the evaluation of the accuracy of NS1 Ag-based test, which eventually might help the clinical decision making process.
Our meta-analysis suggests that NS1 Ag-based test was a good diagnosis method for dengue with a high specificity. However, viral serotype, serological status, clinical severity and illness duration were the main factors influencing the diagnosis accuracy. Moreover, the different test kits have their own advantages/disadvantages. Large, multicenter prospective studies are needed to confirm our results.
Simmons CP, Farrar JJ, Nguyen vV, Wills B. Dengue. N Engl J Med. 2012;366(15):1423–32.
Bhatt S, Gething PW, Brady OJ, Messina JP, Farlow AW, Moyes CL, et al. The global distribution and burden of dengue. Nature. 2013;496(7446):504–7.
WHO. Dengue guidelines for diagnosis, treatment, prevention and control. Accessed 2013 December 16 2009.
Shu PY, Huang JH. Current advances in dengue diagnosis. Clin Diagn Lab Immunol. 2004;11(4):642–50.
Young PR, Hilditch PA, Bletchly C, Halloran W. An antigen capture enzyme-linked immunosorbent assay reveals high levels of the dengue virus protein NS1 in the sera of infected patients. J Clin Microbiol. 2000;38(3):1053–7.
Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60.
Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58(9):882–93.
Wahala WM, Silva AM: The human antibody response to dengue virus infection. Viruses 2011, 3(12):2374-2395.
Hermann LL, Thaisomboonsuk B, Poolpanichupatam Y, Jarman RG, Kalayanarooj S, Nisalak A, et al. Evaluation of a dengue NS1 antigen detection assay sensitivity and specificity for the diagnosis of acute dengue virus infection. PLoS Negl Trop Dis. 2014;8(10):e3193.
Gan VC, Tan LK, Lye DC, Pok KY, Mok SQ, Chua RC, et al. Diagnosing dengue at the point-of-care: utility of a rapid combined diagnostic kit in Singapore. PLoS One. 2014;9(3):e90037.
Naz A, Zahid D, Murky SN. Evaluation of efficacy of various immunochromatographic rapid tests for dengue diagnosis. Pak J Med Sci. 2014;30(1):166–71.
Aryati A, Trimarsanto H, Yohan B, Wardhani P, Fahri S, Sasmono RT. Performance of commercial dengue NS1 ELISA and molecular analysis of NS1 gene of dengue viruses obtained during surveillance in Indonesia. BMC Infect Dis. 2013;13:611.
Huang CH, Kuo LL, Yang KD, Lin PS, Lu PL, Lin CC, et al. Laboratory diagnostics of dengue fever: an emphasis on the role of commercial dengue virus nonstructural protein 1 antigen rapid test. J Microbiol Immunol Infect. 2013;46(5):358–65.
Kosasih H, Alisjahbana B, Widjaja S, Nurhayati, de Mast Q, Parwati I, et al. The diagnostic and prognostic value of dengue non-structural 1 antigen detection in a hyper-endemic region in Indonesia. PLoS One. 2013;8(11):e80891.
Blacksell SD, Jarman RG, Gibbons RV, Tanganuchitcharnchai A, Mammen Jr MP, Nisalak A, et al. Comparison of seven commercial antigen and antibody enzyme-linked immunosorbent assays for detection of acute dengue infection. Clin Vaccine Immunol. 2012;19(5):804–10.
Watthanaworawit W, Turner P, Turner CL, Tanganuchitcharnchai A, Jarman RG, Blacksell SD, et al. A prospective evaluation of diagnostic methodologies for the acute diagnosis of dengue virus infection on the Thailand-Myanmar border. Trans R Soc Trop Med Hyg. 2011;105(1):32–7.
Duong V, Ly S, Lorn Try P, Tuiskunen A, Ong S, Chroeung N, et al. Clinical and virological factors influencing the performance of a NS1 antigen-capture assay and potential use as a marker of dengue disease severity. PLoS Negl Trop Dis. 2011;5(7):e1244.
Chuansumrit A, Chaiyaratana W, Tangnararatchakit K, Yoksan S, Flamand M, Sakuntabhai A. Dengue nonstructural protein 1 antigen in the urine as a rapid and convenient diagnostic test during the febrile stage in patients with dengue infection. Diagn Microbiol Infect Dis. 2011;71(4):467–9.
Blacksell SD, Jarman RG, Bailey MS, Tanganuchitcharnchai A, Jenjaroen K, Gibbons RV, et al. Evaluation of six commercial point-of-care tests for diagnosis of acute dengue infections: the need for combining NS1 antigen and IgM/IgG antibody detection to achieve acceptable levels of accuracy. Clin Vaccine Immunol. 2011;18(12):2095–101.
Fry SR, Meyer M, Semple MG, Simmons CP, Sekaran SD, Huang JX, et al. The diagnostic sensitivity of dengue rapid test assays is significantly enhanced by using a combined antigen and antibody testing approach. PLoS Negl Trop Dis. 2011;5(6):e1199.
Pok KY, Lai YL, Sng J, Ng LC. Evaluation of nonstructural 1 antigen assays for the diagnosis and surveillance of dengue in Singapore. Vector Borne Zoonotic Dis. 2010;10(10):1009–16.
Hang VT, Nguyet NM, Trung DT, Tricou V, Yoksan S, Dung NM, et al. Diagnostic accuracy of NS1 ELISA and lateral flow rapid tests for dengue sensitivity, specificity and relationship to viraemia and antibody responses. PLoS Negl Trop Dis. 2009;3(1):e360.
Zainah S, Wahab AH, Mariam M, Fauziah MK, Khairul AH, Roslina I, et al. Performance of a commercial rapid dengue NS1 antigen immunochromatography test with reference to dengue NS1 antigen-capture ELISA. J Virol Methods. 2009;155(2):157–60.
Blacksell SD, Mammen Jr MP, Thongpaseuth S, Gibbons RV, Jarman RG, Jenjaroen K, et al. Evaluation of the Panbio dengue virus nonstructural 1 antigen detection and immunoglobulin M antibody enzyme-linked immunosorbent assays for the diagnosis of acute dengue infections in Laos. Diagn Microbiol Infect Dis. 2008;60(1):43–9.
Lapphra K, Sangcharaswichai A, Chokephaibulkit K, Tiengrim S, Piriyakarnsakul W, Chakorn T, et al. Evaluation of an NS1 antigen detection for diagnosis of acute dengue infection in patients with acute febrile illness. Diagn Microbiol Infect Dis. 2008;60(4):387–91.
Kumarasamy V, Chua SK, Hassan Z, Wahab AH, Chem YK, Mohamad M, et al. Evaluating the sensitivity of a commercial dengue NS1 antigen-capture ELISA for early diagnosis of acute dengue virus infection. Singap Med J. 2007;48(7):669–73.
This work was supported by National Natural Science Foundation of China (No. 81172692 and 81373036), foundation from Zhejiang Provincial Department of Science and Technology (No. 2013C14016) and foundation from Hangzhou City Commission of Science and Technology (No.20130633B50).
The authors declare that they have no competing interests.
XS drafted the protocol, performed the literature search, quality appraisal and statistical analysis, and drafted the manuscript. YW and JY conceived the design of the study, provided overall supervision and helped revise the manuscript. XW and QY participated in the design of the study and helped revise the manuscript. YZ and HZ assisted in the full text article review and quality appraisal. All authors have approved the manuscript for submission.