 Research article
 Open Access
 Open Peer Review
 Published:
Using Baidu index to nowcast handfootmouth disease in China: a meta learning approach
BMC Infectious Diseasesvolume 18, Article number: 398 (2018)
Abstract
Background
Hand, foot, and mouth disease (HFMD) has been recognized as one of the leading infectious diseases among children in China, which causes hundreds of annual deaths since 2008. In China, the reports of monthly HFMD cases usually have a delay of 1–2 months due to the time needed for collecting and processing clinical information. This time lag is far from optimal for policymakers making decisions. To alleviate this information gap, this study uses a meta learning framework and combines publicly Internetbased information (Baidu search queries) for realtime estimation of HFMD cases.
Methods
We incorporate Baidu index into modeling to nowcast the monthly HFMD incidences in Guangxi, Zhejiang, Henan provinces and the whole China. We develop a meta learning framework to select appropriate predictive model based on the statistical and time series meta features. Our proposed approach is assessed for the HFMD cases within the time period from July 2015 to June 2016 using multiple evaluation metrics including root mean squared error (RMSE) and correlation coefficient (Corr).
Results
For the four areas: whole China, Guangxi, Zhejiang, and Henan, our approach is superior to the best competing models, reducing the RMSE by 37, 20, 20, and 30% respectively. Compared with all the alternative predictive methods, our estimates show the strongest correlation with the observations.
Conclusions
In this study, the proposed meta learning method significantly improves the HFMD prediction accuracy, demonstrating that: (1) the Internetbased information offers the possibility for effective HFMD nowcasts; (2) the meta learning approach is capable of adapting to a wide variety of data, and enables selecting appropriate method for improving the nowcasting accuracy.
Background
Hand, foot and mouth disease (HFMD), usually caused by enterovirus 71 (EV71) and coxsackievirus A16 (Cox a16), is a type of infectious disease that occurs most commonly among children under 5 years old [1–4]. The typical symptoms of HFMD patients include fever, skin eruptions on hands and feet, and vesicles in the mouth. HFMD can cause mild to severe illness. Some patients, especially those infected by EV71, would rapidly deteriorate with lifethreatening neurological and systemic complications, including neurological, cardiovascular and respiratory problems. Several large outbreaks of HFMD have been witnessed in AsiaPacific region in recent decades, such as the 1997 pandemic in Malaysia, 1998 pandemic in Taiwan, 2000 pandemic in Japan, 2008 pandemic in Singapore, Vietnam, Mongolia and Brunei, 2008 to 2012 pandemics in China, 2011 pandemic in Japan, 2012 pandemic in Cambodia and 2015 pandemic in Syria [5–11], posing a heavy burden to public health and socioeconomic system in the affected areas [12]. HFMD has been recognized as one of the leading infectious diseases among children in China, which causes hundreds of annual deaths since 2008 [4, 13]. Realtime epidemiological surveillance and early warning of HFMD could enable the timely interventions to prevent and control HFMD outbreaks, thus effectively minimizing morbidity, mortality, and reducing the cost of public health system.
China has built its surveillance system to report the monthly HFMD cases and mortality, but the report always has a 1–2 months delay which could be a major challenge for policymakers to accurately estimate epidemics in an efficient realtime manner. Therefore, an effective system that enables forecasting current HFMD (i.e. nowcasting) is in urgent need. An uptodate detection of acute disease outbreak means more days gained, more lives and more resources saved. In the previous studies, various time series models have been employed for HFMD prediction based on historical reports, including autoregressive integrated moving average (ARIMA) and season ARIMA (SARIMA) [14–18]. However, ARIMA based models have a disadvantage in common that they are essentially ’backwardlooking’, which results in poor prediction at turning points unless the turning point represents a return to a longrun equilibrium [19]. Several studies discovered the correlation between the trend of HFMD and some external variables, where the prediction models are constructed by incorporating external variables such as meteorological data and calendar variables [20–27]. However, one limitation of those models is that they can only be used in a relatively small area, such as a town, and may not be applicable in larger areas due to geographical variety of those external variables among subareas. Thus, how to predict HFMD epidemics effectively in larger scales, such as in a province or entire China, remains an open question for researchers.
With the arrival of big data era, we are encountering large streaming data in our lives more frequently than ever before. The availability of big data from multiple sources provides new opportunities and tools for evidentsupported decision making, such as infectious diseases prediction. In 2008, Google developed an influenza surveillance webservice, namely GFT (Google Flu Trends) [28], which used the Google search query as external variables to predict weekly influenzalikeliness (ILI) rate. The success of GFT motivated several studies aiming to assess current flu activity based on secondary data such as Internet search queries and electronic health records [29–35]. Several studies have been conducted on HFMD prediction using Baidu search queries [36–38]. In these research works, Baidu search queries are incorporated into forecasting methods, and the HFMD prediction is either at provincial or national level. In fact, both datadriven and knowledgedriven forecasting methods usually work well in specific conditions, which is due to the inherent diversities among data sets. The forecasting accuracy can be completely varied when there exists some difference in data structure, data size, time scale, etc. [39, 40]. Therefore, how to develop a robust method or framework with effective model selection for epidemics prediction is a major concern for many applications of public health surveillance.
Our contribution in this paper is two folds: (1) We comprehensively investigate the predictive utility of search queries from Baidu, a dominating search engine in China, for predicting the number of HFMD cases in China, and (2) We develop a novel meta learning (ML) framework that incorporates Internet big data and various parametric predictive models for improving the nowcasting accuracy of HFMD. We evaluate the prediction performance of our estimates in terms of root mean squared error (RMSE) and correlation coefficients (Corr). The results show that: the prediction performance of the predictive models and methods can be significantly improved by utilizing Internetbased search data; the developed meta learning approach can automatically select befitting model based on the historical information, and is more efficient than using single model in terms of prediction power.
Methods
Data source and process
In this study, we focus on the problem of nowcasting monthly HFMD cases in areas with geographical variety including Guangxi province, Zhejiang province, Henan province, and China. The reason that we choose these provinces is that most HFMD cases occur in central and southern China [13]. The surveillance data in China, Guangxi, and Zhejiang cover four years from July 2012 to June 2016, and the data in Henan are from January 2013 to June 2016. We collect the monthly reported clinical cases of HFMD from Chinese Centers for Disease Control and Prevention (CDC) and CDCs in the specific provinces accordingly. In medical informatics, an HFMD case is defined as having clinical confirmation of popular vesicular rashes on hands, feet, mouth or buttocks, with or without fever [4].
Baidu is the most prevailing Web search engine in China with over 80 percent of market share [41]. Among the various online services provided by Baidu, Baidu Index (https://index.baidu.com) is an online search tool that allows users to view how frequent the specific keywords, subjects and phrases have been queried over a time period. In this study, we use the HFMD related search frequency of keywords obtained from Baidu Index as external variables to predict HFMD epidemics. We select search terms or keywords which are closely correlated with HFMD epidemics from a keyword tool ’Chinaz’ (http://tool.chinaz.com) [12]. The keywords are obtained through calculating their pairwise correlation with HFMD time series data, using semantic correlation analysis on the relevant queries in Baidu from any available portal websites, blogs, and online reports. Finally, 46 top keywords are selected as the most correlated to the China HFMD cases (the selected Chinese keywords are displayed in the Additional file 1: Table S1). We collect daily search query of these keywords via Baidu Index, and then aggregate the data to a monthly basis for consistency. Figure 1 illustrates the HFMD associated queries, where the monthly HMFD cases in China and search frequency of Chinese keyword ‘handfootmouth’ are plotted for comparison. As can be seen in Fig. 1, the two time series are highly correlated.
Study design
In our case, the response variable is the monthly HFMD incidences and the covariates are the Baidu index of the selected search keywords. The correlation coefficients are calculated, and only those search terms whose correlation coefficients are higher than 0.5 are used in the subsequent predictive models. The keywords used thus might be different for predicting the HFMD cases in each month. Our proposed approach also employs autoregressive terms because of the intrinsic time series structure in HFMD observations. Let y_{i} denote the number of HFMD cases in month i, we calculate the correlation coefficients between the HFMD observations at lag 0 (y_{i}) and observations at lag 1, 2, 3, 4, 5, and 6 (y_{i−1},…,y_{i−6}), respectively. As can be seen in Table 1, the HFMD cases at lag 1 is significantly associated with the current HFMD incidence in terms of correlation coefficients. The autoregressive term y_{i−1} together with the Baidu index of search keywords comprise the covariates in our proposed approach.
It is observed that the number of covariates exceeds the number of cases in our data sets, least squares estimation may be illposed when using linear regression [42]. Three methods, including principal component analysis (PCA), least absolute shrinkage and selection operator (LASSO), and ridge regression (RR), are employed in our model to tackle this problem. In addition, we use autoregressive integrated moving average (ARIMA) to predict the incidence of HFMD in four regions, because of the underlying time series structure of HFMD observations.
Since the relationship between HFMD cases and Baidu index is intrinsically dynamic we adopt an adaptive form of outofsample forecasting in this study [43]. For PCA, LASSO, RR, and ARIMA, we use a 24 months window (i.e. two full years) to train statistical models and then the upcoming months to perform outofsample prediction validation. As the available data is limited, the selected 24 months window length can also capture the yearly trend as well as seasonal pattern. The model parameters are recomputed before predicting each point by using the training data from the previous 24 months.
Evaluation metrics
Three metrics are employed to measure the prediction accuracy: root mean square error (RMSE), mean absolute percent error (MAPE) and correlation coefficient (Corr). For a series of predicted values \(\hat {\boldsymbol {Y}}=(\hat {y}_{1}, \hat {y}_{2}, \ldots, \hat {y}_{n})\) and their corresponding real values Y=(y_{1},y_{2},…,y_{n}), these metrics are
Smaller RMSE and MAPE indicates the better prediction performance, while the higher the correlation the better.
Statistical methods
A meta learning approach for HFMD nowcasting
As discussed earlier, one major challenge of health forecasting is that there is no single algorithm performs best for all health conditions. Although four individual models are examined in this study, there is no guarantee that one of them can always outperform the others. To achieve more accurate forecasting result, an important question is how to choose the best model for each time point in each location. Metalearning approach, in this scenario, is a potential approach to automatically acquire empirical knowledge for supporting nonexpert users in algorithm selection task [44]. Metalearning has proven to be effective in many forecasting applications [45–48], but its effectiveness in forecasting infectious diseases has been rarely investigated.
Metalearning is defined as an automatic process of generating knowledge associating the performance of algorithms to the characteristics of problem [49]. The meta learner can simply be a single machine learning algorithm [50]. In this case, we employ support vector machine (SVM) as the meta learner to build the recommendation system in meta learning. SVM is a specific class of algorithms, characterized by the usage of kernels, absence of local minima, sparseness of the solution and number of support vectors, etc [51]. SVM can be applied for both classification and regression purpose. In SVM classification, the goal is to find a maximal margin hyperplane that separates data points from different classes as wide as possible in feature space. Besides linear classification, SVMs also works efficiently in cases of nonlinear separation via kernel transformation, which can automatically map their inputs into the transformed feature spaces.
Figure 2 shows the overall procedure of our meta learning framework. We take the HFMD forecasting in China as an example to illustrate the framework. Let Y = (y_{1}…y_{48})^{⊤} represent the outputs, where y_{i}(i = 1,…,48) denotes the monthly HFMD incidences in China from July 2012 to June 2016. Let X=(x_{1}…x_{48})^{⊤} represent the covariates set, where x_{i} = (1,y_{i−1},b_{i}) denotes the ith input, and b_{i}=(b_{i1},…,b_{ik}) denotes the Baidu index (search frequency) of k (k=46) search keywords related to HFMD activity in the ith month. The procedure of meta learning method mainly consists of the following steps:
Step 1: The dataset is divided into training set T^{(0)} and testing set T^{(1)}. For training set T^{(0)}, \(\boldsymbol {t}_{j}^{(0)}=\left (y_{j},\boldsymbol {x}_{j}\right) (j=1,\ldots,26)\) is the jth point of training set, where x_{j}=(y_{j−1},b_{j}). For testing set T^{(1)}, \(\boldsymbol {t}_{s}^{(1)}=\left (y_{s},\boldsymbol {x}_{s}\right) (s=1,\ldots,22)\) is the sth point of testing set, where x_{s}=(y_{s−1},b_{s}).
Step 2: A set of predictive method candidates {f^{(1)},…,f^{(L)}} for fitting the relationship between Y and X is constructed. For each method, we have the fitted model y_{i}=f^{(l)}(x_{i};θ^{(l)}), where f^{(l)}∈{f^{(1)},…,f^{(L)}} and θ^{(l)} is the parameter set of this method. For each data point in testing set, all the predictive methods are applied for HFMD prediction and an adaptive approach (models are dynamically trained with a 2year time window) is adopted.
Step 3: The MAPE of each predictive method at the first n−1 testing data points is calculated, and the optimal method is selected by minimizing MAPE value, i.e. \(l_{s}^{*}=arg\min \limits _{l\in \{1,\ldots,L\}}{MAPE}_{s}=arg\min \limits _{l\in \{1,\ldots,L\}}\left \hat {y}_{s}^{(l)}y_{s}^{(l)}\right /y_{s}^{(l)}\);
Step 4: For each case in the first n−1 testing data points, 11 statistical, time series and physical features characterizing its training set are extracted based on previous study [46–48, 50]. Let \(\boldsymbol {F}_{s}=\left (F_{s}^{1},\ldots,F_{s}^{m}\right)\) denote the set of features. The description of the features is shown in Table 2.
Step 5: SVM is employed as the meta leaner to train the data set \(\left (l_{s}^{*},\boldsymbol {F}_{s}\right)(s=1,\ldots,n1)\), where \(l_{s}^{*}\) is the response variable, which is the optimal method index for the sth point, and the 11 features F_{s} extracted from the corresponding training set are the covariates. LeaveOneOut Cross Validation is applied for model parameter tuning. The fitted model will be sent to the recommendation system for selecting optimal method on a given data set.
Step 6: To predict new HFMD cases in the nth month, the 11 features associated with its training set will be input to the recommendation system, then the meta learner will return an appropriate method for forecasting HFMD incidences in the nth month. The new HFMD cases will be predicted via the recommended model.
Linear regression (LR) with principle component analysis (PCA)
Linear regression (LR) was the first type of regression method with complete theoretical system, and to be applied widely in practical applications. In this study, the linear regression model is formulated as:
Letting \(\hat {\boldsymbol {\beta }}=\left (\hat {\beta }_{0}\ldots \hat {\beta }_{46}\right)\), where b_{i} are the exogenous variables.
However, as mentioned earlier, LR might be illposed when the number of covariates exceeds the number of cases due to the limitation of least squares estimation. To tackle this problem, we introduce Principal Component Analysis (PCA) to reduce the dimensionality of covariates. PCA works by first computing linear combinations of variables that contribute to variation in the sample, and then ranking the combinations of variables according to the amount of variations they account for. The most contributed combinations of variables are then used as the new covariates for regression. More details of application of PCA can be referred to [52–56]. In this study, we apply PCA on the observed Baidu index matrix of training set to obtain the principal components, and select a subset of the top principal components that explain at least 95% variance.
Least absolute shrinkage and selection operator (LASSO)
LASSO, which is referred to as L1 regularization method, is able to achieve both covariates selection and regression. It works by setting a constraint on the sum of the absolute value of the regression coefficients, forcing certain coefficients to be zeros. In this way, LASSO enables efficient selection of a simpler model without the insignificant features, which could enhance predication accuracy. More technical details of LASSO and its some generalizations and variants can be found in [57, 58]. In this study, the LASSO estimate \((\hat {\alpha },\hat {\boldsymbol {\beta }})_{lasso}\) can be obtained by solving
where g≥0 is a tuning parameter.
Ridge regression (RR)
Ridge regression, which is referred to as L2 regularization method, is also applied for HFMD nowcasting in this study. Ridge regression conducts the least squares estimation by adding a small constant value λ to the diagonal entries of the matrix X^{T}X before taking its inverse. The ridge regression estimate \(\left (\hat {\alpha },\hat {\boldsymbol {\beta }}\right)_{ridge}\) can be obtained by solving
The analytical solution of the ridge regression estimator is given by
where I is an identity matrix.
Different from LASSO, ridge regression is more commonly used to deal with the collinearity among variables. More details of ridge regression and its applications can be found in [59–61].
Autoregressive integrated moving average (ARIMA)
Besides regressionbased approaches, we also consider autoregressive integrated moving average ARIMA(p,d,q) model, where p is the number of autoregressive (AR) terms, q is the order of the nonseasonal moving average (MA) lags, and d is the number of nonseasonal differences [62–64]. ARIMA model can be formulated as:
where y_{t} is the number of HFMD cases at time t and \(\varepsilon _{t}^{arima}\) is white noise random error; φ_{i} (i = 1,2,…,p) and 𝜗_{j} (j = 0,1,2,…,q) are parameters to be estimated via least squares or maximum likelihood estimation. The parameters p, q, and d are selected from a search over all the possible model candidates by minimizing the corrected Akaike Information Criterion (AIC) [65].
Time series models can provide satisfactory forecasting performance when the time series data have clear trend and seasonality. However, the strong assumption of the statistical properties of time series data might limit the reliability of forecast performance.
All of the experiments are implemented in R v3.4.1(64 bit) platform using the “MASS”, “penalized”, “hydroGOF”, “forecast”, “glmnet”, “moments”, “e1071”, and “kernalb” packages [66].
Results
We evaluate and compare the forecasting performance of each method. For the time period from July 2015 to June 2016, the meta learning approach reduces the RMSE of the compared method which has the minimum RMSE by 37%, 20%, 20%, and 30% for the four regions, i.e. China, Guangxi, Zhejiang, and Henan, respectively. Comparing the correlation between the nowcasting results and observations, the prediction of the meta learning approach has the maximum correlation coefficient with the ground truth.
Figures 3 and 4 show the RMSE and correlation coefficient of the compared predictive methods in different regions, respectively. As can be seen from the figures, the result verifies the fact that no single model outperforms other models in the four regions. PCA shows inconsistent forecasting performance as it performs worst in China and is comparable with RR and LASSO in the three provinces. The two regularization methods, i.e. LASSO and RR, are competitive in most of the cases except in Henan, where PCA outperforms LASSO. ARIMA does not perform well in all the four regions compared with the models with Baidu index, especially in the three provinces where it is always the worst among the four individual models, validating the predictive utility of Baidu search queries. Comparing the proposed meta learning approach with each individual model, it performs best in China, Guangxi, and Zhejiang, while it is as good as RR in Henan, indicating the effectiveness of meta learning in selecting the befitting models.
The comparison of the prediction results over the entire forecasting period of all the methods is displayed in Fig. 5 (The numerical results can be found in Additional file 2: Table S2). Clearly, ARIMA model shows delayed (or “off”) prediction performances in all the regions, as ARIMA only relies on the historical time series data and it is not able to capture the irregular turning point which contributes to the delayed prediction of those points. PCA, LASSO, and RR can capture the seasonal pattern of HFMD epidemics more accurately, but PCA greatly overestimates the HFMD cases at some time points. At most of the forecasting points, meta learning can match the best or one of the best two models, and there is few significant overestimation or underestimation throughout the forecasting period.
Furthermore, in order to further demonstrate the predictive utility of models incorporating Baidu search queries, we compare regression based models with and without Baidu index data. For models without Baidu index data, the three models including PCA+LR, LASSO, and RR degrade into classical linear regression (LR), as only the HFMD cases at lag 1 (y_{t−1}) is left as covariate. Tables 3 and 4 show the RMSE and Corr of the four compared forecasting models (LR, PCA+LR, LASSO, RR) in different regions. As can be seen from the results, PCA.LR, LASSO and RR (models with Baidu data) shows better predictive performance than LR (model without Baidu data), indicating the utility of Baidu search queries.
Discussion
In this paper, we evaluated the predictive utility of Baidu search data in nowcasting HFMD cases in China. The conventional linear regression is not appropriate for this problem due to the relatively large number of covariates in the model. Therefore, we employ four parametric models, including PCA, RR, LASSO, and ARIMA, to nowcast monthly HFMD cases in China, Guangxi province, Zhejiang province, and Henan province.
The result suggests that the time series model, ARIMA, underperforms due to its delayed prediction performance. PCA, LASSO, and RR have the competitive performances in most of the regions and produce more accurate prediction than ARIMA. Among the compared methods, PCA overestimates or underestimates the HFMD epidemics at some forecasting points, and performs slightly worse than LASSO and RR. The performance of LASSO and RR are similar.
In general, PCA, LASSO, and RR can be feasible single model to nowcast HFMD cases in province or country scales by using Baidu search data when there are limited observations and a relatively large number of search terms. However, they could not produce consistently accurate HFMD nowcasting results because of the relatively weak robustness of each model. No single predictive method proves to be universally best in the four cases.
This result motivates us to develop a novel model selection approach in order to choose appropriate model in different situations. The meta learning approach is then developed to fulfill the requirement. Specifically, the meta learning framework consists of a twostage learning process: In Stage 1, the features characterizing the problem are extracted based on historical data; In Stage 2, a meta learner module is built to learn the interrelation between the features and model performances from the known facts, and deduce new knowledge and rules. This meta learning approach with automatic model recommendation system is superior to the compared individual methods in the problem of HFMD nowcasting.
In this paper, we focus on HFMD nowcasting with 1month lag data. It should be noted that the prediction power of forecasting method may degrade as time lag increases. In the following, we take the HFMD nowcasting in the whole China as an example for further illustration. Similar to the 1month nowcasting, the metric RMSE is used to evaluate the prediction performance of the nowcasting with varied time lag. Figure 6 shows the evaluation results in terms of RMSE of the five compared forecasting methods including meta learning, ARIMA, PCA+LR, LASSO and RR. As can be seen from Fig. 6, the prediction accuracy of various methods declines with the increase of time lag (i.e. from 1 month to 4 months), which is consistent with our findings in the preliminary analysis that the more recent HFMD activities are more associated with the current HFMD incidence in terms of Corr. In spite of the varied time lag, the proposed ML framework still outperforms the other methods, indicating its robustness and effectiveness; as the time lag increases, the difference between the various predictive models’ performance become smaller. It is worth noting that the proposed meta learning approach is not restricted by data resolution, although monthly data is used to illustrate its effectiveness.
The proposed meta learning framework provides practical guidelines in the design, development, implementation, and testing of a forecasting recommendation system for health forecasting problems. Specifically, it can help nonexperts with predictive methods selection. One is to further examine the features for meta learner. The meta learning framework can incorporate various predictive methods and machine learning algorithms. In fact, there could be some other effective features than those used in our model, and there are also more choices of machine learning methods for training meta learner, such as deep learning. These will be further investigated in our future work.
Conclusions
The result of this study demonstrates that the accuracy of HFMD nowcasting can be significantly improved by incorporating Baidu Index data in predictive model. In addition, the developed meta learning approach for model selection together with Baidu Index data enables credible forecasts and provide helpful information for predicting HFMD incidence. Compared with the four individual predictive methods used in this study, the performance of meta learning is more robust for different forecasting scales. Of course, there is still some room for our approach to improve. For example, we will refine the meta learner by examining various learning algorithms in our future work. Besides, we will evaluate the utility of the developed approach in some other forecasting applications.
Abbreviations
 ARIMA:

Autoregressive integrated moving average
 Corr:

Correlation coefficient
 HFMD:

Hand, foot, and mouth disease
 LASSO:

Least absolute shrinkage and selection operator
 MAPE:

Mean absolute percent error
 ML:

Meta learning
 PCA:

Principle component analysis
 RMSE:

Root mean squared error
 RR:

Ridge regression
References
 1
Solomon T, Lewthwaite P, Perera D, Cardosa MJ, McMinn P, Ooi MH. Virology, epidemiology, pathogenesis, and control of enterovirus 71. Lancet Infect Dis. 2010; 10(11):778–90.
 2
Zhuang ZC, Kou ZQ, Bai YJ, Cong X, Wang LH, Li C, et al.Epidemiological research on hand, foot, and mouth disease in Mainland China. Viruses. 2015; 7(12):6400–11.
 3
Chang PC, Chen SC, Chen KT. The current status of the disease caused by Enterovirus 71 infections: epidemiology, pathogenesis, molecular epidemiology, and vaccine development. Int J Environ Res Public Health. 2016; 13(9):890.
 4
Xing W, Liao Q, Viboud C, Zhang J, Sun J, Wu JT, et al.Hand, foot, and mouth disease in China, 2008–12: an epidemiological study. Lancet Infect Dis. 2014; 14(4):308–18.
 5
Ang LW, Koh BK, Chan KP, Chua LT, James L, Goh KT. Epidemiology and control of hand, foot and mouth disease in Singapore. Ann Acad Med Singapore. 2009; 38(2):106–12.
 6
Chan L, Parashar UD, Lye M, Ong F, Zaki SR, Alexander JP, et al.Deaths of children during an outbreak of hand, foot, and mouth disease in Sarawak, Malaysia: clinical and pathological characteristics of the disease. Clin Infect Dis. 2000; 31(3):678–83.
 7
Fujimoto T, Chikahira M, Yoshida S, Ebira H, Hasegawa A, Totsuka A, et al.Outbreak of central nervous system disease associated with hand, foot, and mouth disease in Japan during the summer of 2000: detection and molecular epidemiology of enterovirus 71. Microbiol Immunol. 2002; 46(9):621–7.
 8
Fujimoto T, Iizuka S, Enomoto M, Abe K, Yamashita K, Hanaoka N, et al.Hand, foot, and mouth disease caused by coxsackievirus A6, Japan, 2011. Emerg Infect Dis. 2012; 18(2):337.
 9
Chen KT, Chang HL, Wang ST, Cheng YT, Yang JY. Epidemiologic features of handfootmouth disease and herpangina caused by enterovirus 71 in Taiwan, 1998–2005. Pediatrics. 2007; 120(2):e244–e252.
 10
Yang F, Ren L, Xiong Z, Li J, Xiao Y, Zhao R, et al.Enterovirus 71 outbreak in the People’s Republic of China in 2008. J Clin Microbiol. 2009; 47(7):2351–2.
 11
Nguyen NT, Pham HV, Hoang CQ, Nguyen TM, Nguyen LT, Phan HC, et al.Epidemiological and clinical characteristics of children who died from hand, foot and mouth disease in Vietnam, 2011. BMC Infect Dis. 2014; 14(1):341.
 12
Lee BY, Wateska AR, Bailey RR, Tai JH, Bacon KM, Smith KJ. Forecasting the economic value of an Enterovirus 71 (EV71) vaccine. Vaccine. 2010; 28(49):7731–6.
 13
Xiao X, Gasparrini A, Huang J, Liao Q, Liu F, Yin F, et al.The exposureresponse relationship between temperature and childhood hand, foot and mouth disease: A multicity study from mainland China. Environ Int. 2017; 100:102–9.
 14
Feng H, Duan G, Zhang R, Zhang W.Time series analysis of handfootmouth disease hospitalization in Zhengzhou: establishment of forecasting models using climate variables as predictors. PLoS ONE. 2014; 9(1):e87916.
 15
Song Y, Wang F, Wang B, Tao S, Zhang H, Liu S, et al. Time series analyses of hand, foot and mouth disease integrating weather variables. PloS ONE. 2015; 10(3):e0117296.
 16
Wang P, Goggins WB, Chan EY. Hand, foot and mouth disease in Hong Kong: A timeseries analysis on its relationship with weather. PloS ONE. 2016; 11(8):e0161006.
 17
Liu L, Luan R, Yin F, Zhu X, Lü Q. Predicting the incidence of hand, foot and mouth disease in Sichuan province, China using the ARIMA model. Epidemiol Infect. 2016; 144(1):144–51.
 18
Cai Xh, Wan Qp, Wu Ys, et al.Application of ARIMA Model in Predicting Incidence Trend of Handfootmouth Disease in Zhabei District, Shanghai. Pract Prev Med. 2012; 3:018.
 19
Meyler A, Kenny G, Quinn T. Forecasting Irish inflation using ARIMA models. Central Bank and Financial Services Authority of Ireland Technical Paper Series. 1998; 1998(3/RT/98):1–48.
 20
Huang R, Bian G, He T, Chen L, Xu G. Effects of meteorological parameters and PM10 on the incidence of hand, foot, and mouth disease in children in China. Int J Environ Res Publ Health. 2016; 13(5):481.
 21
Lin H, Zou H, Wang Q, Liu C, Lang L, Hou X, et al.Shortterm effect of El NinoSouthern Oscillation on pediatric hand, foot and mouth disease in Shenzhen, China. PLoS ONE. 2013; 8(7):e65585.
 22
Chen C, Lin H, Li X, Lang L, Xiao X, Ding P, et al.Shortterm effects of meteorological factors on children hand, foot and mouth disease in Guangzhou, China. Int J Biometeorol. 2014; 58(7):1605–14.
 23
Yu L, Zhou L, Tan L, Jiang H, Wang Y, Wei S, et al.Application of a new hybrid model with seasonal autoregressive integrated moving average (ARIMA) and nonlinear autoregressive neural network (NARNN) in forecasting incidence cases of HFMD in Shenzhen, China. PloS ONE. 2014; 9(6):e98241.
 24
Liao Y, Ouyang R, Wang J, Xu B. A study of spatiotemporal delay in hand, foot and mouth disease in response to weather variations based on SVD: a case study in Shandong Province, China. BMC Public health. 2015; 15(1):71.
 25
Ma E, Lam T, Wong C, Chuang S. Is hand, foot and mouth disease associated with meteorological parametersEpidemiol Infect. 2010; 138(12):1779–88.
 26
Huang Y, Deng T, Yu S, Gu J, Huang C, Xiao G, et al.Effect of meteorological variables on the incidence of hand, foot, and mouth disease in children: a timeseries analysis in Guangzhou, China. BMC Infect Dis. 2013; 13(1):134.
 27
Guo C, Yang J, Guo Y, Ou QQ, Shen SQ, Ou CQ, et al.Shortterm effects of meteorological factors on pediatric hand, foot, and mouth disease in Guangdong, China: a multicity timeseries analysis. BMC Infect Dis. 2016; 16(1):524.
 28
Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature. 2009; 457(7232):1012–4.
 29
McIver DJ, Brownstein JS. Wikipedia usage estimates prevalence of influenzalike illness in the United States in near realtime. PLoS Comput Biol. 2014; 10(4):e1003581.
 30
Xu Q, Gel YR, Ramirez LLR, Nezafati K, Zhang Q, Tsui KL. Forecasting influenza in Hong Kong with Google search queries and statistical model fusion. PloS ONE. 2017; 12(5):e0176690.
 31
Santillana M, Zhang DW, Althouse BM, Ayers JW. What can digital disease detection learn from (an external revision to) Google Flu Trends?Am J Prev Med. 2014; 47(3):341–7.
 32
Yang S, Santillana M, Kou SC. Accurate estimation of influenza epidemics using Google search data via ARGO. Proc Natl Acad Sci. 2015; 112(47):14473–8.
 33
Lampos V, Miller AC, Crossan S, Stefansen C. Advances in nowcasting influenzalike illness rates using search query logs. Sci Rep. 2015; 5:12760.
 34
Kang M, Zhong H, He J, Rutherford S, Yang F. Using google trends for influenza surveillance in South China. PloS ONE. 2013; 8(1):e55205.
 35
Yang S, Santillana M, Brownstein JS, Gray J, Richardson S, Kou S. Using electronic health records and Internet search information for accurate influenza forecasting. BMC Infect Dis. 2017; 17(1):332.
 36
Du Z, Xu L, Zhang W, Zhang D, Yu S, Hao Y. Predicting the hand, foot, and mouth disease incidence using search engine query data and climate variables: an ecological study in Guangdong, China. BMJ open. 2017; 7(10):e016263.
 37
Xiao Q, Liu H, Feldman M. Tracking and predicting hand, foot, and mouth disease (HFMD) epidemics in China by Baidu queries. Epidemiol Infect. 2017; 145(8):1699–707.
 38
Huang DC, Wang JF, Huang JX, Sui DZ, Zhang HY, Hu MG, et al. Towards identifying and reducing the bias of disease information extracted from search engine data. PLoS Comput Biol. 2016; 12(6):e1004876.
 39
Grossglauser M, Saner H. Datadriven healthcare: from patterns to actions. Eur J Prev Cardiol. 2014; 21(2_suppl):14–7.
 40
Abidi SSR. Knowledge management in healthcare: towards ’knowledgedriven’decisionsupport services. Int J Med Inform. 2001; 63(1):5–18.
 41
China Search Engine Market Overview. 2015. Available from: https://www.chinainternetwatch.com/17415/searchengine20122018e/. Accessed 11 July 2018.
 42
Kutner MH, Nachtsheim C, Neter J. Applied linear regression models. New York: McGrawHill/Irwin; 2004.
 43
Burkom HS, Murphy SP, Shmueli G. Automated time series forecasting for biosurveillance. Stat Med. 2007; 26(22):4202–18.
 44
Prudêncio RB, Ludermir TB. Metalearning approaches to selecting time series models. Neurocomputing. 2004; 61:121–37.
 45
Lan Z, Gu J, Zheng Z, Thakur R, Coghlan S. A study of dynamic metalearning for failure prediction in largescale systems. J Parallel Distrib Comput. 2010; 70(6):630–43.
 46
Zhou S, Lai KK, Yen J. A dynamic metalearning ratebased model for gold market forecasting. Expert Syst Appl. 2012; 39(6):6168–73.
 47
Matijaṡ M, Suykens JA, Krajcar S. Load forecasting using a multivariate metalearning system. Expert Syst Appl. 2013; 40(11):4427–37.
 48
Lemke C, Gabrys B. Metalearning for time series forecasting and forecast combination. Neurocomputing. 2010; 73(10):2006–16.
 49
Vilalta R, Drissi Y. A perspective view and survey of metalearning. Artif Intell Rev. 2002; 18(2):77–95.
 50
Cui C, Wu T, Hu M, Weir JD, Li X. Shortterm building energy model recommendation system: A metalearning approach. Appl Energy. 2016; 172:251–63.
 51
Cortes C, Vapnik V. Supportvector networks. Mach Learn. 1995; 20(3):273–97.
 52
Lee YJ, Yeh YR, Wang YCF. Anomaly detection via online oversampling principal component analysis. IEEE Trans Knowl Data Eng. 2013; 25(7):1460–70.
 53
Shlens J. A tutorial on principal component analysis. arXiv:14041100. 2014.
 54
Karamizadeh S, Abdullah SM, Manaf AA, Zamani M, Hooman A. An overview of principal component analysis. J Signal Inf Process. 2013; 4(03):173.
 55
Dunia R, Qin SJ, Edgar TF, McAvoy TJ. Identification of faulty sensors using principal component analysis. AIChE J. 1996; 42(10):2797–812.
 56
Petroni A, Braglia M. Vendor selection using principal component analysis. J Supply Chain Manag. 2000; 36(1):63–9.
 57
Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol. 1996; 58(1):267–88.
 58
Tibshirani R. Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc Ser B Stat Methodol. 2011; 73(3):273–82.
 59
Chatterjee S, Hadi AS. Regression analysis by example. New Jersey: Wiley; 2015.
 60
Hoerl AE, Kennard RW. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics. 1970; 12(1):55–67.
 61
Wan S, Mak MW, Kung SY. R3PLoc: A compact multilabel predictor using ridge regression and random projection for protein subcellular localization. J Theor Biol. 2014; 360:34–45.
 62
Box GE, Jenkins GM, Reinsel GC, Ljung GM. Time series analysis: forecasting and control. New Jersey: Wiley; 2015.
 63
Hamilton JD, Vol. 2. Time series analysis. Princeton: Princeton university press Princeton; 1994.
 64
Chatfield C. Timeseries forecasting. Boca Raton: CRC Press; 2000.
 65
Hyndman RJ, Khandakar Y, et al.Automatic time series for forecasting: the forecast package for R. 6/07. Melbourne: Monash University, Department of Econometrics and Business Statistics; 2007.
 66
R Core Team. R: A Language and Environment for Statistical Computing. 2014. Available from: http://www.Rproject.org/. Accessed 11 July 2018.
Acknowledgements
We acknowledge all the participants in the study. We would also like to appreciate the editors’ and reviewers’ valuable comments and suggestions on improving this paper.
Funding
This project has been funded in part with the RGC ThemeBased Research Scheme (TBRS) No. T32102/14N, and the National Natural Science Foundation of China (NSFC) No. 71420107023.
Availability of data and materials
Baidu index is publicly available at https://index.baidu.com. CDC data is publicly available at http://www.nhfpc.gov.cn/jkj/s3578/new_list.shtml.
Author information
Affiliations
Contributions
YZ, QX, YC and KLT conceived the study, undertook statistical analysis and drafted the manuscript. YZ and QX analyzed and interpreted the results. QX and YC assisted with data collection. All authors read and approved the final manuscript.
Corresponding author
Correspondence to Yang Zhao.
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional files
Additional file 1
Table S1. Contains the selected 46 Baidu key words used in predictive models. (PDF 87 kb)
Additional file 2
Table S2. Provides the nowcasting results of the monthly HFMD incidences in China, Guangxi, Zhejiang and Henan. (PDF 46 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Received
Accepted
Published
DOI
Keywords
 HFMD
 Baidu index
 Predictive model
 Metalearning