Research article | Open | Open Peer Review | Published:
Influence of social contact patterns and demographic factors on influenza simulation results
BMC Infectious Diseasesvolume 16, Article number: 646 (2016)
The demographic composition and the frequency and nature of social contacts may affect the spread of influenza virus in a population, resulting in distinct age-dependent immunity patterns. As demography and social contact rates differ strongly between European countries, this may impact infection incidence and vaccine effectiveness and thus limit the extent to which conclusions derived from observations in one country can be generalized to others. In the current study, we aimed to decipher the impact of social contact patterns and demographic factors on simulation results and, thus, to determine to what extent vaccination results can be generalized.
We simulated the transmission of four influenza strains (A(H1N1), A(H3N2), B/Victoria, B/Yamagata) in Belgium, Finland, Germany, GB, Italy, Luxembourg, Netherlands and Poland, using the simulation tool 4Flu. Individuals were connected in a dynamically evolving age-dependent contact network based on the POLYMOD study.
When averaged over 20 years, simulation results without vaccination ranged from annually 20,984 (Germany) to 31,322 infections (Italy) per 100,000 individuals. QIV annually prevented 1758 (Poland) to 7720 infections (Germany) per 100,000. Variability of prevented cases remained high when the country-specific vaccination was replaced by unified coverage, but was reduced considerably if the same demography was used for all countries, or even more so when the same contact matrix was used.
Contact matrix and demography strongly influence the age-dependent incidence of influenza and the success of vaccination. Projecting simulation results from one country to another can, therefore, lead to erroneous results.
Influenza viruses constantly change and vaccines must be reformulated every year. As a consequence, vaccine efficacy from previous years may not be fully applicable for subsequent years. This problem has been recognized and led the European Medicine Agency (EMA) to draft guidelines for influenza vaccines (EMA/CHMP/VWP/457259/2014) in which they request that vaccine effectiveness (VE) for individual influenza vaccines should routinely be investigated. The epidemiology of influenza should be influenced by the demographic composition of the population and by age-dependent immunity patterns which result from prior influenza waves. The frequency and nature of social contacts is also likely to affect the spread of infections in a population. Contact patterns and rates strongly differ among European countries, and this may limit the extent to which modeling results obtained from one country can be generalized. A population-based prospective survey showed that age dependent contact patterns are highly assortative with age . In all countries, the average contact rates (i.e. the number of contacts per individual per day) strongly varied among age groups, usually peaking for juveniles. Furthermore, contact rates averaged over all ages differ substantially between countries (Italians report the highest, Germans the lowest rates). Many European countries are currently experiencing substantial demographic changes (generally, declining birth rates and increasing life expectancy lead to ageing populations), yet demographic projections indicate that these changes occur at very different pace in the European countries . Specifically, as children and juveniles contribute most to the spread of influenza , these demographic changes may further limit extrapolations from one country to others.
We examined the influence of contact patterns and demography on the epidemiology of influenza by using the previously published tool 4Flu which simulates the simultaneous and independent transmission of four influenza strains (A(H1N1), A(H3N2), B/Victoria, B/Yamagata) in a population with demographic turnover [4, 5]. Individuals were connected in a dynamically evolving age-dependent contact network based on the contact structures which were determined in the EU POLYMOD study .
Mathematical modeling of infectious diseases can help to assess the impact of preventive interventions such as vaccination and increasingly informs public health decisions in this field. Mathematical models influenced the UK’s decision to extend the annual influenza vaccination program to children [6, 7]. They also supported the recommendations to introduce rotavirus vaccination in the childhood vaccination calendar of Germany and human papilloma virus vaccination in Denmark [8, 9]. Mathematical modeling may also help to optimize strategies for influenza vaccine effectiveness assessment. In the current study, we aimed to decipher the impact of social contact patterns and demographic factors on simulation results and, thus, to determine to what extent vaccination results can be generalized. We used a version of 4Flu which has been extended to allow for using demographic data and contact structures of different countries to explore for all eight European countries which participated in the POLYMOD study the influence of contact pattern and demographic changes on the simulation outcomes.
We used a modified version of the previously published individual-based simulation tool 4Flu [4, 5] which extends the standard susceptible-infected-resistant (SIR) model by including maternal protection, loss of immunity, boosting infections and vaccinations Fig. 1; 4Flu is freely available for download . For a list of modeling parameters, see Additional file 1: Table S1.
Initialization and evaluation period
Each simulation ran for 40 years: during the first 20 years (starting on July 1st 1994), the age-dependent immunity pattern of the population was initialized by applying trivalent influenza (TIV) vaccinations (using the recorded vaccination coverage and the B lineage which actually was contained in TIV) and allowing for independent transmission of four influenza strains A(H1N1), A(H3N2), B/Victoria and B/Yamagata in populations with demographic turnover. In 2014, a 20 year evaluation period started during which all TIV vaccinations were replaced by vaccination with tetravalent influenza vaccine (QIV) which contained both B lineages. During the evaluation period, the number of cases was reported on a daily basis.
We applied this study to each one of the eight countries for which contact structures were determined in the POLYMOD project: Belgium, Finland, Germany, Great Britain, Italy, Luxembourg, the Netherlands, and Poland . For each country, the official demographic age distributions from 1994 to 2033 were used (Fig. 2); data were obtained from Deutsches Statistisches Bundesamt  for Germany and from the Eurostat web pages  for the other countries. Age-dependent data were complete for the evaluation period (2014–2033), but in some cases, data on very old individuals were missing for various years of the initialization period (1994–2013). Whenever this was the case, the number of individuals in these cohorts was assumed to be zero (i.e. for Italy 1994–2011: 100+; Luxembourg: 1994–2005 and 2007–8: 95+, 2006: 100+; Poland: 1994–2001: 95+, 2002–13: 100+; GB: 1994–2001: 85+; 2002–11: 90+). In the simulations, the initial population size was chosen such that the simulated population reaches exactly 100,000 individuals at the beginning of the evaluation period (on July 1st, 2014). Births, immigrations, deaths and changes of the individuals’ risk status occurred throughout the simulation time, such that the age distribution of the simulated population corresponded exactly to the observed or (for future years) the projected age distribution of the simulated country. Whenever the number of individuals was predicted to increase from 1 year to the next one, new individuals of the appropriate age were created during the simulation (interpreted as immigrants); such “immigrants” were given the infection status and the immunologic status of randomly chosen individuals of approximately the same age . As the vaccination coverage differs for individuals with underlying medical conditions (“at risk” individuals), we had to consider the risk status of individuals. A percentage of newborn individuals was chosen at random to belong to the at risk group. Due to births, ageing and deaths, the age-dependent percentages at risk continuously changed (i.e. they drifted away from the values given in Additional file 1: Table S1). To correct for these deviations, the risk status of randomly chosen individuals were changed throughout the course of the simulations.
Individuals were interconnected in a contact network which changed over time. This network was constructed so that the population statistics of contacts always corresponded to the average number of contacts given by the country’s contact matrix which was derived from the contact matrices reported in the POLYMOD study . The original POLYMOD matrices use 15 categories which usually group individuals in 5-year age classes, and have an asymmetric structure, i.e. they report contacts in an unidirectional way (in an extreme case this structure could imply that an individual of a given age group can contact and infect somebody from another age group, but cannot be infected by the other person himself or herself). We extended the original matrix to a full 101x101 matrix (to explicitly have contact frequencies for any combination of the simulated 101 age cohorts), while keeping the sum of contacts in the 5-year groups identical to the original values. In the simulations, we then applied the extended POLYMOD matrix such that contacts became bidirectional and that the contact distribution evolved as the country’s age distribution changes (see Additional file 1 for details).
Infection transmission and natural history
Infective individuals passed on the infection to individuals who were in contact with them, but—due to the small basic reproduction number of influenza—usually only a rather small percentage of these contacts were infected. Children have a longer period of infectiousness and can therefore infect a higher percentage of their contact network. The age-dependent average number of contacts between individuals was determined by the country-specific POLYMOD matrix  as summarized in Table 1. The infection probability per individual contact per day was calibrated to be 0.03 for the German model by comparing the simulation output with observed data from the 2006/07 German influenza season ). In order to be able to compare different countries, this value was used for all countries. In the simulations, it was further multiplied by a seasonal factor which depended on calendar time and which reached a maximum around Christmas . This led to seasonal waves which typically peaked in January or February.
Immunity was lost over time, but could be boosted by later infections. At random years, new drift variants of any one of the four influenza types were introduced which only shared part of their immunologic features with the previously circulating variant which then was replaced by the new one. Thus, the occurrence of drift variants mimicked an additional loss of immunity, leading to an average duration of immunity of 6 years (A(H1N1) and B lineages) or 4.5 years (A(H3N2), respectively, in the absence of boosting events . The two B lineages shared some cross-immunity: infection with one influenza B virus could boost the immunity against the other lineage. Vaccination-derived immunity was lost over time (average duration 1.8 years ), but it could be boosted by subsequent infections; vaccinations could also boost infection-derived immunity. The vaccine efficacy depends on the age of the vaccinee [14–16].
Vaccinations were performed annually in October and November, whereby vaccinees of the previous season were preferentially re-vaccinated in the new season (see Additional file 1: Table S1). The vaccination coverage of all countries, except for the Netherlands, Belgium and Luxembourg, came from Blank et al. , using the values for 2007, whereby the values for adults and elderly had to be read from their Fig. 2 using the tool “Data picker” . For the Netherlands, the 2008 vaccination coverage for the total population came from the web pages of the Nationaal Programma Grieppreventie . For Belgium, the 2008 vaccination coverage was calculated from values supplied by the HISIA web pages ; to obtain separate values for people with and without risk, we additionally used the percentage of individuals “at risk” from Table 6 of . For Luxemburg, no detailed information of the vaccination coverage could be found. The composition of TIV (which only contains one B lineage) followed the WHO recommendations of past years (1994–2014). Despite containing only one B lineage, TIV vaccination could protect against the missing B lineage or boost pre-existing immunity against the missing B lineage, yet the age-dependent vaccine efficacy of this cross-protection was reduced by 40 % . In some of the years when a new drift variant occurred, the vaccine efficacy was reduced against the new variant (vaccine mismatch).
In a first analysis, each country’s demography was used in combination with its own contact matrix and its own vaccination coverage (Table 2). To obtain a basic set of results, the country-specific vaccination coverage was then replaced by a unified age and risk specific coverage (i.e. by an arbitrarily chosen coverage which was close to the country-specific coverage of most countries): for people without risk status, the vaccination coverage was assumed to be 20 % (0–2 years), 10 % (3–10), 5 % (11–15), 10 % (16–59) and 50 % (60+), respectively; for people with risk status, it was 30 % (0–59) and 70 % (60+), respectively. In two further analyses, we explored how the contact structure and the demography influenced the results by either modifying the contact matrix or the demography while keeping everything else constant.
Each simulation was set up to have a population size of 100,000 individuals at the start of the evaluation period. Due to demographic turnover, the simulated number of individuals in earlier years (i.e. in the initialization period) and in later years differed from 100,000. The average annual incidence of influenza infections per 100,000 individuals was calculated by (1) determining the number of infections projected by the model for each evaluation year of each simulation, (2) transforming these numbers into incidence values per 100,000 individuals, (3) averaging the annual incidence values of each simulation over the 20 evaluation years, and (4) by calculating the mean of these values over 1000 simulations. The country-specific number of prevented infections was calculated as the difference between the mean annual incidence values of 1000 simulations without vaccination and with unified vaccination coverage. The country-specific vaccination effect was then given as the percentage of prevented infections.
The age distributions of the eight countries differed widely and they underwent rather extreme changes between the start of the initialization phase (1994) and the end of the evaluation period (2033; Fig. 2), e.g. Poland changed from being the country with by far the highest percentage of children and juveniles (31.8 % below 20 years of age in 1994) to a country which ranked among the lowest (18.4 % in 2033), whereas its percentage of elderly (65+) more than doubled. In each of the eight countries, the population’s mean age increased during the simulation (by at least 1.9 years in Luxembourg up to 10.7 years in Poland). In some countries like Germany, the size of the population declined whereas in others it grew. At least part of this growth must be explained by immigration as is the case for Luxembourg which is predicted to grow by over 100 % until 2033, yet whose age distribution stays nearly constant. The demography of Finland was used in simulations where the same demography was used for every country and where only the contact matrix was varied because its demographic features (percentage of children, average age, annual growth rate) were close to the median of the counties depicted in Fig. 2.
Differences in contact distributions
The countries also differed strongly by their numbers of contacts per individual and by the age distribution of these contacts: the 1st column of Fig. 3 shows the results of the POLYMOD study ; black sections denote contacts with children and juveniles (0–19 years), dark grey ones contacts with young adults (20–64 years) and light grey ones contacts with elderly (65+ years). The contact matrix of Belgium was used in simulations where the same matrix was used for every country and where only the demography was varied because this contact matrix appeared to lie between the matrices of the matrices of all the counties depicted in Fig. 3. In the simulation tool 4Flu, the contacts depicted in the first column are interpreted as “outgoing” contacts which are initiated by the individuals whose ages are given on the horizontal axis. To obtain the age distribution of all contacts in the population (shown in the 3rd column of Fig. 3), the contacts per individual must be combined with the age distribution of the population (2nd column of Fig. 3; see Additional file 1 and  for more explanations). Germany had by far the lowest average number of contacts less than half of the value of Italy, which was the other extreme. The 4th column of Fig. 3 summarizes the contact distribution which prevailed during the evaluation period of the simulations (figure areas are proportional to population size and arrow thickness is proportional to number of contacts): young adults (A) form the largest section of the population; the vast majority of contacts are among young adults, followed by contacts among children (C) and between children and adults. The epidemiological importance of children is further enhanced by the fact that they enter the population susceptible (at least after having passed through a short-term period of material protection) and that they are assumed to have a longer period of infectiousness (Additional file 1: Table S1). Although elderly (E) also form a big segment of the population, contacts among them or between elderly and others are comparatively scarce.
Results without vaccination
Simulation results without vaccination for the annual influenza infection incidence of the eight countries differed considerably (Fig. 4a), ranging from 20,984 infections per 100,000 per year in Germany to 31,322 infections in Italy. The deviation of the individual countries from their common mean value was quite high (coefficient of variation CV = 11.8 %). Although clearly visible, the differences in inter-country variances were not statistically significant which presumably was due to the small number of countries on which these comparisons were based (Brown-Forsythe test; p > 0.05).
Results of scenario analyses
To examine the influence of the demography on infection incidence, we ran simulations where the countries’ demography was replaced by the Finish demography (Fig. 4b). This had a unifying effect on the results (CV = 7.8 %), much of which was due to infection incidence in children and juveniles 0–19 years of age (whose CV was reduced from 17.8 to 6.5 %) and adults 20–64 years of age (whose CV was reduced from 12.3 to 7.0 %), indicating that these age groups contributed more to the inter-country variability of incidence than the elderly (whose CV only changed from 20.5 to 16.5 %). To also examine the influence of the contact matrix on infection incidence, we ran simulations where each country’s contact matrix was replaced by the Belgian one (Fig. 4c). The unifying effect of replacing the contact matrix was even stronger (CV = 3.2 %) than that of using the same demography. The unifying effect was most pronounced in adults (whose CV is reduced from 12.3 to 4.1 %) and much less in Children (whose CV was reduced from 17.8 to 15.3 %) and elderly (whose CV was reduced from 20.5 to 14.6 %).
If each country’s own vaccination coverage was used in the simulations, QIV vaccination prevented from 1758 infections (in Poland) to 7720 infections (in Germany; Fig. 5a; Table 3). The CV of the inter-country variability was 41.4 %. Replacing the country-specific vaccination coverage by a unified vaccination coverage (Fig. 5b) reduced the variability to CV = 27.9 %. To allow for a better comparison of countries and to also include Luxembourg (for which the original vaccination coverage was missing), we used the results with unified vaccination coverage in the following analyses. Replacing each country’s contact matrix by the Belgian one (Fig. 5d) again had a large unifying effect on the absolute number of vaccine prevented infections (CV reduced to 8.0 %). The additional unifying effect was most pronounced in children and juveniles (whose CV was reduced from 31.1 to 8.6 %) and younger adults (whose CV was reduced from 34.9 to 7.2 %), but much less in elderly (whose CV was reduced from 22.1 to 17.9 %). Applying the Finish demography for all countries hardly led to any reduction in overall variability of vaccine effects (Fig. 5c; CV = 22.6 %).
Measuring epidemiologic variability among countries allows investigation of the influence of the contact matrix and demography on infection incidence and vaccine prevented infections. When using the same contact matrix for each country, overall variability of infection incidence was greatly reduced. In contrast, if the same age-distribution was used for each country, this had only a minor harmonizing effect. Thus, the contact matrix was the most important factor which determined differences in country-specific infection incidence. The impact of the contact matrix on vaccine prevented infections was particular striking in children, juveniles and young adults which may be due to the high inter-country differences in the number of contacts of young people (Fig. 3). Thus, when applying the same vaccination strategy in countries with different contact matrices, the number of vaccine prevented infections may differ greatly—particularly for the younger age groups—even if exactly the same age and risk group specific vaccination coverage is used.
As is the case with every modeling study, we had to make some simplifying assumptions. We used the same transmission probability irrespective of the country. Although the same contact questionnaire was used in all POLYMOD countries, contacts among people or reporting could be different between countries. For the sake of examining the influence of contact matrix and age-distribution on the effects of influenza vaccination, it seems justified to keep as many parameters constant as possible, but for a more in-depth evaluation of the benefits of influenza vaccination, it may be warranted to separately calibrate the transmission probability and to use as many country-specific parameter values as possible. We used the same age-dependent percentage of individuals at risk for all countries, whereas the definition of who is regarded to be “at risk” may differ between countries. This should not have a large impact on the results, as the values for Belgium and for the Netherlands tend to be rather similar to the German ones [21–23]. The results for Germany reported in this paper slightly differ from the previously reported ones . Reasons for this are (a) the initialization period and the evaluation period start 1 year later than in the previous paper, (b) the composition of TIV could now be fixed until 2014, and (c) we have extended the number of contact matrix age groups (but this had negligible impact on the results; see Additional file 1 for details). Furthermore, the results in the published paper were given for 0.1 % of the German population whereas here we report results per 100,000 individuals to facilitate comparisons between countries.
Taken together, the country’s contact matrix and to a much lesser degree the demography influence infection incidence and vaccine effects. The effects vary by age group; for vaccine effects, contact matrices affect more the younger and demography more the older age groups. Projecting simulation results from one country to another can, therefore, lead to erroneous results.
European Medicine Agency
Quadrivalent influenza vaccine
Trivalent influenza vaccine
Mossong J, et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med. 2008;5(3):e74.
Lanzieri G. The greying of the baby boomers. A century-long view of ageing in European populations. Eurostat Statistics in focus. 2011;(23). Available from: https://bookshop.europa.eu.
Wallinga J, Teunis P, Kretzschmar M. Using data on social contacts to estimate age-specific transmission parameters for respiratory-spread infectious agents. Am J Epidemiol. 2006;164(10):936–44.
Eichner M, et al. 4Flu—an individual based simulation tool to study the effects of quadrivalent vaccination on seasonal influenza in Germany. BMC Infect Dis. 2014;14:365.
Dolk C, et al. Cost-Utility of Quadrivalent Versus Trivalent Influenza Vaccine in Germany, Using an Individual-Based Dynamic Transmission Model. Pharmacoeconomics. 2016. [Epub ahead of print].
Baguelin M, et al. Assessing optimal target populations for influenza vaccination programmes: an evidence synthesis and modelling study. PLoS Med. 2013;10(10):e1001527.
JCVI. JCVI statement on the annual influenza vaccination programme—extension of the programme to children. 25 July 2012. 2012; Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/224775/JCVI-statement-on-the-annual-influenza-vaccination-programme-25-July-2012.pdf.
Olsen J, Jepsen MR. Human papillomavirus transmission and cost-effectiveness of introducing quadrivalent HPV vaccination in Denmark. Int J Technol Assess Health Care. 2010;26(2):183–91.
Horn J, et al. Langfristige epidemiologische und ökonomische Auswirkungen der HPV‐Impfung in Deutschland. Überarbeiteter Abschlussbericht. 2012. Available from http://www.rki.de/DE/Content/Infekt/Impfen/Forschungsprojekte/HPV-Impfung/Abschlussbericht.pdf?__blob=publicationFile.
Epimos GmbH. Download plattform for the individual-based influenza simulation tool 4Flu. 2015; Available from: http://www.4flu.net. Accessed Oct 2016.
Deutsches Statistisches Bundesamt. 2009.
European Commission. Eurostat. 2014 [cited 2015; Available from: http://www.epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home/. Accessed Oct 2016.
Vynnycky E, et al. Estimating the impact of childhood influenza vaccination programmes in England and Wales. Vaccine. 2008;26(41):5321–30.
Jefferson T, et al. Vaccines for preventing influenza in healthy children. Cochrane Database Syst Rev. 2012;8:CD004879.
Jefferson T, et al. Vaccines for preventing influenza in healthy adults. Cochrane Database Syst Rev. 2014;3:CD001269.
Jefferson T, et al. Vaccines for preventing influenza in the elderly. Cochrane Database Syst Rev. 2010;(2):CD004876. doi:10.1002/14651858.CD004876.pub3.
Blank PR, Szucs TD. Increasing influenza vaccination coverage in recommended population groups in Europe. Expert Rev Vaccines. 2009;8(4):425–33.
Aotsuka M. DataPicker Shareware Version 1.2. (http://hp.vector.co.jp/authors/VA019223/).
Tacken, M.J., B.; Mulder, J.; Tiersma, W.; Braspenning, J. Tacken M, Jansen B, Mulder J, Tiersma W, Braspenning J. Monitoring vaccinatiegraad Nationaal Programma Grieppreventie 2013. 2014. 2014; Available from: http://www.iqhealthcare.nl/media/93613/monitoring_vaccinatiegraad_npg_2013-def.pdf.
HISIA. Belgian Health Interview Survey—Interactive Analysis. Available from: https://hisia.wiv-isp.be/SitePages/Home.aspx. Accessed Oct 2016.
Hanquet, G.J., P.; Vlayen, J.; Vijens, F.; Thiry, N.; Beutels P. Hanquet G, Jonckheer P, Vlayen J, Vijens F, Thiry N,Beutels P. Seasonal influenza vaccination: priority target groups—Part I. KCE reports 162C. 2011. 2011; Available from: https://kce.fgov.be/sites/default/files/page_documents/kce_162c_seasonal_influenza_vaccination.pdf.
Robert Koch Institut. Nationaler Pandemieplan. 2007. Available from: http://www.rki.de/DE/Content/InfAZ/I/Influenza/Pandemieplanung/Archiv_Tabelle.html. Accessed Nov 2016.
Postma MJ, et al. Economic evaluation of influenza vaccination. Assessment for The Netherlands. Pharmacoeconomics. 1999;16 Suppl 1:33–40.
Creation of the simulation tool 4Flu was funded by GlaxoSmithKline GmbH & Co. KG, Munich, Germany. We are grateful to Thierry van Effelterre for helpful comments and suggestions. We acknowledge support by Deutsche Forschungsgemeinschaft and Open Access Publishing Fund of Tübingen University.
This work uses the simulation tool which was funded by the GlaxoSmithKline group of companies, but no specific funding was obtained for the work which is presented in this paper.
Availability of data and materials
The individual-based influenza simulation tool 4Flu can be downloaded at https://www.4flu.net.
ME has created the mathematical transmission model, designed the simulation studies and written the major part of the manuscript. MS has designed and programmed the simulation tool and has taken part in the model development. RSO participated to the structural design of the model. RSO drafted parts of all sections of the paper. All authors had full access to the data and gave final approval before submission.
Martin Eichner teaches mathematical modeling, statistics and epidemiology at Tubingen University. Markus Schwehm designs and develops simulation software. Ruprecht Schmidt-Ott is Medical Doctor with board certification in Medical Microbiology and Infectious Disease Epidemiology and works as Director Global Scientific Affairs at GlaxoSmithKline Vaccines.
RSO is employee of the GlaxoSmithKline group of companies, and has restricted shares ownership in the GlaxoSmithKline group of companies. MS is shareholder of ExploSYS GmbH, which has received payments from Epimos GmbH for developing the simulation tool 4Flu. ME is shareholder of Epimos GmbH, which has received research support from the GlaxoSmithKline group of companies and AstraZeneca.
Consent for publication
Ethics approval and consent to participate
The online available file “Supporting_Material_4Flu.pdf” explains (1) the extension and smoothing of the original POLYMOD matrix (2) the translation of the resulting matrix into a contact network (3) the occurrence of super-spreaders in the simulation tool 4Flu (PDF 930 kb)