Limits...
Accuracy of models for the 2001 foot-and-mouth epidemic.

Tildesley MJ, Deardon R, Savill NJ, Bessell PR, Brooks SP, Woolhouse ME, Grenfell BT, Keeling MJ - Proc. Biol. Sci. (2008)

Bottom Line: These claims are generally based on a comparison between model results and epidemic data at fairly coarse spatio-temporal resolution.By contrast, while the accuracy of predicting culls is higher (20-30%), this is lower than expected from the comparison between model epidemics.These results generally support the contention that the type of the model used in 2001 was a reliable representation of the epidemic process, but highlight the difficulties of predicting the complex human response, in terms of control strategies to the perceived epidemic risk.

View Article: PubMed Central - PubMed

Affiliation: Department of Biological Sciences and Mathematics Institute, University of Warwick, Gibbet Hill Road, Coventry CV4 7AL, UK. m.j.tildesley@warwick.ac.uk

ABSTRACT
Since 2001 models of the spread of foot-and-mouth disease, supported by the data from the UK epidemic, have been expounded as some of the best examples of problem-driven epidemic models. These claims are generally based on a comparison between model results and epidemic data at fairly coarse spatio-temporal resolution. Here, we focus on a comparison between model and data at the individual farm level, assessing the potential of the model to predict the infectious status of farms in both the short and long terms. Although the accuracy with which the model predicts farms reporting infection is between 5 and 15%, these low levels are attributable to the expected level of variation between epidemics, and are comparable to the agreement between two independent model simulations. By contrast, while the accuracy of predicting culls is higher (20-30%), this is lower than expected from the comparison between model epidemics. These results generally support the contention that the type of the model used in 2001 was a reliable representation of the epidemic process, but highlight the difficulties of predicting the complex human response, in terms of control strategies to the perceived epidemic risk.

Show MeSH

Related in: MedlinePlus

Results of multiple simulations of the entire epidemic for the whole of Great Britain. (a) The distribution of farms reporting infection in proportion p of simulations. This distribution is partitioned into those farms reporting infection in 2001 (grey) and those not (white). (b) Comparable distributions for culled farms again partitioned into those farms culled in 2001 (grey) and those not (white). In graphs (c,d), we define a threshold proportion Pc, such that only those farms reporting infection in more than Pc simulations are identified as likely to report infection in the 2001 epidemic. (c) The number of correctly identified reports (grey) and the number of false positives and false negatives (hatched lines). (d) The number of false positives and false negatives (hatched lines) relative to the number of correctly identified reports. (Results are from 250 replicate simulations.)
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2376304&req=5

fig3: Results of multiple simulations of the entire epidemic for the whole of Great Britain. (a) The distribution of farms reporting infection in proportion p of simulations. This distribution is partitioned into those farms reporting infection in 2001 (grey) and those not (white). (b) Comparable distributions for culled farms again partitioned into those farms culled in 2001 (grey) and those not (white). In graphs (c,d), we define a threshold proportion Pc, such that only those farms reporting infection in more than Pc simulations are identified as likely to report infection in the 2001 epidemic. (c) The number of correctly identified reports (grey) and the number of false positives and false negatives (hatched lines). (d) The number of false positives and false negatives (hatched lines) relative to the number of correctly identified reports. (Results are from 250 replicate simulations.)

Mentions: The comparisons so far have been between individual replicates and the 2001 epidemic—hence our results are strongly influenced by the stochastic nature of the simulations. An alternative approach is to consider the results of multiple simulations and use the proportion of simulations in which a farm is infected (or culled) as a measure of its risk. These results are shown in figure 3 for simulations of the entire epidemic beginning on 23 February. Figure 3a shows the distribution of the proportion of simulations in which a given farm reports infection; this is partitioned into those farms reporting infection in 2001 (grey) and those not (white). Clearly, the two distributions are very different, with the distribution for farms reporting in 2001 showing a distinct secondary peak. This is an additional evidence that the simulations can partially discriminate between those farms that are likely to become infected and those that are not. Figure 3b shows comparable results for farms culled in the simulations and in 2001; here the distributions are more similar and this re-enforces our belief that human influence in the culling policy makes it far more difficult to simulate.


Accuracy of models for the 2001 foot-and-mouth epidemic.

Tildesley MJ, Deardon R, Savill NJ, Bessell PR, Brooks SP, Woolhouse ME, Grenfell BT, Keeling MJ - Proc. Biol. Sci. (2008)

Results of multiple simulations of the entire epidemic for the whole of Great Britain. (a) The distribution of farms reporting infection in proportion p of simulations. This distribution is partitioned into those farms reporting infection in 2001 (grey) and those not (white). (b) Comparable distributions for culled farms again partitioned into those farms culled in 2001 (grey) and those not (white). In graphs (c,d), we define a threshold proportion Pc, such that only those farms reporting infection in more than Pc simulations are identified as likely to report infection in the 2001 epidemic. (c) The number of correctly identified reports (grey) and the number of false positives and false negatives (hatched lines). (d) The number of false positives and false negatives (hatched lines) relative to the number of correctly identified reports. (Results are from 250 replicate simulations.)
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2376304&req=5

fig3: Results of multiple simulations of the entire epidemic for the whole of Great Britain. (a) The distribution of farms reporting infection in proportion p of simulations. This distribution is partitioned into those farms reporting infection in 2001 (grey) and those not (white). (b) Comparable distributions for culled farms again partitioned into those farms culled in 2001 (grey) and those not (white). In graphs (c,d), we define a threshold proportion Pc, such that only those farms reporting infection in more than Pc simulations are identified as likely to report infection in the 2001 epidemic. (c) The number of correctly identified reports (grey) and the number of false positives and false negatives (hatched lines). (d) The number of false positives and false negatives (hatched lines) relative to the number of correctly identified reports. (Results are from 250 replicate simulations.)
Mentions: The comparisons so far have been between individual replicates and the 2001 epidemic—hence our results are strongly influenced by the stochastic nature of the simulations. An alternative approach is to consider the results of multiple simulations and use the proportion of simulations in which a farm is infected (or culled) as a measure of its risk. These results are shown in figure 3 for simulations of the entire epidemic beginning on 23 February. Figure 3a shows the distribution of the proportion of simulations in which a given farm reports infection; this is partitioned into those farms reporting infection in 2001 (grey) and those not (white). Clearly, the two distributions are very different, with the distribution for farms reporting in 2001 showing a distinct secondary peak. This is an additional evidence that the simulations can partially discriminate between those farms that are likely to become infected and those that are not. Figure 3b shows comparable results for farms culled in the simulations and in 2001; here the distributions are more similar and this re-enforces our belief that human influence in the culling policy makes it far more difficult to simulate.

Bottom Line: These claims are generally based on a comparison between model results and epidemic data at fairly coarse spatio-temporal resolution.By contrast, while the accuracy of predicting culls is higher (20-30%), this is lower than expected from the comparison between model epidemics.These results generally support the contention that the type of the model used in 2001 was a reliable representation of the epidemic process, but highlight the difficulties of predicting the complex human response, in terms of control strategies to the perceived epidemic risk.

View Article: PubMed Central - PubMed

Affiliation: Department of Biological Sciences and Mathematics Institute, University of Warwick, Gibbet Hill Road, Coventry CV4 7AL, UK. m.j.tildesley@warwick.ac.uk

ABSTRACT
Since 2001 models of the spread of foot-and-mouth disease, supported by the data from the UK epidemic, have been expounded as some of the best examples of problem-driven epidemic models. These claims are generally based on a comparison between model results and epidemic data at fairly coarse spatio-temporal resolution. Here, we focus on a comparison between model and data at the individual farm level, assessing the potential of the model to predict the infectious status of farms in both the short and long terms. Although the accuracy with which the model predicts farms reporting infection is between 5 and 15%, these low levels are attributable to the expected level of variation between epidemics, and are comparable to the agreement between two independent model simulations. By contrast, while the accuracy of predicting culls is higher (20-30%), this is lower than expected from the comparison between model epidemics. These results generally support the contention that the type of the model used in 2001 was a reliable representation of the epidemic process, but highlight the difficulties of predicting the complex human response, in terms of control strategies to the perceived epidemic risk.

Show MeSH
Related in: MedlinePlus