Limits...
Accuracy of climate-based forecasts of pathogen spread

View Article: PubMed Central - PubMed

ABSTRACT

Species distribution models (SDMs) are a tool for predicting the eventual geographical range of an emerging pathogen. Most SDMs, however, rely on an assumption of equilibrium with the environment, which an emerging pathogen, by definition, has not reached. To determine if some SDM approaches work better than others for modelling the spread of emerging, non-equilibrium pathogens, we studied time-sensitive predictive performance of SDMs for Batrachochytrium dendrobatidis, a devastating infectious fungus of amphibians, using multiple methods trained on time-incremented subsets of the available data. We split our data into timeline-based training and testing sets, and evaluated models on each set using standard performance criteria, including AUC, kappa, false negative rate and the Boyce index. Of eight models examined, we found that boosted regression trees and random forests performed best, closely followed by MaxEnt. As expected, predictive performance generally improved with the length of time series used for model training. These results provide information on how quickly the potential extent of an emerging disease may be determined, and identify which modelling frameworks are likely to provide useful information during the early phases of pathogen expansion.

No MeSH data available.


AUC values with 95% CIs, from (a) five-part chronological analysis, averaged over time steps, and (b) 10-fold cross-validation, averaged across folds.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC5383844&req=5

RSOS160975F3: AUC values with 95% CIs, from (a) five-part chronological analysis, averaged over time steps, and (b) 10-fold cross-validation, averaged across folds.

Mentions: RF, BRT and MaxEnt had the highest average AUC scores on chronological analyses, although their confidence intervals (CI) overlapped with some of the other methods (figure 3a). Average false negative rates for RF, BRT and MaxEnt did not stand out from the other methods (electronic supplementary material, figure S1a). In the random split and the 10-fold cross-validation, however, RF and BRT performed significantly better than the others (including MaxEnt) on AUC and kappa, with nearly indistinguishable evaluation statistics (figure 3b; electronic supplementary material, figure S2; kappa not shown). False negative rates for RF and BRT were among the lowest achieved, but not significantly so (electronic supplementary material, figure S1b,c). Based on these combined results, RF and BRT were the best models for predicting the spread of Bd, and MaxEnt also performed well (figure 3; electronic supplementary material, figures S1 and S2). On the chronological analyses, k-NN was the worst performing model; GLM was worst on the random split, followed by RB, both of which also performed poorly in 10-fold cross-validation.Figure 3.


Accuracy of climate-based forecasts of pathogen spread
AUC values with 95% CIs, from (a) five-part chronological analysis, averaged over time steps, and (b) 10-fold cross-validation, averaged across folds.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC5383844&req=5

RSOS160975F3: AUC values with 95% CIs, from (a) five-part chronological analysis, averaged over time steps, and (b) 10-fold cross-validation, averaged across folds.
Mentions: RF, BRT and MaxEnt had the highest average AUC scores on chronological analyses, although their confidence intervals (CI) overlapped with some of the other methods (figure 3a). Average false negative rates for RF, BRT and MaxEnt did not stand out from the other methods (electronic supplementary material, figure S1a). In the random split and the 10-fold cross-validation, however, RF and BRT performed significantly better than the others (including MaxEnt) on AUC and kappa, with nearly indistinguishable evaluation statistics (figure 3b; electronic supplementary material, figure S2; kappa not shown). False negative rates for RF and BRT were among the lowest achieved, but not significantly so (electronic supplementary material, figure S1b,c). Based on these combined results, RF and BRT were the best models for predicting the spread of Bd, and MaxEnt also performed well (figure 3; electronic supplementary material, figures S1 and S2). On the chronological analyses, k-NN was the worst performing model; GLM was worst on the random split, followed by RB, both of which also performed poorly in 10-fold cross-validation.Figure 3.

View Article: PubMed Central - PubMed

ABSTRACT

Species distribution models (SDMs) are a tool for predicting the eventual geographical range of an emerging pathogen. Most SDMs, however, rely on an assumption of equilibrium with the environment, which an emerging pathogen, by definition, has not reached. To determine if some SDM approaches work better than others for modelling the spread of emerging, non-equilibrium pathogens, we studied time-sensitive predictive performance of SDMs for Batrachochytrium dendrobatidis, a devastating infectious fungus of amphibians, using multiple methods trained on time-incremented subsets of the available data. We split our data into timeline-based training and testing sets, and evaluated models on each set using standard performance criteria, including AUC, kappa, false negative rate and the Boyce index. Of eight models examined, we found that boosted regression trees and random forests performed best, closely followed by MaxEnt. As expected, predictive performance generally improved with the length of time series used for model training. These results provide information on how quickly the potential extent of an emerging disease may be determined, and identify which modelling frameworks are likely to provide useful information during the early phases of pathogen expansion.

No MeSH data available.