Limits...
Bayesian history matching of complex infectious disease models using emulation: a tutorial and a case study on HIV in Uganda.

Andrianakis I, Vernon IR, McCreesh N, McKinley TJ, Oakley JE, Nsubuga RN, Goldstein M, White RG - PLoS Comput. Biol. (2015)

Bottom Line: History matching is an iterative procedure that reduces the simulator's input space by identifying and discarding areas that are unlikely to provide a good match to the empirical data.Simulator evaluations made within this region were found to have a 65% probability of fitting all 18 outputs.Further research is required to explicitly address the stochastic nature of the simulator as well as to account for correlations between outputs.

View Article: PubMed Central - PubMed

Affiliation: Dept. of Infectious Disease Epidemiology, London School of Hygiene & Tropical Medicine, London, United Kingdom.

ABSTRACT
Advances in scientific computing have allowed the development of complex models that are being routinely applied to problems in disease epidemiology, public health and decision making. The utility of these models depends in part on how well they can reproduce empirical data. However, fitting such models to real world data is greatly hindered both by large numbers of input and output parameters, and by long run times, such that many modelling studies lack a formal calibration methodology. We present a novel method that has the potential to improve the calibration of complex infectious disease models (hereafter called simulators). We present this in the form of a tutorial and a case study where we history match a dynamic, event-driven, individual-based stochastic HIV simulator, using extensive demographic, behavioural and epidemiological data available from Uganda. The tutorial describes history matching and emulation. History matching is an iterative procedure that reduces the simulator's input space by identifying and discarding areas that are unlikely to provide a good match to the empirical data. History matching relies on the computational efficiency of a Bayesian representation of the simulator, known as an emulator. Emulators mimic the simulator's behaviour, but are often several orders of magnitude faster to evaluate. In the case study, we use a 22 input simulator, fitting its 18 outputs simultaneously. After 9 iterations of history matching, a non-implausible region of the simulator input space was identified that was 10(11) times smaller than the original input space. Simulator evaluations made within this region were found to have a 65% probability of fitting all 18 outputs. History matching and emulation are useful additions to the toolbox of infectious disease modellers. Further research is required to explicitly address the stochastic nature of the simulator as well as to account for correlations between outputs.

Show MeSH

Related in: MedlinePlus

Minimum implausibility (below and left of diagonal) and optical depth plots (above and right of diagonal) for 10 key inputs for waves 1,4,7,9.Minimum implausibility plots show an estimate of the minimum implausibility for different values of pairs of inputs. Optical depth plots show an estimate of the  probability of encountering a non-implausible point for different values of pairs of inputs.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4288726&req=5

pcbi-1003968-g006: Minimum implausibility (below and left of diagonal) and optical depth plots (above and right of diagonal) for 10 key inputs for waves 1,4,7,9.Minimum implausibility plots show an estimate of the minimum implausibility for different values of pairs of inputs. Optical depth plots show an estimate of the probability of encountering a non-implausible point for different values of pairs of inputs.

Mentions: Fig. 5 shows the minimum implausibility and depth plots for the percentage of men in high sexual activity group (mhag) and the contact rate for high activity group in the first period (hacr1). This figure shows that a match was unlikely to be found if both inputs take a large value. Fig. 6(a) shows the implausibility and depth plots for 8 of the most active inputs in wave 1. It is noticeable that the contact rates for the low activity groups in the first two periods (lacr1, lacr2), only lead to matches when they take a relatively small value (). Finally, correlation patterns appear to emerge between a few input pairs, such as between inputs mhag and hacr1, and mhag and hacr3.


Bayesian history matching of complex infectious disease models using emulation: a tutorial and a case study on HIV in Uganda.

Andrianakis I, Vernon IR, McCreesh N, McKinley TJ, Oakley JE, Nsubuga RN, Goldstein M, White RG - PLoS Comput. Biol. (2015)

Minimum implausibility (below and left of diagonal) and optical depth plots (above and right of diagonal) for 10 key inputs for waves 1,4,7,9.Minimum implausibility plots show an estimate of the minimum implausibility for different values of pairs of inputs. Optical depth plots show an estimate of the  probability of encountering a non-implausible point for different values of pairs of inputs.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4288726&req=5

pcbi-1003968-g006: Minimum implausibility (below and left of diagonal) and optical depth plots (above and right of diagonal) for 10 key inputs for waves 1,4,7,9.Minimum implausibility plots show an estimate of the minimum implausibility for different values of pairs of inputs. Optical depth plots show an estimate of the probability of encountering a non-implausible point for different values of pairs of inputs.
Mentions: Fig. 5 shows the minimum implausibility and depth plots for the percentage of men in high sexual activity group (mhag) and the contact rate for high activity group in the first period (hacr1). This figure shows that a match was unlikely to be found if both inputs take a large value. Fig. 6(a) shows the implausibility and depth plots for 8 of the most active inputs in wave 1. It is noticeable that the contact rates for the low activity groups in the first two periods (lacr1, lacr2), only lead to matches when they take a relatively small value (). Finally, correlation patterns appear to emerge between a few input pairs, such as between inputs mhag and hacr1, and mhag and hacr3.

Bottom Line: History matching is an iterative procedure that reduces the simulator's input space by identifying and discarding areas that are unlikely to provide a good match to the empirical data.Simulator evaluations made within this region were found to have a 65% probability of fitting all 18 outputs.Further research is required to explicitly address the stochastic nature of the simulator as well as to account for correlations between outputs.

View Article: PubMed Central - PubMed

Affiliation: Dept. of Infectious Disease Epidemiology, London School of Hygiene & Tropical Medicine, London, United Kingdom.

ABSTRACT
Advances in scientific computing have allowed the development of complex models that are being routinely applied to problems in disease epidemiology, public health and decision making. The utility of these models depends in part on how well they can reproduce empirical data. However, fitting such models to real world data is greatly hindered both by large numbers of input and output parameters, and by long run times, such that many modelling studies lack a formal calibration methodology. We present a novel method that has the potential to improve the calibration of complex infectious disease models (hereafter called simulators). We present this in the form of a tutorial and a case study where we history match a dynamic, event-driven, individual-based stochastic HIV simulator, using extensive demographic, behavioural and epidemiological data available from Uganda. The tutorial describes history matching and emulation. History matching is an iterative procedure that reduces the simulator's input space by identifying and discarding areas that are unlikely to provide a good match to the empirical data. History matching relies on the computational efficiency of a Bayesian representation of the simulator, known as an emulator. Emulators mimic the simulator's behaviour, but are often several orders of magnitude faster to evaluate. In the case study, we use a 22 input simulator, fitting its 18 outputs simultaneously. After 9 iterations of history matching, a non-implausible region of the simulator input space was identified that was 10(11) times smaller than the original input space. Simulator evaluations made within this region were found to have a 65% probability of fitting all 18 outputs. History matching and emulation are useful additions to the toolbox of infectious disease modellers. Further research is required to explicitly address the stochastic nature of the simulator as well as to account for correlations between outputs.

Show MeSH
Related in: MedlinePlus