Limits...
Spartan: a comprehensive tool for understanding uncertainty in simulations of biological systems.

Alden K, Read M, Timmis J, Andrews PS, Veiga-Fernandes H, Coles M - PLoS Comput. Biol. (2013)

Bottom Line: Success requires the relationship between simulation and the real-world system to be established: substantial aspects of the biological system are typically unknown, and the abstract nature of simulation can complicate interpretation of in silico results in terms of the biology.The tools comprising spartan help identify which simulation results can be attributed to the dynamics of the modelled biological system, rather than artefacts of biological uncertainty or parametrisation, or simulation stochasticity.We demonstrate the power of spartan in providing critical insight into aspects of lymphoid tissue development in the small intestine through simulation.

View Article: PubMed Central - PubMed

Affiliation: Centre for Systems Biology, School of Biosciences, University of Birmingham, Birmingham, United Kingdom. k.j.alden@bham.ac.uk

ABSTRACT
Integrating computer simulation with conventional wet-lab research has proven to have much potential in furthering the understanding of biological systems. Success requires the relationship between simulation and the real-world system to be established: substantial aspects of the biological system are typically unknown, and the abstract nature of simulation can complicate interpretation of in silico results in terms of the biology. Here we present spartan (Simulation Parameter Analysis RToolkit ApplicatioN), a package of statistical techniques specifically designed to help researchers understand this relationship and provide novel biological insight. The tools comprising spartan help identify which simulation results can be attributed to the dynamics of the modelled biological system, rather than artefacts of biological uncertainty or parametrisation, or simulation stochasticity. Statistical analyses reveal the influence that pathways and components have on simulation behaviour, offering valuable biological insight into aspects of the system under study. We demonstrate the power of spartan in providing critical insight into aspects of lymphoid tissue development in the small intestine through simulation. Spartan is released under a GPLv2 license, implemented within the open source R statistical environment, and freely available from both the Comprehensive R Archive Network (CRAN) and http://www.cs.york.ac.uk/spartan. The techniques within the package can be applied to traditional ordinary or partial differential equation simulations as well as agent-based implementations. Manuals, comprehensive tutorials, and example simulation data upon which spartan can be applied are available from the website.

Show MeSH
Use of eFAST method within spartan to partition variance in simulation results between parameters.Si (black): the fraction of output variance that can be explained by the value assigned to that parameter; STi (grey): the variance caused by higher-order non-linear effects between that parameter and the others explored. Error bars are standard error over three resample curves. A: Velocity response. B: Displacement response.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3585389&req=5

pcbi-1002916-g004: Use of eFAST method within spartan to partition variance in simulation results between parameters.Si (black): the fraction of output variance that can be explained by the value assigned to that parameter; STi (grey): the variance caused by higher-order non-linear effects between that parameter and the others explored. Error bars are standard error over three resample curves. A: Velocity response. B: Displacement response.

Mentions: Simulation responses are analysed using the Fourier frequency approach [7], [15]. 500 runs were performed for each set of parameter values, as suggested by results generated using Technique 1, and median responses calculated. With 1,365 individual parameter sets, this analysis therefore required 682,500 runs in our case. Plots are created for each simulation output response (velocity and displacement), detailing the first-order (Si) and total-order (STi) sensitivity indexes calculated for each parameter of interest. Indications of significance of each parameters sensitivity index, contrasted to those calculated for the ‘Dummy’ parameter using a two-sample t-test, are output to a CSV file in the directory specified by the user prior to running the analysis. For online tutorial 4, spartan produces the plots in Figure 4. In our published study [9], we utilised our simulator and statistical methods described in techniques one to three, and determined no significant role for chemoattractant factors at an early stage of tissue development. Results shown in Figure 4 examine the same time-point with this additional analysis method, and support these conclusions. We suggest that the factor that influences cell velocity at this early stage in development is the level of expression of cellular adhesion factors (maxVCAMeffectProbabilityCutoff parameter). This supports predictions made by use of Technique 3, but counters the accepted view in the literature, where chemokines are known to have an influential role in tissue development [19], [20]. However results in the literature draw these conclusions from an examination of the whole tissue development time-period (72 hours), rather than just the early stage (12 hours) focused on here and in our previous study [9]. Thus potentially our analysis, using spartan, suggests that different factors could be important at different stages of development. Examining a later time-point in development, both in the lab and through use of spartan, may suggest that this is indeed the case, and the tissue development period is more complex than previously thought.


Spartan: a comprehensive tool for understanding uncertainty in simulations of biological systems.

Alden K, Read M, Timmis J, Andrews PS, Veiga-Fernandes H, Coles M - PLoS Comput. Biol. (2013)

Use of eFAST method within spartan to partition variance in simulation results between parameters.Si (black): the fraction of output variance that can be explained by the value assigned to that parameter; STi (grey): the variance caused by higher-order non-linear effects between that parameter and the others explored. Error bars are standard error over three resample curves. A: Velocity response. B: Displacement response.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3585389&req=5

pcbi-1002916-g004: Use of eFAST method within spartan to partition variance in simulation results between parameters.Si (black): the fraction of output variance that can be explained by the value assigned to that parameter; STi (grey): the variance caused by higher-order non-linear effects between that parameter and the others explored. Error bars are standard error over three resample curves. A: Velocity response. B: Displacement response.
Mentions: Simulation responses are analysed using the Fourier frequency approach [7], [15]. 500 runs were performed for each set of parameter values, as suggested by results generated using Technique 1, and median responses calculated. With 1,365 individual parameter sets, this analysis therefore required 682,500 runs in our case. Plots are created for each simulation output response (velocity and displacement), detailing the first-order (Si) and total-order (STi) sensitivity indexes calculated for each parameter of interest. Indications of significance of each parameters sensitivity index, contrasted to those calculated for the ‘Dummy’ parameter using a two-sample t-test, are output to a CSV file in the directory specified by the user prior to running the analysis. For online tutorial 4, spartan produces the plots in Figure 4. In our published study [9], we utilised our simulator and statistical methods described in techniques one to three, and determined no significant role for chemoattractant factors at an early stage of tissue development. Results shown in Figure 4 examine the same time-point with this additional analysis method, and support these conclusions. We suggest that the factor that influences cell velocity at this early stage in development is the level of expression of cellular adhesion factors (maxVCAMeffectProbabilityCutoff parameter). This supports predictions made by use of Technique 3, but counters the accepted view in the literature, where chemokines are known to have an influential role in tissue development [19], [20]. However results in the literature draw these conclusions from an examination of the whole tissue development time-period (72 hours), rather than just the early stage (12 hours) focused on here and in our previous study [9]. Thus potentially our analysis, using spartan, suggests that different factors could be important at different stages of development. Examining a later time-point in development, both in the lab and through use of spartan, may suggest that this is indeed the case, and the tissue development period is more complex than previously thought.

Bottom Line: Success requires the relationship between simulation and the real-world system to be established: substantial aspects of the biological system are typically unknown, and the abstract nature of simulation can complicate interpretation of in silico results in terms of the biology.The tools comprising spartan help identify which simulation results can be attributed to the dynamics of the modelled biological system, rather than artefacts of biological uncertainty or parametrisation, or simulation stochasticity.We demonstrate the power of spartan in providing critical insight into aspects of lymphoid tissue development in the small intestine through simulation.

View Article: PubMed Central - PubMed

Affiliation: Centre for Systems Biology, School of Biosciences, University of Birmingham, Birmingham, United Kingdom. k.j.alden@bham.ac.uk

ABSTRACT
Integrating computer simulation with conventional wet-lab research has proven to have much potential in furthering the understanding of biological systems. Success requires the relationship between simulation and the real-world system to be established: substantial aspects of the biological system are typically unknown, and the abstract nature of simulation can complicate interpretation of in silico results in terms of the biology. Here we present spartan (Simulation Parameter Analysis RToolkit ApplicatioN), a package of statistical techniques specifically designed to help researchers understand this relationship and provide novel biological insight. The tools comprising spartan help identify which simulation results can be attributed to the dynamics of the modelled biological system, rather than artefacts of biological uncertainty or parametrisation, or simulation stochasticity. Statistical analyses reveal the influence that pathways and components have on simulation behaviour, offering valuable biological insight into aspects of the system under study. We demonstrate the power of spartan in providing critical insight into aspects of lymphoid tissue development in the small intestine through simulation. Spartan is released under a GPLv2 license, implemented within the open source R statistical environment, and freely available from both the Comprehensive R Archive Network (CRAN) and http://www.cs.york.ac.uk/spartan. The techniques within the package can be applied to traditional ordinary or partial differential equation simulations as well as agent-based implementations. Manuals, comprehensive tutorials, and example simulation data upon which spartan can be applied are available from the website.

Show MeSH