Limits...
Estimating confidence intervals in predicted responses for oscillatory biological models.

St John PC, Doyle FJ - BMC Syst Biol (2013)

Bottom Line: However, since a model's parameter values are uncertain, and since dynamic responses to inputs are highly parameter-dependent, it is difficult to assess the confidence associated with these in silico predictions.Additionally, we show how this method can be used for model discrimination by comparing the output identifiability of two candidate model structures to published literature data.Our method permits modellers of oscillatory systems to confidently show that a model's dynamic characteristics follow directly from experimental data and model structure, relaxing assumptions on the particular parameters chosen.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Chemical Engineering, University of California, Santa Barbara, CA 93106-5080, USA.

ABSTRACT

Background: The dynamics of gene regulation play a crucial role in a cellular control: allowing the cell to express the right proteins to meet changing needs. Some needs, such as correctly anticipating the day-night cycle, require complicated oscillatory features. In the analysis of gene regulatory networks, mathematical models are frequently used to understand how a network's structure enables it to respond appropriately to external inputs. These models typically consist of a set of ordinary differential equations, describing a network of biochemical reactions, and unknown kinetic parameters, chosen such that the model best captures experimental data. However, since a model's parameter values are uncertain, and since dynamic responses to inputs are highly parameter-dependent, it is difficult to assess the confidence associated with these in silico predictions. In particular, models with complex dynamics - such as oscillations - must be fit with computationally expensive global optimization routines, and cannot take advantage of existing measures of identifiability. Despite their difficulty to model mathematically, limit cycle oscillations play a key role in many biological processes, including cell cycling, metabolism, neuron firing, and circadian rhythms.

Results: In this study, we employ an efficient parameter estimation technique to enable a bootstrap uncertainty analysis for limit cycle models. Since the primary role of systems biology models is the insight they provide on responses to rate perturbations, we extend our uncertainty analysis to include first order sensitivity coefficients. Using a literature model of circadian rhythms, we show how predictive precision is degraded with decreasing sample points and increasing relative error. Additionally, we show how this method can be used for model discrimination by comparing the output identifiability of two candidate model structures to published literature data.

Conclusions: Our method permits modellers of oscillatory systems to confidently show that a model's dynamic characteristics follow directly from experimental data and model structure, relaxing assumptions on the particular parameters chosen. Ultimately, this work highlights the importance of continued collection of high-resolution data on gene and protein activity levels, as they allow the development of predictive mathematical models.

Show MeSH
Identifiability comparison of two model structures. (A) Bootstrap parameter estimations on two model structures using literature time-series data with estimated errors (box plots). Resulting regions of model trajectories are shaded between the 5 th and 95 th percentile. Per species are shown in purple, Cry1 in red, and Cry2 in green. While both models were able to approximately reproduce the same dynamic response, the expanded model was better able to capture differences between the Cry1 and Cry2 profiles. (B) Parameter and sensitivity identifiability for the base and expanded models. Violin plots show the parameter and sensitivity distributions, with unidentifiable sensitivities (90% confidence level) highlighted in red. Despite containing more parameters, the expanded model shows better parameter identifiability and higher confidence in its predicted sensitivities. The PER translation rate (1) and PER-CRY association rate (2) sensitivities are consistent across model equations and are highlighted.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3733791&req=5

Figure 5: Identifiability comparison of two model structures. (A) Bootstrap parameter estimations on two model structures using literature time-series data with estimated errors (box plots). Resulting regions of model trajectories are shaded between the 5 th and 95 th percentile. Per species are shown in purple, Cry1 in red, and Cry2 in green. While both models were able to approximately reproduce the same dynamic response, the expanded model was better able to capture differences between the Cry1 and Cry2 profiles. (B) Parameter and sensitivity identifiability for the base and expanded models. Violin plots show the parameter and sensitivity distributions, with unidentifiable sensitivities (90% confidence level) highlighted in red. Despite containing more parameters, the expanded model shows better parameter identifiability and higher confidence in its predicted sensitivities. The PER translation rate (1) and PER-CRY association rate (2) sensitivities are consistent across model equations and are highlighted.

Mentions: We next apply the method to literature time-course data for core clock components[26]. When modeling a genetic regulatory network, many candidate model equations are often considered. We show that a bootstrap uncertainty analysis can also be useful in discriminating between potential model structures based on predictive confidence. Here two variations of the same model are fit, see Additional file2. The first model (Figure5, base) was originally optimized using a genetic algorithm approach, and thus contains a minimal number of parameters to reduce optimization complexity. The second model considered (Figure5, expanded) contains independent parameters for each rate expression, increasing the number of parameters from 23 to 35.


Estimating confidence intervals in predicted responses for oscillatory biological models.

St John PC, Doyle FJ - BMC Syst Biol (2013)

Identifiability comparison of two model structures. (A) Bootstrap parameter estimations on two model structures using literature time-series data with estimated errors (box plots). Resulting regions of model trajectories are shaded between the 5 th and 95 th percentile. Per species are shown in purple, Cry1 in red, and Cry2 in green. While both models were able to approximately reproduce the same dynamic response, the expanded model was better able to capture differences between the Cry1 and Cry2 profiles. (B) Parameter and sensitivity identifiability for the base and expanded models. Violin plots show the parameter and sensitivity distributions, with unidentifiable sensitivities (90% confidence level) highlighted in red. Despite containing more parameters, the expanded model shows better parameter identifiability and higher confidence in its predicted sensitivities. The PER translation rate (1) and PER-CRY association rate (2) sensitivities are consistent across model equations and are highlighted.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3733791&req=5

Figure 5: Identifiability comparison of two model structures. (A) Bootstrap parameter estimations on two model structures using literature time-series data with estimated errors (box plots). Resulting regions of model trajectories are shaded between the 5 th and 95 th percentile. Per species are shown in purple, Cry1 in red, and Cry2 in green. While both models were able to approximately reproduce the same dynamic response, the expanded model was better able to capture differences between the Cry1 and Cry2 profiles. (B) Parameter and sensitivity identifiability for the base and expanded models. Violin plots show the parameter and sensitivity distributions, with unidentifiable sensitivities (90% confidence level) highlighted in red. Despite containing more parameters, the expanded model shows better parameter identifiability and higher confidence in its predicted sensitivities. The PER translation rate (1) and PER-CRY association rate (2) sensitivities are consistent across model equations and are highlighted.
Mentions: We next apply the method to literature time-course data for core clock components[26]. When modeling a genetic regulatory network, many candidate model equations are often considered. We show that a bootstrap uncertainty analysis can also be useful in discriminating between potential model structures based on predictive confidence. Here two variations of the same model are fit, see Additional file2. The first model (Figure5, base) was originally optimized using a genetic algorithm approach, and thus contains a minimal number of parameters to reduce optimization complexity. The second model considered (Figure5, expanded) contains independent parameters for each rate expression, increasing the number of parameters from 23 to 35.

Bottom Line: However, since a model's parameter values are uncertain, and since dynamic responses to inputs are highly parameter-dependent, it is difficult to assess the confidence associated with these in silico predictions.Additionally, we show how this method can be used for model discrimination by comparing the output identifiability of two candidate model structures to published literature data.Our method permits modellers of oscillatory systems to confidently show that a model's dynamic characteristics follow directly from experimental data and model structure, relaxing assumptions on the particular parameters chosen.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Chemical Engineering, University of California, Santa Barbara, CA 93106-5080, USA.

ABSTRACT

Background: The dynamics of gene regulation play a crucial role in a cellular control: allowing the cell to express the right proteins to meet changing needs. Some needs, such as correctly anticipating the day-night cycle, require complicated oscillatory features. In the analysis of gene regulatory networks, mathematical models are frequently used to understand how a network's structure enables it to respond appropriately to external inputs. These models typically consist of a set of ordinary differential equations, describing a network of biochemical reactions, and unknown kinetic parameters, chosen such that the model best captures experimental data. However, since a model's parameter values are uncertain, and since dynamic responses to inputs are highly parameter-dependent, it is difficult to assess the confidence associated with these in silico predictions. In particular, models with complex dynamics - such as oscillations - must be fit with computationally expensive global optimization routines, and cannot take advantage of existing measures of identifiability. Despite their difficulty to model mathematically, limit cycle oscillations play a key role in many biological processes, including cell cycling, metabolism, neuron firing, and circadian rhythms.

Results: In this study, we employ an efficient parameter estimation technique to enable a bootstrap uncertainty analysis for limit cycle models. Since the primary role of systems biology models is the insight they provide on responses to rate perturbations, we extend our uncertainty analysis to include first order sensitivity coefficients. Using a literature model of circadian rhythms, we show how predictive precision is degraded with decreasing sample points and increasing relative error. Additionally, we show how this method can be used for model discrimination by comparing the output identifiability of two candidate model structures to published literature data.

Conclusions: Our method permits modellers of oscillatory systems to confidently show that a model's dynamic characteristics follow directly from experimental data and model structure, relaxing assumptions on the particular parameters chosen. Ultimately, this work highlights the importance of continued collection of high-resolution data on gene and protein activity levels, as they allow the development of predictive mathematical models.

Show MeSH