Limits...
Domain-specific model selection for structural identification of the Rab5-Rab7 dynamics in endocytosis.

Tanevski J, Todorovski L, Kalaidzidis Y, Džeroski S - BMC Syst Biol (2015)

Bottom Line: Furthermore, taking into account the complexity of the model does not lead to better model selection.However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria.We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

View Article: PubMed Central - PubMed

Affiliation: Jožef Stefan Institute, Jamova cesta 39, Ljubljana, 1000, Slovenia. jovan.tanevski@ijs.si.

ABSTRACT

Background: Given its recent rapid development and the central role that modeling plays in the discipline, systems biology clearly needs methods for automated modeling of dynamical systems. Process-based modeling focuses on explanatory models of dynamical systems; it constructs such models from measured time-course data and formalized modeling knowledge. In this paper, we apply process-based modeling to the practically relevant task of modeling the Rab5-Rab7 conversion switch in endocytosis. The task is difficult due to the limited observability of the system variables and the noisy measurements, which pose serious challenges to the process of model selection. To address these issues, we propose a domain-specific model selection criteria that take into account knowledge about the necessary properties of the simulated model behavior.

Results: In a series of modeling experiments, we compare the results of process-based modeling obtained with different model selection criteria. The first is the standard maximum likelihood criterion based solely on least-squares model error. The second one is a parsimony-based criterion that also takes into account model complexity. We also introduce three domain-specific criteria based on domain expert expectations about the simulated behavior of an endocytosis model. According to the first criterion, 90 of the candidate models are indistinguishable. Furthermore, taking into account the complexity of the model does not lead to better model selection. However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria.

Conclusions: We demonstrate the applicability of process-based modeling to the task of modeling the Rab5-Rab7 dynamics in endocytosis. Our experiments show that the domain-specific criteria outperform the standard domain-independent criteria for model selection. We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

No MeSH data available.


Error profile. Sorted ranking of the 126 models according to the estimated values of the ER criterion. The trade-off parameter setting is α=0.5. Two long and two short plateaus can be identified in this error profile
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4482292&req=5

Fig4: Error profile. Sorted ranking of the 126 models according to the estimated values of the ER criterion. The trade-off parameter setting is α=0.5. Two long and two short plateaus can be identified in this error profile

Mentions: The first performance metric describes the ability of the model selection method to discriminate between the 126 model structures considered by ProBMoT. To measure the discriminative power of a particular model selection criterion, we run a ProBMoT experiment where the given criterion is used to rank the models. We then depict the error profile, i.e., plot the value of the given criterion for each model against the increasing model rank; see Fig. 4 for an example. Furthermore, we refer to the initial flat region of the error profile as the plateau; its length equals the number of models it contains. A simple heuristic for detecting the plateau is the test whether there is more than 10 % error difference between two consecutive points. The first such difference indicates the end of the plateau. For example, the plateau of the error profile in Fig. 4 contains 62 models. Note that the plateau represents the set of top-ranked model structures that are indistinguishable in terms of the model selection criterion used to rank them. The fewer models in the plateau, the better the performance of the model selection criterion, i.e., its ability to discriminate between the candidate model structures.Fig. 4


Domain-specific model selection for structural identification of the Rab5-Rab7 dynamics in endocytosis.

Tanevski J, Todorovski L, Kalaidzidis Y, Džeroski S - BMC Syst Biol (2015)

Error profile. Sorted ranking of the 126 models according to the estimated values of the ER criterion. The trade-off parameter setting is α=0.5. Two long and two short plateaus can be identified in this error profile
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4482292&req=5

Fig4: Error profile. Sorted ranking of the 126 models according to the estimated values of the ER criterion. The trade-off parameter setting is α=0.5. Two long and two short plateaus can be identified in this error profile
Mentions: The first performance metric describes the ability of the model selection method to discriminate between the 126 model structures considered by ProBMoT. To measure the discriminative power of a particular model selection criterion, we run a ProBMoT experiment where the given criterion is used to rank the models. We then depict the error profile, i.e., plot the value of the given criterion for each model against the increasing model rank; see Fig. 4 for an example. Furthermore, we refer to the initial flat region of the error profile as the plateau; its length equals the number of models it contains. A simple heuristic for detecting the plateau is the test whether there is more than 10 % error difference between two consecutive points. The first such difference indicates the end of the plateau. For example, the plateau of the error profile in Fig. 4 contains 62 models. Note that the plateau represents the set of top-ranked model structures that are indistinguishable in terms of the model selection criterion used to rank them. The fewer models in the plateau, the better the performance of the model selection criterion, i.e., its ability to discriminate between the candidate model structures.Fig. 4

Bottom Line: Furthermore, taking into account the complexity of the model does not lead to better model selection.However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria.We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

View Article: PubMed Central - PubMed

Affiliation: Jožef Stefan Institute, Jamova cesta 39, Ljubljana, 1000, Slovenia. jovan.tanevski@ijs.si.

ABSTRACT

Background: Given its recent rapid development and the central role that modeling plays in the discipline, systems biology clearly needs methods for automated modeling of dynamical systems. Process-based modeling focuses on explanatory models of dynamical systems; it constructs such models from measured time-course data and formalized modeling knowledge. In this paper, we apply process-based modeling to the practically relevant task of modeling the Rab5-Rab7 conversion switch in endocytosis. The task is difficult due to the limited observability of the system variables and the noisy measurements, which pose serious challenges to the process of model selection. To address these issues, we propose a domain-specific model selection criteria that take into account knowledge about the necessary properties of the simulated model behavior.

Results: In a series of modeling experiments, we compare the results of process-based modeling obtained with different model selection criteria. The first is the standard maximum likelihood criterion based solely on least-squares model error. The second one is a parsimony-based criterion that also takes into account model complexity. We also introduce three domain-specific criteria based on domain expert expectations about the simulated behavior of an endocytosis model. According to the first criterion, 90 of the candidate models are indistinguishable. Furthermore, taking into account the complexity of the model does not lead to better model selection. However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria.

Conclusions: We demonstrate the applicability of process-based modeling to the task of modeling the Rab5-Rab7 dynamics in endocytosis. Our experiments show that the domain-specific criteria outperform the standard domain-independent criteria for model selection. We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

No MeSH data available.