Limits...
Domain-specific model selection for structural identification of the Rab5-Rab7 dynamics in endocytosis.

Tanevski J, Todorovski L, Kalaidzidis Y, Džeroski S - BMC Syst Biol (2015)

Bottom Line: Furthermore, taking into account the complexity of the model does not lead to better model selection.However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria.We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

View Article: PubMed Central - PubMed

Affiliation: Jožef Stefan Institute, Jamova cesta 39, Ljubljana, 1000, Slovenia. jovan.tanevski@ijs.si.

ABSTRACT

Background: Given its recent rapid development and the central role that modeling plays in the discipline, systems biology clearly needs methods for automated modeling of dynamical systems. Process-based modeling focuses on explanatory models of dynamical systems; it constructs such models from measured time-course data and formalized modeling knowledge. In this paper, we apply process-based modeling to the practically relevant task of modeling the Rab5-Rab7 conversion switch in endocytosis. The task is difficult due to the limited observability of the system variables and the noisy measurements, which pose serious challenges to the process of model selection. To address these issues, we propose a domain-specific model selection criteria that take into account knowledge about the necessary properties of the simulated model behavior.

Results: In a series of modeling experiments, we compare the results of process-based modeling obtained with different model selection criteria. The first is the standard maximum likelihood criterion based solely on least-squares model error. The second one is a parsimony-based criterion that also takes into account model complexity. We also introduce three domain-specific criteria based on domain expert expectations about the simulated behavior of an endocytosis model. According to the first criterion, 90 of the candidate models are indistinguishable. Furthermore, taking into account the complexity of the model does not lead to better model selection. However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria.

Conclusions: We demonstrate the applicability of process-based modeling to the task of modeling the Rab5-Rab7 dynamics in endocytosis. Our experiments show that the domain-specific criteria outperform the standard domain-independent criteria for model selection. We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

No MeSH data available.


The size of the error-profile plateau (black line) and the average ranks of the structures belonging to the COT (green line), IP (yellow line) and NOBS (red line) group obtained using the criterion ER. The plot is obtained by varying the values of the α trade-off parameter in the range [0,1] with an increment of 0.1
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4482292&req=5

Fig7: The size of the error-profile plateau (black line) and the average ranks of the structures belonging to the COT (green line), IP (yellow line) and NOBS (red line) group obtained using the criterion ER. The plot is obtained by varying the values of the α trade-off parameter in the range [0,1] with an increment of 0.1

Mentions: Figure 7 shows the influence of the change of the trade-off parameter α on the plateau size and the average ranks of the COT, IP and NOBS models in the list of models ranked using the ER criterion. Small and large values of α lead to large plateaus, with a significant drop of the plateau size for α=0.4 and a minimum at α=0.5. Note that this value also leads to the smallest average ranks of the plausible model structures. Additional file 1: Figure S3 provides further details on the results of the modeling experiment using the ER criterion with α=0.5. The size of the plateau is 62, i.e., less than 50 % of all the candidate models; a significant improvement in discriminative power over the 90 % obtained with E. Out of these 62 models, 13 have structures belonging to the COT group, 8 to the IP group, and 6 to the NOBS group. The range of errors is tight with a mean value of 0.42, a median of 0.45 and a standard deviation of 0.04. Note that the obtained behavior of some of the models in the first plateau can be considered as unsatisfactory, for example, the simulation of one of the active-state concentrations of the proteins can be uncorrelated to the corresponding measured density even though the correlation is taken into account within the ER criterion during optimization. We believe that this is due to the strong influence of the E component in the used criterion, combined with the imperfect optimization and the identifiability issues presented below.Fig. 7


Domain-specific model selection for structural identification of the Rab5-Rab7 dynamics in endocytosis.

Tanevski J, Todorovski L, Kalaidzidis Y, Džeroski S - BMC Syst Biol (2015)

The size of the error-profile plateau (black line) and the average ranks of the structures belonging to the COT (green line), IP (yellow line) and NOBS (red line) group obtained using the criterion ER. The plot is obtained by varying the values of the α trade-off parameter in the range [0,1] with an increment of 0.1
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4482292&req=5

Fig7: The size of the error-profile plateau (black line) and the average ranks of the structures belonging to the COT (green line), IP (yellow line) and NOBS (red line) group obtained using the criterion ER. The plot is obtained by varying the values of the α trade-off parameter in the range [0,1] with an increment of 0.1
Mentions: Figure 7 shows the influence of the change of the trade-off parameter α on the plateau size and the average ranks of the COT, IP and NOBS models in the list of models ranked using the ER criterion. Small and large values of α lead to large plateaus, with a significant drop of the plateau size for α=0.4 and a minimum at α=0.5. Note that this value also leads to the smallest average ranks of the plausible model structures. Additional file 1: Figure S3 provides further details on the results of the modeling experiment using the ER criterion with α=0.5. The size of the plateau is 62, i.e., less than 50 % of all the candidate models; a significant improvement in discriminative power over the 90 % obtained with E. Out of these 62 models, 13 have structures belonging to the COT group, 8 to the IP group, and 6 to the NOBS group. The range of errors is tight with a mean value of 0.42, a median of 0.45 and a standard deviation of 0.04. Note that the obtained behavior of some of the models in the first plateau can be considered as unsatisfactory, for example, the simulation of one of the active-state concentrations of the proteins can be uncorrelated to the corresponding measured density even though the correlation is taken into account within the ER criterion during optimization. We believe that this is due to the strong influence of the E component in the used criterion, combined with the imperfect optimization and the identifiability issues presented below.Fig. 7

Bottom Line: Furthermore, taking into account the complexity of the model does not lead to better model selection.However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria.We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

View Article: PubMed Central - PubMed

Affiliation: Jožef Stefan Institute, Jamova cesta 39, Ljubljana, 1000, Slovenia. jovan.tanevski@ijs.si.

ABSTRACT

Background: Given its recent rapid development and the central role that modeling plays in the discipline, systems biology clearly needs methods for automated modeling of dynamical systems. Process-based modeling focuses on explanatory models of dynamical systems; it constructs such models from measured time-course data and formalized modeling knowledge. In this paper, we apply process-based modeling to the practically relevant task of modeling the Rab5-Rab7 conversion switch in endocytosis. The task is difficult due to the limited observability of the system variables and the noisy measurements, which pose serious challenges to the process of model selection. To address these issues, we propose a domain-specific model selection criteria that take into account knowledge about the necessary properties of the simulated model behavior.

Results: In a series of modeling experiments, we compare the results of process-based modeling obtained with different model selection criteria. The first is the standard maximum likelihood criterion based solely on least-squares model error. The second one is a parsimony-based criterion that also takes into account model complexity. We also introduce three domain-specific criteria based on domain expert expectations about the simulated behavior of an endocytosis model. According to the first criterion, 90 of the candidate models are indistinguishable. Furthermore, taking into account the complexity of the model does not lead to better model selection. However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria.

Conclusions: We demonstrate the applicability of process-based modeling to the task of modeling the Rab5-Rab7 dynamics in endocytosis. Our experiments show that the domain-specific criteria outperform the standard domain-independent criteria for model selection. We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

No MeSH data available.