Limits...
When the optimal is not the best: parameter estimation in complex biological models.

Fernández Slezak D, Suárez C, Cecchi GA, Marshall G, Stolovitzky G - PLoS ONE (2010)

Bottom Line: This cost function has many local minima with unrealistic solutions, including the global minimum corresponding to the best fit.To avoid force-fitting a model to a dataset, we propose that the best model parameters should be found by choosing, among suboptimal parameters, those that match criteria other than the ones used to fit the model.We also conclude that the model, data and optimization approach form a new complex system and point to the need of a theory that addresses this problem more generally.

View Article: PubMed Central - PubMed

Affiliation: Laboratorio de Sistemas Complejos, Depto de Computación, FCEyN, Buenos Aires University, Buenos Aires, Argentina. dfslezak@dc.uba.ar

ABSTRACT

Background: The vast computational resources that became available during the past decade enabled the development and simulation of increasingly complex mathematical models of cancer growth. These models typically involve many free parameters whose determination is a substantial obstacle to model development. Direct measurement of biochemical parameters in vivo is often difficult and sometimes impracticable, while fitting them under data-poor conditions may result in biologically implausible values.

Results: We discuss different methodological approaches to estimate parameters in complex biological models. We make use of the high computational power of the Blue Gene technology to perform an extensive study of the parameter space in a model of avascular tumor growth. We explicitly show that the landscape of the cost function used to optimize the model to the data has a very rugged surface in parameter space. This cost function has many local minima with unrealistic solutions, including the global minimum corresponding to the best fit.

Conclusions: The case studied in this paper shows one example in which model parameters that optimally fit the data are not necessarily the best ones from a biological point of view. To avoid force-fitting a model to a dataset, we propose that the best model parameters should be found by choosing, among suboptimal parameters, those that match criteria other than the ones used to fit the model. We also conclude that the model, data and optimization approach form a new complex system and point to the need of a theory that addresses this problem more generally.

Show MeSH

Related in: MedlinePlus

The landscape of the cost function as a function of the 6th () and 4th () parameters.The cost function value has been averaged over the values that correspond to the same values of  and , but for which the other coordinates differed. This data was taken from all runs available of all methods.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC2963600&req=5

pone-0013283-g004: The landscape of the cost function as a function of the 6th () and 4th () parameters.The cost function value has been averaged over the values that correspond to the same values of and , but for which the other coordinates differed. This data was taken from all runs available of all methods.

Mentions: The location and number of local minima shown in Figure 3 suggest that the cost function landscape is a rather rugged one, plagued with local minima. In Figure 4 we explicitly constructed the surface of the cost function as a function of parameters 4 and 6, using all the runs available from all the optimization methods used, yielding a total of more than 100,000 evaluations. This figure shows an extremely rugged landscape with many peaks and valleys permeating the parameter space. This ruggedness is smoothed out by the fact that the value of the cost function shown in Figure 4 is the average over of the cost function values with the same values of parameters and , but different values for the other four parameters. We also tried different strategies to represent the cost function landscape, such as plotting the minimum over of the cost function with the same values of parameters and . This alternative representation of the cost function yielded a similarly rugged landscape (data not shown).


When the optimal is not the best: parameter estimation in complex biological models.

Fernández Slezak D, Suárez C, Cecchi GA, Marshall G, Stolovitzky G - PLoS ONE (2010)

The landscape of the cost function as a function of the 6th () and 4th () parameters.The cost function value has been averaged over the values that correspond to the same values of  and , but for which the other coordinates differed. This data was taken from all runs available of all methods.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC2963600&req=5

pone-0013283-g004: The landscape of the cost function as a function of the 6th () and 4th () parameters.The cost function value has been averaged over the values that correspond to the same values of and , but for which the other coordinates differed. This data was taken from all runs available of all methods.
Mentions: The location and number of local minima shown in Figure 3 suggest that the cost function landscape is a rather rugged one, plagued with local minima. In Figure 4 we explicitly constructed the surface of the cost function as a function of parameters 4 and 6, using all the runs available from all the optimization methods used, yielding a total of more than 100,000 evaluations. This figure shows an extremely rugged landscape with many peaks and valleys permeating the parameter space. This ruggedness is smoothed out by the fact that the value of the cost function shown in Figure 4 is the average over of the cost function values with the same values of parameters and , but different values for the other four parameters. We also tried different strategies to represent the cost function landscape, such as plotting the minimum over of the cost function with the same values of parameters and . This alternative representation of the cost function yielded a similarly rugged landscape (data not shown).

Bottom Line: This cost function has many local minima with unrealistic solutions, including the global minimum corresponding to the best fit.To avoid force-fitting a model to a dataset, we propose that the best model parameters should be found by choosing, among suboptimal parameters, those that match criteria other than the ones used to fit the model.We also conclude that the model, data and optimization approach form a new complex system and point to the need of a theory that addresses this problem more generally.

View Article: PubMed Central - PubMed

Affiliation: Laboratorio de Sistemas Complejos, Depto de Computación, FCEyN, Buenos Aires University, Buenos Aires, Argentina. dfslezak@dc.uba.ar

ABSTRACT

Background: The vast computational resources that became available during the past decade enabled the development and simulation of increasingly complex mathematical models of cancer growth. These models typically involve many free parameters whose determination is a substantial obstacle to model development. Direct measurement of biochemical parameters in vivo is often difficult and sometimes impracticable, while fitting them under data-poor conditions may result in biologically implausible values.

Results: We discuss different methodological approaches to estimate parameters in complex biological models. We make use of the high computational power of the Blue Gene technology to perform an extensive study of the parameter space in a model of avascular tumor growth. We explicitly show that the landscape of the cost function used to optimize the model to the data has a very rugged surface in parameter space. This cost function has many local minima with unrealistic solutions, including the global minimum corresponding to the best fit.

Conclusions: The case studied in this paper shows one example in which model parameters that optimally fit the data are not necessarily the best ones from a biological point of view. To avoid force-fitting a model to a dataset, we propose that the best model parameters should be found by choosing, among suboptimal parameters, those that match criteria other than the ones used to fit the model. We also conclude that the model, data and optimization approach form a new complex system and point to the need of a theory that addresses this problem more generally.

Show MeSH
Related in: MedlinePlus