Limits...
Understanding the limits of animal models as predictors of human biology: lessons learned from the sbv IMPROVER Species Translation Challenge.

Rhrissorrakrai K, Belcastro V, Bilal E, Norel R, Poussin C, Mathis C, Dulize RH, Ivanov NV, Alexopoulos L, Rice JJ, Peitsch MC, Stolovitzky G, Meyer P, Hoeng J - Bioinformatics (2014)

Bottom Line: Participating teams submitted 49 different solutions across the sub-challenges, two-thirds of which were statistically significantly better than random.Additionally, similar computational methods were found to range widely in their performance within the same challenge, and no single method emerged as a clear winner across all sub-challenges.Finally, computational methods were able to effectively translate some specific stimuli and biological processes in the lung epithelial system, such as DNA synthesis, cytoskeleton and extracellular matrix, translation, immune/inflammation and growth factor/proliferation pathways, better than the expected response similarity between species. pmeyerr@us.ibm.com or Julia.Hoeng@pmi.com Supplementary data are available at Bioinformatics online.

View Article: PubMed Central - PubMed

Affiliation: IBM T.J. Watson Research Center, Computational Biology Center, Yorktown Heights, NY 10003, USA, Philip Morris International R&D, Philip Morris Products S.A., 2000 Neuchâtel, Switzerland, Telethon Institute of Genetics and Medicine, Via Pietro Castellino, 111, 80131 Naples, Italy, ProtATonce Ltd, Scientific Park Lefkippos, Patriarchou Grigoriou & Neapoleos 15343 Ag. Paraskevi, Attiki and National Technical University of Athens, Heroon Polytechniou 9, Zografou 15780, Greece.

Show MeSH

Related in: MedlinePlus

Predictability versus species similarity for stimuli. (A) The y-axis indicates for each stimulus the mean predictability Prs of all team predictions when considering gene set activation in SC3. The x-axis is species similarity Ss of gene set activation. In red are stimuli where Prs > Ss > 0. (B) The y-axis indicates for each stimulus the mean predictability Prs of all team predictions when considering protein phosphorylation activation in SC2. The x-axis is Sp of phosphoprotein activation. In red are stimuli where Prs > Ss > 0. (C, D) Plots showing the percentage of teams where Prs > Ss for each stimulus when predicting gene set activation (C) or phosphoprotein activation (D). Stimuli are ordered by percentage of teams and the number of activated gene sets or phosphorylated proteins is indicated on top of each stimulus. The number of active calls per gene set is shown on the top of the graph. Nineteen stimuli are not shown in (B) and (D) because no proteins were measured as phosphorylated
© Copyright Policy - creative-commons
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4325540&req=5

btu611-F3: Predictability versus species similarity for stimuli. (A) The y-axis indicates for each stimulus the mean predictability Prs of all team predictions when considering gene set activation in SC3. The x-axis is species similarity Ss of gene set activation. In red are stimuli where Prs > Ss > 0. (B) The y-axis indicates for each stimulus the mean predictability Prs of all team predictions when considering protein phosphorylation activation in SC2. The x-axis is Sp of phosphoprotein activation. In red are stimuli where Prs > Ss > 0. (C, D) Plots showing the percentage of teams where Prs > Ss for each stimulus when predicting gene set activation (C) or phosphoprotein activation (D). Stimuli are ordered by percentage of teams and the number of activated gene sets or phosphorylated proteins is indicated on top of each stimulus. The number of active calls per gene set is shown on the top of the graph. Nineteen stimuli are not shown in (B) and (D) because no proteins were measured as phosphorylated

Mentions: To assess how the accuracy of the participants’ predictions depended on the nature of the stimulus applied, we defined the species similarity S and the predictability or teams’ prediction performance Pr. Briefly, S is the MCC between rat and human GS, and Pr is the MCC between a team’s submission and the human GS. A high S value would indicate a putatively conserved response between rat and human; a high Pr suggests the signal is well translated by participants. S and Pr could be defined for stimuli based on gene set or protein phosphorylation activation. S and Pr could also be defined for gene set and phosphorylation activation based on response to stimuli (see methods for details). Figure 3 shows the mean Prs for all participants plotted against Ss based on the activation of gene sets (Fig. 3A) and protein phosphorylation (Fig. 3B).Fig. 3.


Understanding the limits of animal models as predictors of human biology: lessons learned from the sbv IMPROVER Species Translation Challenge.

Rhrissorrakrai K, Belcastro V, Bilal E, Norel R, Poussin C, Mathis C, Dulize RH, Ivanov NV, Alexopoulos L, Rice JJ, Peitsch MC, Stolovitzky G, Meyer P, Hoeng J - Bioinformatics (2014)

Predictability versus species similarity for stimuli. (A) The y-axis indicates for each stimulus the mean predictability Prs of all team predictions when considering gene set activation in SC3. The x-axis is species similarity Ss of gene set activation. In red are stimuli where Prs > Ss > 0. (B) The y-axis indicates for each stimulus the mean predictability Prs of all team predictions when considering protein phosphorylation activation in SC2. The x-axis is Sp of phosphoprotein activation. In red are stimuli where Prs > Ss > 0. (C, D) Plots showing the percentage of teams where Prs > Ss for each stimulus when predicting gene set activation (C) or phosphoprotein activation (D). Stimuli are ordered by percentage of teams and the number of activated gene sets or phosphorylated proteins is indicated on top of each stimulus. The number of active calls per gene set is shown on the top of the graph. Nineteen stimuli are not shown in (B) and (D) because no proteins were measured as phosphorylated
© Copyright Policy - creative-commons
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4325540&req=5

btu611-F3: Predictability versus species similarity for stimuli. (A) The y-axis indicates for each stimulus the mean predictability Prs of all team predictions when considering gene set activation in SC3. The x-axis is species similarity Ss of gene set activation. In red are stimuli where Prs > Ss > 0. (B) The y-axis indicates for each stimulus the mean predictability Prs of all team predictions when considering protein phosphorylation activation in SC2. The x-axis is Sp of phosphoprotein activation. In red are stimuli where Prs > Ss > 0. (C, D) Plots showing the percentage of teams where Prs > Ss for each stimulus when predicting gene set activation (C) or phosphoprotein activation (D). Stimuli are ordered by percentage of teams and the number of activated gene sets or phosphorylated proteins is indicated on top of each stimulus. The number of active calls per gene set is shown on the top of the graph. Nineteen stimuli are not shown in (B) and (D) because no proteins were measured as phosphorylated
Mentions: To assess how the accuracy of the participants’ predictions depended on the nature of the stimulus applied, we defined the species similarity S and the predictability or teams’ prediction performance Pr. Briefly, S is the MCC between rat and human GS, and Pr is the MCC between a team’s submission and the human GS. A high S value would indicate a putatively conserved response between rat and human; a high Pr suggests the signal is well translated by participants. S and Pr could be defined for stimuli based on gene set or protein phosphorylation activation. S and Pr could also be defined for gene set and phosphorylation activation based on response to stimuli (see methods for details). Figure 3 shows the mean Prs for all participants plotted against Ss based on the activation of gene sets (Fig. 3A) and protein phosphorylation (Fig. 3B).Fig. 3.

Bottom Line: Participating teams submitted 49 different solutions across the sub-challenges, two-thirds of which were statistically significantly better than random.Additionally, similar computational methods were found to range widely in their performance within the same challenge, and no single method emerged as a clear winner across all sub-challenges.Finally, computational methods were able to effectively translate some specific stimuli and biological processes in the lung epithelial system, such as DNA synthesis, cytoskeleton and extracellular matrix, translation, immune/inflammation and growth factor/proliferation pathways, better than the expected response similarity between species. pmeyerr@us.ibm.com or Julia.Hoeng@pmi.com Supplementary data are available at Bioinformatics online.

View Article: PubMed Central - PubMed

Affiliation: IBM T.J. Watson Research Center, Computational Biology Center, Yorktown Heights, NY 10003, USA, Philip Morris International R&D, Philip Morris Products S.A., 2000 Neuchâtel, Switzerland, Telethon Institute of Genetics and Medicine, Via Pietro Castellino, 111, 80131 Naples, Italy, ProtATonce Ltd, Scientific Park Lefkippos, Patriarchou Grigoriou & Neapoleos 15343 Ag. Paraskevi, Attiki and National Technical University of Athens, Heroon Polytechniou 9, Zografou 15780, Greece.

Show MeSH
Related in: MedlinePlus