Limits...
On the precarious path of reverse neuro-engineering.

Marom S, Meir R, Braun E, Gal A, Kermany E, Eytan D - Front Comput Neurosci (2009)

Bottom Line: We demonstrate that application of reverse engineering to the study of the design principle of a functional neuro-system with a known mechanism, may result in a perfectly valid but wrong induction of the system's design principle.If in the very simple setup we bring here (static environment, primitive task and practically unlimited access to every piece of relevant information), it is difficult to induce a design principle, what are our chances of exposing biological design principles when more realistic conditions are examined?Implications to the way we do Biology are discussed.

View Article: PubMed Central - PubMed

Affiliation: Network Biology Research Laboratories, Technion - Israel Institute of Technology Haifa, Israel. marom@technion.ac.il

ABSTRACT
In this perspective we provide an example for the limits of reverse engineering in neuroscience. We demonstrate that application of reverse engineering to the study of the design principle of a functional neuro-system with a known mechanism, may result in a perfectly valid but wrong induction of the system's design principle. If in the very simple setup we bring here (static environment, primitive task and practically unlimited access to every piece of relevant information), it is difficult to induce a design principle, what are our chances of exposing biological design principles when more realistic conditions are examined? Implications to the way we do Biology are discussed.

No MeSH data available.


Accuracy of input classification (i.e. classification of the eye that triggered the network stimulation). A classifier (Gaussian kernel) was constructed using network spike profiles of the kind shown in the bottom panel of Figure 2. Each blue point is the average classification accuracy obtained by 30 independent executions of the classification procedure (80% training set); error bars depict standard deviation. Parameters of the network spike histograms are depicted inside the plot: (x; y) is a network spike histogram computed over x ms, starting 10 ms following the stimulus, using y ms bin size. Analyses of (100;25) and (10;5) are shown, indicating that the result obtained by analyzing (100;5) is by and large valid under more restricted conditions. Gray points depict the classification of same data set using Euclidean distance based cluster analysis. All computations were carried out within Mathematica (Wolfram Research) environment.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2691154&req=5

Figure 3: Accuracy of input classification (i.e. classification of the eye that triggered the network stimulation). A classifier (Gaussian kernel) was constructed using network spike profiles of the kind shown in the bottom panel of Figure 2. Each blue point is the average classification accuracy obtained by 30 independent executions of the classification procedure (80% training set); error bars depict standard deviation. Parameters of the network spike histograms are depicted inside the plot: (x; y) is a network spike histogram computed over x ms, starting 10 ms following the stimulus, using y ms bin size. Analyses of (100;25) and (10;5) are shown, indicating that the result obtained by analyzing (100;5) is by and large valid under more restricted conditions. Gray points depict the classification of same data set using Euclidean distance based cluster analysis. All computations were carried out within Mathematica (Wolfram Research) environment.

Mentions: There are several ways to test an hypothesis about the validity of a given representation scheme in neurophysiology. One very efficient and bias-free way is to use state-of-the-art non-linear classifiers. Indeed, our dedicated neurophysiologist uses the non-linear version of Support Vector Machine approach (Vapnik, 1998): Data is transformed to a space where linear classification is performed. To avoid over-fitting, only a fraction of the data is used for the construction of the classifier, and the efficacy of categorization by population response rate is evaluated by testing the classifier on the complementary (unseen) set of the data. The blue point of Figure 3, denoted (100;5), shows the efficacy of categorization using vectors of population spike rate constructed over a 100 ms time window at 5 ms bin size; categorization is very good (accuracy 0.9), for all practical purposes. In other words, population response rate provides an accurate input categorization. So, concludes the neurophysiologist, population response rate is (or, “may be,” as a less cavalier physiologists would say) the scheme of representation, the “neural code.” But it is wrong; we know it is wrong because we have designed the machine otherwise. Of course, one might say that the neurophysiologist is too hasty in jumping to conclusions; but honestly, how many of us (physiologists) try to find an alternative design principle to one at hand that is 80–90% accurate in predicting the results? Moreover, if in the very simple neural setup examined here (static environment, primitive task and practically unlimited access to every piece of relevant information), it is difficult to induce a design principle, what are our chances of exposing biological design principles when more realistic conditions are examined?


On the precarious path of reverse neuro-engineering.

Marom S, Meir R, Braun E, Gal A, Kermany E, Eytan D - Front Comput Neurosci (2009)

Accuracy of input classification (i.e. classification of the eye that triggered the network stimulation). A classifier (Gaussian kernel) was constructed using network spike profiles of the kind shown in the bottom panel of Figure 2. Each blue point is the average classification accuracy obtained by 30 independent executions of the classification procedure (80% training set); error bars depict standard deviation. Parameters of the network spike histograms are depicted inside the plot: (x; y) is a network spike histogram computed over x ms, starting 10 ms following the stimulus, using y ms bin size. Analyses of (100;25) and (10;5) are shown, indicating that the result obtained by analyzing (100;5) is by and large valid under more restricted conditions. Gray points depict the classification of same data set using Euclidean distance based cluster analysis. All computations were carried out within Mathematica (Wolfram Research) environment.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2691154&req=5

Figure 3: Accuracy of input classification (i.e. classification of the eye that triggered the network stimulation). A classifier (Gaussian kernel) was constructed using network spike profiles of the kind shown in the bottom panel of Figure 2. Each blue point is the average classification accuracy obtained by 30 independent executions of the classification procedure (80% training set); error bars depict standard deviation. Parameters of the network spike histograms are depicted inside the plot: (x; y) is a network spike histogram computed over x ms, starting 10 ms following the stimulus, using y ms bin size. Analyses of (100;25) and (10;5) are shown, indicating that the result obtained by analyzing (100;5) is by and large valid under more restricted conditions. Gray points depict the classification of same data set using Euclidean distance based cluster analysis. All computations were carried out within Mathematica (Wolfram Research) environment.
Mentions: There are several ways to test an hypothesis about the validity of a given representation scheme in neurophysiology. One very efficient and bias-free way is to use state-of-the-art non-linear classifiers. Indeed, our dedicated neurophysiologist uses the non-linear version of Support Vector Machine approach (Vapnik, 1998): Data is transformed to a space where linear classification is performed. To avoid over-fitting, only a fraction of the data is used for the construction of the classifier, and the efficacy of categorization by population response rate is evaluated by testing the classifier on the complementary (unseen) set of the data. The blue point of Figure 3, denoted (100;5), shows the efficacy of categorization using vectors of population spike rate constructed over a 100 ms time window at 5 ms bin size; categorization is very good (accuracy 0.9), for all practical purposes. In other words, population response rate provides an accurate input categorization. So, concludes the neurophysiologist, population response rate is (or, “may be,” as a less cavalier physiologists would say) the scheme of representation, the “neural code.” But it is wrong; we know it is wrong because we have designed the machine otherwise. Of course, one might say that the neurophysiologist is too hasty in jumping to conclusions; but honestly, how many of us (physiologists) try to find an alternative design principle to one at hand that is 80–90% accurate in predicting the results? Moreover, if in the very simple neural setup examined here (static environment, primitive task and practically unlimited access to every piece of relevant information), it is difficult to induce a design principle, what are our chances of exposing biological design principles when more realistic conditions are examined?

Bottom Line: We demonstrate that application of reverse engineering to the study of the design principle of a functional neuro-system with a known mechanism, may result in a perfectly valid but wrong induction of the system's design principle.If in the very simple setup we bring here (static environment, primitive task and practically unlimited access to every piece of relevant information), it is difficult to induce a design principle, what are our chances of exposing biological design principles when more realistic conditions are examined?Implications to the way we do Biology are discussed.

View Article: PubMed Central - PubMed

Affiliation: Network Biology Research Laboratories, Technion - Israel Institute of Technology Haifa, Israel. marom@technion.ac.il

ABSTRACT
In this perspective we provide an example for the limits of reverse engineering in neuroscience. We demonstrate that application of reverse engineering to the study of the design principle of a functional neuro-system with a known mechanism, may result in a perfectly valid but wrong induction of the system's design principle. If in the very simple setup we bring here (static environment, primitive task and practically unlimited access to every piece of relevant information), it is difficult to induce a design principle, what are our chances of exposing biological design principles when more realistic conditions are examined? Implications to the way we do Biology are discussed.

No MeSH data available.