Limits...
The effect of target and non-target similarity on neural classification performance: a boost from confidence.

Marathe AR, Ries AJ, Lawhern VJ, Lance BJ, Touryan J, McDowell K, Cecotti H - Front Neurosci (2015)

Bottom Line: In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes.It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented.This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets.

View Article: PubMed Central - PubMed

Affiliation: Translational Neuroscience Branch, US Army Research Laboratory, Human Research and Engineering Directorate Aberdeen Proving Grounds, MD, USA.

ABSTRACT
Brain computer interaction (BCI) technologies have proven effective in utilizing single-trial classification algorithms to detect target images in rapid serial visualization presentation tasks. While many factors contribute to the accuracy of these algorithms, a critical aspect that is often overlooked concerns the feature similarity between target and non-target images. In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes. It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented. This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets. The resulting findings show that behavior is slower and less accurate when targets are presented together with similar non-targets; moreover, single-trial classification yielded high levels of misclassification when infrequent non-targets are included. Furthermore, we present an approach to mitigate the image misclassification. We use confidence measures to assess the quality of single-trial classification, and demonstrate that a system in which low confidence trials are reclassified through a secondary process can result in improved performance.

No MeSH data available.


Behavioral Performance. (A) Shows error rates for each stimulus type for both TO (light gray) and TN (dark gray) conditions. (B) Shows target reaction time for both conditions. (C) Shows d-prime measures for both conditions. Error bars show the highest and lowest data point within 1.5 times the inter-quartile range of the upper and lower quartiles, respectively. Within each box, crosses indicate mean values and horizontal lines indicate median values.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4544215&req=5

Figure 2: Behavioral Performance. (A) Shows error rates for each stimulus type for both TO (light gray) and TN (dark gray) conditions. (B) Shows target reaction time for both conditions. (C) Shows d-prime measures for both conditions. Error bars show the highest and lowest data point within 1.5 times the inter-quartile range of the upper and lower quartiles, respectively. Within each box, crosses indicate mean values and horizontal lines indicate median values.

Mentions: Behavioral performance was characterized by comparing the error rate by stimulus type, reaction time, and d-prime across the TO and TN conditions (Figure 2). Across all three measures, behavioral performance declined when non-targets were included. Adding non-targets more than doubled the average error rate for target stimuli (difference significant, Wilcoxon signed rank test, p < 0.01, Figure 2A). Reaction times obtained from correct target trials were significantly faster in the TO condition (median RT of 514.67 ms) when compared to the TN condition (median RT of 602.82 ms) (Wilcoxon signed rank test, p < 0.001, Figure 2B). D-prime analysis showed that target discrimination performance was significantly better for TO trials (median d-prime of 4.25) over TN trials (median d-prime of 3.49) (Wilcoxon signed rank test, p < 0.01, Figure 2C).


The effect of target and non-target similarity on neural classification performance: a boost from confidence.

Marathe AR, Ries AJ, Lawhern VJ, Lance BJ, Touryan J, McDowell K, Cecotti H - Front Neurosci (2015)

Behavioral Performance. (A) Shows error rates for each stimulus type for both TO (light gray) and TN (dark gray) conditions. (B) Shows target reaction time for both conditions. (C) Shows d-prime measures for both conditions. Error bars show the highest and lowest data point within 1.5 times the inter-quartile range of the upper and lower quartiles, respectively. Within each box, crosses indicate mean values and horizontal lines indicate median values.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4544215&req=5

Figure 2: Behavioral Performance. (A) Shows error rates for each stimulus type for both TO (light gray) and TN (dark gray) conditions. (B) Shows target reaction time for both conditions. (C) Shows d-prime measures for both conditions. Error bars show the highest and lowest data point within 1.5 times the inter-quartile range of the upper and lower quartiles, respectively. Within each box, crosses indicate mean values and horizontal lines indicate median values.
Mentions: Behavioral performance was characterized by comparing the error rate by stimulus type, reaction time, and d-prime across the TO and TN conditions (Figure 2). Across all three measures, behavioral performance declined when non-targets were included. Adding non-targets more than doubled the average error rate for target stimuli (difference significant, Wilcoxon signed rank test, p < 0.01, Figure 2A). Reaction times obtained from correct target trials were significantly faster in the TO condition (median RT of 514.67 ms) when compared to the TN condition (median RT of 602.82 ms) (Wilcoxon signed rank test, p < 0.001, Figure 2B). D-prime analysis showed that target discrimination performance was significantly better for TO trials (median d-prime of 4.25) over TN trials (median d-prime of 3.49) (Wilcoxon signed rank test, p < 0.01, Figure 2C).

Bottom Line: In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes.It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented.This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets.

View Article: PubMed Central - PubMed

Affiliation: Translational Neuroscience Branch, US Army Research Laboratory, Human Research and Engineering Directorate Aberdeen Proving Grounds, MD, USA.

ABSTRACT
Brain computer interaction (BCI) technologies have proven effective in utilizing single-trial classification algorithms to detect target images in rapid serial visualization presentation tasks. While many factors contribute to the accuracy of these algorithms, a critical aspect that is often overlooked concerns the feature similarity between target and non-target images. In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes. It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented. This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets. The resulting findings show that behavior is slower and less accurate when targets are presented together with similar non-targets; moreover, single-trial classification yielded high levels of misclassification when infrequent non-targets are included. Furthermore, we present an approach to mitigate the image misclassification. We use confidence measures to assess the quality of single-trial classification, and demonstrate that a system in which low confidence trials are reclassified through a secondary process can result in improved performance.

No MeSH data available.