Limits...
The effect of target and non-target similarity on neural classification performance: a boost from confidence.

Marathe AR, Ries AJ, Lawhern VJ, Lance BJ, Touryan J, McDowell K, Cecotti H - Front Neurosci (2015)

Bottom Line: In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes.It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented.This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets.

View Article: PubMed Central - PubMed

Affiliation: Translational Neuroscience Branch, US Army Research Laboratory, Human Research and Engineering Directorate Aberdeen Proving Grounds, MD, USA.

ABSTRACT
Brain computer interaction (BCI) technologies have proven effective in utilizing single-trial classification algorithms to detect target images in rapid serial visualization presentation tasks. While many factors contribute to the accuracy of these algorithms, a critical aspect that is often overlooked concerns the feature similarity between target and non-target images. In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes. It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented. This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets. The resulting findings show that behavior is slower and less accurate when targets are presented together with similar non-targets; moreover, single-trial classification yielded high levels of misclassification when infrequent non-targets are included. Furthermore, we present an approach to mitigate the image misclassification. We use confidence measures to assess the quality of single-trial classification, and demonstrate that a system in which low confidence trials are reclassified through a secondary process can result in improved performance.

No MeSH data available.


Grand-average ERP waveforms at electrode Pz and topographic voltage maps (400–800 ms); white dot indicates location of electrode Pz. (A) Shows grand-average ERP waveforms and topographic maps to target and background distractor stimuli in the Target Only (TO) condition. (B) Shows grand-average ERP waveforms and topographic maps to target, non-target and background distractor stimuli in the Target plus Non-target (TN) condition. (C) Shows difference waves created by subtracting the background distractor from targets in the TO condition and the background distractor from targets and non-targets in the TN condition.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4544215&req=5

Figure 3: Grand-average ERP waveforms at electrode Pz and topographic voltage maps (400–800 ms); white dot indicates location of electrode Pz. (A) Shows grand-average ERP waveforms and topographic maps to target and background distractor stimuli in the Target Only (TO) condition. (B) Shows grand-average ERP waveforms and topographic maps to target, non-target and background distractor stimuli in the Target plus Non-target (TN) condition. (C) Shows difference waves created by subtracting the background distractor from targets in the TO condition and the background distractor from targets and non-targets in the TN condition.

Mentions: Statistical comparisons of grand average ERP waveforms demonstrated that ERPs were significantly different across stimulus type, with visually-similar non-targets generating ERPs with amplitudes between those of target stimuli and background distracters. In addition, ERPs for background distractor and target stimuli were not significantly different across the TO and TN conditions. A one-way ANOVA was used to analyze the mean amplitude (400–800 ms) from electrode Pz with stimulus (background distractor, target, non-target) as a main factor. There was a main effect for stimulus in the TO condition, [F(1, 16) = 111.34, p < 0.001], indicating a significantly larger P3 amplitude for targets (mean amplitude: 13.66 μV) relative to background distractors (mean amplitude: −0.44 μV, Figure 3A). A main effect was also obtained in the TN condition [F(2, 32) = 83.01, p < 0.001]. Subsequent multiple comparison tests using the Tukey-Kramer method showed that amplitudes from background distractors, targets, and non-targets were all significantly different from each other (Figure 3B). A Two-Way ANOVA was run with the factors of Condition (TO or TN) and stimulus (distractor or target) to assess any differences between target P3 amplitude in the two conditions. There was a main effect of stimulus [F(1, 16) = 344.33, p < 0.001] but no main effect for condition [F(1, 16) = 0.001, p = 0.978] or interaction [F(1, 16) = 0.002, p = 0.964] indicating that both the background distractor and target activity was similar between the TO and TN conditions, and that there were significant differences between background distractor and target activity in both the TO and TN conditions (Figure 3).


The effect of target and non-target similarity on neural classification performance: a boost from confidence.

Marathe AR, Ries AJ, Lawhern VJ, Lance BJ, Touryan J, McDowell K, Cecotti H - Front Neurosci (2015)

Grand-average ERP waveforms at electrode Pz and topographic voltage maps (400–800 ms); white dot indicates location of electrode Pz. (A) Shows grand-average ERP waveforms and topographic maps to target and background distractor stimuli in the Target Only (TO) condition. (B) Shows grand-average ERP waveforms and topographic maps to target, non-target and background distractor stimuli in the Target plus Non-target (TN) condition. (C) Shows difference waves created by subtracting the background distractor from targets in the TO condition and the background distractor from targets and non-targets in the TN condition.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4544215&req=5

Figure 3: Grand-average ERP waveforms at electrode Pz and topographic voltage maps (400–800 ms); white dot indicates location of electrode Pz. (A) Shows grand-average ERP waveforms and topographic maps to target and background distractor stimuli in the Target Only (TO) condition. (B) Shows grand-average ERP waveforms and topographic maps to target, non-target and background distractor stimuli in the Target plus Non-target (TN) condition. (C) Shows difference waves created by subtracting the background distractor from targets in the TO condition and the background distractor from targets and non-targets in the TN condition.
Mentions: Statistical comparisons of grand average ERP waveforms demonstrated that ERPs were significantly different across stimulus type, with visually-similar non-targets generating ERPs with amplitudes between those of target stimuli and background distracters. In addition, ERPs for background distractor and target stimuli were not significantly different across the TO and TN conditions. A one-way ANOVA was used to analyze the mean amplitude (400–800 ms) from electrode Pz with stimulus (background distractor, target, non-target) as a main factor. There was a main effect for stimulus in the TO condition, [F(1, 16) = 111.34, p < 0.001], indicating a significantly larger P3 amplitude for targets (mean amplitude: 13.66 μV) relative to background distractors (mean amplitude: −0.44 μV, Figure 3A). A main effect was also obtained in the TN condition [F(2, 32) = 83.01, p < 0.001]. Subsequent multiple comparison tests using the Tukey-Kramer method showed that amplitudes from background distractors, targets, and non-targets were all significantly different from each other (Figure 3B). A Two-Way ANOVA was run with the factors of Condition (TO or TN) and stimulus (distractor or target) to assess any differences between target P3 amplitude in the two conditions. There was a main effect of stimulus [F(1, 16) = 344.33, p < 0.001] but no main effect for condition [F(1, 16) = 0.001, p = 0.978] or interaction [F(1, 16) = 0.002, p = 0.964] indicating that both the background distractor and target activity was similar between the TO and TN conditions, and that there were significant differences between background distractor and target activity in both the TO and TN conditions (Figure 3).

Bottom Line: In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes.It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented.This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets.

View Article: PubMed Central - PubMed

Affiliation: Translational Neuroscience Branch, US Army Research Laboratory, Human Research and Engineering Directorate Aberdeen Proving Grounds, MD, USA.

ABSTRACT
Brain computer interaction (BCI) technologies have proven effective in utilizing single-trial classification algorithms to detect target images in rapid serial visualization presentation tasks. While many factors contribute to the accuracy of these algorithms, a critical aspect that is often overlooked concerns the feature similarity between target and non-target images. In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes. It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented. This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets. The resulting findings show that behavior is slower and less accurate when targets are presented together with similar non-targets; moreover, single-trial classification yielded high levels of misclassification when infrequent non-targets are included. Furthermore, we present an approach to mitigate the image misclassification. We use confidence measures to assess the quality of single-trial classification, and demonstrate that a system in which low confidence trials are reclassified through a secondary process can result in improved performance.

No MeSH data available.