Limits...
Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Aranyi G, Pecune F, Charles F, Pelachaud C, Cavazza M - Front Comput Neurosci (2016)

Bottom Line: We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance.We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition.Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

View Article: PubMed Central - PubMed

Affiliation: School of Computing, Teesside University Middlesbrough, UK.

ABSTRACT
Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent's facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

No MeSH data available.


Example of a successful block, where asymmetry during NF is significantly larger than during View.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4940367&req=5

Figure 5: Example of a successful block, where asymmetry during NF is significantly larger than during View.

Mentions: We included the View epoch in each block as a reference (with the mental arithmetic task unrelated to asymmetry) to support feedback mapping for NF within the same block, in the following way. We defined the threshold (i.e., minimum asymmetry value resulting in feedback) during NF based on asymmetry values collected during View within the same block (Figure 5). The minimum point (Min) for mapping was defined as the mean of asymmetry values during the View epoch plus 1.28 times their SD. Assuming normally distributed asymmetry scores, this threshold would result in no feedback for 90% of the asymmetry values during View. This approach to determine NF threshold is consistent with the original one of Rosenfeld et al. (1995) for EEG-based frontal-asymmetry NF. To determine the maximum point (Max) for mapping (i.e., the asymmetry value resulting in feedback with maximum magnitude), we added the variation range of asymmetry values during View to the threshold Min we defined above. Outliers (values outside three SDs from the mean) were removed for calculating the threshold and range in order to prevent extreme values, likely resulting from movement artifacts, exerting an unduly influence on NF mapping. Asymmetry values within the range [Min; Max] during NF were mapped linearly onto the virtual agent’s facial expression, with the same 2 Hz frequency as the acquisition of asymmetry values.


Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Aranyi G, Pecune F, Charles F, Pelachaud C, Cavazza M - Front Comput Neurosci (2016)

Example of a successful block, where asymmetry during NF is significantly larger than during View.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4940367&req=5

Figure 5: Example of a successful block, where asymmetry during NF is significantly larger than during View.
Mentions: We included the View epoch in each block as a reference (with the mental arithmetic task unrelated to asymmetry) to support feedback mapping for NF within the same block, in the following way. We defined the threshold (i.e., minimum asymmetry value resulting in feedback) during NF based on asymmetry values collected during View within the same block (Figure 5). The minimum point (Min) for mapping was defined as the mean of asymmetry values during the View epoch plus 1.28 times their SD. Assuming normally distributed asymmetry scores, this threshold would result in no feedback for 90% of the asymmetry values during View. This approach to determine NF threshold is consistent with the original one of Rosenfeld et al. (1995) for EEG-based frontal-asymmetry NF. To determine the maximum point (Max) for mapping (i.e., the asymmetry value resulting in feedback with maximum magnitude), we added the variation range of asymmetry values during View to the threshold Min we defined above. Outliers (values outside three SDs from the mean) were removed for calculating the threshold and range in order to prevent extreme values, likely resulting from movement artifacts, exerting an unduly influence on NF mapping. Asymmetry values within the range [Min; Max] during NF were mapped linearly onto the virtual agent’s facial expression, with the same 2 Hz frequency as the acquisition of asymmetry values.

Bottom Line: We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance.We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition.Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

View Article: PubMed Central - PubMed

Affiliation: School of Computing, Teesside University Middlesbrough, UK.

ABSTRACT
Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent's facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

No MeSH data available.