Limits...
Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Aranyi G, Pecune F, Charles F, Pelachaud C, Cavazza M - Front Comput Neurosci (2016)

Bottom Line: We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance.We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition.Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

View Article: PubMed Central - PubMed

Affiliation: School of Computing, Teesside University Middlesbrough, UK.

ABSTRACT
Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent's facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

No MeSH data available.


System overview. Brain signals are collected through functional near infrared spectroscopy (fNIRS) system (A) where left-most and right-most channels are processed to generate a left-asymmetry score (1). During the View epoch (2), the left-asymmetry values are used to define the Min and Max bounds (3) to be used during the Neurofeedback (NF) epoch where the real-time left-asymmetry scores (4) are normalized (B) before being used as single input (5) to the virtual agent’s facial expressions action units (AUs) and body action parameters under the neural network’s control (C).
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4940367&req=5

Figure 1: System overview. Brain signals are collected through functional near infrared spectroscopy (fNIRS) system (A) where left-most and right-most channels are processed to generate a left-asymmetry score (1). During the View epoch (2), the left-asymmetry values are used to define the Min and Max bounds (3) to be used during the Neurofeedback (NF) epoch where the real-time left-asymmetry scores (4) are normalized (B) before being used as single input (5) to the virtual agent’s facial expressions action units (AUs) and body action parameters under the neural network’s control (C).

Mentions: To support our experiments with virtual agent’s control from a single affective dimension, we designed a complete software platform, which is presented in Figure 1. From the user’s perspective, the virtual agent behaves autonomously as a response to what it perceives as the user’s mental disposition towards itself. Users are instructed to express positive feelings towards the agent in order to capture its interest. This should in turn result in the agent responding to the user with an expression matching the perceived interest in both valence and intensity.


Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Aranyi G, Pecune F, Charles F, Pelachaud C, Cavazza M - Front Comput Neurosci (2016)

System overview. Brain signals are collected through functional near infrared spectroscopy (fNIRS) system (A) where left-most and right-most channels are processed to generate a left-asymmetry score (1). During the View epoch (2), the left-asymmetry values are used to define the Min and Max bounds (3) to be used during the Neurofeedback (NF) epoch where the real-time left-asymmetry scores (4) are normalized (B) before being used as single input (5) to the virtual agent’s facial expressions action units (AUs) and body action parameters under the neural network’s control (C).
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4940367&req=5

Figure 1: System overview. Brain signals are collected through functional near infrared spectroscopy (fNIRS) system (A) where left-most and right-most channels are processed to generate a left-asymmetry score (1). During the View epoch (2), the left-asymmetry values are used to define the Min and Max bounds (3) to be used during the Neurofeedback (NF) epoch where the real-time left-asymmetry scores (4) are normalized (B) before being used as single input (5) to the virtual agent’s facial expressions action units (AUs) and body action parameters under the neural network’s control (C).
Mentions: To support our experiments with virtual agent’s control from a single affective dimension, we designed a complete software platform, which is presented in Figure 1. From the user’s perspective, the virtual agent behaves autonomously as a response to what it perceives as the user’s mental disposition towards itself. Users are instructed to express positive feelings towards the agent in order to capture its interest. This should in turn result in the agent responding to the user with an expression matching the perceived interest in both valence and intensity.

Bottom Line: We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance.We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition.Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

View Article: PubMed Central - PubMed

Affiliation: School of Computing, Teesside University Middlesbrough, UK.

ABSTRACT
Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent's facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

No MeSH data available.