Limits...
Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Aranyi G, Pecune F, Charles F, Pelachaud C, Cavazza M - Front Comput Neurosci (2016)

Bottom Line: We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance.We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition.Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

View Article: PubMed Central - PubMed

Affiliation: School of Computing, Teesside University Middlesbrough, UK.

ABSTRACT
Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent's facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

No MeSH data available.


Valence ratings associated with AU and BAP combinations. Note that the four facial-expression categories (negative, neutral, mildly positive, and highly positive) were rated in the intended order.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4940367&req=5

Figure 3: Valence ratings associated with AU and BAP combinations. Note that the four facial-expression categories (negative, neutral, mildly positive, and highly positive) were rated in the intended order.

Mentions: In order to test H1, we ran a one-way repeated measures analysis of variance (ANOVA) with perceived valence as dependent variable. The within-subjects factor was AU and BAP combinations (with eight levels; Figure 2). There was a statistically significant effect of AU and BAP combinations on perceived valence, F(7,105) = 53.91, p < 0.001, η2 is 0.78 (large). Figure 3 shows average valence ratings for each AU and BAP combination with 95% confidence interval (CI). To assess whether the four categories of agent behavior (negative, neutral, mildly positive, and highly positive) were perceived reliably differently, we performed three post hoc pairwise comparisons using Bonferroni correction (one-tailed p < 0.05 criterion adjusted for three comparisons: p < 0.017): (1) highest-rated negative vs. neutral; (2) neutral vs. lowest-rated mild positive; and (3) highest-rated mild positive vs. lowest-rated high positive. Because the parametric assumption of normality was violated for half of the AU and BAP combinations, we report pairwise related-samples comparisons using Wilcoxon Signed Ranks Test.


Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Aranyi G, Pecune F, Charles F, Pelachaud C, Cavazza M - Front Comput Neurosci (2016)

Valence ratings associated with AU and BAP combinations. Note that the four facial-expression categories (negative, neutral, mildly positive, and highly positive) were rated in the intended order.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4940367&req=5

Figure 3: Valence ratings associated with AU and BAP combinations. Note that the four facial-expression categories (negative, neutral, mildly positive, and highly positive) were rated in the intended order.
Mentions: In order to test H1, we ran a one-way repeated measures analysis of variance (ANOVA) with perceived valence as dependent variable. The within-subjects factor was AU and BAP combinations (with eight levels; Figure 2). There was a statistically significant effect of AU and BAP combinations on perceived valence, F(7,105) = 53.91, p < 0.001, η2 is 0.78 (large). Figure 3 shows average valence ratings for each AU and BAP combination with 95% confidence interval (CI). To assess whether the four categories of agent behavior (negative, neutral, mildly positive, and highly positive) were perceived reliably differently, we performed three post hoc pairwise comparisons using Bonferroni correction (one-tailed p < 0.05 criterion adjusted for three comparisons: p < 0.017): (1) highest-rated negative vs. neutral; (2) neutral vs. lowest-rated mild positive; and (3) highest-rated mild positive vs. lowest-rated high positive. Because the parametric assumption of normality was violated for half of the AU and BAP combinations, we report pairwise related-samples comparisons using Wilcoxon Signed Ranks Test.

Bottom Line: We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance.We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition.Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

View Article: PubMed Central - PubMed

Affiliation: School of Computing, Teesside University Middlesbrough, UK.

ABSTRACT
Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent's facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

No MeSH data available.