Limits...
Computational state space models for activity and intention recognition. A feasibility study.

Krüger F, Nyolt M, Yordanova K, Hein A, Kirste T - PLoS ONE (2014)

Bottom Line: Our results suggest that the combinatorial explosion caused by rich CSSM models does not inevitably lead to intractable inference or inferior performance.This means that the potential benefits of CSSM models (knowledge-based model construction, model reusability, reduced need for training data) are available without performance penalty.However, our results also show that research on CSSMs needs to consider sufficiently complex domains in order to understand the effects of design decisions such as choice of heuristics or inference procedure on performance.

View Article: PubMed Central - PubMed

Affiliation: Computer Science Institute, University of Rostock, Rostock, Germany.

ABSTRACT

Background: Computational state space models (CSSMs) enable the knowledge-based construction of Bayesian filters for recognizing intentions and reconstructing activities of human protagonists in application domains such as smart environments, assisted living, or security. Computational, i. e., algorithmic, representations allow the construction of increasingly complex human behaviour models. However, the symbolic models used in CSSMs potentially suffer from combinatorial explosion, rendering inference intractable outside of the limited experimental settings investigated in present research. The objective of this study was to obtain data on the feasibility of CSSM-based inference in domains of realistic complexity.

Methods: A typical instrumental activity of daily living was used as a trial scenario. As primary sensor modality, wearable inertial measurement units were employed. The results achievable by CSSM methods were evaluated by comparison with those obtained from established training-based methods (hidden Markov models, HMMs) using Wilcoxon signed rank tests. The influence of modeling factors on CSSM performance was analyzed via repeated measures analysis of variance.

Results: The symbolic domain model was found to have more than 10(8) states, exceeding the complexity of models considered in previous research by at least three orders of magnitude. Nevertheless, if factors and procedures governing the inference process were suitably chosen, CSSMs outperformed HMMs. Specifically, inference methods used in previous studies (particle filters) were found to perform substantially inferior in comparison to a marginal filtering procedure.

Conclusions: Our results suggest that the combinatorial explosion caused by rich CSSM models does not inevitably lead to intractable inference or inferior performance. This means that the potential benefits of CSSM models (knowledge-based model construction, model reusability, reduced need for training data) are available without performance penalty. However, our results also show that research on CSSMs needs to consider sufficiently complex domains in order to understand the effects of design decisions such as choice of heuristics or inference procedure on performance.

Show MeSH

Related in: MedlinePlus

Per-class performance measures.Detailed accuracies for the configuration (OS21s, , L1, ).
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4220990&req=5

pone-0109381-g006: Per-class performance measures.Detailed accuracies for the configuration (OS21s, , L1, ).

Mentions: For testing , the configurations (CM {f, s}, O21s, , L1, ) were chosen. These configurations use the least amount of information from the training data, are the least restrictive considering the state space, and employ the most rational action selection heuristic available in this study. Comparison of HMMf and CMf showed a significant median increase of accuracy for CMf by 3.63pp (Wilcoxon signed rank test, , ). For HMMs and CMs a median increase of 6.78pp was found (, ). Both results do not support the hypothesis that CSSM models perform at the same level of accuracy as training-based models, instead they suggest that CSSM performance exceeds baseline performance. (For comparisons of the other configurations, see Table S8.) This is also supported by the per-class performance data shown in Fig. 6. (The confusion matrices from which this data has been computed are included in Fig. S5.) Considering the per-class performance there was no strong difference in overall classification behavior between QDA, HMM, and CSSM. However, the score suggests an overall more balanced performance for CSSM. We also see that there are fundamentally “difficult” action classes (TAKE, PUT, WAIT). Plots of the estimated action class sequences versus ground truth for all (O21s, , L1, ) configurations and the corresponding baseline results are given in Fig. S6.


Computational state space models for activity and intention recognition. A feasibility study.

Krüger F, Nyolt M, Yordanova K, Hein A, Kirste T - PLoS ONE (2014)

Per-class performance measures.Detailed accuracies for the configuration (OS21s, , L1, ).
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4220990&req=5

pone-0109381-g006: Per-class performance measures.Detailed accuracies for the configuration (OS21s, , L1, ).
Mentions: For testing , the configurations (CM {f, s}, O21s, , L1, ) were chosen. These configurations use the least amount of information from the training data, are the least restrictive considering the state space, and employ the most rational action selection heuristic available in this study. Comparison of HMMf and CMf showed a significant median increase of accuracy for CMf by 3.63pp (Wilcoxon signed rank test, , ). For HMMs and CMs a median increase of 6.78pp was found (, ). Both results do not support the hypothesis that CSSM models perform at the same level of accuracy as training-based models, instead they suggest that CSSM performance exceeds baseline performance. (For comparisons of the other configurations, see Table S8.) This is also supported by the per-class performance data shown in Fig. 6. (The confusion matrices from which this data has been computed are included in Fig. S5.) Considering the per-class performance there was no strong difference in overall classification behavior between QDA, HMM, and CSSM. However, the score suggests an overall more balanced performance for CSSM. We also see that there are fundamentally “difficult” action classes (TAKE, PUT, WAIT). Plots of the estimated action class sequences versus ground truth for all (O21s, , L1, ) configurations and the corresponding baseline results are given in Fig. S6.

Bottom Line: Our results suggest that the combinatorial explosion caused by rich CSSM models does not inevitably lead to intractable inference or inferior performance.This means that the potential benefits of CSSM models (knowledge-based model construction, model reusability, reduced need for training data) are available without performance penalty.However, our results also show that research on CSSMs needs to consider sufficiently complex domains in order to understand the effects of design decisions such as choice of heuristics or inference procedure on performance.

View Article: PubMed Central - PubMed

Affiliation: Computer Science Institute, University of Rostock, Rostock, Germany.

ABSTRACT

Background: Computational state space models (CSSMs) enable the knowledge-based construction of Bayesian filters for recognizing intentions and reconstructing activities of human protagonists in application domains such as smart environments, assisted living, or security. Computational, i. e., algorithmic, representations allow the construction of increasingly complex human behaviour models. However, the symbolic models used in CSSMs potentially suffer from combinatorial explosion, rendering inference intractable outside of the limited experimental settings investigated in present research. The objective of this study was to obtain data on the feasibility of CSSM-based inference in domains of realistic complexity.

Methods: A typical instrumental activity of daily living was used as a trial scenario. As primary sensor modality, wearable inertial measurement units were employed. The results achievable by CSSM methods were evaluated by comparison with those obtained from established training-based methods (hidden Markov models, HMMs) using Wilcoxon signed rank tests. The influence of modeling factors on CSSM performance was analyzed via repeated measures analysis of variance.

Results: The symbolic domain model was found to have more than 10(8) states, exceeding the complexity of models considered in previous research by at least three orders of magnitude. Nevertheless, if factors and procedures governing the inference process were suitably chosen, CSSMs outperformed HMMs. Specifically, inference methods used in previous studies (particle filters) were found to perform substantially inferior in comparison to a marginal filtering procedure.

Conclusions: Our results suggest that the combinatorial explosion caused by rich CSSM models does not inevitably lead to intractable inference or inferior performance. This means that the potential benefits of CSSM models (knowledge-based model construction, model reusability, reduced need for training data) are available without performance penalty. However, our results also show that research on CSSMs needs to consider sufficiently complex domains in order to understand the effects of design decisions such as choice of heuristics or inference procedure on performance.

Show MeSH
Related in: MedlinePlus