• Japanese
• Main Page
• Lab HP

• Research Top
• to appear
• Media Timing
• Facial Score
• Hybrid Dynamical System
• Multi-viewpoint Gesture Recognition
• Dissertation

• Publication List
• Top Page

Multi-Viewpoint Gesture Recognition by an Integrated Continuous State Machine

Dynamic Integration of Recognition Modules for Robust Event Recognition with Occluded Observations

This research proposes a system architecture for event recognition that dynamically integrates information from multiple sources (e.g., multi-modal data from visual and auditory sensors). The proposed system consists of multiple event classifiers named Continuous State Machines (CSMs). Each CSM has a state transition rule in a continuous state space and classifies time-varying patterns from a different single source. Since the rule is defined as a extension of Kalman filters (i.e., the next state is deduced from the trade-off scheme between input data and model's prediction), CSMs support dynamic time warping and robustness against noise.

We then introduce an interaction method among CSMs to classify events from multiple sources (Fig. 1). A continuous state space (i.e., vector space) allows us to design interaction as minimization of an energy function. This interaction enables the system to dynamically suppress unreliable classifiers and improves the system reliability and accuracy of classifying events in dynamically changing situations (e.g., the object is temporary occluded from one of multiple cameras in a gesture recognition task) (Fig. 2). Experimental results on gesture recognition by two cameras show the effectiveness of our proposed system.

References

1. Hiroaki Kawashima, Takashi Matsuyama, "Multi-viewpoint gesture recognition by an integrated continuous state machine", Systems and Computers in Japan, Vol.34, No.14, pp.1--12, 2003 (in English, translation of the paper below)
2. [PDF] Hiroaki Kawashima, Takashi Matsuyama, "Multi-Viewpoint Gesture Recognition by an Integrated Continuous State Machine", The Transaction of The Institute of Electronics, Information and Communication Engineers, Vol.J85-D-II, No.12, pp.1801--1812, 2002 (in Japanese).
3. [PDF] Hiroaki Kawashima, Takashi Matsuyama: "Integrated Event Recognition from Multiple Sources", 16th International Conference on Pattern Recognition (ICPR), Vol.2, pp785-789, 2002.