Beyond tracking: modelling activity and understanding behaviour

• Main Authors: Tao Xiang, Shaogang Gong
• Other Authors: The Pennsylvania State University CiteSeerX Archives
• Format: Text
• Language: English
• Published: 2006
• Subjects: Dynamic scene modelling; Graph models; Discrete event recognition; Activity representa- tion; Behaviour recognition; Dynamic probabilistic networks; Bayesian model selection; DML
• Online Access: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.109.7294; http://www.dcs.qmul.ac.uk/~txiang/xiang-gong-VISI790-041-accepted.pdf

In this work, we present a unified bottom-up and top-down automatic model selection based approach for modelling complex activities of multiple objects in cluttered scenes. An activity of multiple objects is represented based on discrete scene events and their behaviours are modelled by reasoning about the temporal and causal correlations among different events. This is significantly different from the ma-jority of the existing techniques that are centred on object tracking followed by trajectory matching. In our approach, object-independent events are detected and classified by unsupervised clustering us-ing Expectation-Maximisation (EM) and classified using automatic model selection based on Schwarz’s Bayesian Information Criterion (BIC). Dynamic Probabilistic Networks (DPNs) are formulated for mod-elling the temporal and causal correlations among discrete events for robust and holistic scene-level be-haviour interpretation. In particular, we developed a Dynamically Multi-Linked Hidden Markov Model (DML-HMM) based on the discovery of salient dynamic interlinks among multiple temporal processes corresponding to multiple event classes. A DML-HMM is built using BIC based factorisation result-ing in its topology being intrinsically determined by the underlying causality and temporal order among events. Extensive experiments are conducted on modelling activities captured in different indoor and