STAP: Spatial-Temporal Attention-Aware Pooling for Action Recognition

Citation data:

IEEE Transactions on Circuits and Systems for Video Technology, ISSN: 1051-8215, Vol: 25, Issue: 1, Page: 77-86

Publication Year:
Usage 253
Abstract Views 250
Link-outs 3
Captures 33
Readers 22
Exports-Saves 11
Citations 17
Citation Indexes 17
Repository URL:
Tam V. Nguyen; Zheng Song; Shuicheng Yan
Institute of Electrical and Electronics Engineers (IEEE); IEEE; eCommons
Engineering; Computer Sciences; Databases and Information Systems; Graphics and Human Computer Interfaces; Other Computer Sciences
article description
Human action recognition is valuable for numerous practical applications, e.g., gaming, video surveillance, and video search. In this paper we hypothesize that the classification of actions can be boosted by designing a smart feature pooling strategy under the prevalently used bag-of-words-based representation. Founded on automatic video saliency analysis, we propose the spatial-temporal attention-aware pooling scheme for feature pooling. First, the video saliencies are predicted using the video saliency model, and the localized spatial-temporal features are pooled at different saliency levels and video-saliency-guided channels are formed. Saliency-aware matching kernels are thus derived as the similarity measurement of these channels. Intuitively, the proposed kernels calculate the similarities of the video foreground (salient areas) or background (nonsalient areas) at different levels. Finally, the kernels are fed into popular support vector machines for action classification. Extensive experiments on three popular data sets for action classification validate the effectiveness of our proposed method, which outperforms the state-of-the-art methods, namely 95.3% on UCF Sports (better by 4.0%), 87.9% on YouTube data set (better by 2.5%), and achieves comparable results on Hollywood2 dataset.