STAP: Spatial-Temporal Attention-Aware Pooling for Action Recognition

Citation data:

IEEE Transactions on Circuits and Systems for Video Technology, ISSN: 1051-8215, Vol: 25, Issue: 1, Page: 77-86

Publication Year:
Usage 258
Abstract Views 255
Link-outs 3
Captures 35
Readers 24
Exports-Saves 11
Citations 20
Citation Indexes 20
Repository URL:;
Tam V. Nguyen; Zheng Song; Shuicheng Yan
Institute of Electrical and Electronics Engineers (IEEE); IEEE; eCommons
Engineering; Computer Sciences; Databases and Information Systems; Graphics and Human Computer Interfaces; Other Computer Sciences
article description
Human action recognition is valuable for numerous practical applications, e.g., gaming, video surveillance, and video search. In this paper we hypothesize that the classification of actions can be boosted by designing a smart feature pooling strategy under the prevalently used bag-of-words-based representation. Founded on automatic video saliency analysis, we propose the spatial-temporal attention-aware pooling scheme for feature pooling. First, the video saliencies are predicted using the video saliency model, and the localized spatial-temporal features are pooled at different saliency levels and video-saliency-guided channels are formed. Saliency-aware matching kernels are thus derived as the similarity measurement of these channels. Intuitively, the proposed kernels calculate the similarities of the video foreground (salient areas) or background (nonsalient areas) at different levels. Finally, the kernels are fed into popular support vector machines for action classification. Extensive experiments on three popular data sets for action classification validate the effectiveness of our proposed method, which outperforms the state-of-the-art methods, namely 95.3% on UCF Sports (better by 4.0%), 87.9% on YouTube data set (better by 2.5%), and achieves comparable results on Hollywood2 dataset.