Micro-expression Recognition Based on Dual-Stream Spatiotemporal Transformer

Micro-expressions, imperceptible spontaneous facial movements reflecting underlying emotions, hold significant importance in emotion recognition. Due to their short duration and low intensity, micro-expression recognition (MER) remains challenging. The collection of micro-expressions poses difficulties due to their characteristics, leading to a scarcity of spontaneous micro-expression datasets. Furthermore, existing methods typically utilize only one type of input for MER, thus failing to fully exploit the limited micro-expression samples. To address these issues, we propose a new dual-stream spatiotemporal transformer network combining optical flow and magnified micro-expression, enabling to handle different types of information, thereby providing richer and more comprehensive representations. By simultaneously inputting both original micro-expression images and the corresponding optical flow change images into the dual-stream net-work, we obtain a diverse range of micro-expression information, consequently mitigating the impact of the scarcity of micro-expression datasets. Experimental evaluations conducted on three public datasets, namely SMIC, SAMM, and CASME II, demonstrate the superiority of our approach over other methods.