A Hybrid-Attention-LSTM-Based Deep Convolutional Neural Network to Extract Modal Frequencies from Limited Data Using Transfer Learning

Current computer video-based vibration modal analysis approaches typically decompose video frames into representations and then adjust them that allow to magnify motions to extract motion representations for vibration modal analysis. Their decomposition usually relies upon handcrafted designed kernels, such as the complex steerable kernels, which typically may not be optimally designed for the extraction of subtle motions specially in higher frequency domains. In this paper, optimal decomposition kernel is learned and designed directly from baseline dataset images using deep convolutional neural network (CNN) models. Each subpixel of an image obtained from a digital camera is included when computing the spatiotemporal information, which serves similar to an individual motion sensor to acquire the modal frequencies of a vibrating structure. A hybrid-attention-LSTM-based deep convolutional neural network architecture is developed to take advantage of attention and LSTM blocks to discover subtle motions from a specific source to visualize high resolution of dynamic properties of the structures in the existence of high amounts of noise. The idea of transfer learning is utilized to transfer the knowledge previously learned to new limited dataset. Transfer learning is used to take advantage of limited existing dataset to avoid underfitting in the training of the network, considering the current publicly available modal frequency datasets are insufficient to train a generalized network. The proposed deep learning architecture is designed in such a way that has capability of transferring the trained model from baseline dataset on a simple structure to a complicated structure using transfer learning perspective. After training, the model takes the video of a vibrating structure as input and outputs the fundamental modal frequencies. By showing reliable empirical results, the proposed model is autonomous, efficient, and accurate.