A generalized deep learning model for heart failure diagnosis using dynamic and static ultrasound

Objective: Echocardiography (ECG) is the most common method used to diagnose heart failure (HF). However, its accuracy relies on the experience of the operator. Additionally, the video format of the data makes it challenging for patients to bring them to referrals and reexaminations. Therefore, this study used a deep learning approach to assist physicians in assessing cardiac function to promote the standardization of echocardiographic findings and compatibility of dynamic and static ultrasound data. Methods: A deep spatio-temporal convolutional model r2plus1d-Pan (trained on dynamic data and applied to static data) was improved and trained using the idea of "regression training combined with classification application,"which can be generalized to dynamic ECG and static cardiac ultrasound views to identify HF with a reduced ejection fraction (EF < 40%). Additionally, three independent datasets containing 8976 cardiac ultrasound views and 10085 cardiac ultrasound videos were established. Subsequently, a multinational, multi-center dataset of EF was labeled. Furthermore, model training and independent validation were performed. Finally, 15 registered ultrasonographers and cardiologists with different working years in three regional hospitals specialized in cardiovascular disease were recruited to compare the results. Results: The proposed deep spatio-temporal convolutional model achieved an area under the receiveroperating characteristic curve (AUC) value of 0.95 (95% confidence interval [CI]: 0.947 to 0.953) on the training set of dynamic ultrasound data and an AUC of 1 (95% CI, 1 to 1) on the independent validation set. Subsequently, the model was applied to the static cardiac ultrasound view (validation set) with simultaneous input of 1, 2, 4, and 8 images of the same heart, with classification accuracies of 85%, 81%, 93%, and 92%, respectively. On the static data, the classification accuracy of the artificial intelligence (AI) model was comparable with the best performance of ultrasonographers and cardiologists with more than 3 working years (P = 0.344), but significantly better than the median level (P = 0.0000008). Conclusion: A new deep spatio-temporal convolution model was constructed to identify patients with HF with reduced EF accurately (< 40%) using dynamic and static cardiac ultrasound images. The model outperformed the diagnostic performance of most senior specialists. This may be the first HF-related AI diagnostic model compatible with multi-dimensional cardiac ultrasound data, and may thereby contribute to the improvement of HF diagnosis. Additionally, the model enables patients to carry "on-the-go"static ultrasound reports for referral and reexamination, thus saving healthcare resources.