CBDN: A Chinese Short-Text Classification Model Based on Chinese BERT and Fused Deep Neural Networks

To address the common issues in Chinese short-text classification caused by the lack of contextual information, ambiguity, and sparsity of semantic features due to the short length of the text, a feature fusion-based Chinese short-text classification model CBDN is proposed. Firstly, the Chinese-BERT-wwm pre-trained model, improved by the full-word masking technique, is selected as the embedding layer to output the vector representation of the short text. Secondly, to fully extract the limited semantic features of the short text, the model employs a multi-head self-attention module and a long connected bidirectional LSTM (LC-BiLSTM) network to further learn the semantic features, and then fuses the hidden layer output vector with the feature vector further processed by these two methods. Finally, to improve the classification performance, the fused features are input into an improved “pyramid CNN” (PCNN) layer, and the short-text classification result is obtained through the classifier. The CBDN model is experimentally compared with various baseline models on the THUCNews dataset. The experimental results show that the proposed model achieves an accuracy and precision of 94.38% and 94.37%, respectively, outperforming other baseline models, indicating that the model better extracts the semantic information of short text and effectively improves the classification performance of Chinese short text.