Investigating the Impact of Utilizing the ChatGPT for Arabic Sentiment Analysis

In the field of artificial intelligence (A), there has been a remarkable breakthrough with the emergence of large language models (LLMs) that are fine-tuned to follow human instructions. One such model is OpenAI's ChatGPT (Chat Generative Pre-trained Transformer), which has proven to be a highly capable tool for various tasks including question answering, code debugging, and dialogue generation. However, while these models are touted for their multilingual proficiency, their ability to accurately analyze sentiment, particularly in the Arabic language, has not been extensively investigated. Recognizing this limitation, we aim to address this gap by conducting a comprehensive evaluation of ChatGPT’ sentiment analysis capabilities specifically for Arabic text. We investigate the impact of utilizing the ChatGPT variants for Arabic sentiment analysis (ASA) and propose a new active labeling methods for ChatGPT. We evaluate the performance of four machine learning (ML) techniques, including Naive Bayes (NB), K-Nearest Neighbors (K-NN), Support Vector Machine (SVM), and Random Forest (FR), using accuracy, recall, precision, and F-score measure. We also compare six methods of labeling the data for ASA, manual labeling by humans, labeling using ChatGPT by Assistant-Poe, labeling using ChatGPT by Bing-Edge, labeling using ChatGPT by Assistant-Poe with humans, labeling using ChatGPT by Bing-Edge with humans, and labeling using ChatGPT by Assistant-Poe with Bing-Edge. Our experimental results show that the NB technique performed the best, achieving an accuracy of 91.22%, recall of 89.62%, precision of 88.90%, and F-score of 89.26% by using multiple Bing-Edge models for ASA. Moreover, utilizing our proposed active labeling method with ChatGPT achieved higher accuracy compared to other labeling methods. Our study suggests that the NB technique with multiple Bing-Edge models and our proposed active labeling method are effective approaches for ASA using ChatGPT. Our study contributes to the advancement of sentiment analysis in Arabic text and offers valuable insights into effective approaches for this task.