Improving Arabic Text Classification Accuracy Using Lightweight NLP Techniques

Abstract

Arabic text classification is a challenging task because of the complex morphology of the language, the existence of different writing forms and a multitude of dialects, which can result in sparser common text representations. While transformer models such as AraBERT have obtained superior results on many Arabic NLP tasks, their high computational requirements make them difficult to deploy in environments with limited hardware resources. In some cases this can also make the model less practical for researchers working with basic computer systems.

This study focuses on a more practical issue: how much accuracy a simple classifier may lose when the amount of required computation is reduced. We use a combined TF-IDF representation based on both words and characters, then reduce the number of features using Chi-Square selection. The selected features are finally used with a linear SVM to create a model that is faster and more efficient, while still keeping good predictive performance. However, the method do not always provide the same level of accuracy as more complex models, especially with difficult text.

The proposed lightweight method is tested against several other approaches, including word-based TF-IDF, character-based TF-IDF, Naive Bayes, Logistic Regression, BiLSTM, and AraBERT. All methods are tested using the same data distribution to make the comparison fair. The dataset used is the arbml/arabic_100k_reviews corpus, which originally contains 99,999 balanced reviews from three sentiment categories. After cleaning the data and removing very short reviews, the dataset was reduced to 99,759 samples. The training, validation, and testing sets includes 69,831, 9,976, and 19,952 reviews, respectively. This setup allows the different models to be compared under similar conditions and with the same evaluation process.

 After applying Chi-Square feature selection, the hybrid feature set was reduced from 250,000 to 50,000 dimensions, which represents an 80% reduction. That proposed method achieved 67.90% accuracy and a Macro-F1 score of 67.75%. Its training time was around 41 seconds, while the complete test set was processed in about 0.03 seconds. Among all the evaluated models, AraBERT achieved the best overall performance, reaching 74.08% accuracy and 74.25% Macro-F1. However, its computational requirements were much higher, with nearly 1,760 seconds needed for training and about 47 seconds for inference on the same test set. Depending on the processing stage, this makes AraBERT approximately 40 to 1,500 times slower than the proposed lightweight approach. The proposed method is therefore not presented as a replacement for AraBERT in terms of accuracy, since its accuracy is lower. Instead, its main advantage is the considerable and measurable reduction in computational cost. This trade-off can be useful in situations where GPU availability, memory capacity, or response time are limited.