A COMPARATIVE STUDY OF MACHINE LEARNING TECHNIQUES FOR SENTIMENT CLASSIFICATION OF SOCIAL MEDIA TEXT
Abstract
Social media platforms generate vast volumes of opinionated text, making automated sentiment classification a core task in natural language processing. One high-stakes application is the analysis of public opinion on India's National Education Policy 2020 (NEP-2020), which has sustained online debate through its fifth anniversary in 2025 and whose comments need classifiers of well-understood accuracy, cost, and robustness. This study compares eight machine learning techniques, spanning classical classifiers (multinomial Naïve Bayes, logistic regression, support vector machines, and random forests), neural architectures (convolutional and bidirectional LSTM networks), and pre-trained transformers (BERT and RoBERTa), on a three-class Twitter sentiment benchmark (TweetEval) that serves as a controlled proxy for NEP-2020 public comments. Models were evaluated using accuracy, macro-averaged F1, training cost, and sensitivity to training-set size. The dataset is documented in detail, including a marked shift in class distribution between training and test partitions, and a preprocessing ablation examines how text normalization choices affect each model family. Paired significance tests with multiple-comparison correction assess whether observed differences exceed seed-to-seed and test-sampling variability. Transformer models achieved the strongest performance, with RoBERTa reaching a macro-F1 of 74.5%, while a linear support vector machine remained competitive at a fraction of the computational cost when labeled data were scarce or compute was limited. Errors across all models concentrated on the neutral class, reflecting the inherent ambiguity of short informal text. The findings offer practical guidance for selecting sentiment classifiers under different resource constraints, including for the analysis of public comments on NEP-2020.
Downloads
References
[2]Batra, P. (2020). NEP 2020: Undermining the constitutional education agenda? Social Change, 50(4), 594–598. https://doi.org/10.1177/0049085720958809
[3]Berg-Kirkpatrick, T., Burkett, D., & Klein, D. (2012). An empirical investigation of statistical significance in NLP. Proceedings of EMNLP-CoNLL 2012, 995–1005.
[4]Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
[5]Chaudhary, D., Das, D., Dutta, R., Kapoor, S. K., Singh, S., & Prajapat, G. N. (2026). Bridging public opinion and educational reform through sentiment analysis. In J. C. Bansal, S. Borah, S. Hussain, & S. Salhi (Eds.), Computing and machine learning: CML 2025 (Lecture Notes in Networks and Systems, Vol. 1614, pp. 691–701). Springer. https://doi.org/10.1007/978-981-95-2878-3_49
[6]Chaudhary, D., Dutta, R., & Ghosh, P. (2024). A review on public acceptance towards new education policy in India based on sentiment analysis using machine learning approaches. In A. Joshi, M. Mahmud, R. G. Ragel, & S. Kartik (Eds.), ICT: Cyber security and applications: ICTCS 2022 (Lecture Notes in Networks and Systems, Vol. 916, pp. 465–474). Springer. https://doi.org/10.1007/978-981-97-0744-7_38
[7]Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
[8]Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT 2019, 4171–4186.
[9]Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7), 1895–1923.
[10]Dong, X., & Lian, Y. (2021). A review of social media-based public opinion analyses: Challenges and recommendations. Technology in Society, 67, Article 101724. https://doi.org/10.1016/j.techsoc.2021.101724
[11]Dror, R., Baumer, G., Shlomov, S., & Reichart, R. (2018). The hitchhiker's guide to testing statistical significance in natural language processing. Proceedings of ACL 2018, 1383–1392.
[12]Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Chapman & Hall.
[13]Go, A., Bhayani, R., & Huang, L. (2009). Twitter sentiment classification using distant supervision (CS224N Project Report). Stanford University.
[14]Göçen, A., Ibrahim, M. M., & Khan, A. U. I. (2024). Public attitudes toward higher education using sentiment analysis and topic modeling. Discover Artificial Intelligence, 4, Article 83. https://doi.org/10.1007/s44163-024-00195-4
[15]Govinda, R. (2020). NEP 2020: A critical examination. Social Change, 50(4), 603–607. https://doi.org/10.1177/0049085720958804
[16]Haddi, E., Liu, X., & Shi, Y. (2013). The role of text pre-processing in sentiment analysis. Procedia Computer Science, 17, 26–32.
[17]Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
[18]Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70.
[19]Jianqiang, Z., & Xiaolin, G. (2017). Comparison research on text pre-processing methods on Twitter sentiment analysis. IEEE Access, 5, 2870–2879.
[20]Kaurav, R. P. S., Narula, S., Baber, R., & Tiwari, P. (2021). Theoretical extension of the new education policy 2020 using Twitter mining. Journal of Content, Community and Communication, 13(7), 16–26. https://doi.org/10.31620/jccc.06.21/03
[21]Kaurav, R. P. S., Suresh, K. G., Narula, S., & Baber, R. (2020). New education policy, 2020: Qualitative (contents) analysis and Twitter mining (sentiment analysis). Journal of Content, Community and Communication, 12, 4–13. https://doi.org/10.31620/jccc.12.20/02
[22]Kim, Y. (2014). Convolutional neural networks for sentence classification. Proceedings of EMNLP 2014, 1746–1751.
[23]Kulal, A., Abhishek, N., Dinesh, S., Bhat, D. C., & Girish, A. (2024). Evaluating the promise and pitfalls of India's National Education Policy 2020: Insights from the perspectives of students, teachers, and experts. SAGE Open, 14(4). https://doi.org/10.1177/21582440241279367
[24]Liu, B. (2012). Sentiment analysis and opinion mining. Morgan & Claypool.
[25]Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach (arXiv:1907.11692). arXiv.
[26]Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011). Learning word vectors for sentiment analysis. Proceedings of ACL-HLT 2011, 142–150.
[27]McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153–157.
[28]Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space (arXiv:1301.3781). arXiv.
[29]Ministry of Education. (2020). National Education Policy 2020. Government of India. https://www.education.gov.in/sites/upload_files/mhrd/files/NEP_Final_English_0.pdf
[30]Ministry of Education. (2025, July 28). Press release on the Akhil Bharatiya Shiksha Samagam 2025 marking five years of NEP 2020 [Press release]. Press Information Bureau, Government of India. https://www.education.gov.in/sites/upload_files/mhrd/files/PIB2149319.pdf
[31]Mohammad, S. M., Kiritchenko, S., & Zhu, X. (2013). NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval 2013), 321–327.
[32]Nguyen, D. Q., Vu, T., & Nguyen, A. T. (2020). BERTweet: A pre-trained language model for English tweets. Proceedings of EMNLP 2020: System Demonstrations, 9–14.
[33]Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2), 1–135.
[34]Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? Sentiment classification using machine learning techniques. Proceedings of EMNLP 2002, 79–86.
[35]Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
[36]Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global vectors for word representation. Proceedings of EMNLP 2014, 1532–1543.
[37]Rosenthal, S., Farra, N., & Nakov, P. (2017). SemEval-2017 Task 4: Sentiment analysis in Twitter. Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 502–518.
[38]Saini, M., Singh, M., Kaur, M., & Kaur, M. (2021). Analysing the tweets to examine the behavioural response of Indian citizens over the approval of National Education Policy 2020. International Journal of Educational Development, 82, Article 102356. https://doi.org/10.1016/j.ijedudev.2021.102356
[39]Sengupta, P. (2021). NEP 2020 and the language-in-education policy in India: A critical assessment. Economic and Political Weekly, 56(43), 45–52.
[40]Sharma, B., & Vasuja, K. (2021). Investigating social media induced polarization on National Education Policy 2020. In Causes and symptoms of socio-cultural polarization (Chap. 8). Springer. https://doi.org/10.1007/978-981-16-5268-4_8
[41]Siddique, M. M., & Kumar, S. (2023). Sentiment analysis on educational tweets: A case of National Education Policy 2020. Proceedings of the 2023 IEEE International Conference on Contemporary Computing and Communications (InC4). IEEE. https://doi.org/10.1109/InC457730.2023.10262944
[42]Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., & Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of EMNLP 2013, 1631–1642.
[43]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998–6008.
[44]Wang, S., & Manning, C. D. (2012). Baselines and bigrams: Simple, good sentiment and topic classification. Proceedings of ACL 2012, 90–94.
[45]Zhang, W., Deng, Y., Liu, B., Pan, S., & Bing, L. (2024). Sentiment analysis in the era of large language models: A reality check. Findings of the Association for Computational Linguistics: NAACL 2024, 3881–3906. https://doi.org/10.18653/v1/2024.findings-naacl.246
Copyright (c) 2026 IJRDO -Journal of Computer Science Engineering
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Author(s) and co-author(s) jointly and severally represent and warrant that the Article is original with the author(s) and does not infringe any copyright or violate any other right of any third parties, and that the Article has not been published elsewhere. Author(s) agree to the terms that the IJRDO Journal will have the full right to remove the published article on any misconduct found in the published article.
