Improving Sentiment Analysis Using Transformer-Based NLP Models

  • Authors

    • Dr. Ibrahim Lawal Federal University of Technology Minna, Niger state. Author

    DOI:

    https://doi.org/10.67228/30713498/IJADSMC-2020PI1W6N

    Published 05-05-2020

  • Sentiment Analysis, Natural Language Processing, Transformers, BERT, Self-Attention, Deep Learning, Text Classification, Opinion Mining

    Issue

    Vol. 3 No. 1 (2020)

    Section

    Articles

  • Abstract

    Sentiment analysis is an important area of Natural Language Processing (NLP) used to interpret opinions from text such as reviews and social media. Traditional methods, including rule-based and machine learning approaches, struggled with complex language features like sarcasm and context. Deep learning models like RNNs and CNNs improved performance but had limitations in capturing long-range dependencies. Transformer-based models such as BERT, RoBERTa, DistilBERT, and XLNet overcome these issues using self-attention mechanisms to better understand context. This study explores how these models enhance sentiment analysis accuracy through transfer learning, fine-tuning, and domain adaptation. Experimental results on benchmark datasets show that transformer models outperform traditional methods in accuracy, precision, recall, and F1-score. The findings highlight that transformer-based approaches provide more efficient and scalable solutions for real-world sentiment analysis applications.

  • References

    [1] B. Liu, Sentiment Analysis and Opinion Mining, Morgan & Claypool Publishers, 2012.

    [2] S. M. Mohammad and P. D. Turney, “Crowdsourcing a word–emotion association lexicon,” Computational Intelligence, vol. 29, no. 3, pp. 436–465, 2013.

    [3] A. Esuli and F. Sebastiani, “SentiWordNet: A publicly available lexical resource for opinion mining,” in Proc. 5th Conf. Language Resources and Evaluation (LREC), 2006, pp. 417–422.

    [4] J. W. Pennebaker, M. E. Francis, and R. J. Booth, “Linguistic Inquiry and Word Count (LIWC): LIWC2001,” Mahway: Lawrence Erlbaum Associates, 2001.

    [5] B . Pang, L. Lee, and S. Vaithyanathan, “Thumbs up? Sentiment classification using machine learning techniques,” in Proc. ACL, 2002, pp. 79–86.

    [6] B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Foundations and Trends in Information Retrieval, vol. 2, no. 1–2, pp. 1–135, 2008.

    [7] T. Joachims, “Text categorization with support vector machines: Learning with many relevant features,” in Proc. ECML, 1998, pp. 137–142.

    [8] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” in Proc. ICLR, 2013.

    [9] J. Pennington, R. Socher, and C. Manning, “GloVe: Global vectors for word representation,” in Proc. EMNLP, 2014, pp. 1532–1543.

    [10] Y. Kim, “Convolutional neural networks for sentence classification,” in Proc. EMNLP, 2014, pp. 1746–1751.

    [11] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.

    [12] K. Cho et al., “Learning phrase representations using RNN encoder–decoder for statistical machine translation,” in Proc. EMNLP, 2014, pp. 1724–1734.

    [13] A. Vaswani et al., “Attention is all you need,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5998–6008.

    [14] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186.

    [15] Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv preprint arXiv:1907.11692, 2019.

  • Downloads