Sentiment Analysis Using Transformer-Based Language Models

Abstract

Sentiment analysis has emerged as a critical field in natural language processing (NLP), enabling machines to understand human emotions embedded within text. Transformer-based architectures, such as BERT, RoBERTa, and GPT variants, have revolutionized this task through contextual embeddings and self-attention mechanisms. Unlike traditional models relying on static word embeddings, transformers dynamically capture semantic dependencies and linguistic nuances. This paper explores the methodologies, performance benchmarks, and optimization strategies of transformer-based models for sentiment classification across diverse datasets, including IMDB, Twitter, and Yelp. Comparative results demonstrate that fine-tuned transformer architectures consistently outperform classical machine learning models such as SVM and LSTM, achieving higher accuracy, robustness, and generalization. The study concludes with future research prospects, emphasizing multilingual sentiment analysis, zero-shot learning, and cross-domain transferability