News Classification Using Transformer Based Models

Abhinaya, Muhammad Abrar (2026) News Classification Using Transformer Based Models. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025201124-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (7MB) | Request a copy

Abstract

The rapid growth of digital news content published online every day creates challenges in organizing and categorizing information accurately. To address this issue, this research applies Transformer-based models for news classification, as they are capable of capturing contextual representations more effectively than traditional machine learning approaches. In this study, three Transformer architectures, namely Bidirectional Encoder Representations from Transformers (BERT), Robustly Optimized BERT (RoBERTa), and DistilBERT, are evaluated for their performance in classifying news categories using the cleaned HuffPost News dataset obtained from Kaggle. The training process involves comparing the effectiveness of each model in understanding textual structure and handling the complexity of news content. The final results demonstrate that Transformer-based models provide better performance than conventional machine learning methods in text understanding and classification tasks. The bestperforming model is further compared with an ensemble learning approach from previous work as a performance benchmark. The experiment utilizes an 80:20 train-test split along with commonly used evaluation metrics, including accuracy, precision, recall, and F1-score, to ensure reliable performance measurement. Overall, this study highlights the capability of Transformer architectures to improve classification accuracy and serves as a reference for the development of more effective text-based news classification systems.
=============================================================================================================================
Perkembangan media digital menyebabkan jumlah berita yang dipublikasikan secara daring meningkat sangat cepat setiap harinya, sehingga menimbulkan tantangan dalam proses pengelolaan dan pengelompokan konten secara akurat. Salah satu pendekatan yang efektif untuk mengatasi permasalahan tersebut adalah pemanfaatan model Transformer, yang mampu menangkap konteks kalimat secara lebih mendalam dibandingkan metode pembelajaran mesin tradisional. Pada penelitian ini, dilakukan klasifikasi berita menggunakan model berbasis Transformer, yaitu Bidirectional Encoder Representations from Transformers (BERT), Robustly Optimized BERT (RoBERTa), dan DistilBERT. Penelitian ini menggunakan dataset HuffPost News dari Kaggle yang telah melalui tahap pembersihan data untuk memastikan kualitas input. Proses pelatihan dilakukan dengan membandingkan performa ketiga model dalam mengklasifikasikan kategori berita. Hasil penelitian menunjukkan bahwa RoBERTa memberikan performa terbaik di antara model individual, diikuti oleh BERT dan DistilBERT. Selain itu, pada tahap akhir, hasil model terbaik dibandingkan dengan metode ensemble learning dari penelitian sebelumnya sebagai tolok ukur peningkatan performa klasifikasi. Pengujian dilakukan menggunakan pembagian data pelatihan dan data uji (80:20) serta metrik evaluasi berupa akurasi, precision, recall, dan F1-score untuk memastikan kemampuan generalisasi model. Hasil yang diperoleh menunjukkan bahwa model Transformer memiliki kemampuan yang lebih baik dalam memahami konteks teks serta mengatasi kerumitan struktur bahasa pada konten berita dibanding model Machine Learning tradisional. Penelitian ini diharapkan dapat menjadi acuan dalam pengembangan sistem klasifikasi berita dengan performa lebih akurat dan efisien di masa mendatang.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	News Classification, Transformer-based Models, BERT, RoBERTa, DistilBERT, Text Classification, Deep Learning.
Subjects:	Q Science Q Science > Q Science (General) > Q325.5 Machine learning. Support vector machines. Q Science > QA Mathematics > QA336 Artificial Intelligence Q Science > QA Mathematics > QA76.758 Software engineering T Technology > T Technology (General) > T57.5 Data Processing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Muhammad Abrar Abhinaya
Date Deposited:	18 Feb 2026 00:52
Last Modified:	18 Feb 2026 00:52
URI:	http://repository.its.ac.id/id/eprint/132476

Actions (login required)

View Item