Fine-Tuning BERT Transformer Untuk Automated Essay Scoring Dengan Analisis Multi-Head Attention Pada Komponen Argumentatif Esai Persuasif

Dewani, Alma Amira (2026) Fine-Tuning BERT Transformer Untuk Automated Essay Scoring Dengan Analisis Multi-Head Attention Pada Komponen Argumentatif Esai Persuasif. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5027221054-Undergraduate_Thesis.pdf Restricted to Repository staff only Download (2MB) | Request a copy

Abstract

Penilaian esai secara manual membutuhkan waktu dan tenaga yang besar serta rentan terhadap inkonsistensi antar penilai, sehingga mendorong pengembangan sistem Automated Essay Scoring (AES). Penelitian ini mengembangkan BERTMultiTaskAES, sebuah model berbasis BERT dengan pendekatan multi-task untuk menilai esai persuasif berbahasa Indonesia pada dataset PERSUADE 2.0. Model menangani dua tugas sekaligus, yaitu penilaian holistik (skor 1 hingga 6) dan klasifikasi efektivitas komponen argumentatif (Ineffective, Adequate, Effective), dengan fokus utama pada analisis mekanisme multi-head attention. Esai berbahasa Inggris diterjemahkan ke bahasa Indonesia, kemudian model dilatih menggunakan strategi freeze-then-unfreeze dan dievaluasi dengan tiga seed. Hasil penelitian menunjukkan model mencapai Quadratic Weighted Kappa (QWK) 0,6426 pada penilaian holistik. Perbandingan dengan IndoBERT yang mencapai QWK 0,7769 menunjukkan bahwa kesesuaian bahasa korpus pre-training merupakan faktor dominan yang menentukan performa model. Analisis attention mengidentifikasi spesialisasi head terhadap tujuh tipe discourse element serta keberadaan head umum yang konsisten lintas tipe. Selain itu, ditemukan dominasi korelasi negatif antara konsentrasi attention dan kualitas argumentatif, dengan head yang berkorelasi paling kuat dengan skor holistik adalah Layer 12 Head 10 (ρ = −0,4293). Temuan ini menunjukkan bahwa konsentrasi perhatian yang tinggi pada suatu komponen berasosiasi dengan kualitas yang lebih rendah, yang mencerminkan kompleksitas hubungan antara mekanisme attention dan penilaian kualitas argumentatif.
========================================================================================================================================
Manual essay scoring is time-consuming, labor-intensive, and prone to inter-rater inconsistency, motivating the development of Automated Essay Scoring (AES) systems. This study develops BERTMultiTaskAES, a BERT-based model with a multi-task approach for scoring Indonesian persuasive essays on the PERSUADE 2.0 dataset. The model handles two tasks simultaneously, namely holistic scoring (scores 1 to 6) and effectiveness classification of argumentative components (Ineffective, Adequate, Effective), with a primary focus on analyzing the multi-head attention mechanism. English essays were translated into Indonesian, after which the model was trained using a freeze-then-unfreeze strategy and evaluated across three seeds. The results show that the model achieved a Quadratic Weighted Kappa (QWK) of 0.6426 on holistic scoring. A comparison with IndoBERT, which achieved a QWK of 0.7769, indicates that the language alignment of the pre-training corpus is the dominant factor limiting model performance. The attention analysis identified head specialization toward the seven discourse element types as well as the presence of general heads consistent across types. Furthermore, a dominant negative correlation was found between attention concentration and argumentative quality, with the head most strongly correlated with the holistic score being Layer 12 Head 10 (ρ = −0.4293). These findings indicate that high attention concentration on a component is associated with lower quality, reflecting the complexity of the relationship between the attention mechanism and argumentative quality assessment.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Automated Essay Scoring, BERT, Multi-task Learning, Multi-head Attention, Esai Persuasif, Persuasive Essay
Subjects:	T Technology > T Technology (General) > T58.5 Information technology. IT--Auditing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information Technology > 59201-(S1) Undergraduate Thesis
Depositing User:	Alma Amira Dewani
Date Deposited:	30 Jul 2026 01:44
Last Modified:	30 Jul 2026 01:44
URI:	http://repository.its.ac.id/id/eprint/136668

Actions (login required)

View Item