Pengembangan Sistem Penilaian Otomatis Dan Umpan Balik Untuk Esai Ielts Berbasis Model Transformer

Febrianti, Sylvia (2026) Pengembangan Sistem Penilaian Otomatis Dan Umpan Balik Untuk Esai Ielts Berbasis Model Transformer. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5027221019_Undergraduate Thesis.pdf Restricted to Repository staff only Download (3MB) | Request a copy

Abstract

Penilaian esai otomatis (Automated Essay Scoring/AES) merupakan tantangan penting dalam teknologi pendidikan, khususnya pada penilaian berstandar internasional seperti IELTS Writing Task 2. Penelitian ini mengimplementasikan dan membandingkan performa dua model Transformer, RoBERTa-base dan DistilBERT-base, dalam memprediksi skor esai IELTS berdasarkan empat dimensi penilaian resmi Task Achievement, Coherence and Cohesion, Lexical Resource, dan Grammatical Range and Accuracy serta Overall Band Score. Selain itu, dikembangkan prototipe sistem berbasis web yang menampilkan skor prediksi dan umpan balik berbasis template sesuai deskriptor resmi IELTS Writing. Penelitian menggunakan pendekatan eksperimen kuantitatif dengan melakukan fine-tuning kedua model pada dataset IELTS-writing-task-2-evaluation dari Hugging Face yang berisi sekitar 9.000 esai berlabel penilai manusia. Skor dinormalisasi ke rentang 0–1 dan diproyeksikan kembali ke skala band 4–9 pada tahap inferensi. Evaluasi dilakukan menggunakan metrik MAE, RMSE, dan QWK. Hasil eksperimen menunjukkan bahwa RoBERTa memiliki performa relatif lebih baik dengan nilai peak QWK sebesar 0,6425, dibandingkan DistilBERT sebesar 0,6118. Namun, prediksi model masih cenderung terpusat pada skor menengah akibat distribusi data yang tidak seimbang. Prototipe berbasis Streamlit mampu menjalankan inferensi dan menampilkan skor serta umpan balik statis, namun belum layak digunakan sebagai sistem penilaian berisiko tinggi dan lebih sesuai sebagai sarana latihan mandiri.
==============================================================================================================================
Automated Essay Scoring (AES) is a significant challenge in educational technology, particularly for international standardized assessments such as IELTS Writing Task 2. This study implements and compares the performance of two Transformer-based models, RoBERTa-base and DistilBERT-base, in predicting IELTS essay scores based on four official scoring dimensions Task Achievement, Coherence and Cohesion, Lexical Resource, and Grammatical Range and Accuracy as well as the Overall Band Score. In addition, a web-based prototype system is developed to present predicted scores and provide template-based feedback aligned with the official IELTS Writing descriptors. The study adopts a quantitative experimental approach by fine-tuning both models on the IELTS-writing-task-2-evaluation dataset from Hugging Face, which contains approximately 9,000 essays labeled by human raters. Scores are normalized to the 0–1 range and projected back to the IELTS band scale of 4–9 during inference. Model performance is evaluated using Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Quadratic Weighted Kappa (QWK). Experimental results show that RoBERTa achieves relatively better performance with a peak QWK of 0.6425, compared to 0.6118 for DistilBERT. However, model predictions tend to concentrate around mid-range scores due to imbalanced data distribution. The Streamlit-based prototype is capable of performing inference and displaying scores along with static feedback, but it is not yet suitable for high-stakes assessment and is better positioned as a self-training and exploratory learning tool.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Automated Essay Scoring, IELTS, Transformer Models, RoBERTa, DistilBERT, Natural Language Processing, Automated Essay Scoring, IELTS, Transformer Models, RoBERTa, DistilBERT, Natural Language Processing
Subjects:	Q Science > Q Science (General) > Q325.5 Machine learning. Support vector machines.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information Technology > 59201-(S1) Undergraduate Thesis
Depositing User:	Sylvia Febrianti
Date Deposited:	22 Jan 2026 07:59
Last Modified:	22 Jan 2026 07:59
URI:	http://repository.its.ac.id/id/eprint/130077

Actions (login required)

View Item