Nurhadi, Imam (2026) Automated Essay Scoring untuk Esai Singkat dan Jawaban Teks Terbatas Berbahasa Indonesia Berbasis Large Language Model Dengan Retrieval-Augmented Generation: Studi Kasus Biologi. Other thesis, Institut Teknologi Sepuluh Nopember.
![[thumbnail of 5027221046_Imam Nurhadi_BukuTA.pdf]](http://repository.its.ac.id/style/images/fileicons/text.png) |
Text
5027221046_Imam Nurhadi_BukuTA.pdf Restricted to Repository staff only Download (3MB) | Request a copy |
Abstract
Penilaian jawaban berbasis teks, seperti esai singkat dan jawaban teks terbatas, penting dalam evaluasi pembelajaran karena mampu mengukur pemahaman konsep dan kemampuan berpikir kritis siswa. Namun, penilaian manual terhadap jawaban tersebut cenderung memakan waktu, bersifat subjektif, dan sulit dilakukan secara konsisten, terutama pada mata pelajaran Biologi yang menuntut penjelasan konseptual dan argumentatif. Penelitian ini mengembangkan sistem Automated Essay Scoring (AES) berbasis Large Language Model (LLM) dan Retrieval-Augmented Generation (RAG) yang diimplementasikan dalam aplikasi berbasis web menggunakan Vue.js dan FastAPI. Tiga LLM open-source, yaitu LLaMA 3 8B, Gemma 3 4B, dan Qwen 3 4B, dievaluasi menggunakan empat metrik, yaitu Akurasi, Mean Absolute Error (MAE), Quadratic Weighted Kappa (QWK), dan execution time (ET). Hasil pengujian menunjukkan bahwa Qwen 3 4B memberikan performa terbaik pada seluruh metrik. Evaluasi metode embedding RAG menunjukkan bahwa dense retrieval menghasilkan performa optimal dengan akurasi 0,682, MAE 0,406, dan QWK 0,736. Fine-tuning menggunakan Low-Rank Adaptation meningkatkan QWK sekitar 0,02 dan menurunkan MAE sekitar 0,01, sementara ablation study menunjukkan bahwa integrasi RAG meningkatkan QWK sekitar 0,13 dengan peningkatan ET sekitar 0,47 menit. Berdasarkan hasil tersebut, konfigurasi Qwen 3 4B dengan dense retrieval ditetapkan sebagai konfigurasi paling optimal untuk sistem AES pada konteks pendidikan Biologi.
=============================================================================================================================
Text-based answer assessment, such as short essays and limited text answers, is important in learning evaluation because it can measure students' conceptual understanding and critical thinking skills. However, manual assessment of these answers tends to be time-consuming, subjective, and difficult to perform consistently, especially in Biology subjects that require conceptual and argumentative explanations. This study developed an Automated Essay Scoring (AES) system based on Large Language Model (LLM) and Retrieval-Augmented Generation (RAG), implemented in a web-based application using Vue.js and FastAPI. Three open-source LLMs, namely LLaMA 3 8B, Gemma 3 4B, and Qwen 3 4B, were evaluated using four metrics, namely Accuracy, Mean Absolute Error (MAE), Quadratic Weighted Kappa (QWK), and execution time (ET). The test results show that Qwen 3 4B performs best on all metrics. Evaluation of the RAG embedding method shows that dense retrieval produces optimal performance with an accuracy of 0,682, MAE of 0,406, and QWK of 0,736. Fine-tuning using Low-Rank Adaptation increased QWK by approximately 0,02 and decreased MAE by approximately 0,01, while the ablation study showed that RAG integration increased QWK by approximately 0,13 with an ET increase of approximately 0,47 minutes. Based on these results, the Qwen 3 4B configuration with dense retrieval was determined to be the most optimal configuration for the AES system in the context of biology education.
| Item Type: | Thesis (Other) |
|---|---|
| Uncontrolled Keywords: | Automated Essay Scoring, Fine-tuning LoRA, Large Language Model, QWK, Retrieval-Augmented Generation, |
| Subjects: | Q Science > QA Mathematics > QA336 Artificial Intelligence T Technology > T Technology (General) T Technology > T Technology (General) > T57.5 Data Processing |
| Divisions: | Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information Technology > 59201-(S1) Undergraduate Thesis |
| Depositing User: | Imam Nurhadi Nurhadi |
| Date Deposited: | 20 Jan 2026 05:30 |
| Last Modified: | 20 Jan 2026 05:30 |
| URI: | http://repository.its.ac.id/id/eprint/129815 |
Actions (login required)
 |
View Item |