Pembangkitan Narasi Dokumen Laporan Evaluasi Diri (LED) Menggunakan Large Language Models (LLM) dengan Pendekatan Retrieval-Augmented Generation

Amoriza, Muhammad Nabil Akhtar Raya (2026) Pembangkitan Narasi Dokumen Laporan Evaluasi Diri (LED) Menggunakan Large Language Models (LLM) dengan Pendekatan Retrieval-Augmented Generation. Project Report. [s.n.]. (Unpublished)

Text 5025221021-Project_Report.pdf - Accepted Version Restricted to Repository staff only Download (4MB) | Request a copy

Abstract

Penyusunan dokumen Laporan Evaluasi Diri (LED) merupakan tahapan krusial dalam akreditasi perguruan tinggi yang menuntut ketepatan data, koherensi narasi, dan kepatuhan terhadap standar instrumen akreditasi. Proses penyusunan yang dilakukan secara manual seringkali memakan waktu lama dan rentan terhadap inkonsistensi. Penelitian ini bertujuan untuk mengembangkan sistem semi-otomatis berbasis Large Language Models (LLM) dengan arsitektur Retrieval-Augmented Generation (RAG) untuk membantu penyusunan draf narasi LED. Sistem ini mengintegrasikan dokumen LED historis dan referensi pendukung ke dalam pipeline generatif untuk menghasilkan teks yang faktual dan sesuai gaya bahasa penulisan akreditasi. Penelitian menerapkan strategi eksperimen bertingkat, dimulai dari penapisan umum (general screening) dan dilanjutkan dengan evaluasi mendalam (in-depth evaluation) pada 9 kriteria akreditasi. Hasil eksperimen fase kedua menunjukkan bahwa model Llama-SEA-LION-v3.5-R (varian reasoning) mencatat kinerja terbaik, mengungguli varian instruction-tuned (Llama-SEA-LION-v3-IT). Model v3.5-R mencapai skor BERTScore 0.8174 dan Rouge-L 0.5801, menunjukkan tingkat kesesuaian semantik dan leksikal yang lebih tinggi dibandingkan v3-IT (BERTScore 0.7976). Meskipun demikian, evaluasi kualitatif masih mengindikasikan adanya tantangan berupa kecenderungan replikasi narasi yang perlu penanganan lebih lanjut. Luaran akhir penelitian ini berupa purwarupa aplikasi berbasis web yang mengintegrasikan sistem pencarian dokumen (retrieval) dan generasi naskah (generation), yang berfungsi sebagai alat bantu efisiensi bagi tim penyusun akreditasi.
================================================================================================================================
The preparation of a Self-Assessment Report (SAR) or Laporan Evaluasi Diri (LED) is a critical process in higher education accreditation, necessitating data accuracy, narrative coherence, and strict adherence to accreditation instrument standards. The traditional manual drafting process is time-consuming and susceptible to inconsistencies. This study aims to develop a semi-automated system leveraging Large Language Models (LLMs) integrated with a Retrieval-Augmented Generation (RAG) architecture to facilitate the drafting of LED narratives. The proposed system integrates historical LED documents and supporting references into a generative pipeline to produce factual text that aligns with the formal stylistic requirements of accreditation documentation. This research employs a multi-tiered experimental strategy, commencing with a general screening phase and followed by an in-depth evaluation across nine accreditation criteria. Results from the second experimental phase indicate that the Llama-SEA-LION-v3.5-R (reasoning variant) model achieved optimal performance, outperforming the instruction-tuned variant (Llama-SEA-LION-v3-IT). The v3.5-R model attained a BERTScore of 0.8174 and a ROUGE-L score of 0.5801, demonstrating superior semantic and lexical congruence compared to the v3-IT model (BERTScore 0.7976). Nevertheless, qualitative evaluations reveal remaining challenges, notably a tendency toward narrative replication that requires further mitigation. The final output of this research is a web-based application prototype integrating document retrieval and text generation systems, serving as an efficiency-enhancing tool for accreditation preparation teams.

Item Type:	Monograph (Project Report)
Uncontrolled Keywords:	Laporan Evaluasi Diri (LED), Large Language Models (LLM), Retrieval-Augmented Generation (RAG), Akreditasi Perguruan Tinggi, Prompt Engineering
Subjects:	Q Science > Q Science (General) > Q325.5 Machine learning. Support vector machines. T Technology > T Technology (General) > T11 Technical writing. Scientific Writing T Technology > T Technology (General) > T58.5 Information technology. IT--Auditing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering
Depositing User:	Muhammad Nabil Akhtar Raya Amoriza
Date Deposited:	09 Jul 2026 03:51
Last Modified:	09 Jul 2026 03:51
URI:	http://repository.its.ac.id/id/eprint/134530

Actions (login required)

View Item