Implementasi Agentic Retrieval Berbasis Large Language Model (LLM) Untuk Penelusuran Bukti Laporan Evaluasi Diri (LED)

Tee, Hardy (2026) Implementasi Agentic Retrieval Berbasis Large Language Model (LLM) Untuk Penelusuran Bukti Laporan Evaluasi Diri (LED). Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025221271-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (2MB) | Request a copy

Abstract

Penyusunan Laporan Evaluasi Diri (LED) akreditasi membutuhkan penelusuran bukti dari banyak dokumen pendukung. Proses ini sulit karena nama dokumen, struktur sitasi, dan isi bukti antarprogram studi tidak selalu identik. Retrieval non-agentik dapat membantu pencarian awal, tetapi cenderung statis ketika kandidat dokumen atau konteks yang ditemukan masih lemah. Penelitian ini mengimplementasikan pipeline Agentic Retrieval berbasis Large Language Model (LLM) untuk memetakan bukti LED RPL dengan memanfaatkan data RKA, S2SI, S1TI, dan S1SD.
Pipeline terdiri dari pra-pemrosesan LED dan dokumen referensi, penyimpanan embedding pada Qdrant, Agent 1 untuk pencocokan nama dokumen, dan Agent 2 untuk penelusuran konteks. Agent 1 memanfaatkan pola nama file, sitasi, dan pencarian fuzzy untuk menemukan kandidat dokumen target. Agent 2 melakukan pencarian semantik, mencoba beberapa jalur penelusuran, dan memeriksa apakah konteks target mendukung kebutuhan bukti. Evaluasi dilakukan pada fase PPEPP menggunakan F1-score nama file, Contextual Precision, dan Contextual Recall, serta dibandingkan dengan baseline retrieval non-agentik.
Hasil pengujian menunjukkan bahwa pendekatan agentik meningkatkan performa pada metrik yang sesuai dengan karakter bukti. Pada fase Penetapan, RKA tunggal menghasilkan F1-score terbaik sebesar 0,712, lebih tinggi daripada baseline gabungan RKA+S2SI+S1TI sebesar 0,287. Pada fase Pelaksanaan, gabungan RKA+S2SI+S1TI+S1SD menghasilkan Contextual Recall tertinggi sebesar 0,826, lebih tinggi daripada baseline sebesar 0,605, dengan Contextual Precision sebesar 0,983. Penambahan sumber tidak selalu linear karena dapat memperluas cakupan sekaligus menambah noise. Dengan demikian, keunggulan Agentic Retrieval terletak pada kemampuan agent untuk mencoba beberapa jalur penelusuran dan menyesuaikan strategi retrieval dengan karakter bukti pada setiap fase PPEPP.
=====================================================================================================================================
Preparing an accreditation Self-Evaluation Report (LED) requires evidence tracing across many supporting documents. This process is difficult because document names, citation structures, and evidence content are not always identical across study programs. Non-agentic retrieval can support initial search, but it tends to operate statically when document candidates or retrieved contexts remain weak. This study implements a Large Language Model (LLM)-based Agentic Retrieval pipeline to map RPL LED evidence by using historical data from RKA, S2SI, S1TI, and S1SD.
The pipeline consists of LED and reference document preprocessing, embedding storage in Qdrant, Agent 1 for document name matching, and Agent 2 for context tracing. Agent 1 uses filename patterns, citations, and fuzzy search to find target document candidates. Agent 2 performs semantic search, tries several tracing routes, and checks whether the target context supports the required evidence. The evaluation is conducted across PPEPP phases using filename F1-score, Contextual Precision, and Contextual Recall, and is compared with a non-agentic retrieval baseline.
The results show that the agentic approach improves performance on metrics that match the evidence characteristics. In the Penetapan phase, the single-source RKA configuration achieves the best F1-score of 0.712, outperforming the combined RKA+S2SI+S1TI baseline score of 0.287. In the Pelaksanaan phase, the combined RKA+S2SI+S1TI+S1SD configuration achieves the highest Contextual Recall of 0.826, higher than the baseline score of 0.605, with a Contextual Precision of 0.983. Adding historical sources does not always improve performance linearly because it can expand coverage while introducing noise. Thus, the advantage of Agentic Retrieval lies in the agent's ability to try multiple tracing routes and adapt the retrieval strategy to the evidence characteristics of each PPEPP phase.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Akreditasi, Agentic Retrieval, Laporan Evaluasi Diri, Large Language Model, Accreditation, Agentic Retrieval, Self-Evaluation Report, Large Language Model.
Subjects:	T Technology > T Technology (General) > T58.62 Decision support systems T Technology > T Technology (General) > T58.8 Productivity. Efficiency
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Hardy Tee
Date Deposited:	23 Jul 2026 15:11
Last Modified:	23 Jul 2026 15:11
URI:	http://repository.its.ac.id/id/eprint/137581

Actions (login required)

View Item