Pengembangan Chatbot Berbasis Agentic AI dan Retrieval-Augmented Generation (RAG) Untuk Sistem Informasi Akademik di Departemen Teknologi Informasi ITS

Yuniar, Midyanisa (2025) Pengembangan Chatbot Berbasis Agentic AI dan Retrieval-Augmented Generation (RAG) Untuk Sistem Informasi Akademik di Departemen Teknologi Informasi ITS. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5027211025-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (9MB) | Request a copy

Abstract

Sistem informasi akademik yang kurang terstruktur kerap menyulitkan mahasiswa dalam mengakses informasi penting seperti jadwal, pengumuman, dan prosedur administrasi, termasuk di Departemen Teknologi Informasi ITS. Untuk menjawab masalah tersebut, penelitian ini mengusulkan pengembangan chatbot WhatsApp berbasis Agentic AI dan Retrieval-Augmented Generation (RAG) guna menyajikan informasi secara cepat, akurat, dan kontekstual. Sistem dikembangkan menggunakan FastAPI sebagai backend dan React.js untuk dashboard admin, serta diintegrasikan dengan Twilio API sebagai penghubung ke WhatsApp. Model GPT-4o digunakan untuk pemrosesan bahasa alami, sementara teknik Hybrid Retriever dengan LLM-based Contextual Reranking dan prompt engineering diterapkan untuk meningkatkan relevansi jawaban berdasarkan dokumen internal. Evaluasi dilakukan menggunakan metrik RAGAS, uji fungsionalitas sistem, serta user testing terhadap 41 pengguna. Hasil pengujian menunjukkan bahwa kombinasi penggunaan retriever hybrid dan prompting terintegrasi (Prompt B) menghasilkan skor RAGAS rata-rata sebesar 0,80, yang mencerminkan relevansi, akurasi, dan kesesuaian konteks jawaban. Penerapan retriever hybrid serta teknik prompt engineering memberikan peningkatan signifikan pada kualitas jawaban dibandingkan dengan baseline retrieval. Selain itu, penerapan Agentic AI memungkinkan sistem melakukan aksi spesifik (tools function) seperti memberikan data terkait magang dan penyimpanan umpan balik secara otomatis. Integrasi Agentic AI dan RAG tidak hanya menghasilkan respons berbasis fakta, tetapi juga memperluas fungsionalitas chatbot secara signifikan. Penelitian ini menunjukkan bahwa chatbot dapat meningkatkan efisiensi akses informasi di lingkungan akademik. Pengembangan selanjutnya disarankan menggunakan Twilio production dan evaluasi model embedding yang lebih unggul.
===================================================================================================================================
Poorly structured academic information systems often hinder students' access to important information such as schedules, announcements, and administrative procedures, including those in the ITS Information Technology Department. To address this issue, this study proposes the development of a WhatsApp chatbot based on Agentic AI and Retrieval-Augmented Generation (RAG) to deliver information quickly, accurately, and contextually. The system was developed using FastAPI for the backend and React.js for the admin dashboard, and is integrated with the Twilio API to connect to WhatsApp. The GPT-4o model was utilized for natural language processing, while a Hybrid Retriever technique, featuring LLM-based contextual reranking and prompt engineering, was employed to enhance the relevance of answers based on internal documents. Evaluation was conducted using RAGAS metrics, system functionality tests, and user testing involving 41 participants. The results indicated that the combination of the hybrid retriever and integrated prompting (Prompt B) achieved an average RAGAS score of 0,80, reflecting the relevance, accuracy, and contextual appropriateness of the answers. The implementation of hybrid retrievers and prompt engineering techniques led to a significant improvement in response quality compared to baseline retrieval. Additionally, the integration of Agentic AI enables the system to perform specific actions (tool functions), such as automatically providing internship-related data and storing feedback. The combination of Agentic AI and RAG not only generates fact-based responses but also significantly enhances the chatbot's functionality. This research demonstrates that chatbots can improve the efficiency of information access in academic settings. Further development is recommended to utilize the production version of Twilio and evaluate superior embedding models.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Agentic AI, Chatbot, Large Language Model (LLM), Retrieval-Augmented Generation (RAG), Sistem Informasi Akademik, Academic Information System, Agentic AI, Chatbot, Large Language Model (LLM), Retrieval-Augmented Generation (RAG).
Subjects:	T Technology > T Technology (General) > T58.8 Productivity. Efficiency
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information Technology > 59201-(S1) Undergraduate Thesis
Depositing User:	Midyanisa Yuniar
Date Deposited:	18 Jul 2025 06:36
Last Modified:	18 Jul 2025 06:36
URI:	http://repository.its.ac.id/id/eprint/120041

Actions (login required)

View Item