Automatic Quiz Generator Berbasis Large Language Model Menggunakan Metode Retrieval Augmented Generation

Fikar, Muhammad Detri Abdul (2026) Automatic Quiz Generator Berbasis Large Language Model Menggunakan Metode Retrieval Augmented Generation. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025221236-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (8MB) | Request a copy

Abstract

Perkembangan Large Language Model (LLM) membuka peluang pembuatan kuis otomatis untuk mendukung pembelajaran mandiri. Namun, penggunaan LLM secara langsung masih dapat menghasilkan soal yang kurang sesuai dengan dokumen sumber. Penelitian ini bertujuan merancang dan mengimplementasikan aplikasi Automatic Quiz Generator berbasis LLM menggunakan metode Retrieval-Augmented Generation (RAG). Sistem memproses dokumen PDF melalui ekstraksi teks, preprocessing, chunking, embedding, retrieval konteks, dan generasi soal pilihan ganda serta esai. Evaluasi dilakukan menggunakan dua buku, yaitu Software Engineering Bab 2 Process Models dan Database Management Systems Bab 7 Internet Applications, dengan empat skenario berdasarkan buku dan bahasa soal. Konfigurasi RAG menggunakan embedding nvidia/llama-nemotron-embed-vl-1b-v2:free, top_k 20 chunk, chunk size 4000, chunk overlap 400, dan relevance threshold 0,40. Evaluasi menggunakan ROUGE, BLEU, Precision, Recall, F1-Score, Mean Similarity, SUS, serta black box testing. Hasil terbaik generasi soal diperoleh pada skenario Database Management Systems Bahasa Inggris dengan ROUGE-1 0,1143 dan BLEU-1 0,2811, sedangkan Software Engineering Bahasa Indonesia memperoleh Mean Similarity tertinggi 0,6098 dan Precision 1,0000. Skor SUS sebesar 79,75 termasuk kategori Good, dan seluruh skenario black box testing berhasil. Dengan demikian, aplikasi ini dapat menghasilkan kuis dari PDF dan digunakan sebagai alat bantu pembelajaran mandiri.
===============================================================================================================================
The development of Large Language Models (LLMs) enables automatic quiz generation to support self-directed learning. However, directly using an LLM may produce questions that are not sufficiently grounded in the source document. This research aims to design and implement an LLM-based Automatic Quiz Generator application using the Retrieval-Augmented Generation (RAG) method. The system processes PDF documents through text extraction, preprocessing, chunking, embedding generation, context retrieval, and the generation of multiple-choice and essay questions. The evaluation uses two books, namely Software Engineering Chapter 2 Process Models and Database Management Systems Chapter 7 Internet Applications, with four scenarios based on the book source and question output language. The RAG configuration uses nvidia/llama-nemotron-embed-vl-1b-v2:free embedding, top_k of 20 chunks, chunk size of 4000, chunk overlap of 400, and a relevance threshold of 0.40. The evaluation uses ROUGE, BLEU, Precision, Recall, F1-Score, Mean Similarity, SUS, and black box testing. The best question generation result is obtained in the Database Management Systems English scenario with ROUGE-1 of 0.1143 and BLEU-1 of 0.2811, while the Software Engineering Indonesian scenario obtains the highest Mean Similarity of 0.6098 and Precision of 1.0000. The SUS score of 79.75 is categorized as Good, and all black box testing scenarios are successful. Therefore, the application can generate quizzes from PDF documents and be used as a supporting tool for self-directed learning.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Automatic Quiz Generator, Large Language Model, Retrieval-Augmented Generation, Evaluasi Soal, Question Evaluation, System Usability Scale.
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK5105.888 Web sites--Design. Web site development.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Muhammad Detri Abdul Fikar
Date Deposited:	28 Jul 2026 02:29
Last Modified:	28 Jul 2026 02:29
URI:	http://repository.its.ac.id/id/eprint/138281

Actions (login required)

View Item