Model Intent Recognition Berbasis Bert Pada Chatbot Edukasi Etika Penggunaan AI Di Lingkungan Akademik

Wardani, Ayunda Kusuma (2026) Model Intent Recognition Berbasis Bert Pada Chatbot Edukasi Etika Penggunaan AI Di Lingkungan Akademik. Masters thesis, Institut Teknologi Sepuluh Nopember.

Text 6026242004-Master_Thesis.pdf Restricted to Repository staff only Download (2MB) | Request a copy

Abstract

Perkembangan Artificial Intelligence (AI) di lingkungan pendidikan semakin meningkat. Namun, penggunaan tersebut belum sepenuhnya disertai dengan pemahaman yang baik mengenai etika penggunaannya. Hal ini dapat menimbulkan risiko seperti plagiarisme, ketergantungan berlebihan pada AI, serta penyalahgunaan informasi. Oleh karena itu, dibutuhkan media edukasi yang interaktif untuk membantu mahasiswa memahami etika penggunaan AI secara tepat dan bertanggung jawab. Penelitian ini bertujuan untuk mengembangkan dan mengevaluasi model intent recognition berbasis BERT pada chatbot edukasi etika penggunaan Artificial Intelligence (AI) di lingkungan akademik, khususnya perguruan tinggi, termasuk dalam menangani pertanyaan di luar topik (out-of-scope). Penelitian ini menggunakan Design Science Research Methodology (DSRM). Dataset disusun berdasarkan Panduan Penggunaan Generative AI pada Pembelajaran di Perguruan Tinggi dari Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi Tahun 2024 menggunakan pendekatan Retrieval-Augmented Generation (RAG) dengan model DeepSeek. Selanjutnya dilakukan fine-tuning pada tiga varian model BERT, yaitu Multilingual BERT, IndoBERT, dan DistilBERT menggunakan skema 5-Fold Stratified Cross Validation. Evaluasi model dilakukan menggunakan accuracy, precision, recall, dan F1-score, sedangkan evaluasi chatbot menggunakan Intent Recognition Rate (IRR). Hasil penelitian menunjukkan bahwa IndoBERT memberikan performa terbaik dengan accuracy 0,9571, precision 0,9593, recall 0,9571, dan F1-score 0,9575, serta memiliki penggunaan memori GPU paling rendah sebesar 1463.06 MB. Untuk penanganan pertanyaan OOS, metode Entropy-Based Detection dengan threshold 0,9 memberikan performa terbaik dibandingkan metode Confidence Threshold, dengan F1-score sebesar 0.783. Hasil evaluasi chatbot menunjukkan nilai IRR sebesar 87,00%, yang menunjukkan bahwa chatbot mampu mengenali intent dan memberikan respons yang sesuai pada sebagian besar pertanyaan yang diuji. Hasil penelitian ini menunjukkan bahwa kombinasi model IndoBERT dan Entropy-Based Detection efektif dalam meningkatkan keandalan chatbot edukasi etika penggunaan AI di lingkungan akademik.
===================================================================================================================================
The use of Artificial Intelligence (AI) in education has grown rapidly. However, its adoption has not always been accompanied by a proper understanding of the ethical principles governing its use. This may lead to risks such as plagiarism, excessive dependence on AI, and misuse of information. Therefore, an interactive educational medium is needed to help students understand the ethical use of AI in an appropriate and responsible manner. This study aims to develop and evaluate a BERT-based intent recognition model for an educational chatbot on the ethical use of Artificial Intelligence (AI) in academic environments, particularly in higher education, including the handling of out-of-scope (OOS) queries. This study employed the Design Science Research Methodology (DSRM). The dataset was developed based on the Guidelines for the Use of Generative AI in Higher Education Learning published by the Ministry of Education, Culture, Research, and Technology of Indonesia in 2024 using a Retrieval-Augmented Generation (RAG) approach with the DeepSeek model. Fine-tuning was then performed on three BERT variants, namely Multilingual BERT, IndoBERT, and DistilBERT, using a 5-Fold Stratified Cross Validation scheme. Model performance was evaluated using accuracy, precision, recall, and F1-score, while chatbot performance was evaluated using the Intent Recognition Rate (IRR). The results showed that IndoBERT achieved the best performance, with an accuracy of 0,9571, precision of 0,9593, recall of 0,9571, and F1-score of 0,9575, while also requiring the lowest GPU memory usage of 1463.06 MB. For handling OOS queries, the Entropy-Based Detection method with a threshold of 0.9 outperformed the Confidence Threshold method, achieving the highest F1-score of 0,740. The chatbot evaluation yielded an IRR of 87.00%, indicating that the chatbot was able to correctly recognize user intents and provide appropriate responses for most of the evaluated queries. These findings demonstrate that the combination of IndoBERT and Entropy-Based Detection effectively improves the reliability of educational chatbots in understanding user intent within the domain of AI ethics in academic environments.

Item Type:	Thesis (Masters)
Uncontrolled Keywords:	Chatbot, Intent Recognition, Etika Artificial Intelligence Akademik, BERT, Design Science Research Methodology, Academic Artificial Intelligence Ethics
Subjects:	T Technology > T Technology (General) > T57.5 Data Processing T Technology > T Technology (General) > T59.7 Human-machine systems.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information System > 59101-(S2) Master Thesis
Depositing User:	Ayunda Kusuma Wardani
Date Deposited:	29 Jul 2026 06:25
Last Modified:	29 Jul 2026 06:25
URI:	http://repository.its.ac.id/id/eprint/139378

Actions (login required)

View Item