Narana, Yeremia Maydinata (2026) Multi-label Intent Classification untuk Pengembangan Chatbot Konseling Psikologi dengan Pendekatan Religius Berbasis Retrieval Augmented Generation. Other thesis, Institut Teknologi Sepuluh Nopember.
![[thumbnail of 5026221068-Undergraduate_Thesis.pdf]](http://repository.its.ac.id/style/images/fileicons/text.png) |
Text
5026221068-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (4MB) | Request a copy |
Abstract
Salah satu isu kesehatan global yang menyumbang 4,9% beban penyakit dunia adalah gangguan mental. Indonesia juga turut menanggung beban penyakit tersebut dengan jumlah kasus yang meningkat secara signifikan dari 6,1% populasi Indonesia di 2018 menjadi sekitar 20% populasi di 2023. Tercatat layanan profesional masih terbilang jauh dari kata cukup di mana hanya 0,3 psikiater eksisting per 100.000 penduduk, dan didorong oleh masalah stigma sosial yang menyebabkan orang enggan untuk mencari bantuan turut meningkatkan jumlah kebutuhan akan layanan profesional. Selain itu stigma negatif yang melekat di masyarakat Indonesia terkait gangguan mental turut menyumbang angka kasus di Indonesia. Chatbot menawarkan dukungan psikologis yang bisa diakses di manapun dan kapanpun, dan juga karena sifat anonim mendorong banyak orang merasa lebih nyaman untuk menceritakan masalahnya ke Chatbot dibandingkan manusia. Penambahan aspek spiritual dalam Chatbot diharapkan bisa meningkatkan relevansi di masyarakat Indonesia. Namun, sebagian besar Chatbot yang sudah eksisting sering kali gagal untuk menangkap secara akurat kueri pengguna yang kompleks. Diperlukan sebuah model di mana model tersebut mampu memahami teks secara semantik dan bisa mengklasifikasikan teks tersebut ke beberapa inten sekaligus, sehingga chatbot bisa memberikan respon yang relevan dan tepat. Multi-label Intent Classification merupakan jawaban yang tepat untuk masalah tersebut. Dengan menerapkan arsitektur Label-Aware BERT Attention Network, klasifikasi dilakukan dengan melakukan vector embedding untuk label dan juga input kueri menggunakan IndoBERT, lalu diproyeksikan antar vektor untuk model bisa mendeteksi inten dari input pengguna. Inten yang terdeteksi selanjutnya akan dijadikan prompt bersamaan hasil retrieval dari data answer dan ayat Alkitab dalam pipeline Retrieval-Augmented Generation¸lalu diolah oleh Large Language Model (LLM) Gemini-2.5 Flash. Pengoperasian chatbot akan berbasis website dengan framework Flask. Diharapkan dengan adanya penelitian ini bisa mengembangkan prototipe Chatbot Konseling berbasis Religi yang mampu memahami inten-inten pengguna secara akurat dan memberi respons yang relevan, kontekstual, dan spiritual kepada pengguna. Performa model sangat mumpuni untuk mengerjakan klasifikasi inten multi-label dengan skor Macro-F1 0,8909 dan skor Micro-F1 0,9243 yang menjadikan model IndoBERT sebagai baseline. Untuk aspek religius, ayat yang di-retrieve oleh sistem cukup baik di mana penilaian bukan hanya kebetulan belaka. Model memiliki performa yang cukup dengan pengambilan data kuesioner metode Cohen's Kappa dengan skor rata-rata 0,5346443353. Secara medis, chatbot konseling ini sudah tervalidasi oleh 1 ahli dengan metode Content Validity Index (CVI) dengan skor 0,7777777778. Secara keseluruhan, chatbot mampu menentukan inten dengan akurat dan mampu berperan sebagai konselor virtual. Namun, perlu dicatat bahwa chatbot ini hanya divalidasi mampu mengerjakan fungsi dasar konselor virtual dan belum terbukti secara klinis mampu menangani masalah sedang. Hal tersebut terjadi karena ahli yang memvalidasi hanya 1 dan belum ada eksplorasi beberapa skenario. Dengan itu, terdapat potensi pengembangan untuk chatbot yaitu meningkatkan keamanan uji chatbot konseling dengan menambahkan jumlah ahli dan skenario pengujian, dan penyempurnaan sistem retrieval ayat untuk menambah keakuratan retrieval ayat.
======================================================================================================================================
One of the global health issues that contributes to 4.9% of the global disease burden is mental disorders. Indonesia also bears the burden of this disease, with the number of cases increasing significantly from 6.1% of the Indonesian population in 2018 to around 20% of the population in 2023. It is noted that professional services are still far from sufficient, with only 0.3 existing psychiatrists per 100,000 inhabitants, and driven by social stigma issues that cause people to be reluctant to seek help, further increasing the need for professional services. In addition, the negative stigma attached to mental disorders in Indonesian society also contributes to the number of cases in Indonesia. Chatbots offer psychological support that can be accessed anywhere and anytime, and their anonymous nature encourages many people to feel more comfortable sharing their problems with a chatbot than with a human being. The addition of a spiritual aspect to chatbots is expected to increase their relevance in Indonesian society. However, most existing chatbots often fail to accurately capture complex user queries. A model is needed that can understand text semantically and classify it into several intents simultaneously, so that the chatbot can provide relevant and appropriate responses. Multi-label Intent Classification is the right answer to this problem. By applying the Label-Aware BERT Attention Network architecture, classification is carried out by performing vector embedding for labels and query inputs using IndoBERT, then projecting between vectors so that the model can detect the intent of the user's input. The detected intent is then used as a prompt along with the retrieval results from the answer data and Bible verses in the Retrieval-Augmented Generation pipeline, then processed by the Gemini-2.5 Flash Large Language Model (LLM). The chatbot will operate on a website using the Flask framework. It is hoped that this research will develop a prototype Religious Counselling Chatbot that can accurately understand user intentions and provide relevant, contextual, and spiritual responses to users. The model’s performance is highly capable of handling multi-label intent classification, with a Macro-F1 score of 0.8909 and a Micro-F1 score of 0.9243, making the IndoBERT model the baseline. In terms of religious content, the verses retrieved by the system are of a sufficiently high standard, indicating that the assessments are not merely coincidental. The model performs adequately when using data from questionnaires, with a Cohen’s Kappa score averaging 0.5346443353. From a medical perspective, this counselling chatbot has been validated by one expert using the Content Validity Index (CVI) method, achieving a score of 0.7777777778. Overall, the chatbot is capable of accurately determining intent and can function as a virtual counsellor. However, it should be noted that this chatbot has only been validated to perform the basic functions of a virtual counsellor and has not yet been clinically proven to handle moderate-severity issues. This is because only one expert carried out the validation and several scenarios have not yet been explored. Consequently, there is potential for further development of the chatbot, namely by enhancing the reliability of the counselling chatbot tests by increasing the number of experts and test scenarios, and by refining the verse retrieval system to improve the accuracy of verse retrieval.
| Item Type: | Thesis (Other) |
|---|---|
| Uncontrolled Keywords: | Multi-label Intent Classification, Label-Aware BERT Attention Network, Chatbot, Retrieval-Augmented Generation, Konseling. Multi-label Intent Classification, Label-Aware BERT Attention Network, Chatbot, Retrieval-Augmented Generation, Counselling. |
| Subjects: | T Technology > T Technology (General) > T57.5 Data Processing T Technology > T Technology (General) > T59.7 Human-machine systems. |
| Divisions: | Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information System > 57201-(S1) Undergraduate Thesis |
| Depositing User: | Yeremia Maydinata Narana |
| Date Deposited: | 29 Jul 2026 01:26 |
| Last Modified: | 29 Jul 2026 01:26 |
| URI: | http://repository.its.ac.id/id/eprint/139028 |
Actions (login required)
 |
View Item |