Pengembangan Fine-Tuning Large Language Model untuk Wawancara Diagnosis Klinis Multi-Level Berbasis DSM-5

Yantono, Hans Sanjaya (2026) Pengembangan Fine-Tuning Large Language Model untuk Wawancara Diagnosis Klinis Multi-Level Berbasis DSM-5. Project Report. [s.n.]. (Unpublished)

Text 5025231034-Project_Report.pdf Restricted to Repository staff only Download (2MB) | Request a copy

Abstract

Pemanfaatan Large Language Model (LLM) dalam bidang informatika medis membuka peluang untuk membangun sistem yang mampu melakukan penalaran diagnostik secara terstruktur. Namun, model bahasa umum cenderung gagal mempertahankan alur wawancara klinis dan menghasilkan keluaran yang konsisten dengan standar diagnostik resmi. Kerja praktik ini bertujuan mengembangkan sistem diagnostik klinis berbasis teks melalui fine-tuning LLM menggunakan metode Low-Rank Adaptation (LoRA). Tahap awal penelitian dilakukan dengan mengevaluasi secara komparatif beberapa model open-source untuk menentukan model dasar, kemudian melatih model terpilih, yaitu Qwen2.5-14B-Instruct, menggunakan dataset dialog klinis sintetis yang disusun berdasarkan kriteria DSM-5. Model hasil pelatihan dikonversi ke format GGUF dengan kuantisasi 4-bit (q4_k_m) dan dipublikasikan melalui Hugging Face Hub. Proses deployment dilakukan dengan menjalankan model pada mesin inferensi Ollama di server laboratorium, yang diintegrasikan dengan backend Flask dan diekspos ke jaringan publik menggunakan ngrok. Riwayat percakapan dan laporan diagnostik disimpan pada basis data Supabase, sedangkan antarmuka pengguna dikembangkan sebagai aplikasi web statis yang di-deploy melalui GitHub dan Netlify. Pengujian dilakukan terhadap 40 kasus uji yang terdiri atas 20 skenario generasi laporan dan 20 skenario dialog. Hasil evaluasi menunjukkan bahwa model mencapai akurasi diagnosis sebesar 100% pada skenario generasi laporan (20 dari 20 kasus) dan 100% pada skenario dialog yang berhasil dievaluasi (4 dari 4 kasus). Evaluasi kualitas penalaran menggunakan metode LLM-as-Judge menghasilkan rata-rata skor koherensi sebesar 4,25 dari 5 dan skor faktualitas sebesar 4,25 dari 5. Namun, rata-rata cosine similarity pada reasoning trace untuk skenario dialog masih sebesar 0,40, yang menunjukkan bahwa model masih memiliki keterbatasan dalam menjaga konsistensi proses penalarannya.
================================================================================================================================
The utilization of Large Language Models (LLMs) in medical informatics provides opportunities to develop systems capable of performing structured diagnostic reasoning. However, general-purpose language models often struggle to maintain the flow of clinical interviews and generate outputs that are consistent with official diagnostic standards. This internship project aimed to develop a text-based clinical diagnostic system by fine-tuning an LLM using the Low-Rank Adaptation (LoRA) method. The study began with a comparative evaluation of several open-source models to determine the most suitable base model, followed by training the selected model, Qwen2.5-14B-Instruct, on a synthetic clinical dialogue dataset constructed according to DSM-5 criteria. The trained model was converted into the GGUF format using 4-bit (q4_k_m) quantization and published on the Hugging Face Hub. Deployment was performed by running the model on the Ollama inference engine hosted on a laboratory server, integrated with a Flask backend, and exposed to the public network through ngrok. Conversation histories and diagnostic reports were stored in a Supabase database, while the user interface was implemented as a static web application deployed via GitHub and Netlify. The system was evaluated using 40 test cases consisting of 20 report generation scenarios and 20 dialogue scenarios. The evaluation results showed that the model achieved 100% diagnostic accuracy in the report generation scenarios (20/20 cases) and 100% accuracy in the evaluated dialogue scenarios (4/4 cases). Reasoning quality was assessed using the LLM-as-Judge method, yielding an average coherence score of 4.25/5 and a factuality score of 4.25/5. However, the average cosine similarity of the reasoning traces in the dialogue scenarios was only 0.40, indicating that the model still has limitations in maintaining reasoning consistency.

Item Type:	Monograph (Project Report)
Uncontrolled Keywords:	Diagnostik Klinis, DSM-5, GGUF, Large Language Model, Low-Rank Adaptation (LoRA), Ollama.
Subjects:	T Technology > T Technology (General) > T58.5 Information technology. IT--Auditing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Hans Yantono
Date Deposited:	09 Jul 2026 04:26
Last Modified:	09 Jul 2026 05:23
URI:	http://repository.its.ac.id/id/eprint/134547

Actions (login required)

View Item