Klasifikasi Usulan Aspirasi Masyarakat untuk Program Prioritas Pembangunan Daerah Berbasis Large Language Model

Putratama, Muhammad Fariz Arya Brahmantya (2026) Klasifikasi Usulan Aspirasi Masyarakat untuk Program Prioritas Pembangunan Daerah Berbasis Large Language Model. Masters thesis, Institut Teknologi Sepuluh Nopember.

Text 6022241127-Master_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (2MB) | Request a copy

Abstract

Transformasi digital melalui e-government telah mempercepat administrasi publik, tetapi tingginya volume proposal aspirasi masyarakat masih menimbulkan hambatan pada verifikasi administratif dan penyelarasan prioritas pembangunan daerah. Ruang lingkup penelitian dibatasi pada dokumen usulan aspirasi masyarakat yang digunakan dalam proses perencanaan pembangunan pada satu pemerintah daerah tingkat kabupaten. Penelitian ini mengevaluasi pemanfaatan GPT-5 Nano sebagai model LLM utama untuk ekstraksi informasi, pemeriksaan administratif, peringkasan, dan klasifikasi dokumen ke dalam kategori program prioritas pembangunan daerah, serta membandingkannya dengan Gemini 2.5 Flash-Lite, Gemini 3.5 Flash, Multinomial Naïve Bayes (MNB), dan Complement Naïve Bayes (CNB) pada tahap klasifikasi. Sistem dibangun melalui empat tahapan: ekstraksi teks menggunakan Docling dan Tesseract OCR, pemeriksaan administratif berdasarkan lima kriteria berbasis teks, peringkasan dokumen, dan klasifikasi dokumen. Evaluasi menggunakan 259 dokumen ground truth untuk pemeriksaan administratif, sampel representatif dari 415 dokumen untuk evaluasi kualitas ekstraksi teks, serta 415 dokumen untuk evaluasi peringkasan dan klasifikasi dengan Stratified 10-Fold Cross-Validation pada tahap klasifikasi. Hasil menunjukkan bahwa peringkasan mencapai ROUGE-1 Precision 87,02% dan SBERT Cosine Similarity 80,00%. Pada pemeriksaan administratif, GPT-5 Nano mencapai F1-Score ≥ 0,90 pada empat dari lima kriteria administratif berbasis teks, kriteria rincian anggaran biaya memperoleh F1-Score lebih rendah, yaitu sekitar 0,67. Pada klasifikasi, F1-Score GPT-5 Nano mencapai 87,59%, Gemini 2.5 Flash-Lite 87,24%, Gemini 3.5 Flash 85,21%, CNB 87,60%, dan MNB 84,30%. GPT-5 Nano memperoleh Accuracy tertinggi sebesar 87,72% dan Precision tertinggi sebesar 90,12%, sedangkan CNB memperoleh Recall dan F1-Score tertinggi, masing-masing sebesar 87,61% dan 87,60%. Hasil ini menunjukkan bahwa CNB sangat kompetitif secara kuantitatif, sementara GPT-5 Nano tetap layak diposisikan sebagai model utama karena menunjukkan performa yang paling seimbang dan dapat menyertakan alasan klasifikasi yang membantu interpretasi hasil prediksi.
=====================================================================================================================================
Digital transformation through e-government has accelerated public administration; however, the high volume of community aspiration proposals continues to create bottlenecks in administrative verification and alignment with regional development priorities. The scope of this study is limited to community aspiration proposal documents used in the development planning process of a single regency-level local government. This study evaluates the use of GPT-5 Nano as the primary Large Language Model (LLM) for information extraction, administrative verification, document summarization, and classification of documents into regional development priority program categories. Its performance is also compared with Gemini 2.5 Flash-Lite, Gemini 3.5 Flash, Multinomial Naïve Bayes (MNB), and Complement Naïve Bayes (CNB) at the classification stage. The system is developed through four stages: text extraction using Docling and Tesseract OCR, administrative verification based on five text-based criteria, document summarization, and document classification. The evaluation uses 259 ground-truth documents for administrative verification, a representative sample drawn from 415 documents for evaluating text extraction quality, and 415 documents for evaluating summarization and classification, with Stratified 10-Fold Cross-Validation applied at the classification stage. The results show that the summarization module achieves a ROUGE-1 Precision of 87.02% and an SBERT Cosine Similarity of 80.00%. In administrative verification, GPT-5 Nano achieves an F1-score of ≥ 0.90 in four out of five text-based administrative criteria, while the budget details criterion obtains a lower F1-score of approximately 0.67. In classification, GPT-5 Nano achieves an F1-score of 87.59%, Gemini 2.5 Flash-Lite 87.24%, Gemini 3.5 Flash 85.21%, CNB 87.60%, and MNB 84.30%. GPT-5 Nano obtains the highest Accuracy of 87.72% and the highest Precision of 90.12%, while CNB achieves the highest Recall and F1-score, at 87.61% and 87.60%, respectively. These results indicate that CNB is highly competitive quantitatively, while GPT-5 Nano remains suitable as the primary model because it demonstrates the most balanced performance and can provide classification reasons that support the interpretation of prediction results.

Item Type:	Thesis (Masters)
Uncontrolled Keywords:	Dokumen Administratif, Complement Naïve Bayes, Klasifikasi Dokumen, Large Language Model, Multinomial Naïve Bayes, Administrative Document, Complement Naïve Bayes, Document Classification, Large Language Model, Multinomial Naïve Bayes
Subjects:	J Political Science > JF Political institutions (General) > JF1351.H6 Public administration. Q Science > QA Mathematics > QA279.5 Bayesian statistical decision theory. T Technology > T Technology (General) > T58.62 Decision support systems
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Electrical Engineering > 20101-(S2) Master Thesis
Depositing User:	Muhammad Fariz Arya Brahmantya Putratama
Date Deposited:	20 Jul 2026 03:11
Last Modified:	20 Jul 2026 03:11
URI:	http://repository.its.ac.id/id/eprint/135509

Actions (login required)

View Item