Klasifikasi Multimodal Konten Bencana Alam Di Indonesia Menggunakan Integrasi BERT, ResNet, Dan Fitur Konteks Sosial

Farizky, Dovy Adeeb (2026) Klasifikasi Multimodal Konten Bencana Alam Di Indonesia Menggunakan Integrasi BERT, ResNet, Dan Fitur Konteks Sosial. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5026221033-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (10MB) | Request a copy

Abstract

Peningkatan dampak bencana alam di Indonesia membutuhkan respons darurat yang efektif, yang saat ini sering kali bergantung pada arus informasi vital dari platform media sosial. Namun, tingginya noise serta variasi format data teks dan gambar, ditambah dengan kerumitan komputasi dalam arsitektur klasifikasi lintas-modal, menjadi permasalahan utama. Sebagai solusi, penelitian ini mengusulkan model klasifikasi multimodal yang disederhanakan dengan menerapkan pendekatan early fusion. Model ini mengintegrasikan ekstraksi fitur leksikal dari IndoBERT, fitur visual spasial dari ResNet-50, serta fitur konteks sosial yang digabungkan melalui konkatenasi matriks menggunakan jaringan Multilayer Perceptron atau MLP. Dampak setiap jenis data dievaluasi melalui enam skenario ablation study untuk menyelesaikan dua penugasan utama. Hasil pengujian menunjukkan bahwa arsitektur Multimodal Penuh mencatatkan performa kokoh, dengan meraih tingkat akurasi sebesar 90.28% dan Macro F1-Score sebesar 89.30% pada penugasan penyaringan informasi bencana Task 1, serta Macro F1-Score 84.54% pada penugasan kategorisasi spesifik bencana Task 2. Berdasarkan studi ablasi tersebut, skenario kombinasi teks dan fitur sosial tampil lebih baik pada Task 1, sementara skenario teks murni menjadi yang paling unggul pada Task 2. Dinamika performa ini secara matematis mengekspos adanya fenomena visual noise dari gambar serta terjadinya contextual dilution akibat penggunaan metrik sosial. Secara statistik, arsitektur multimodal yang diusulkan berhasil memproses data yang penuh noise menjadi luaran klasifikasi presisi dan andal untuk mendukung respons tanggap darurat yang efisien.
================================================================================================================================
The increasing impact of natural disasters in Indonesia requires an effective emergency response, which currently often relies on the vital flow of information from social media platforms. However, the high noise and varying formats of text and image data, coupled with the computational complexity of cross-modal classification architectures, pose a major challenge. As a solution, this study proposes a simplified multimodal classification model employing an early fusion approach. The model integrates lexical feature extraction from IndoBERT, spatial visual features from ResNet-50, and social context features combined through matrix concatenation using a Multilayer Perceptron or MLP network. The impact of each data modality is evaluated through six ablation study scenarios to accomplish two main tasks. The experimental results show that the Full Multimodal architecture achieves robust performance, recording an accuracy of 90.28% and a Macro F1-Score of 89.30% on the disaster information filtering assignment Task 1, as well as a Macro F1-Score of 84.54% on the specific disaster categorization assignment Task 2. Based on the ablation study, the scenario combining text and social features performs better on Task 1, while the pure text scenario emerges as the most superior on Task 2. This performance dynamic mathematically exposes the phenomenon of visual noise from images and the occurrence of contextual dilution due to the utilization of social metrics. Statistically, the proposed multimodal architecture successfully processes highly noisy data into precise and reliable classification outputs to support an efficient emergency response.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Klasifikasi konten bencana, Manajemen bencana, Data multimodal, BERT, ResNet, Fusi data, Disaster content classification, Crisis management, Multimodal data, BERT, ResNet-50, Data fusion
Subjects:	H Social Sciences > HV Social pathology. Social and public welfare > HV551.5.I4 Hazard mitigation Q Science > QA Mathematics > QA76.87 Neural networks (Computer Science)
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information System > 57201-(S1) Undergraduate Thesis
Depositing User:	Dovy Adeeb Farizky
Date Deposited:	29 Jul 2026 03:39
Last Modified:	29 Jul 2026 03:39
URI:	http://repository.its.ac.id/id/eprint/138971

Actions (login required)

View Item