Klasifikasi Kekuatan Struktur Bangunan Berbasis Citra Menggunakan Geo-Aware Convolutional Neural Network dengan Explainable AI

Simanihuruk, Willyam Brordus Dominikus (2026) Klasifikasi Kekuatan Struktur Bangunan Berbasis Citra Menggunakan Geo-Aware Convolutional Neural Network dengan Explainable AI. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025221121-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (8MB) | Request a copy

Abstract

Indonesia berada pada wilayah pertemuan tiga lempeng tektonik aktif sehingga kerentanan bangunan terhadap gempa menjadi faktor penting dalam penilaian risiko bencana. Pendataan struktur bangunan secara konvensional masih menghadapi kendala dalam hal biaya, waktu, dan cakupan wilayah yang luas. Penelitian ini mengembangkan model klasifikasi kekuatan struktur bangunan menggunakan pendekatan *geo-aware* Convolutional Neural Network (CNN) yang menggabungkan fitur visual dari citra jalan dengan fitur geografis dan parameter seismik. Data yang digunakan meliputi citra bangunan dari Mapillary dan Google Street View, metadata geografis, serta nilai Modified Mercalli Intensity (MMI) yang diperoleh dari Badan Meteorologi, Klimatologi, dan Geofisika (BMKG). Penelitian ini mengimplementasikan dua pendekatan multimodal, yaitu *hierarchical fusion* dan *intermediate fusion*, dengan memanfaatkan arsitektur ResNet50, DenseNet121, dan EfficientNetB0 sebagai model ekstraksi fitur. Hasil pengujian menunjukkan bahwa DenseNet121 dengan *hierarchical fusion* memberikan performa terbaik, dengan nilai akurasi sebesar 0,9912 dan F1-score sebesar 0,9911, sedangkan EfficientNetB0 dengan *intermediate fusion* mencapai akurasi sebesar 0,9474 dan F1-score sebesar 0,9440. Interpretasi model menggunakan Grad-CAM dan SHAP menunjukkan bahwa model mampu memanfaatkan fitur visual bangunan serta informasi seismik secara efektif, dengan fitur MMI menjadi kontributor dominan dalam klasifikasi kelas *High*. Hasil penelitian ini menunjukkan bahwa pendekatan multimodal dapat digunakan untuk menilai kekuatan struktur bangunan terhadap gempa sekaligus memberikan penjelasan mengenai kontribusi setiap fitur melalui penerapan Explainable Artificial Intelligence (XAI).
===================================================================================================================================
Indonesia is located at the convergence of three active tectonic plates, making the vulnerability of buildings to earthquakes a critical factor in disaster risk assessment. Conventional building structure surveys still face challenges related to cost, time, and the coverage of large urban areas. This study develops a building structural strength classification model using a geo-aware Convolutional Neural Network (CNN) that integrates visual features extracted from street-level imagery with geographic features and seismic parameters. The dataset consists of building images collected from Mapillary and Google Street View, geographic metadata, and Modified Mercalli Intensity (MMI) values obtained from the Badan Meteorologi, Klimatologi, dan Geofisika (BMKG). Two multimodal approaches, namely hierarchical fusion and intermediate fusion, are implemented using ResNet50, DenseNet121, and EfficientNetB0 as feature extraction architectures. Experimental results show that DenseNet121 with hierarchical fusion achieved the best performance, with an accuracy of 0.9912 and an F1-score of 0.9911, while EfficientNetB0 with intermediate fusion achieved an accuracy of 0.9474 and an F1-score of 0.9440. Model interpretation using Grad-CAM and SHAP demonstrates that the proposed model effectively utilizes both visual building features and seismic information, with the MMI feature serving as the dominant contributor to the classification of the High class. These findings indicate that the proposed multimodal approach can effectively assess the structural strength of buildings against earthquakes while providing interpretable explanations of each feature's contribution through the application of Explainable Artificial Intelligence (XAI).

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Convolutional Neural Network, Explainable AI, Klasifikasi Kekuatan Bangunan, Klasifikasi Multimodal, Building Strength Classification, Convolutional Neural Network, Explainable AI, Multimodal Classification
Subjects:	Q Science > QA Mathematics > QA336 Artificial Intelligence Q Science > QA Mathematics > QA76.87 Neural networks (Computer Science) T Technology > TA Engineering (General). Civil engineering (General) > TA1637 Image processing--Digital techniques. Image analysis--Data processing.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Willyam Brordus Dominikus Simanihuruk
Date Deposited:	28 Jul 2026 04:05
Last Modified:	28 Jul 2026 04:05
URI:	http://repository.its.ac.id/id/eprint/138363

Actions (login required)

View Item