Explainable Personalized Depression Detection from Online Social Networks Through Personality Trait-Aware Multimodal Deep Learning Approach

Pradnyana, Gede Aditra (2025) Explainable Personalized Depression Detection from Online Social Networks Through Personality Trait-Aware Multimodal Deep Learning Approach. Doctoral thesis, Institut Teknologi Sepuluh Nopember.

Text 7022221005-Doctoral.pdf - Accepted Version Restricted to Repository staff only Download (16MB) | Request a copy

Abstract

Depresi telah menjadi tantangan kesehatan global yang mendesak. Oleh karena itu, diperlukan pendekatan komputasional yang andal dengan hasil yang transparan untuk mendukung deteksi dini serta pemantauan berkelanjutan. Online Social Networks(OSNs), seperti sosial media, menawarkan sumber data yang kaya karena mampu merekam ekspresi alami pengguna terkait pikiran, emosi, dan perilaku sehari-hari. Penelitian ini mengusulkan sebuah kerangka deteksi depresi yang bersifat personal serta dilengkapi penjelasan melalui integrasi ciri kepribadian ke dalam pendekatan multimodal deep learning. Data OSN yang digunakan bersifat heterogen, mencakup konten teks, konten visual, serta pola polaritas sentimen. Proses pembelajaran dilakukan dengan memanfaatkan transformer-based language models, strategi ensemble learning, dan mekanisme adaptive gated fusion untuk representasi lintas-modal. Ciri kepribadian diprediksi secara otomatis dari konten yang dihasilkan pengguna, kemudian digunakan sebagai pengetahuan tambahan untuk membangun model personal. Integrasi ini didasarkan pada temuan psikologi yang menunjukkan adanya hubungan erat antara dimensi kepribadian dan kerentanan seseorang untuk mengalami depresi, sehingga memungkinkan model menangkap pola afektif dan kognitif yang lebih sesuai dengan karakteristik individu. Hasil eksperimen menunjukkan bahwa pemanfaatan informasi kepribadian secara signifikan meningkatkan akurasi deteksi. Model transformer berukuran kompak, seperti Robustly Optimized BERT Pretraining Approach (RoBERTa) yang diperkaya dengan fitur kepribadian, mampu melampaui kinerja model bahasa berukuran besar, seperti LLaMA, Mistral, dan GPT-3.5. Peningkatan kinerja lebih lanjut diperoleh melalui pemanfaatan model bahasa domain spesifik, seperti MentalRoBERTa, pola polaritas sentimen, serta modalitas visual, yang membuktikan pentingnya strategi yang adaptif terhadap domain dan modalitas. Studi ablasi juga menegaskan peran krusial fitur kepribadian. Kombinasi modalitas berupa teks, visual, dan kepribadian yang difusikan melalui mekanisme cross-modal attention dan ensemble learning secara konsisten mengungguli model unimodal maupun metode fusi konvensional, baik dari sisi akurasi maupun F1-score. Secara keseluruhan, penelitian ini menegaskan potensi penggabungan multimodal deep learning dengan pemodelan berbasis kepribadian dalam memajukan deteksi kesehatan mental otomatis, khususnya depresi. Kerangka yang diusulkan tidak hanya meningkatkan kinerja deteksi, tetapi juga memberikan interpretasi hasil yang lebih komprehensif melalui metode post-hoc explainability. Hasil penelitian ini diharapkan dapat mendukung deteksi dini dan pemantauan kesehatan mental secara nonintrusif, serta menjadi alternatif sekaligus pelengkap bagi pendekatan dari sisi klinis.
====================================================================================================================================
Depression has become a pressing global health challenge, highlighting the need for reliable and explainable computational approaches for early detection and monitoring. With the ubiquity of online platforms, particularly Online Social Networks (OSNs), social media provides a rich data source for capturing users’ naturalistic expressions of thoughts, emotions, and behaviors. This study introduces an explainable and personalized depression detection framework that integrates user personality traits with multimodal deep learning. The proposed approach leverages heterogeneous OSN data, including textual content, visual content, and sentiment polarity patterns, while employing transformer-based language models, ensemble learning strategies, and adaptive gated fusion for cross-modal representation learning. Personality traits are automatically inferred from user-generated content and incorporated into the detection pipeline, enabling personalized modeling of individual emotional and cognitive tendencies. This integration was justified by established psychological research linking personality dimensions with vulnerability to depression, thereby allowing the model to capture user-specific affective and behavioral patterns that traditional text or image-only methods cannot fully represent. Experimental findings reveal that incorporating personality traits significantly improves detection accuracy. Notably, even compact transformer-based models such as Robustly Optimized BERT Pretraining Approach (RoBERTa) method, when enriched with personality features, surpassed larger large language models including LLaMA, Mistral, and GPT-3.5. Further improvements were achieved through the integration of domain-specific models such as MentalRoBERTa, sentiment polarity patterns, and visual modalities, demonstrating the value of modality-aware and domain-adaptive strategies. Ablation studies confirmed the complementary role of textual, visual, and personality-derived features, with cross-modal attention and ensemble learning proving effective for knowledge integration. The proposed multimodal personalized framework consistently outperformed unimodal baselines and conventional fusion methods, in terms of both Accuracy and F1 Score. Overall, this study underscores the potential of combining multimodal deep learning with personality-aware modeling to advance automated mental health detection, particularly for depression. The proposed framework not only enhances detection performance but also provides more comprehensive interpretability through post-hoc explainability methods. The findings are expected to support early and nonintrusive mental health detection and monitoring, offering an alternative and complementary approach to clinical practices

Item Type:	Thesis (Doctoral)
Uncontrolled Keywords:	adaptive gated fusion, ciri kepribadian, cross-modal fusion, deteksi depresi, ensemble learning, multimodal deep learning, online social networks, depression detection, personality traits
Subjects:	T Technology > T Technology (General) T Technology > T Technology (General) > T174 Technological forecasting T Technology > T Technology (General) > T57.5 Data Processing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Electrical Engineering > 20001-(S3) PhD Thesis
Depositing User:	Gede Aditra Pradnyana
Date Deposited:	16 Dec 2025 07:01
Last Modified:	16 Dec 2025 07:01
URI:	http://repository.its.ac.id/id/eprint/128991

Actions (login required)

View Item