Prediksi Kepribadian Berbasis Gambar Wajah Dengan Arsitektur Dual-Branch Adaptive Distribution Fusion Dan Integrasi Fitur Emosi

Wardana, Aryan Shafa (2025) Prediksi Kepribadian Berbasis Gambar Wajah Dengan Arsitektur Dual-Branch Adaptive Distribution Fusion Dan Integrasi Fitur Emosi. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025211031-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (17MB) | Request a copy

Abstract

Prediksi kepribadian berbasis gambar wajah menawarkan alternatif terhadap metode tradisional yang rentan bias. Tugas akhir ini mengusulkan arsitektur dual-branch Adaptive Distribution Fusion (Ada-DF) untuk memprediksi lima dimensi kepribadian Big Five, yaitu extraversion (ekstraversi), neuroticism (neurotisisme), agreeableness (kesesuaian), conscientiousness (ketaatan), dan openness (keterbukaan). Model dikembangkan dalam dua skenario, yaitu tanpa dan dengan integrasi fitur emosi untuk memperkaya representasi data. Dataset Chalearn First Impressions digunakan, berisi gambar wajah yang dilabeli skor dimensi kepribadian Big Five. Proses meliputi persiapan data, ekstraksi frame, pemilihan gambar wajah, ekstraksi emosi, praproses, pengembangan dan pelatihan model, serta evaluasi model klasifikasi multi-output multikelas. Evaluasi dilakukan menggunakan top-1 accuracy, precision, recall, dan F1-score untuk menilai ketepatan dan keseimbangan klasifikasi. Model dasar menghasilkan rata-rata top-1 accuracy 0,3995, precision 0,3666, recall 0,3985, dan F1-score 0,3614, serta lebih akurat untuk kelas sangat tinggi dan sangat rendah. Setelah integrasi fitur emosi, kinerja meningkat menjadi top-1 accuracy 0,4058, precision 0,3870, recall 0,4040, dan F1-score 0,3863, serta memperbaiki akurasi klasifikasi pada kelas tinggi dan rendah. Tugas akhir ini berkontribusi melalui integrasi fitur emosi ke dalam arsitektur Ada-DF, yang terbukti memperkaya representasi data dan meningkatkan kinerja identifikasi kepribadian. Diharapkan temuan ini dapat memperluas pengembangan metode identifikasi kepribadian yang lebih objektif berbasis kecerdasan artifisial.
===================================================================================================================================
Personality prediction based on facial images offers an alternative to traditional methods such as questionnaires and interviews, which are prone to bias. This thesis proposes a dual-branch Adaptive Distribution Fusion (Ada-DF) architecture to predict the five dimensions of the Big Five personality traits, namely extraversion, neuroticism, agreeableness, conscientiousness, and openness. The model is developed in two scenarios: without and with the integration of emotional features to enrich data representation. The ChaLearn First Impressions dataset is used, consisting of facial images annotated with continuous personality scores. The process involves data preparation, frame extraction, face selection, emotion extraction, preprocessing, model development and training, as well as evaluation of a multi-output multi-class classification model. Evaluation uses top-1 accuracy, precision, recall, and F1-score to assess classification accuracy and balance. The baseline model achieves an average top-1 accuracy of 0.3995, precision of 0.3666, recall of 0.3985, and F1-score of 0.3614, performing better on extreme personality classes. After integrating emotional features, the performance improves to a top-1 accuracy of 0.4058, precision of 0.3870, recall of 0.4040, and F1-score of 0.3863, with better classification on mid-range classes. This thesis contributes by integrating emotional features into the Ada-DF architecture, which proves to enrich data representation and improve personality identification performance. The findings are expected to broaden the development of more objective personality identification methods based on artificial intelligence.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	big five, dual-branch adaptive distribution fusion, fitur emosi, gambar wajah, klasifikasi multi-output multikelas, prediksi kepribadian, big five, dual-branch adaptive distribution fusion, emotion features, face image, multi-output multi-class classification, personality prediction
Subjects:	T Technology > T Technology (General) > T57.5 Data Processing T Technology > T Technology (General) > T59.7 Human-machine systems.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Aryan Shafa Wardana
Date Deposited:	25 Jul 2025 03:58
Last Modified:	30 Jul 2025 07:19
URI:	http://repository.its.ac.id/id/eprint/121084

Actions (login required)

View Item