Prediksi Big Five Personality pada Video Monolog Singkat Berbasis Video Transformer

Murdianto, Adyuta Prajahita (2026) Prediksi Big Five Personality pada Video Monolog Singkat Berbasis Video Transformer. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025221186-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (30MB) | Request a copy

Abstract

Perkembangan teknologi digital mendorong pemanfaatan kecerdasan buatan untuk menganalisis data tidak terstruktur, termasuk video, dalam memahami karakteristik manusia. Salah satu karakteristik yang dapat dianalisis adalah kepribadian, yang pada penelitian ini direpresentasikan melalui lima dimensi Big Five Personality. Video memiliki keunggulan dibandingkan citra statis karena memuat perubahan ekspresi, arah pandang, dan gerakan kepala yang merepresentasikan informasi spasial dan temporal. Penelitian ini mengimplementasikan tiga model Video Transformer, yaitu ViViT, TimeSformer, dan VideoMAE, untuk memprediksi lima trait Big Five Personality pada dataset ChaLearn First Impressions yang berisi 10.000 video monolog singkat. Tahapan penelitian meliputi ekstraksi frame, seleksi kualitas frame, segmentasi wajah, pembuatan dataset tersegmentasi bersih, sampling frame, praproses input, dan pelatihan model. Setiap model menggunakan backbone pralatih Kinetics-400 dan diuji melalui dua skenario, yaitu frozen dan fine-tune, pada dataset tersegmentasi dan tersegmentasi bersih. Evaluasi dilakukan menggunakan MAE, RMSE, R², dan akurasi berbasis 1−MAE. Hasil penelitian menunjukkan bahwa skenario fine-tune memberikan performa lebih baik dibandingkan frozen. Performa terbaik diperoleh oleh ViViT fine-tune pada dataset tersegmentasi dengan MAE 0,0856, RMSE 0,1075, R² 0,4576, dan akurasi 0,9144 pada data uji. Perbandingan hasil evaluasi menunjukkan bahwa fine-tune dapat meningkatkan performa seluruh model, dataset tersegmentasi lebih unggul pada skenario akhir, dan ViViT menjadi model dengan performa paling konsisten.
=====================================================================================================================================
The development of digital technology has encouraged the use of artificial intelligence to analyze unstructured data, including video, to understand human characteristics. One characteristic that can be analyzed is personality, which in this study is represented through the five dimensions of the Big Five Personality. Compared with static images, video provides changes in facial expressions, gaze direction, and head movements that represent spatial and temporal information. This study implements three Video Transformer models, namely ViViT, TimeSformer, and VideoMAE, to predict the five Big Five Personality traits using the ChaLearn First Impressions dataset, which consists of 10,000 short monologue videos. The research stages include frame extraction, frame quality selection, face segmentation, construction of a cleaned segmented dataset, frame sampling, input preprocessing, and model training. Each model uses a Kinetics-400 pretrained backbone and is evaluated through two scenarios, namely frozen and fine-tune, on segmented and cleaned segmented datasets. Evaluation is conducted using MAE, RMSE, R², and accuracy based on 1−MAE. The results show that the fine-tune scenario performs better than the frozen scenario. The best performance is achieved by ViViT fine-tune on the segmented dataset, with MAE of 0.0856, RMSE of 0.1075, R² of 0.4576, and accuracy of 0.9144 on the test data. The evaluation results indicate that fine-tune improves the performance of all models, the segmented dataset performs better in the final scenario, and ViViT is the most consistent model.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Prediksi Kepribadian, Big Five Personality, Video Transformer, Video Vision Transformer, TimeSformer, VideoMAE, Personality Prediction, Big Five Personality, Video Transformer, Video Vision Transformer, TimeSformer, VideoMAE
Subjects:	T Technology > T Technology (General) T Technology > T Technology (General) > T57.5 Data Processing T Technology > T Technology (General) > T58.5 Information technology. IT--Auditing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Adyuta Prajahita Murdianto
Date Deposited:	28 Jul 2026 02:21
Last Modified:	28 Jul 2026 02:21
URI:	http://repository.its.ac.id/id/eprint/138093

Actions (login required)

View Item