Rafiqan, Ahmad (2025) Sistem Deteksi Stres Multimodal Menggunakan Radar 24GHz, Analisis Suara, Dan Ekspresi Wajah. Masters thesis, Institut Teknologi Sepuluh Nopember.
![[thumbnail of 6022232019-Master_Thesis.pdf]](http://repository.its.ac.id/style/images/fileicons/text.png) |
Text
6022232019-Master_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (7MB) | Request a copy |
Abstract
Stres telah menjadi masalah kesehatan yang signifikan dalam masyarakat modern, dengan 77% penduduk mengalami gangguan fisik dan 73% mengalami gangguan psikologis terkait stres. Penelitian ini mengusulkan sistem deteksi stres multimodal non-contact yang mengintegrasikan tiga parameter: vital sign dari radar Continuous Wave 24GHz, analisis suara, dan ekspresi wajah melalui extended 41-minute Trier Social Stress Test (TSST) protocol. Sistem menggunakan sensor radar K-MC1 untuk mendeteksi pergerakan mikro dari dada, Raspberry Pi Camera Module V3 untuk analisis ekspresi wajah menggunakan MediaPipe Face Mesh dengan 468 facial landmarks, dan Rexus Microphone Snare CM10 untuk analisis pola suara. Implementasi pada Raspberry Pi Compute Module 4 multicore memungkinkan akuisisi data tersinkronisasi dan pengolahan optimal. Pengolahan data meliputi Bandpass Filter untuk ekstraksi sinyal vital dari radar, speaking-aware video processing yang adaptif memilih fitur wajah (Eye Aspect Ratio, Mouth Aspect Ratio, Face Movement) berdasarkan deteksi bicara untuk meningkatkan performa 4.24%, dan analisis 15-dimensional audio feature vector termasuk MFCC coefficients, Zero Crossing Rate, Spectral Centroid, dan fitur prosodik. Metodologi late fusion menggabungkan klasifikasi LightGBM untuk audio, XGBoost untuk video, dan XGBoost untuk radar menggunakan stacked generalization. Validasi eksperimental dengan 71 partisipan menunjukkan: modalitas audio 89.74% F1-score, modalitas video 83.16%, modalitas radar (71%), dan fusion multimodal (94%) dengan generalisasi cross-participant yang robust. Sistem ini memberikan deteksi stres objektif untuk membantu praktisi kesehatan mental dalam asesmen stres.
===================================================================================================================================
Stress has become a significant health issue in modern society, with 77% of the population experiencing physical disorders and 73% experiencing psychological disorders related to stress. This research proposes a non-contact multimodal stress detection system integrating three parameters: vital signs from 24GHz Continuous Wave radar, audio analysis, and facial expressions through an extended 41-minute Trier Social Stress Test (TSST) protocol. The system employs K-MC1 radar sensor for detecting micro-movements from the chest, Raspberry Pi Camera Module V3 for facial expression analysis using MediaPipe Face Mesh with 468 facial landmarks, and Rexus Microphone Snare CM10 for audio pattern analysis. Implementation on Raspberry Pi Compute Module 4 multicore enables synchronized data acquisition and optimal processing. Data processing includes Bandpass Filter for vital sign extraction from radar, speaking-aware video processing that adaptively selects facial features (Eye Aspect Ratio, Mouth Aspect Ratio, Face Movement) achieving 4.24% performance improvement, and analysis of 15-dimensional audio feature vector including MFCC coefficients, Zero Crossing Rate, Spectral Centroid, and prosodic features. Late fusion methodology combines optimized LightGBM classification for audio, XGBoost for video, and XGBoost for radar using stacked generalization. Experimental validation with 71 participants demonstrates: audio modality 89.74% F1-score, video modality 83.16%, radar modality (71%), and multimodal fusion (94%) with robust cross-participant generalization. This system provides objective stress detection to assist mental health practitioners in stress assessment.
Actions (login required)
 |
View Item |