Deteksi Kantuk Berdasarkan Analisis Citra Mata Dan Mulut Menggunakan Metode Ensemble Deep Learning

Naveeda, Shazia Ingeyla (2025) Deteksi Kantuk Berdasarkan Analisis Citra Mata Dan Mulut Menggunakan Metode Ensemble Deep Learning. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025211203-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (8MB) | Request a copy

Abstract

Kantuk adalah kondisi saat tingkat kesadaran seseorang mulai menurun akibat kurang tidur atau kelelahan yang dapat menyebabkan kecelakaan fatal dalam berkendara. Terdapat penelitian terdahulu yang telah membuat sistem deteksi kantuk berdasarkan sinyal EEG dan EKG. Namun deteksi kantuk dengan parameter fisiologis tentunya tidak dapat dilakukan secara real-time. Penelitian ini membangun sistem untuk mendeteksi kantuk pengemudi berdasarkan citra pada area mata dan mulut menggunakan pendekatan deep learning dengan ensemble untuk meningkatkan akurasi deteksi. Penelitian ini menggunakan tiga dataset publik yang memiliki karakteristik berbeda yaitu NTHUDDD, YawDD dan DDD. Data citra berupa urutan frame yang diproses menggunakan Multi-Task Cascaded Convolutional Network (MTCNN) untuk mendeteksi wajah dan melakukan ekstraksi fitur mata dan mulut. Hasil ekstraksi mata dan mulut diproses secara terpisah menggunakan tiga arsitektur Convolutional Neural Network (CNN) yaitu InceptionV3, MobileNetV3, dan ResNet50. Fine-tuning dilakukan pada lapisan akhir untuk memungkinkan klasifikasi dua kelas secara spesifik yaitu drowsy dan non-drowsy. Model CNN dilatih secara terpisah untuk fitur mata dan mulut, kemudian prediksi kelas probabilitas dari kedua fitur digabungkan menggunakan metode ensemble stacking dengan single layer perceptron. Evaluasi menunjukkan bahwa fitur mata memberikan performa lebih baik dimana menghasilkan akurasi sebesar 83,75% jika dibandingkan fitur mulut yang mendapatkan akurasi sebesar 66,96%. Namun, penggabungan kedua fitur melalui ensemble stacking menghasilkan peningkatan signifikan dengan akurasi 88,79%, recall 94,67%, precision 85,22%, dan f1-score 89,69%. Hasil ini menunjukkan bahwa kombinasi fitur mata dan mulut dengan pendekatan ensemble mampu meningkatkan akurasi deteksi kantuk secara efektif dibandingkan penggunaan model individual dari fitur mata dan mulut.
===============================================================================================================================
Drowsiness is a condition where a person's level of alertness begins to decline due to lack of sleep or fatigue, which can lead to fatal accidents while driving. Previous studies have developed drowsiness detection systems based on EEG and ECG signals. However, detecting drowsiness using physiological parameters cannot be performed in real-time. This study builds a system to detect driver drowsiness based on eye and mouth region images using a deep learning approach with ensemble methods to improve detection accuracy. The study uses three public datasets with different characteristics: NTHUDDD, YawDD, and DDD. The image data consists of frame sequences processed using the Multi-Task Cascaded Convolutional Network (MTCNN) to detect faces and extract eye and mouth features. The extracted eye and mouth features are processed separately using three Convolutional Neural Network (CNN) architectures: InceptionV3, MobileNetV3, and ResNet50. Fine-tuning is performed on the final layers to enable specific binary classification: drowsy and non-drowsy. The CNN models are trained separately for eye and mouth features, then the probability class predictions from both features are combined using an ensemble stacking method with a single-layer perceptron. Evaluation shows that eye features provide better performance, achieving an accuracy of 83.75%, compared to mouth features, which achieve an accuracy of 66.96%. However, combining both features through ensemble stacking yields a significant improvement with 88.79% accuracy, 94.67% recall, 85.22% precision, and an F1-score of 89.69%. These results indicate that combining eye and mouth features using an ensemble approach can effectively enhance drowsiness detection accuracy compared to using individual models for eye or mouth features alone.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	CNN, deep learning, deteksi kantuk, ensemble stacking, MTCNN, CNN, deep learning, drowsiness detection, ensemble stacking, MTCNN
Subjects:	Q Science > QA Mathematics > QA76.87 Neural networks (Computer Science) R Medicine > R Medicine (General) > R858 Deep Learning
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Shazia Ingeyla Naveeda
Date Deposited:	31 Jul 2025 07:11
Last Modified:	31 Jul 2025 07:11
URI:	http://repository.its.ac.id/id/eprint/124145

Actions (login required)

View Item