Smart Glove Menggunakan Sensor Flex, FSR, Dan IMU Untuk Pengenalan Gestur Dinamis Sistem Isyarat Bahasa Indonesia Berbasis Deep Learning

Wulandari, Aryati Dwi (2026) Smart Glove Menggunakan Sensor Flex, FSR, Dan IMU Untuk Pengenalan Gestur Dinamis Sistem Isyarat Bahasa Indonesia Berbasis Deep Learning. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5023221042-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (6MB) | Request a copy

Abstract

Komunikasi antara penyandang tunarungu-wicara dan masyarakat umum sering kali terhambat karena tidak semua orang memahami bahasa isyarat. Berbagai penelitian telah mengembangkan teknologi smart glove untuk menerjemahkan Sistem Isyarat Bahasa Indonesia (SIBI), namun sebagian besar masih berfokus pada pengenalan huruf alfabet yang didominasi oleh gestur statis. Penelitian ini bertujuan mengembangkan smart glove berbasis sensor flex, Force Sensitive Resistor (FSR), dan Inertial Measurement Unit (IMU) untuk mengenali kosakata dinamis SIBI secara langsung menggunakan arsitektur CNN-Transformer. Dataset dikumpulkan dari 21 subjek yang melakukan 26 huruf alfabet dan lima kosakata dinamis. Evaluasi metode pembagian dataset menunjukkan bahwa repetition split menghasilkan performa terbaik dengan akurasi sebesar 97.08%, lebih tinggi dibandingkan random split (96.87%) dan cross-subject split (84.98%). Selain itu, evaluasi pengaruh jumlah sensor IMU menunjukkan bahwa penggunaan tiga sensor IMU mampu merepresentasikan karakteristik gestur dinamis dengan lebih baik dibandingkan konfigurasi satu maupun dua sensor. Model yang diimplementasikan pada sistem memperoleh accuracy, precision, recall, dan F1-score masing-masing sebesar 96.34%, 96.43%, 96.34%, dan 96.33% untuk pengenalan huruf, serta 100% untuk seluruh metrik pada pengenalan kosakata. Pada pengujian sistem secara real-time terhadap dua subjek, sistem memperoleh rata-rata akurasi sebesar 96.15% untuk pengenalan huruf, 100% untuk pengenalan kosakata dinamis dengan rata-rata waktu klasifikasi sebesar 2.05 detik, serta 90.77% pada pengujian protokol kata dengan rata-rata waktu penyusunan kata sebesar 15.80 detik. Hasil penelitian menunjukkan bahwa pendekatan pengenalan kosakata dinamis secara langsung dapat diimplementasikan pada smart glove dengan performa yang baik serta berpotensi meningkatkan efisiensi komunikasi dibandingkan pendekatan penyusunan kata melalui rangkaian huruf. ======================================================================================================================================
Communication between individuals with hearing and speech impairments and the general public is often hindered because not everyone understands sign language. Various studies have developed smart glove technology to translate the Indonesian Sign Language System (SIBI). However, most have focused on recognizing alphabet letters, which are predominantly static gestures. This study aims to develop a smart glove based on flex sensors, Force Sensitive Resistors (FSRs), and Inertial Measurement Units (IMUs) to recognize dynamic SIBI vocabulary directly using a CNN-Transformer architecture. The dataset was collected from 21 participants performing 26 alphabet letters and five dynamic vocabulary gestures. The evaluation of dataset splitting methods showed that the repetition split achieved the best performance, with an accuracy of 97.08%, outperforming the random split (96.87%) and the cross-subject split (84.98%). In addition, the evaluation of the number of IMU sensors demonstrated that the three-IMU configuration was more effective in representing the characteristics of dynamic gestures than configurations using one or two IMUs. The model implemented in the system achieved an accuracy, precision, recall, and F1-score of 96.34%, 96.43%, 96.34%, and 96.33%, respectively, for alphabet letter recognition, while all four metrics reached 100.00% for dynamic vocabulary recognition. In real-time testing involving two participants, the system achieved an average accuracy of 96.15% for alphabet letter recognition, 100.00% for dynamic vocabulary recognition with an average classification time of 2.05 seconds, and 90.77% for the word protocol test with an average word composition time of 15.80 seconds. The results demonstrate that the proposed direct dynamic vocabulary recognition approach can be effectively implemented on a smart glove with high performance and has the potential to improve communication efficiency compared with the conventional letter-by-letter word composition approach.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Bahasa Isyarat, Deep Learning, Gestur Dinamis, Sarung Tangan, Tunarungu-wicara. Deaf-Mute, Deep Learning, Dynamic Gestures, Sign Language, Smart Glove.
Subjects:	R Medicine > R Medicine (General) > R858 Deep Learning
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Biomedical Engineering > 11410-(S1) Undergraduate Thesis
Depositing User:	Aryati Dwi Wulandari
Date Deposited:	01 Aug 2026 07:05
Last Modified:	01 Aug 2026 07:05
URI:	http://repository.its.ac.id/id/eprint/141472

Actions (login required)

View Item