Pengenalan Kata Isyarat dalam Bahasa Indonesia Sesuai Aturan SIBI Menggunakan Convolutional Neural Network dan Long Short-Term Memory

Yosyam, Naufal Ariq Putra (2024) Pengenalan Kata Isyarat dalam Bahasa Indonesia Sesuai Aturan SIBI Menggunakan Convolutional Neural Network dan Long Short-Term Memory. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025201112_Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only until 1 October 2026. Download (10MB) | Request a copy

Abstract

Bahasa isyarat adalah bentuk komunikasi yang menggunakan gerakan tangan, mimik wajah, dan postur badan yang sering digunakan oleh penyandang tuna rungu untuk dapat berkomunikasi dan memperoleh informasi. Di Indonesia sendiri bahasa isyarat yang diakui secara resmi oleh pemerintah adalah Sistem Isyarat Bahasa Indonesia (SIBI). Walaupun sudah diakui secara resmi oleh pemerintah, SIBI sendiri masih kurang dikenali dan diketahui maknanya oleh khalayak umum. Oleh karena itu, akan dikembangkan pengenalan kata isyarat sesuai aturan Sistem Isyarat Bahasa Indonesia untuk memudahkan komunikasi antara para peyandang tuna-wicara dan non tuna-wicara. Penelitian ini akan mengadopsi metode Convolutional Neural Network (CNN) dan Long Short-Term Memory (LSTM). Penelitian akan dimulai dengan membagi-bagi dataset video yang ada menjadi potongan 30 frame dan 60 frame per video. Lalu, dilakukan preprocessing gambar yang diawali dengan pengenalan fitur manusia di dalam gambar frame yang telah didapatkan melalui proses human detection menggunakan YOLOv7. Setelah itu, dilanjutkan dengan pengambilan fitur tensor pada gambar menggunakan CNN. Kemudian, menggunakan LSTM potongan gambar tadi dijadikan sebuah rangkaian agar dapat dikenali sebagai satu kelas kata. Hyperparameter tuning juga dilakukan untuk mencari hyperparameter terbaik sebagai penyusun arsitektur model. Pada percobaan ini didapatkan hasil terbaik dari skenario penggunaan ekstraksi 30 frame per video yang dilakukan human detection menggunakan YOLOv7 dengan hasil adalah 64.1% akurasi, 64.1% presisi, recall 64.1%, dan nilai F1-score 61.2%.
=========================================================================
Sign language is a form of communication that uses hand gestures, facial expressions, and posture that is often used by deaf people to communicate and obtain information. In Indonesia, the sign language officially recognized by the government is the Indonesian Sign Language System (SIBI). Although it has been officially recognized by the government, SIBI itself is still not recognized and known by the general public. Therefore, the introduction of sign words according to the rules of the Indonesian Sign Language System will be developed to facilitate communication between speech-impaired and non-speech-impaired people. This research will adopt Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) methods. The research will start by splitting the existing video dataset into 30 frames and 60 frames per video. Then, image preprocessing is carried out starting with the recognition of human features in the image frames that have been obtained through the human detection process using YOLOv7. After that, proceed with taking tensor features in the image using CNN. Then, using LSTM, the image pieces are made into a series so that they can be recognized as a word class. Hyperparameter tuning is also done to find the best hyperparameter as a constituent of the model architecture. In this experiment, the best results were obtained from the scenario of using 30 frames of extraction per video with human detection using YOLOv7 with the results being 64.1% accuracy, 64.1% precision, 64.1% recall, and 61.2% F1-score value.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Sign Language, CNN, LSTM, Human Detection, YOLOv7, Image Recognition,Bahasa Isyarat, Deteksi Manusia, Pengenalan Citra
Subjects:	T Technology > T Technology (General) T Technology > T Technology (General) > T57.5 Data Processing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Naufal Ariq Putra Yosyam
Date Deposited:	02 Aug 2024 08:38
Last Modified:	02 Aug 2024 08:38
URI:	http://repository.its.ac.id/id/eprint/111502

Actions (login required)

View Item