Kontrol Permainan Komputer Berbasis Hand Pose Menggunakan Machine Learning

Gabriel, Joseph Saido (2023) Kontrol Permainan Komputer Berbasis Hand Pose Menggunakan Machine Learning. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 07211940000061-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only until 1 October 2025. Download (10MB) | Request a copy

Abstract

Industri permainan komputer berkembang dengan cepat belakangan ini, dan hal ini juga diiringi dengan cepatnya perkembangan teknologi, namun, perangkat yang digunakan untuk bermain suatu permainan komputer, seperti misalnya mouse, keyboard, dan joystick, masih kurang interaktif dan natural, karena gerakan tidak sama dengan apa yang terjadi pada permainan. Teknologi gesture recognition atau motion capture yang memungkinkan untuk hal itu, masih kurang umum, dan tidak banyak digunakan. Dibuatlah suatu sistem untuk menangkap gerakan pemain melalui webcam, atau kamera pada laptop, menggunakan estimasi pose dan pembelajaran mesin. Dilakukan uji coba menggunakan beberapa model termasuk transfer learning. Ditemukan model paling akurat merupakan model EfficientNetB0 yang di-training menggunakan dataset citra asli tangan dengan rata-rata tingkat akurasi 98%. Model yang memiliki performa paling cepat adalah model single layer perceptron yang menghasilkan rata-rata 20 FPS ketika menggunakan sistem. Dilakukan juga pengujian untuk beberapa webcam, dan ditemukan bahwa webcam dengan resolusi yang tinggi memiliki performa yang lebih baik. Sistem yang telah dibuat ini kemudian diuji coba kepada pengguna. Respons yang didapatkan dari pengguna bersifat positif, dan dapat menggunakan sistem dengan baik.
=================================================================================================================================
The video game industry is rapidly growing in recent times. And this growth is accompanied by the growth of technology. But, the devices used for controlling video games, such as keyboard, mouse, and joystick, have not changed in a long time. In terms of how natural they feel, it seems that they are lacking. Because they dont give the players the ability to do the actions that are similar to what is happening inside the video game. And the technology to make that happen, that is, motion capture and gesture recognition is still not very common and not widely used. Therefore, in this research, a system to capture the player’s hand pose through webcam is made. Several tests were conducted to find the best performing machine learning models and a few webcam types are also tested. It is found that through transfer learning, EfficientNetB0 model that is trained with real hands dataset has an accuracy of 98%. A webcam that has a high resolution performs much faster and is much accurate. Surveys were also conducted to evaluate the user experience when using the system. Responses from the survei shows that the user understands well how to use the system, able to achieve the intended goal, and while doing so, receive a positive emotional stimulation(e.g. fun, interesting).

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Gestur tangan, visi komputer, pembelajaran mesin, interaksi komputer, hand gesture, computer vision, machine learning, human-computer interaction
Subjects:	Q Science > Q Science (General) > Q325.5 Machine learning.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Computer Engineering > 90243-(S1) Undergraduate Thesis
Depositing User:	Joseph Saido Gabriel
Date Deposited:	10 Oct 2023 04:06
Last Modified:	10 Oct 2023 04:06
URI:	http://repository.its.ac.id/id/eprint/101003

Actions (login required)

View Item