Implementasi Deep Reinforcement Learning pada Hexagonal Grid Turn-Based Strategy Game

Asqav, Dafa Fidini (2023) Implementasi Deep Reinforcement Learning pada Hexagonal Grid Turn-Based Strategy Game. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 07211840000035_Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only until 1 October 2025. Download (7MB) | Request a copy

Abstract

Game strategi merupakan permainan di mana pemain mengambil keputusan strategis di dalam game untuk menyelesaikan tujuan. Salah satu game tersebut merupakan Civilization VI (Civ6). Dalam game ini, pemain melakukan aksi bergilir dengan lawannya dalam area map yang tersusun dari lantai segi enam (hexagonal grid). Civ6 merupakan game dengan aspek 4X (Exploration, exploitation, expansion, extermination). Aspek-aspek 4X tersebut menyebabkan game ini menjadi kompleks. Hal ini menjadi tantangan bagi game developer untuk membuat lawan AI yang dapat memberikan tantangan yang cukup terhadap pemain. Akan tetapi, game strategi seperti Civ6 masih memiliki kapabilitas agen berbasis AI yang belum optimal. Berkembangnya bidang Deep Reinforcement Learning (DRL) menawarkan teknologi AI yang belum memungkinkan sebelumnya. Dalam penelitian ini, dirancang sebuah environment yang mengikuti mekanisme combat dalam Civ6 sebagai media implementasi DRL. Terdapat dua agen dalam environment ini: agen attacker dan defender. Kedua agen memiliki tujuan yang berbeda (asymmetrical) dan saling berlawanan (adversarial). Terdapat empat algoritma state of the art (SOTA) yang digunakan dalam eksperimen penelitian ini: Deep Q-Learning (DQN), Distributed Prioritized Experience Replay Deep Q-Networks (APE-X DQN), Proximal Policy Optimization (PPO), and Importance Weighted Actor-Learner Architecture (IMPALA). Dari hasil experimen, didapatkan bahwa APE-X DQN memiliki performa terbaik bagi agen attacker dan agen defender. Agen attacker APE-X DQN mampu menghancurkan kota secara konsisten sebelum 2 juta environment steps saat training. APE-X DQN juga merupakan algoritma yang memiliki performa paling baik saat evaluasi. Akan tetapi, beberapa generasi APE-X DQN tidak dapat dievaluasi pada evaluasi ke dua (Skenario environment 16x16). APE-X DQN menggunakan CPU dan RAM lebih banyak dari algoritma lain, dengan penggunaan CPU sebanyak 79.95% dan RAM sebanyak 82.3%.
================================================================================================================================
Strategy games are games where the player takes strategic decision in the game to finish an objective. One of these games is Civilization 6 (Civ6). In this game, the players
take turn in action in a map area made of hexagonal tiles (hexagonal grid). Civ6 is also a 4X game. These 4X aspects make the game rather complicated. This has become a
challenge for game developers to create AI opponents that are capable to give enough challenge for the players. However, strategy games like Civ6 are still having less than
optimal AI agent capability. The advancement in Deep Reinforcement Learning (DRL) allows AI technology that was not possible before. In this research, an environment that
follows the combat mechanics in Civ6 is devised as the DRL implementation media. There are two agents in this environment: attacker and defender. Both agent has differing
objectives (asymmetrical) and oppose each other (adversarial). There are four state of the art algorithms used in this research experiment: Deep Q-Learning (DQN), Distributed Prioritized Experience Replay Deep Q-Networks (APE-X DQN), Proximal Policy Optimization (PPO), and Importance Weighted Actor-Learner Architecture (IMPALA). From the experiment result, APE-X DQN performed the best for both attacker agent and defender agent. Attacker agent with APE-X DQN was able to consistently destroying the city before 2 million environment steps during training. During inter generation evaluation, APE-X DQN was also the best performing algorithm. However, some of the APE-X DQN generations failed to be evaluated on second evaluation stage (16x16 environment scenario). APE-X DQN required higher CPU and RAM utilization than other algorithms, with 79.95% CPU utilization and 82.3% RAM utilization.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Game strategi, Deep Reinforcement Learning, Artificial Intelligence, Machine Learning, Civilization VI, DQN, APE-X, PPO, IMPALA, Strategy games
Subjects:	G Geography. Anthropology. Recreation > GV Recreation Leisure > GV1469.2 Computer games Q Science > Q Science (General) > Q325.5 Machine learning. Q Science > QA Mathematics > QA336 Artificial Intelligence Q Science > QA Mathematics > QA76.87 Neural networks (Computer Science) T Technology > T Technology (General) > T57.62 Simulation
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Computer Engineering > 90243-(S1) Undergraduate Thesis
Depositing User:	Dafa Fidini Asqav
Date Deposited:	03 Jul 2023 02:28
Last Modified:	03 Jul 2023 02:28
URI:	http://repository.its.ac.id/id/eprint/98254

Actions (login required)

View Item