Sistem Pengenalan Aktivitas Manusia Berbasis FMCW Radar Dan Large Language Model Pada Mobile Robot

Ardi, Riyu Zaki Rizqulloh Putra (2026) Sistem Pengenalan Aktivitas Manusia Berbasis FMCW Radar Dan Large Language Model Pada Mobile Robot. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5023221047-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (7MB) | Request a copy

Abstract

Peningkatan populasi lansia secara global mendorong kebutuhan akan sistem pemantauan aktivitas yang non-invasif dan menjaga privasi. Metode berbasis kamera memiliki keterbatasan pada kondisi pencahayaan dan isu privasi, sedangkan sensor wearable dapat mengurangi kenyamanan penggunaan jangka panjang. Penelitian ini mengembangkan sistem pengenalan aktivitas manusia berbasis radar Frequency modulated continuous wave (FMCW) yang diintegrasikan dengan Large Language Model (LLM) pada platform mobile robot sebagai solusi pemantauan non-kontak. Sensor IWR6843ISK-ODS digunakan untuk memperoleh sinyal pantulan yang diproses melalui Range-fft, Doppler-FFT, MTI, CFAR, dan estimasi Angle of Arrival untuk menghasilkan representasi 3d radar map. Fitur statistik yang diekstraksi dari representasi tersebut digunakan sebagai masukan algoritma Random forest untuk mengklasifikasikan empat aktivitas, yaitu berjalan, duduk, berdiri, dan jatuh. Secara paralel, model LLM gemma2:2b yang dijalankan secara lokal pada Nvidia Jetson Orin NX melalui Ollama menginterpretasikan data tanda vital berupa detak jantung dan laju pernapasan menjadi deskripsi kondisi fisiologis berbasis bahasa alami. Hasil pengujian menunjukkan radar mampu mengukur jarak dengan error 0,25%–13,4% dan sudut dengan rata-rata error 0,449%, serta tetap mendeteksi subjek melalui material non-konduktif. Algoritma Random forest memperoleh akurasi 0,926 dan F1-macro 0,922 pada evaluasi GroupKFold 5-fold terhadap 176 rekaman, sedangkan modul LLM berhasil menginterpretasikan seluruh skenario kondisi fisiologis (normal, abnormal, dan kritis) yang diuji sesuai dengan kategori rule-based yang ditetapkan, dengan latensi respons rata-rata 3,25 detik. ================================================================================================================================
The increasing global elderly population has created a growing demand for activity monitoring systems that are non-invasive and preserve user privacy. Camera-based methods are limited by lighting conditions and privacy concerns, while wearable sensors may reduce user comfort during long-term use. This study develops a human activity recognition system based on Frequency modulated continuous wave (FMCW) radar integrated with a Large Language Model (LLM) on a mobile robot platform as a non-contact monitoring solution. The IWR6843ISK-ODS sensor was used to acquire reflected signals, which were processed through Range-fft, Doppler-FFT, MTI, CFAR, and Angle of Arrival estimation to generate a 3d radar map representation. Statistical features extracted from this representation were used as input to a Random forest algorithm to classify four activities: walking, sitting, standing, and falling. In parallel, the gemma2:2b LLM model running locally on Nvidia Jetson Orin NX through Ollama interpreted vital sign data, including heart rate and respiratory rate, into natural language descriptions of physiological conditions. Experimental results showed that the radar achieved distance measurement errors ranging from 0.25% to 13.4% and an average angular error of 0.449%, while maintaining detection capability through non-conductive materials. The Random forest algorithm achieved an accuracy of 0.926 and an F1-macro score of 0.922 using GroupKFold 5-fold evaluation on 176 recordings, while the LLM module successfully interpreted all tested physiological condition scenarios (normal, abnormal, and critical) in accordance with the predefined rule-based categories, with an average response latency of 3.25 seconds.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	FMCW Radar, Pengenalan Aktivitas Manusia, 3d radar map, Large Language Model, Mobile robot ; FMCW Radar, Human Activity Recognition, 3d radar map, Large Language Model, Mobile robot
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK5102.9 Signal processing.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Biomedical Engineering > 11410-(S1) Undergraduate Thesis
Depositing User:	Riyu Zaki Rizqulloh Putra Ardi
Date Deposited:	31 Jul 2026 07:17
Last Modified:	31 Jul 2026 07:17
URI:	http://repository.its.ac.id/id/eprint/140605

Actions (login required)

View Item