Ardhana, Rafif Dhimaz (2026) Implementasi AI Agent Berbasis OpenClaw dengan Antarmuka Suara Melalui Perangkat IoT dan Local LLM via Ollama untuk Pengendalian Smart Classroom. Other thesis, Institut Teknologi Sepuluh Nopember.
![[thumbnail of 5027221066-Undergraduate_Thesis.pdf]](http://repository.its.ac.id/style/images/fileicons/text.png) |
Text
5027221066-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (2MB) | Request a copy |
Abstract
Perkembangan Large Language Model (LLM) dan paradigma AI Agent membuka peluang baru dalam pengembangan sistem otomasi yang lebih cerdas dan intuitif. Penelitian ini mengimplementasikan AI Agent berbasis OpenClaw dengan antarmuka suara (HeyClawy pada ESP32-S3) dan Local LLM melalui Ollama untuk mengendalikan sistem Smart Classroom di Departemen Teknologi Informasi ITS. Berbeda dari pendekatan terdahulu yang mengandalkan Assist Pipeline Home Assistant dengan NLP berbasis pencocokan pola rigid, penelitian ini menempatkan OpenClaw sebagai AI Agent orkestrasi yang mampu melakukan reasoning multi-langkah, memanggil tools secara dinamis, serta memahami perintah suara dalam bahasa alami. Seluruh inferensi LLM, STT (faster-whisper), dan TTS (EdgeTTS) berjalan lokal tanpa layanan cloud, menjamin privasi data dan ketersediaan sistem yang tidak bergantung koneksi internet. Hasil pengujian menunjukkan sistem mampu memahami perintah suara dalam bahasa alami yang bervariasi, mengontrol perangkat ruang kelas melalui Home Assistant, serta memberikan respons verbal yang kontekstual, dengan akurasi pengenalan niat 95%, Task Completion Rate rata-rata 90%, latensi end-to-end rata-rata 20,7 detik (tanpa tool call) dan 32,75 detik (dengan tool call), serta ketahanan penuh terhadap seluruh skenario pengujian keamanan representatif.
===========================================================================================================================================
The rapid advancement of Large Language Models (LLMs) and the AI Agent paradigm opens new opportunities for developing smarter and more intuitive automation systems. This research implements an OpenClaw-based AI Agent with a voice interface (HeyClawy on ESP32-S3) and a Local LLM via Ollama to control a Smart Classroom system in the Department of Information Technology, ITS. Unlike previous approaches relying on Home Assistant's built-in Assist Pipeline with rigid pattern-matching NLP, this research positions OpenClaw as the orchestrating AI Agent capable of multi-step reasoning, dynamic tool invocation, and natural language voice command understanding. All LLM inference, STT (faster-whisper), and TTS (EdgeTTS) run locally with no cloud dependencies, ensuring data privacy and internet-independent system availability. Testing results show the system accurately understands varied natural language voice commands, controls classroom devices through Home Assistant, and delivers contextual verbal responses, achieving 95% intent recognition accuracy, an average Task Completion Rate of 90%, an average end-to-end latency of 20.7 seconds (without tool calls) and 32.75 seconds (with tool calls), and full resilience across all representative security test scenarios.
| Item Type: | Thesis (Other) |
|---|---|
| Uncontrolled Keywords: | AI Agent, OpenClaw, HeyClawy, ESP32-S3, Voice Assistant, Ollama, Local LLM, Home Assistant, Smart Classroom, WakeNet, faster-whisper, EdgeTTS |
| Subjects: | T Technology > T Technology (General) > T58.8 Productivity. Efficiency |
| Divisions: | Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information Technology > 59201-(S1) Undergraduate Thesis |
| Depositing User: | Rafif Dhimaz Ardhana |
| Date Deposited: | 04 Aug 2026 01:29 |
| Last Modified: | 04 Aug 2026 01:29 |
| URI: | http://repository.its.ac.id/id/eprint/141805 |
Actions (login required)
 |
View Item |