Orkestrasi LLM Terkontainerisasi pada Arsitektur Self-Hosted di Lingkungan Proxmox VE

Renanda, Raditya Rakha (2026) Orkestrasi LLM Terkontainerisasi pada Arsitektur Self-Hosted di Lingkungan Proxmox VE. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5024221031-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (18MB) | Request a copy

Abstract

Penelitian ini merancang dan memvalidasi sebuah blueprint arsitektur orkestrasi untuk layanan Large Language Model (LLM) self-hosted pada infrastruktur consumer-grade, yang mengintegrasikan lapisan virtualisasi (Proxmox VE), kontainerisasi (Docker Compose), dan workflow (n8n) sebagai satu kesatuan sistem yang kohesif. Blueprint ini divalidasi melalui purwarupa SAKAI, sebuah layanan helpdesk akademik berbasis kecerdasan buatan yang diimplementasikan di Departemen Teknik Komputer Institut Teknologi Sepuluh Nopember (ITS), mencakup tiga dimensi tantangan orkestrasi yang saling berkaitan: stabilitas, akurasi, dan ketahanan. Pada dimensi stabilitas, pengujian grid search terhadap konfigurasi daya dan frekuensi GPU NVIDIA RTX 2080 Ti menetapkan parameter optimal pada 1400 MHz dan power limit 120 Watt, menghasilkan throughput sebesar 113 token per second tanpa risiko kegagalan sistem. Pada dimensi akurasi, integrasi pipeline Retrieval-Augmented Generation (RAG) berbasis ChromaDB menetapkan Qwen3-Embedding (0.6B) sebagai model embedding final dengan akurasi retrieval 80%, serta Qwen 3.5 (4B) kuantisasi 4-bit sebagai model LLM final yang konsisten berhasil mengeksekusi tool calling. Pada dimensi ketahanan, mekanisme Global Concurrency Limit dan Rate Limiting berbasis Redis terbukti mempertahankan Success Rate 100% pada beban 15 permintaan konkuren serta mencegah pemborosan sumber daya akibat spamming. Hasil validasi ini menghasilkan sebuah blueprint orkestrasi yang teruji secara empiris dan dapat diadopsi oleh institusi pendidikan lain dengan infrastruktur serupa.
==============================================================================================================================
This research designs and validates an orchestration blueprint for self-hosted Large Language Model (LLM) services on consumer-grade infrastructure, integrating the virtualization (Proxmox VE), containerization (Docker Compose), and workflow (n8n) layers into a single, cohesive system. This blueprint is validated through the SAKAI prototype, an AI-based academic helpdesk service implemented at the Department of Computer Engineering, Institut Teknologi Sepuluh Nopember (ITS), covering three interrelated dimensions of orchestration challenges: stability, accuracy, and resilience. In the stability dimension, grid search testing on the power and clock configuration of the NVIDIA RTX 2080 Ti GPU established an optimal configuration of 1400 MHz and a 120 Watt power limit, yielding a throughput of 113 tokens per second without risk of system failure. In the accuracy dimension, integration of a ChromaDB-based Retrieval-Augmented Generation (RAG) pipeline established Qwen3-Embedding (0.6B) as the final embedding model with 80% retrieval accuracy, and Qwen 3.5 (4B) in its 4-bit quantized variant as the final LLM, which consistently succeeded in executing tool calling. In the resilience dimension, Redis-based Global Concurrency Limit and Rate Limiting mechanisms maintained a 100% success rate under a load of 15 concurrent requests while preventing resource waste caused by spamming. These validation results yield an empirically tested orchestration blueprint that can be adopted by other educational institutions with similar infrastructure.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Orkestrasi Sistem, Self-Hosted, Large Language Model, Retrieval-Augmented Generation, Proxmox VE, Docker. System Orchestration, Self-Hosted, Large Language Model, Retrieval-Augmented Generation, Proxmox VE, Docker.
Subjects:	Q Science > QA Mathematics > QA336 Artificial Intelligence Q Science > QA Mathematics > QA76.585 Cloud computing. Mobile computing. Q Science > QA Mathematics > QA76.754 Software architecture. Computer software T Technology > T Technology (General) > T58.5 Information technology. IT--Auditing Z Bibliography. Library Science. Information Resources > ZA Information resources > Z699.5 Information storage and retrieval systems
Divisions:	Faculty of Electrical Technology > Computer Engineering > 90243-(S1) Undergraduate Thesis
Depositing User:	Raditya Rakha Renanda
Date Deposited:	22 Jul 2026 01:31
Last Modified:	22 Jul 2026 01:31
URI:	http://repository.its.ac.id/id/eprint/136488

Actions (login required)

View Item