Prediksi Beban Kerja Di Ray Cluster Heterogen Melalui Analisis Kode Sumber Dan Dataset Menggunakan LLM

Taufan, Keanu Fortuna (2026) Prediksi Beban Kerja Di Ray Cluster Heterogen Melalui Analisis Kode Sumber Dan Dataset Menggunakan LLM. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025221042_Undergraduate_Thesis.pdf Restricted to Repository staff only Download (9MB) | Request a copy

Abstract

Prediksi beban kerja diperlukan agar kebutuhan sumber daya suatu job dapat diketahui sebelum eksekusi. Kebutuhan ini menonjol pada penjadwalan job di kluster Ray heterogen, yang bergantung pada permintaan sumber daya eksplisit dan heuristik umum tanpa menganalisis struktur kode, sehingga dua job dengan permintaan serupa tetapi pola komputasi berbeda diperlakukan sama. Prediksi beban kerja sebelumnya bertumpu pada runtime logs dan riwayat eksekusi sehingga tidak dapat diterapkan pada job yang belum pernah dijalankan, terbatas pada satu metrik, dan belum mengkarakterisasi kualitas prediksi LLM secara sistematis. Penelitian ini memprediksi lima metrik sekaligus secara pra-eksekusi tanpa riwayat job, dengan hanya memanfaatkan kode sumber, metadata dataset, dan spesifikasi node target. Mekanisme yang dibangun menyusun ketiga masukan tersebut menjadi prompt terstruktur dan meminta Large Language Model (LLM) mengestimasi runtime, utilisasi CPU, utilisasi GPU, penggunaan RAM, dan penggunaan VRAM, murni melalui teknik prompting tanpa pelatihan model. Kumpulan 30 job dari Kaggle dipraproses menjadi shard WebDataset beserta indeks Parquet dan diberi instrumentasi pengumpulan metrik, lalu dieksekusi pada kluster Ray heterogen (A4000, L4, dan A100) untuk memperoleh nilai acuan. Inferensi dijalankan menggunakan tujuh model LLM dan empat konfigurasi prompting, kemudian dievaluasi menggunakan Estimation Accuracy (EA), Mean Absolute Error, dan Mean Squared Error, beserta stabilitas keluaran melalui koefisien variasi (CV). Pengujian fungsionalitas menunjukkan seluruh kombinasi job dan node berhasil melalui alur praproses hingga eksekusi dan menghasilkan metrik yang valid sebagai nilai acuan. Pengujian performa prediksi menunjukkan Claude 4.6 Sonnet sebagai model terbaik dengan rata-rata Estimation Accuracy (EA) lintas metrik 0.32-0.40 (skala 0-1, semakin tinggi semakin baik). Pengujian stabilitas dengan koefisien variasi (CV) menunjukkan model yang sama juga menghasilkan keluaran paling stabil dengan CV 0.017-0.046 (tak berdimensi, semakin rendah semakin stabil).
===================================================================================================================================
Workload prediction is needed so job's resource requirements can be known before execution. This need is prominent in job scheduling on heterogeneous Ray clusters, which relies on explicit resource requests and generic heuristics without analyzing code structure, causing two jobs with similar requests but different computational patterns to be treated identically. Prior workload prediction relies on runtime logs and execution history, making it inapplicable to jobs that have never been run, limited to a single metric, and lacking systematic characterization of LLM prediction quality. This research predicts five metrics at once, pre-execution and without job history, using only source code, dataset metadata, and target node specifications. The proposed mechanism composes these three inputs into a structured prompt and asks a Large Language Model (LLM) to estimate runtime, CPU utilization, GPU utilization, RAM usage, and VRAM usage, purely through prompting techniques without model training. A collection of 30 jobs from Kaggle was preprocessed into WebDataset shards with Parquet indices and instrumented for metric collection, then executed on a heterogeneous Ray cluster (A4000, L4, and A100) to obtain reference values. Inference was run using seven LLM models and four prompting configurations, then evaluated using Estimation Accuracy (EA), Mean Absolute Error, and Mean Squared Error, along with output stability via the coefficient of variation (CV). Functional testing showed that all job-node combinations successfully passed the pipeline from preprocessing through execution and produced valid metrics as reference values. Prediction performance testing showed Claude 4.6 Sonnet as the best model, with an average Estimation Accuracy (EA) across metrics of 0.32–0.40 (0–1 scale, higher is better). Stability testing using the coefficient of variation (CV) showed that the same model also produced the most stable output, with a CV of 0.017–0.046 (dimensionless, lower is more stable).

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Kluster heterogen, Large Language Model, Prediksi beban kerja, Prompt engineering, Ray, Heterogeneous cluster, Large Language Model, Prompt engineering, Ray, Workload prediction
Subjects:	T Technology > T Technology (General) > T58.62 Decision support systems
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Keanu Fortuna Taufan
Date Deposited:	28 Jul 2026 06:38
Last Modified:	28 Jul 2026 06:38
URI:	http://repository.its.ac.id/id/eprint/138649

Actions (login required)

View Item