Pendekatan Multimodal Retrieval-Augmented Generation (RAG) dan Stable Diffusion untuk Otomatisasi Storyboard Konten YouTube ITS TV

Wijaya, Agas Ananta (2026) Pendekatan Multimodal Retrieval-Augmented Generation (RAG) dan Stable Diffusion untuk Otomatisasi Storyboard Konten YouTube ITS TV. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5027221004-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (5MB) | Request a copy

Abstract

Pembuatan storyboard merupakan tahap penting dalam pra-produksi konten YouTube ITS TV karena berfungsi mengubah naskah menjadi rancangan adegan yang terstruktur. Proses ini masih membutuhkan interpretasi naskah, pencarian referensi visual, dan penyusunan elemen produksi secara manual. Penelitian ini mengembangkan ITS TV Storyboard AI, yaitu aplikasi web berbasis Multimodal Retrieval-Augmented Generation (RAG), Large Language Model (LLM), dan Stable Diffusion untuk membantu otomatisasi penyusunan storyboard. Sistem menyediakan dua alur kerja, yaitu Manual Scene untuk mengekstrak naskah menjadi storyboard terstruktur dan Concept Auto Mode untuk membangkitkan storyboard dari prompt konsep. RAG digunakan untuk mengambil referensi visual dari dataset aset ITS TV melalui ChromaDB, sedangkan Stable Diffusion digunakan untuk menghasilkan visual storyboard. Evaluasi sistem dilakukan melalui enam skenario utama dan satu skenario tambahan. Pada Manual Scene, ekstraksi naskah dievaluasi menggunakan kelengkapan ekstraksi shot dan Skor Cakupan Informasi Ekstraksi karena keluaran LLM dapat menambahkan detail deskriptif tanpa menghilangkan informasi utama. Hasil lima tema menunjukkan rata-rata cakupan informasi sebesar 93,04%. Pada Concept Auto Mode, evaluasi difokuskan pada relevansi retrieval aset visual sebagai grounding, dengan rata-rata Top-1 Vector Similarity sebesar 83,25%. Pengujian RAG menunjukkan bahwa secara global Cosine Similarity meningkat dari 12,09% menjadi 17,58%, atau naik 5,49 poin. Pada tema berdomain ITS, peningkatan lebih besar terjadi dari 13,82% menjadi 25,77%, atau naik 11,95 poin. Skenario tambahan menunjukkan bahwa penambahan dataset dari 312 menjadi 412 gambar belum memberikan peningkatan yang jelas terhadap hasil generasi visual. Penilaian pengguna menghasilkan skor storyboard AI 3,50 dibanding storyboard manual 4,05, sementara UAT menunjukkan keberhasilan fungsional 98,00% dan penerimaan pengguna 4,04. Hasil tersebut menunjukkan bahwa sistem layak digunakan sebagai alat bantu awal pra-produksi storyboard ITS TV, tetapi tetap memerlukan kurasi manusia.
=====================================================================================================================================
Storyboard creation is an essential stage in the pre-production process of ITS TV YouTube content because it transforms a script into a structured scene design. This process still requires manual script interpretation, visual reference searching, and production element planning. This study develops ITS TV Storyboard AI, a web-based application that integrates Multimodal Retrieval-Augmented Generation (RAG), a Large Language Model (LLM), and Stable Diffusion to support storyboard automation. The system provides two workflows: Manual Scene, which extracts a script into a structured storyboard, and Concept Auto Mode, which generates a storyboard from a concept prompt. RAG is used to retrieve visual references from the ITS TV asset dataset through ChromaDB, while Stable Diffusion is used to generate storyboard visuals. The system was evaluated through six main testing scenarios and one additional scenario. In Manual Scene, script extraction was evaluated using shot extraction completeness and Information Coverage Score because LLM outputs may add descriptive details without removing the main information from the reference script. The evaluation across five themes achieved an average information coverage of 93.04%. In Concept Auto Mode, the evaluation focused on the relevance of visual asset retrieval as grounding, with an average Top-1 Vector Similarity of 83.25%. RAG testing showed that global Cosine Similarity increased from 12.09% to 17.58%, or by 5.49 points. On ITS-domain themes, the improvement was higher, increasing from 13.82% to 25.77%, or by 11.95 points. The additional scenario showed that increasing the dataset from 312 to 412 images did not produce a clear improvement in the generated visual results. User evaluation showed that AI-generated storyboards received an average score of 3.50, compared to 4.05 for manual storyboards, while User Acceptance Testing achieved a functional success rate of 98.00% and a user acceptance score of 4.04. These results indicate that the system is feasible as an initial pre-production aid for ITS TV storyboard creation, although human curation remains necessary.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Large Language Model, Multimodal RAG, Stable Diffusion, Storyboard, ITS TV
Subjects:	T Technology > T Technology (General) T Technology > T Technology (General) > T58.8 Productivity. Efficiency
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information Technology > 59201-(S1) Undergraduate Thesis
Depositing User:	Agas Ananta Wijaya
Date Deposited:	30 Jul 2026 01:32
Last Modified:	30 Jul 2026 01:32
URI:	http://repository.its.ac.id/id/eprint/137221

Actions (login required)

View Item