Integrasi SERVQUAL Framework dalam Model Generative Aspect Sentiment Quadruple Prediction (ASQP) Berbasis Large Language Model

Gorter, Muhammad Jerino (2026) Integrasi SERVQUAL Framework dalam Model Generative Aspect Sentiment Quadruple Prediction (ASQP) Berbasis Large Language Model. Masters thesis, Institut Teknologi Sepuluh Nopember.

Text 6025241054-Master_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (3MB) | Request a copy

Abstract

Pertumbuhan ulasan digital pada industri hotel menghasilkan data teks dalam jumlah besar untuk evaluasi kualitas layanan, namun tingginya volume dan keragaman ulasan membuat analisis manual tidak efisien. Penelitian ini mengusulkan integrasi Aspect Sentiment Quadruple Prediction (ASQP) dengan framework SERVQUAL untuk menghasilkan representasi Category–Aspect–Opinion–Polarity (CAOP) ke dalam lima dimensi Tangibles, Reliability, Responsiveness, Assurance, dan Empathy, sekaligus membangun Dataset ASQP-SERVQUAL pertama untuk ulasan hotel berbahasa Indonesia. Dataset ground truth disusun melalui anotasi manual terhadap 998 ulasan hotel yang menghasilkan 4.417 quadruple CAOP-SERVQUAL. Validasi oleh ahli hotel menghasilkan Cohen's Kappa sebesar 0,901 untuk Category dan 0,913 untuk Polarity, serta F1-score sebesar 87,3% untuk Aspect dan 82,2% untuk Opinion. Untuk mengatasi keterbatasan data berlabel, pseudo-labeling menggunakan LLaMA 3.3 70B menghasilkan 8.384 pseudo-label valid dari 8.389 ulasan (99,94%), yang digunakan untuk fine-tuning model generatif IndoT5 dan mT5. Hasil eksperimen menunjukkan bahwa model baseline zero-shot memperoleh F1-score 0%, sedangkan setelah fine-tuning IndoT5 mencapai F1 Exact 23% dan F1 Fuzzy 30%, sementara mT5 mencapai F1 Exact 21% dan F1 Fuzzy 31%. Model mT5 menunjukkan performa terbaik pada Category (F1-score 78%) dan Polarity (F1-score 88%), setara dengan 86,5% dan 96,7% dari kemampuan anotator manusia. Hasil penelitian menunjukkan bahwa integrasi ASQP–SERVQUAL dengan pseudo-labeling semi-supervised efektif untuk analisis kualitas layanan hotel berbahasa Indonesia.
===============================================================================================================================
The growth of digital reviews in the hotel industry generates large volumes of textual data for service quality evaluation however, the high volume and diversity of reviews make manual analysis inefficient. This study proposes integrating Aspect Sentiment Quadruple Prediction (ASQP) with the SERVQUAL framework to produce a Category – Aspect – Opinion – Polarity (CAOP) representation across the five dimensions of Tangibles, Reliability, Responsiveness, Assurance, and Empathy, while also constructing the first ASQP-SERVQUAL dataset for Indonesian-language hotel reviews. The ground truth dataset was developed through manual annotation of 998 hotel reviews, yielding 4,417 CAOP-SERVQUAL quadruples. Validation by a hotel expert produced a Cohen's Kappa of 0.901 for Category and 0.913 for Polarity, as well as F1-scores of 87.3% for Aspect and 82.2% for Opinion. To address the limited amount of labelled data, pseudo-labelling using LLaMA 3.3 70B produced 8,384 valid pseudo-labels from 8,389 reviews (99.94%), which were used to fine-tune the generative models IndoT5 and mT5. Experimental results show that the zero-shot baseline models obtained an F1-score of 0%, whereas after fine-tuning IndoT5 achieved an F1 Exact of 23% and F1 Fuzzy of 30%, while mT5 achieved an F1 Exact of 21% and F1 Fuzzy of 31%. The mT5 model showed the best performance on Category (F1-score 78%) and Polarity (F1-score 88%), equivalent to 86.5% and 96.7% of human annotator performance. These findings demonstrate that integrating ASQP–SERVQUAL with semi-supervised pseudo-labelling is effective for service quality analysis of Indonesian-language hotel reviews.

Item Type:	Thesis (Masters)
Uncontrolled Keywords:	ABSA, ASQP, LLM, Transfomer, SERVQUAL, Pseudo-labeling ABSA, ASQP, LLM, Transfomer, SERVQUAL, Pseudo-labeling
Subjects:	T Technology > T Technology (General) > T57.5 Data Processing T Technology > T Technology (General) > T58.6 Management information systems
Divisions:	Faculty of Industrial Technology > Informatics Engineering > 55101-(S2) Master Thesis
Depositing User:	Muhammad Jerino Gorter
Date Deposited:	29 Jul 2026 03:57
Last Modified:	29 Jul 2026 03:57
URI:	http://repository.its.ac.id/id/eprint/138954

Actions (login required)

View Item