Audio-to-Ayat Retrieval Al-Qur'an Menggunakan Hybrid Self-Supervised Speech Embedding dan Acoustic Tokenization

Cantika, Dhia Auzie (2026) Audio-to-Ayat Retrieval Al-Qur'an Menggunakan Hybrid Self-Supervised Speech Embedding dan Acoustic Tokenization. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025221166-Dhia-Auzie-Cantika-Buku.pdf - Accepted Version Restricted to Repository staff only Download (7MB) | Request a copy

Abstract

Pencarian ayat Al-Qur'an berbasis audio (audio-to-ayat retrieval) tanpa memanfaatkan Automatic Speech Recognition (ASR) masih menghadapi tantangan akibat variasi gaya bacaan, tempo pelafalan, karakteristik vokal, dan durasi keheningan antar-qari. Penelitian ini bertujuan mengembangkan sistem audio-to-ayat retrieval yang mampu melakukan pencarian ayat secara langsung berdasarkan informasi akustik tanpa melalui proses transkripsi. Metode yang diusulkan mengintegrasikan self-supervised speech embedding dari model wav2vec 2.0 dengan acoustic tokenization berbasis algoritma K-Means untuk membentuk representasi audio hibrida. Representasi kontinu diperoleh melalui kombinasi mean, max, min, dan standard deviation pooling, kemudian digabungkan dengan histogram token akustik diskret. Selanjutnya, proses pencarian dilakukan dengan menghitung kemiripan antara embedding kueri dan embedding referensi menggunakan cosine similarity. Hasil evaluasi pada skenario pengujian lintas qari menunjukkan bahwa sistem yang diusulkan memperoleh rata-rata Recall@1 sebesar 97,90%, Recall@5 sebesar 99,43%, Recall@10 sebesar 99,63%, dan Mean Reciprocal Rank (MRR) sebesar 98,62%. Selain itu, pengujian generalisasi pada dataset qari non-profesional yang memiliki variasi kualitas audio dan tingkat kefasihan yang lebih rendah menunjukkan bahwa sistem tetap mampu mempertahankan performa yang kompetitif dengan Recall@1 sebesar 84,38%, Recall@5 sebesar 92,28%, Recall@10 sebesar 94,18%, serta MRR sebesar 87,96%. Hasil tersebut menunjukkan bahwa representasi audio hibrida mampu mendukung proses pencarian ayat secara akurat pada berbagai karakteristik bacaan qari tanpa memerlukan modul ASR.
===================================================================================================================================
Audio-to-ayat retrieval without relying on Automatic Speech Recognition (ASR) remains challenging due to variations in recitation style, speaking rate, vocal characteristics, and silence duration across Quran reciters (qaris). This study aims to develop an audio-to-ayat retrieval system capable of retrieving Quranic verses directly from acoustic information without requiring speech transcription. The proposed method integrates self-supervised speech embeddings extracted using wav2vec 2.0 with K-Means-based acoustic tokenization to construct a hybrid audio representation. Continuous representations are obtained by combining mean, max, min, and standard deviation pooling, and are subsequently fused with discrete acoustic token histograms. Retrieval is then performed by measuring the similarity between query embeddings and reference embeddings using cosine similarity. Evaluation results in the cross-qari testing scenario show that the proposed system achieved an average Recall@1 of 97.90%, Recall@5 of 99.43%, Recall@10 of 99.63%, and a Mean Reciprocal Rank (MRR) of 98.62%. Furthermore, generalization experiments on a non-professional qari dataset, which contains greater variations in audio quality and lower recitation fluency, demonstrated that the system maintained competitive performance, achieving a Recall@1 of 84.38%, Recall@5 of 92.28%, Recall@10 of 94.18%, and an MRR of 87.96%. These results indicate that the proposed hybrid audio representation effectively supports accurate audio-to-ayat retrieval across diverse recitation characteristics without relying on an ASR module.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Acoustic Tokenization, Audio-to-Ayat Retrieval, Cross-Modal Retrieval, Pooling, Self-Supervised Learning, Wav2vec 2.0, Acoustic Tokenization, Audio-to-Ayat Retrieval, Cross-Modal Retrieval, Pooling, Self-Supervised Learning, Wav2vec 2.0
Subjects:	T Technology > T Technology (General) > T57.5 Data Processing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Dhia Auzie Cantika
Date Deposited:	28 Jul 2026 01:19
Last Modified:	28 Jul 2026 01:19
URI:	http://repository.its.ac.id/id/eprint/138220

Actions (login required)

View Item