Pembuatan Sitasi Paper dengan LLM Berbasis Vector Embeddings dan Query Expansion

Prawira, Dwiyana Yudha (2026) Pembuatan Sitasi Paper dengan LLM Berbasis Vector Embeddings dan Query Expansion. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025221230-Undergraduate_Thesis.pdf Restricted to Repository staff only Download (6MB) | Request a copy

Abstract

Jumlah publikasi ilmiah yang terus bertambah membuat pencarian referensi secara manual semakin sulit. Alat pencarian umum masih mengandalkan pencocokan kata sehingga rujukan relevan dapat terlewat karena perbedaan istilah, sementara penggunaan kecerdasan buatan secara langsung berisiko menghasilkan informasi yang tidak akurat. Penelitian ini mengembangkan sistem pembuatan sitasi ilmiah bernama SitasiAI berbasis Retrieval-Augmented Generation (RAG). Makalah ilmiah diubah menjadi representasi vektor menggunakan SPECTER2, model open source yang dilatih khusus pada hubungan sitasi antar-makalah sehingga makalah berkaitan mudah ditemukan tanpa pelatihan ulang. Pada tahap pencarian, HyDE (query expansion) mengubah paragraf pengguna menjadi abstrak hipotetis agar lebih sesuai dengan bentuk makalah yang dicari, lalu GPT-4o-mini menyusun kalimat sitasi dengan penalaran bertahap (chain-of-thought) agar tetap sesuai dengan isi rujukan. Sistem dievaluasi menggunakan Precision, Recall, dan Hit terhadap data acuan berlabel manusia, serta Faithfulness dan Answer Relevancy dari kerangka RAGAS. HyDE meningkatkan Precision dari 0,688 menjadi 0,705, dengan penurunan terkendali pada Recall, Hit, dan Faithfulness, sebuah trade-off yang sejalan dengan kebutuhan sistem yang mengutamakan ketepatan referensi. Uji signifikansi menunjukkan selisih tersebut tidak signifikan setelah koreksi multiple comparison, sehingga dibaca sebagai kecenderungan arah. SitasiAI berpotensi membantu peneliti menemukan dan menyusun referensi secara lebih efisien.
================================================================================================================================
The rapidly growing volume of scientific publications makes manual reference searching increasingly difficult. Common search tools still rely on keyword matching, so relevant references may be missed due to terminological differences, while using artificial intelligence directly risks producing inaccurate information. This study develops a scientific citation generation system named SitasiAI based on Retrieval-Augmented Generation (RAG). Scientific papers are converted into vector representations using SPECTER2, an open-source model trained specifically on inter-paper citation relationships, so that related papers can be retrieved without additional fine-tuning. During retrieval, HyDE (query expansion) transforms a user's paragraph into a hypothetical abstract to better match the form of the target papers, after which GPT-4o-mini composes the citation sentence using step-by-step reasoning (chain-of-thought) to remain faithful to the referenced papers. The system is evaluated using Precision, Recall, and Hit against human-labeled ground truth, together with Faithfulness and Answer Relevancy from the RAGAS framework. HyDE improves Precision from 0.688 to 0.705, with a controlled decrease in Recall, Hit, and Faithfulness, a trade-off consistent with a system that prioritizes reference precision. Statistical significance testing indicates that these differences are not significant after multiple comparison correction and should therefore be read as a directional tendency. SitasiAI has the potential to help researchers find and compose references more efficiently.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Retrieval-Augmented Generation, citation recommendation, HyDE, query expansion, Chain-of-thought
Subjects:	T Technology > T Technology (General) > T57.5 Data Processing T Technology > T Technology (General) > T57.8 Nonlinear programming. Support vector machine. Wavelets. Hidden Markov models. T Technology > T Technology (General) > T58.5 Information technology. IT--Auditing T Technology > T Technology (General) > T58.64 Information resources management
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering
Depositing User:	Dwiyana Yudha Prawira
Date Deposited:	28 Jul 2026 03:51
Last Modified:	28 Jul 2026 03:51
URI:	http://repository.its.ac.id/id/eprint/138387

Actions (login required)

View Item