Sistem Penilaian Otomatis Presentasi Bahasa Inggris Bagi Pembelajar EFL: Evaluasi Aspek Konten Dan Delivery

Purba, Etha Felisya Br (2026) Sistem Penilaian Otomatis Presentasi Bahasa Inggris Bagi Pembelajar EFL: Evaluasi Aspek Konten Dan Delivery. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5027221017-Undergraduate-Thesis.pdf Restricted to Repository staff only Download (8MB) | Request a copy

Abstract

Kemampuan presentasi berbahasa Inggris merupakan keterampilan krusial bagi mahasiswa English as a Foreign Language (EFL), namun akses terhadap umpan balik berkualitas dari evaluator ahli masih terbatas karena beban kerja tinggi dan jumlah mahasiswa yang besar. Evaluasi manual juga cenderung subjektif dan memerlukan waktu lama, sehingga menghambat latihan iteratif untuk meningkatkan kemampuan public speaking. Sementara itu, sistem evaluasi otomatis yang ada umumnya bergantung pada data visual atau hanya menilai aspek linguistik tanpa menilai delivery secara komprehensif. Penelitian ini mengembangkan sistem evaluasi presentasi otomatis berbasis kecerdasan buatan yang menilai konten dan delivery menggunakan data audio dan teks tanpa ketergantungan visual. Dataset terdiri dari 15 presentasi TED Talks dan 16 rekaman mahasiswa berdurasi 3-5 menit yang dianotasi oleh dua evaluator independen. Pipeline meliputi transkripsi dengan Whisper ASR, evaluasi konten berbasis Sentence-BERT (struktur, koherensi melalui semantic similarity, kompleksitas melalui Type-Token Ratio), serta evaluasi delivery berbasis fitur prosodik Librosa (speaking rate, pitch variation, intensity, pause detection), lalu digabungkan dengan weighted average (60% konten, 40% delivery) pada aplikasi web Streamlit. Hasil evaluasi empat algoritma regresi menunjukkan model konten memiliki performa lebih baik dengan rata-rata korelasi Pearson 0,8583 dan MAE 0,6423, dengan Gradient Boosting terbaik pada Struktur (0,8772) dan Random Forest pada Koherensi (0,8913). Model delivery bersifat lebih bervariasi, dengan Volume sangat baik (Random Forest: Spearman 0,9169) namun aspek Jeda mengalami kegagalan sistemik (MAE > 1,1). Sistem memproses audio dalam 26,86-68,88 detik dan tahap penilaian otomatis berjalan cepat pada 1,96–8,83 detik, sehingga layak untuk kebutuhan pembelajaran praktis.
===================================================================================================================================
English presentation skills are crucial for English as a Foreign Language (EFL) students, yet access to high-quality and consistent feedback from expert evaluators remains limited due to heavy workloads and large class sizes. Manual assessment is often subjective and time-consuming, which hinders the iterative practice needed to improve public speaking. Meanwhile, existing automated assessment systems commonly rely on visual data or evaluate linguistic aspects only, without comprehensively addressing the delivery dimension. This research develops an artificial intelligence-based automated presentation assessment system that evaluates both content and delivery using audio and text without visual dependency. The dataset consists of 15 TED Talks presentations and 16 student recordings with 3–5 minute duration, manually annotated by two independent evaluators. The pipeline includes transcription with Whisper ASR, Sentence-BERT-based content evaluation (structure, coherence via semantic similarity, and complexity via Type-Token Ratio), and Librosa-based prosodic feature extraction for delivery evaluation (speaking rate, pitch variation, intensity, and pause detection), integrated using a weighted average (60% content, 40% delivery) in a Streamlit web application. Results from a comparative evaluation of four regression algorithms show stronger performance for content assessment, achieving an average Pearson correlation of 0.8583 and MAE of 0.6423, with Gradient Boosting performing best for Structure (0.8772) and Random Forest for Coherence (0.8913). Delivery models exhibit higher variability, with excellent performance on Volume (Random Forest: Spearman 0.9169) but systematic failure on Pause assessment (MAE > 1.1). The system processes audio within 26.86–68.88 seconds, while automated scoring is completed rapidly in 1.96–8.83 seconds, indicating practical feasibility for learning applications.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Evaluasi Presentasi Otomatis, English as a Foreign Language (EFL), Natural Language Processing, Analisis Prosodi, Sentence-BERT, Librosa, Automated Presentation Assessment, English as a Foreign Language (EFL), Natural Language Processing, Prosodic Analysis, Sentence-BERT, Librosa
Subjects:	T Technology > T Technology (General) > T58.5 Information technology. IT--Auditing
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information Technology > 59201-(S1) Undergraduate Thesis
Depositing User:	Etha Felisya Br Purba Purba
Date Deposited:	19 Jan 2026 00:38
Last Modified:	19 Jan 2026 00:38
URI:	http://repository.its.ac.id/id/eprint/129671

Actions (login required)

View Item