Pengembangan Antarmuka Multimodal Untuk Aplikasi Pembelajaran Asistif Bahasa Oral Anak Tuli Di SLB Karya Mulia Surabaya

Amana, Nadya Zuhria (2025) Pengembangan Antarmuka Multimodal Untuk Aplikasi Pembelajaran Asistif Bahasa Oral Anak Tuli Di SLB Karya Mulia Surabaya. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025211058-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (17MB) | Request a copy

Abstract

Perkembangan Anak dengan disabilitas pendengaran (tuli) menghadapi tantangan dalam pemerolehan bahasa lisan karena keterbatasan akses suara sejak dini. Berdasarkan observasi dan wawancara di SLB, diketahui bahwa sebagian besar anak hanya mengandalkan bahasa isyarat dan belum mendapat pelatihan bahasa oral secara terstruktur. Padahal, kemampuan bahasa oral sangat penting untuk berkomunikasi di lingkungan mayoritas pendengar.
Penelitian ini bertujuan merancang dan mengembangkan antarmuka aplikasi pembelajaran bahasa oral berbasis multimodal untuk anak tuli usia sekolah dasar. Aplikasi ini menggabungkan modalitas visual (gerakan mulut, animasi, teks), auditori (contoh suara), kinestetik (aktivitas meniru dan menekan tombol), serta interaktif (fitur deteksi suara dan pengenal benda melalui kamera). Metode yang digunakan adalah Design Thinking melalui lima tahap: empathize, define, ideate, prototype, dan test.
Pada tahap empathize dilakukan observasi dan wawancara dengan stakeholder. Hasilnya dipetakan dalam persona, affinity map, dan user journey. Ide fitur meliputi latihan pelafalan, umpan balik visual, gamifikasi, dan kamera pengenal benda. Prototipe diuji kepada lima anak, lima orang tua, dan lima guru.
Evaluasi menunjukkan peningkatan usability dari iterasi pertama ke kedua. Skor SUS anak meningkat dari 76,5 menjadi 84. Orang tua menunjukkan kenaikan SEQ dari 6,0–6,6 menjadi 6,8, dan SUS dari 95 ke 96. Pada guru, skor SUS naik dari 70 ke 81,5 dan SEQ dari 4,4–4,8 menjadi 5,2–6,0. Anak juga menunjukkan lebih banyak ekspresi positif dan membutuhkan bantuan lebih sedikit. Hasil ini menunjukkan bahwa aplikasi Voca efektif, inklusif, dan mendukung pembelajaran bahasa oral anak tuli secara menyenangkan melalui pendekatan multimodal.
==================================================================================================================================================
Children with hearing disabilities (deaf) face significant challenges in acquiring oral language due to limited access to sound from an early age. Observations and interviews with the head of a special school, teachers, and parents revealed that many children rely solely on sign language and lack structured oral language training. In fact, developing oral language skills early is crucial to help them communicate more easily in predominantly hearing environments.
This study aims to design and develop a multimodal-based oral language learning application interface for deaf elementary school children. The application utilizes multimodal approaches, combining visual (mouth movement videos, animations, texts), auditory (voice examples for children with residual hearing), kinesthetic (mimicking movements, pressing buttons), and interactive modalities (voice detection and object recognition via camera) to enrich learning experiences. The Design Thinking method was applied through five stages: empathize, define, ideate, prototype, and test. The empathize stage involved direct observation and in-depth interviews with stakeholders. From this, user personas, affinity diagrams, and user journeys were developed. Ideation generated feature ideas such as pronunciation practice with voice detection, visual feedback, gamification, and camera-based object recognition. A prototype was created using Figma and tested with five children, five parents, and five teachers.
Usability evaluations using SUS and SEQ (or expression observation for children) showed improvements from the first to the second iteration across all user groups. For children, increased positive expressions and reduced assistance needs were observed, with SUS scores rising from 76.5 to 84. For parents, SEQ scores improved from 6.0–6.6 to 6.8, and SUS from 95 to 96. Teachers showed progress as well, with SUS increasing from 70 to 81.5, and SEQ from 4.4–4.8 to 5.2–6.0. These results indicate enhanced usability and user experience in the second iteration.This application is expected to serve as an engaging and inclusive educational solution that supports oral language acquisition for deaf children through a multimodal approach.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Educational App, Deaf Children, Oral Language, Design Thinking, Multimodal Interface, Usability, Aplikasi Edukasi, Anak Tuli, Bahasa Oral, Design Thinking, Multimodal, Antarmuka Pengguna
Subjects:	L Education > L Education (General) L Education > LB Theory and practice of education > LB1029.S53 Educational games. Simulation methods P Language and Literature > PZ Childrens literature T Technology > T Technology (General)
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Nadya Zuhria Amana
Date Deposited:	30 Jul 2025 10:35
Last Modified:	30 Jul 2025 10:35
URI:	http://repository.its.ac.id/id/eprint/123841

Actions (login required)

View Item