Pengembangan Sistem Pengenalan Kemasan Obat Berbasis Visual dengan Dukungan Generative AI untuk Tunanetra

Ulfa, Indri Aulia (2026) Pengembangan Sistem Pengenalan Kemasan Obat Berbasis Visual dengan Dukungan Generative AI untuk Tunanetra. Masters thesis, Institut Teknologi Sepuluh Nopember.

Text 6022242002-Master_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (8MB) | Request a copy

Abstract

Menurut perkiraan Kementerian Kesehatan Republik Indonesia, jumlah tunanetra mencapai 1,5% dari total populasi. Dengan populasi Indonesia lebih dari 270 juta, diperkirakan ada sekitar 4 juta penyandang tunanetra. Individu dengan gangguan penglihatan menghadapi tantangan besar, terutama terkait kebutuhan untuk mengidentifikasi obat dengan akurat. Penelitian ini mengembangkan sebuah assistive technology berupa sistem pengenalan obat berbasis kecerdasan buatan. Sistem dirancang untuk mendeteksi kemasan obat secara real-time menggunakan kamera perangkat dengan memanfaatkan model YOLOv8 yang telah dikonversi ke TensorFlow Lite dan Optical Character Recognition (OCR) untuk mengekstrak teks pada kemasan. Proses deteksi dengan menganalisis frame dari kamera untuk mengidentifikasi objek kemasan obat dan hasil deteksi ditampilkan pada layar dalam bentuk bounding box beserta tingkat kepercayaan, kemudian hasil deteksi digunakan untuk mengidentifikasi nama obat menggunakan OCR. Nama obat yang berhasil dikenali kemudian disampaikan kepada pengguna melalui Text-to-Speech. Selain itu, sistem menyediakan fitur interaksi tanya jawab berbasis suara dengan memanfaatkan OpenAI API dan Firebase Realtime Database. Serta fitur pengingat minum obat membantu pengguna dalam mengatur jadwal konsumsi obat melalui perintah suara yang diproses menjadi alarm otomatis. Hasil pengujian menunjukkan bahwa model deteksi memiliki performa dengan nilai precision sebesar 0.999, recall sebesar 0.999, mAP@0.5 sebesar 0.995, dan mAP@0.5:0.95 sebesar 0.933. Selain itu, integrasi OpenAI API menunjukkan keberhasilan pada proses komunikasi API dengan penggunaan token yang efisien. Dengan demikian, sistem ini diharapkan dapat membantu tunanetra dalam mengenali dan memperoleh informasi obat serta meningkatkan kemandirian dalam penggunaan obat sehari-hari.
===============================================================================================================================
According to estimates by the Ministry of Health of the Republic of Indonesia, the number of visually impaired individuals accounts for 1.5% of the total population. With Indonesia’s population exceeding 270 million, it is estimated that there are approximately 4 million visually impaired individuals. Individuals with visual impairments face significant challenges, particularly regarding the need to accurately identify medications. This study developed an assistive technology in the form of an artificial intelligence-based medication recognition system. The system is designed to detect medication packaging in real-time using a device camera by leveraging the YOLOv8 model, which has been converted to TensorFlow Lite, and Optical Character Recognition (OCR) to extract text from the packaging. The detection process analyses camera frames to identify medication packaging objects, and the detection results are displayed on the screen as bounding boxes along with confidence scores. These results are then used to identify the medication name via OCR. The successfully recognized medication name is subsequently conveyed to the user via Text-to-Speech. Additionally, the system offers a voice-based question and answer feature utilizing the OpenAI API and Firebase Realtime Database, as well as a medication reminder feature that allows users to set consumption schedules via voice commands processed into automatic alarms. Test results show that the detection model achieves a precision of 0.999, a recall of 0.999, a mAP@0.5 of 0.995, and a mAP@0.5:0.95 of 0.933. Furthermore, the integration of the OpenAI API successfully facilitated API communication with efficient token utilization. Therefore, this system is expected to assist visually impaired individuals in identifying and obtaining medication information, as well as enhancing their independence in the daily use of medications.

Item Type:	Thesis (Masters)
Uncontrolled Keywords:	Tunanetra, Kemasan Obat, assistive technology, Generative AI, Firebase Realtime Database Visually Impaired, Medication Packaging, Assistive Technology, Generative AI, Firebase Realtime Database
Subjects:	T Technology > T Technology (General) > T57.5 Data Processing T Technology > TA Engineering (General). Civil engineering (General) > TA1637 Image processing--Digital techniques. Image analysis--Data processing.
Divisions:	Faculty of Electrical Technology > Electrical Engineering > 20101-(S2) Master Thesis
Depositing User:	Indri Aulia Ulfa
Date Deposited:	28 Jul 2026 02:19
Last Modified:	28 Jul 2026 02:19
URI:	http://repository.its.ac.id/id/eprint/137709

Actions (login required)

View Item