Rancang Bangun Sistem Pencarian Lokasi Obat Pada Apotek Berbasis Pengenalan Citra Kemasan Obat Dan Ucapan

Rizqi, Danish Ahza (2026) Rancang Bangun Sistem Pencarian Lokasi Obat Pada Apotek Berbasis Pengenalan Citra Kemasan Obat Dan Ucapan. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5022221217-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (8MB) | Request a copy

Abstract

Efisiensi pelayanan apotek sering terhambat oleh proses identifikasi dan pencarian obat yang masih dilakukan secara manual, terutama ketika pelanggan membawa kemasan obat yang rusak. Penelitian ini mengembangkan sistem pencarian lokasi obat berbasis multimodal yang mengintegrasikan klasifikasi citra menggunakan Convolutional Neural Network (CNN) dan pengenalan ucapan (speech recognition) untuk mengenali 15 jenis kemasan obat serta menerima masukan suara pengguna. Penelitian mengevaluasi tiga arsitektur CNN (Custom CNN, MobileNetV2, dan ResNet18) menggunakan visualisasi Grad-CAM untuk menginterpretasikan proses pengambilan keputusan model, serta tiga varian Faster-Whisper (Tiny, Base, dan Small). Modul pengenalan ucapan dioptimalkan menggunakan initial prompting, fuzzy matching, dan kamus alias, sedangkan sistem diimplementasikan menggunakan arsitektur client-server berbasis FastAPI. Hasil penelitian menunjukkan bahwa ResNet18 memberikan performa klasifikasi terbaik, di mana setelah hyperparameter tuning, model mencapai test accuracy sebesar 99,47% dan F1-Macro sebesar 99,51%, serta menekan error klasifikasi menjadi 20 kejadian. Selain itu, model ini memiliki ketahanan terbaik terhadap variasi kerusakan kemasan, sementara analisis Grad-CAM menunjukkan bahwa ResNet18 dan MobileNetV2 lebih berfokus pada area kemasan dibandingkan Custom CNN. Pada modul pengenalan suara, Faster-Whisper Base dipilih karena memberikan keseimbangan terbaik antara akurasi transkripsi dan latensi dengan rata-rata waktu inferensi 1,96 detik. Integrasi kedua modul berhasil mempercepat identifikasi dan pencarian lokasi obat serta meminimalkan risiko kesalahan pemberian obat.
========================================================================================================================================
Pharmacy service efficiency is often hindered by the manual process of medicine identification and retrieval, particularly when customers bring damaged medicine packaging. This study develops a multimodal medicine location retrieval system that integrates image classification using a Convolutional Neural Network (CNN) and speech recognition to identify 15 types of medicine packages and process users' voice input. The study evaluates three CNN architectures (Custom CNN, MobileNetV2, and ResNet18) using Grad-CAM visualization to interpret the models' decision-making process, as well as three Faster-Whisper variants (Tiny, Base, and Small). The speech recognition module is optimized using initial prompting, fuzzy matching, and an alias dictionary, while the system is implemented using a FastAPI-based client-server architecture. The results show that ResNet18 achieves the best classification performance, where after hyperparameter tuning, the model attains a test accuracy of 99.47% and an F1-macro score of 99.51%, while reducing classification errors to only 20 cases. Furthermore, ResNet18 demonstrates the highest robustness against variations in package damage. Grad-CAM analysis also indicates that ResNet18 and MobileNetV2 focus more effectively on the medicine package regions than Custom CNN. For the speech recognition module, Faster-Whisper Base is selected as it provides the best balance between transcription accuracy and latency, with an average inference time of 1.96 seconds. The integrated system successfully combines image-based identification, speech recognition, and medicine location retrieval, thereby accelerating the identification process and reducing the risk of medication errors.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Convolutional Neural Network, Faster-Whisper, Grad-CAM, Klasifikasi Citra, Pengenalan Ucapan, Pencarian Lokasi Obat, Image Classification, Speech Recognition, Drug Location Retrieval.
Subjects:	R Medicine > RS Pharmacy and materia medica T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7882.P3 Pattern recognition systems T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7895.S65 Speech recognition systems
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Electrical Engineering > 20201-(S1) Undergraduate Thesis
Depositing User:	Danish Ahza Rizqi
Date Deposited:	27 Jul 2026 01:02
Last Modified:	27 Jul 2026 01:02
URI:	http://repository.its.ac.id/id/eprint/137549

Actions (login required)

View Item