Design and Evaluation of a Human-In- The-Loop AI-Assisted Annotation System for Underwater Ecological Survey Imagery

Triatmono, Muhammad Aulia (2026) Design and Evaluation of a Human-In- The-Loop AI-Assisted Annotation System for Underwater Ecological Survey Imagery. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5025221069-Undergraduate_thesis.pdf - Accepted Version Restricted to Repository staff only Download (4MB) | Request a copy

Abstract

Pemantauan ekologi bawah air menghasilkan citra survei dalam jumlah besar yang memerlukan inspeksi, anotasi, validasi, dan kurasi sebelum dapat digunakan sebagai bukti ilmiah. Proses tersebut masih bergantung pada tenaga ahli karena citra bawah air sering dipengaruhi oleh kekeruhan, distorsi warna, keburaman, oklusi, kemunculan organisme berulang, dan ambiguitas visual. Proyek akhir ini merancang dan mengevaluasi Seaseek, prototipe sistem anotasi citra bawah air berbasis human-in-the-loop, di mana kecerdasan buatan memberikan saran awal sementara keputusan akhir tetap berada di bawah kendali manusia. Sistem ini menggunakan frontend Next.js, backend FastAPI, database PostgreSQL, dan layanan model terpisah. Fitur yang dievaluasi meliputi deteksi ikan dan teripang berbasis YOLO, analisis cakupan bentik berbasis DINOv2/KNN untuk kategori karang, pasir, dan unknown, pencarian semantik CLIP/FAISS, serta fondasi teknis untuk pelacakan dan estimasi kepadatan. Evaluasi dilakukan melalui analisis artefak perangkat lunak, dokumentasi repositori, pengujian teknis, dan batasan evaluasi yang eksplisit. Hasil menunjukkan bahwa prototipe mendukung peninjauan berbantuan AI, koreksi manusia, pencatatan keputusan review, ekspor anotasi terkurasi, serta dokumentasi risiko etika dan keamanan. Model deteksi ikan melaporkan F1 sebesar 0,5895, sedangkan model Line B melaporkan akurasi 90,0% 5,5% pada 1.875 titik berlabel ahli. Namun, sistem masih berupa prototipe penelitian dan belum membuktikan pengurangan waktu anotasi melalui studi pengguna eksternal. Kontribusi utama penelitian ini adalah alur kerja anotasi yang menjaga kontrol manusia, ketertelusuran, dan penggunaan ulang anotasi terverifikasi.
=======================================================================================================================================
Underwater ecological monitoring can produce large volumes of survey imagery, yet the scientific value of those images depends on post-processing activities such as inspection, annotation, validation, and curation. This work remains expert-intensive because underwater imagery is affected by turbidity, colour distortion, blur, occlusion, repeated organisms across adjacent frames, and high visual ambiguity. This final project designs and evaluates Seaseek, a human-in-the-loop underwater image annotation prototype in which artificial intelligence provides first-pass suggestions while final decisions remain under human control. The system uses a Next.js frontend, a FastAPI backend, PostgreSQL persistence, and separate model services for object detection and benthic analysis. The evaluated features include Line A YOLO-based detection for fish and sea cucumber review, Line B DINOv2/KNN-based benthic coverage analysis for coral, sand, and unknown labels, CLIP/FAISS semantic frame search, and technical foundations for Line C tracking and Line D density estimation. The methodology follows a design-and-evaluation approach using software artefacts, repository documentation, technical tests, and explicit evaluation boundaries. The prototype supports AI-assisted review, human correction, persistence of review decisions, export of curated annotations, and documentation of ethical and security limitations. The fish model card reports an F1 score of 0.5895 for the packaged fish detector, while the Line B model card reports 90.0% +- 5.5% leave-one-bag-out accuracy over 1875 expert-labelled points. Nevertheless, the system remains a research prototype and has not yet demonstrated measured annotation-time reduction through an external user study. The main contribution is a review-centred workflow that preserves human control, traceability, and reusable verified annotations for dataset curation and future model improvement.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Anotasi gambar bawah air, keterlibatan manusia, visi komputer, YOLO, DINOV2, CLIP, FAISS, FastAPI, PostgreSQL, ekologi laut Underwater image annotation, human-in-the-loop, computer vision, YOLO, DINOv2, CLIP, FAISS, FastAPI, PostgreSQL, marine ecology
Subjects:	Q Science > QA Mathematics > QA336 Artificial Intelligence T Technology > T Technology (General) > T59.7 Human-machine systems. T Technology > TA Engineering (General). Civil engineering (General) > TA1637 Image processing--Digital techniques. Image analysis--Data processing.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55201-(S1) Undergraduate Thesis
Depositing User:	Muhammad Aulia Triatmono
Date Deposited:	23 Jul 2026 14:59
Last Modified:	23 Jul 2026 14:59
URI:	http://repository.its.ac.id/id/eprint/137715

Actions (login required)

View Item