Maulana, Avin (2020) Facial Inpainting Menggunakan Generative Adversarial Network dengan Mempertahankan Keterkaitan Spasial. Masters thesis, Institut Teknologi Sepuluh Nopember.
Preview |
Text
05111850010029-Master_Thesis.pdf Download (6MB) | Preview |
Abstract
Facial inpainting merupakan proeses merekonstruksi kembali bagian yang hilang pada citra wajah sedemikian sehingga citra hasil rekonstruksi dapat tetap terlihat realistis, serta pihak pengamat tidak dapat mengenali bagian yang merupakan hasil rekontsruksi. Facial inpainting dapat menjadi masalah yang menantang, karena untuk melakukan proses rekonstruksi diperlukan pengetahuan perseptual dari wajah, tidak cukup jika hanya dengan melihat kemiripan dengan bagian sekitar dari bagian yang hilang, seperti algoritma inpainting konvensional. Seiring dengan perkembangan teknologi dan ketersediaan data, facial inpainting dapat dilakukan dengan menggunakan konsep deep learning.
Penelitian sebelumnya melakukan inpainting Generative Adversarial Network (GAN). Namun terdapat masalah yang timbul pada saat inpainting dilakukan. Masalah pertama adalah piksel yang tidak konsisten antara bagian hasil inpainting dengan bagian sekitarnya ketika proses inpainting dilakukan pada citra wajah unaligned. Hasil inpainting terlihat tidak realistis karena perbedaan yang dapat terlihat antara bagian hasil rekonstruksi dengan bagian asli. Masalah ini dapat dilihat sebagai masalah keterkaitan spasial. Piksel hasil inpainting juga mungkin menunjukkan perbedaan warna dengan bagian piksel sekitarnya, seperti ketika bagian citra yang dilakukan rekonstruksi terletak pada setengah bibir.
Penelitian ini bertujuan mengembangkan metode inpainting berbasis GAN dengan tambahan loss berupa feature reconstruction loss dan face landmark loss untuk mengatasi hasil inpainting yang terlihat tidak realistis pada citra wajah unaligned. Feature reconstruction loss adalah loss yang diperoleh dari pre-trained network VGG-Net. Loss ini dapat digunakan untuk membantu mempertahankan keterkaitan spasial pada citra, terlebih ketika citra yang digunakan merupakan citra wajah unaligned. Face landmark loss juga merupakan loss dari pre-trained network, dan dapat digunakan untuk membantu meningkatkan kualitas perseptual dari citra hasil inpainting.
Proses training dilakukan dengan skenario curriculum learning. Secara kualitatif hasil yang diperleh menunjukkan bahwa metode inpainting yang diajukan tetap dapat dilakukan pada citra wajah unaligned dengan tetap mempertahankan keterkaitan spasial. Secara kuantitatif, metode yang diajukan mampu memperoleh rata-rata PSNR dan SSIM 21.528 dan 0.665, sementara maksimum PSNR dan SSIM yang diperoleh 29.922 dan 0.908.
==========================================================
Facial inpainting is a process to reconstruct some missing or damaged pixels in the facial image, and the reconstructed pixels should still be realistic so the observer could not differentiate between the reconstructed pixels or the original one. Facial inpainting can be a challenging problem to solve in machine learning. The reconstruction process on the human face's image should consider perceptual knowledge of faces, besides its similarity with its neighbor region, unlike another conventional inpainting algorithm. Along with technology development and data availability, facial inpainting can be done using deep learning methods.
Some of the previous researches have done inpainting using generative network, such as Generative Adversarial Network. However, there are a few problems that may arise when the inpainting algorithm has been done. The first problem was an inconsistency between adjacent pixels when facial inpainting was done on unaligned face images. The inpainting result would be seen as unrealistic because of its difference between the reconstructed and original regions. It can be seen as a spatial correlation problem between adjacent pixels. Inpainting results may also show different colors between the generated region and its adjacent original regions, for instance, when the missing regions were half of the lips area. Therefore, an improvement method in facial inpainting based on deep-learning is proposed to reduce the effect of the stated problems before, using GAN with additional loss from feature reconstruction loss and face landmark loss. Feature reconstruction loss is a loss obtained by using pre-trained network VGG-Net. It can be used to help in preserving spatial consistency within images, especially when it comes to unaligned faces. Face landmark loss is also a loss from a pre-trained network, and able to help the perceptual quality of the inpainting result.
Training process have been done using curriculum learning scenario. Qualitative results show that our inpainting method can reconstruct missing area on unaligned face images, and still preserve consistent colours between its adjacent pixels. From the quantitative results, our proposed method can achieve average score 21.528 and 0.665 on PSNR and SSIM metrics, respectively. While the maximum score achieved for PSNR and SSIM are 29.922 and 0.908, respectively.
| Item Type: | Thesis (Masters) |
|---|---|
| Uncontrolled Keywords: | curriculum learning, facial inpainting, feature reconstruction loss, generative adversarial network, keterkaitan spasial, spatial correlation |
| Subjects: | T Technology > TA Engineering (General). Civil engineering (General) > TA1637 Image processing--Digital techniques |
| Divisions: | Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55101-(S2) Master Thesis |
| Depositing User: | AVIN MAULANA |
| Date Deposited: | 12 Aug 2020 02:51 |
| Last Modified: | 26 May 2023 07:54 |
| URI: | http://repository.its.ac.id/id/eprint/77552 |
Actions (login required)
 |
View Item |