Object Removal System for Urban Imagery Using Image Segmentation and Inpainting with a Deep Learning Approach

EasyChair Preprint 15152

Abstract

Digital imagery is a two-dimensional visual representation, which often holds emotional significance and crucial information. However, in images, specifically urban imagery, unwanted objects frequently appear. To address this issue, a system capable of automatically selecting areas with unwanted objects, removing these areas, and reconstructing the removed regions is essential. The object removal system is developed by implementing and integrating an image segmentation module, image inpainting module, and graphical user interface application. The pre-trained image segmentation model, DeepLabv3+, is used for the image segmentation module. On the other hand, there are seven pre-trained image inpainting models, including DeepFillv2, EdgeConnect (Places), EdgeConnect (PSV), MADF (Places), MADF (PSV), MAT, and CoModGAN, which are compared across several testing aspects to be used in the image inpainting module. Based on the analysis of the test results on the test data, the DeepLabv3+ model is proven to perform accurate segmentation with a mIoU value reaching 0.936. The CoModGAN model is chosen as the pre-trained model of the image inpainting module due to its average PSNR score of 26.59dB, SSIM of 0.8908, FID of 39.99, and subjective evaluation of 4.105. The graphical user interface application developed and integrated with the image segmentation and image inpainting modules successfully provides flexibility to users and shows increased performance compared to previous studies.

Keyphrases: Object Removal, deep learning, image inpainting, image segmentation, urban imagery

@booklet{EasyChair:15152,
  author    = {Eiffel A. Amarendra and Rinaldi Munir},
  title     = {Object Removal System for Urban Imagery Using Image Segmentation and Inpainting with a Deep Learning Approach},
  howpublished = {EasyChair Preprint 15152},
  year      = {EasyChair, 2024}}