DEEP LEARNING EDGE DETECTION IN IMAGE INPAINTING

In recent years, deep learning has grown rapidly, and it has been creatively implemented for various applications. In 2019, deep learning based EdgeConnect image inpainting algorithm came out and occupied a place in the image inpainting field. Unlike traditional image inpainting methods which mainly read and use the color information of the remaining part of the image to fill the missing regions of the image, EdgeConnect uses the innovative edge-first and color-next approach. It uses an edge detector to generate an edge map of an image with missing regions, then the missing edges are completed by an edge model, finally the completed edge map is recolored by an inpaint model. The result of this algorithm has a significant improvement in the smoothness of the image, compared with conventional image inpainting methods.In this project, EdgeConnect is improved to become a completely deep learning-based image inpainting method.This project first implements the EdgeConnect approach. In the implementation, the project selects the optimal training parameters for the three model training phases included EdgeConnect: edge model, inpainting model and joint model, based on the original research paper and the discussions online. Then the EdgeConnect approach is improved by replacing the traditional Canny edge-detection with the deep learning algorithm, Holistically-Nested Edge Detection (HED). With the integration of HED, the accuracy of image inpainting is improved. To compare the performance, the original EdgeConnect and the modified EdgeConnect are both trained on the same set of data and the results are scored using the image inpainting quality assessment metrics such as PSNR, SSIM, MAE and FID.The results show that the modified EdgeConnect approach with the integration of HED not only improves the learning performance of edge detection, but also improves the overall quality of the final image inpainting.The improved EdgeConnect approach proposed and implemented in this project has higher learning efficiency and better image inpainting performance.