DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement
— Additional Results —

Satoshi Iizuka  Edgar Simo-Serra

SIGGRAPH Asia 2019

Overview

We present additional results for remastering, colorization-only, and restoration-only. Results are shown on a test subset of the Youtube-8M dataset. Please refer to the paper for more details.

The approaches we compare to are as follows:

• Richard Zhang, Jun-Yan Zhu, Phillip Isola, Xinyang Geng, Angela S Lin, Tianhe Yu, and Alexei A Efros. 2017a. Real-Time User-Guided Image Colorization with Learned Deep Priors. ACM Transactions on Graphics (Proceedings of SIGGRAPH) 9, 4 (2017).
• Carl Vondrick, Abhinav Shrivastava, Alireza Fathi, Sergio Guadarrama, and Kevin Murphy. 2018. Tracking emerges by colorizing videos. In European Conference on Computer Vision (ECCV).
• Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017b. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing 26, 7 (2017), 3142–3155.
• Yuchen Fan, Jiahui Yu, and Thomas S Huang. 2018. Wide-activated Deep Residual Networks based Restoration for BPG-compressed Images. In IEEE Conference on Computer Vision and Pattern Recognition Workshops.

Remastering Results:

Comparison with combinations of existing restoration and colorization approaches, i.e., the approach of [Zhang et al. 2017b] and [Yu et al. 2018] for restoration, and [Zhang et al. 2017a] and [Vondrick et al. 2018] for colorization. We randomly sample a subset of 300 frames for videos from Youtube-8M dataset, and apply both example-based and algorithm-based deterioration effects. For the reference color images, we provide every 60th frame starting from the first frame as a reference image. In the following results, each video is an input video, [Zhang et al. 2017b] and [Zhang et al. 2017a], [Yu et al. 2018] and [Zhang et al. 2017a], [Zhang et al. 2017b] and [Vondrick et al. 2018], [Yu et al. 2018] and [Vondrick et al. 2018], and ours, in order from left to right, top to bottom.

Colorization Only Results:

Comparison with existing colorization approaches of [Zhang et al. 2017a] and [Vondrick et al. 2018] on the videos from Youtube8M dataset. For the reference color images, we provide every 60th frame starting from the first frame as a reference image. In the following results, each video is an input video, [Zhang et al. 2017a], [Vondrick et al. 2018], and ours, in order from left to right.

Restoration Only Results:

Comparison with existing restoration approaches of [Zhang et al. 2017b] and [Yu et al. 2018] on synthetically-generated highly-deteriorated videos from Youtube8M dataset. In the following results, each video is an input video, [Zhang et al. 2017b], [Yu et al. 2018], and ours, in order from left to right.

This work was partially supported by JST ACT-I (Iizuka, Grant Number: JPMJPR16U3), JST PRESTO (Simo-Serra, Grant Number: JPMJPR1756), and JST CREST (Iizuka and Simo-Serra, Grant Number: JPMJCR14D1).