Learning to Restore Deteriorated Line Drawing

Kazuma Sasaki Satoshi Iizuka Edgar Simo-Serra Hiroshi Ishikawa

Computer Graphics International 2018


We propose a fully automatic approach to restore aged old line drawings. We decompose the task into two subtasks: the line extraction subtask, which aims to extract line fragments and remove the paper texture background, and the restoration subtask, which fills in possible gaps and deterioration of the lines to produce a clean line drawing. Our approach is based on a convolutional neural network that consists of two sub-networks corresponding to the two subtasks. They are trained as part of a single framework in an end-to-end fashion. We also introduce a new dataset consisting of manually annotated sketches by Leonardo da Vinci which, in combination with a synthetic data generation approach, allows training the network to restore deteriorated line drawings. We evaluate our method on challenging 500-year-old sketches and compare with existing approaches with a user study, in which it is found that our approach is preferred 72.7% of the time.

Paper (14.7MB) Dataset (126MB) BibTex

Line Restoration Architecture:

Our model consists of two submodels: a line extraction network, and a restoration network. The output of the line extraction network is used as an input to the restoration network, and the output of both the line extraction network and restoration network is added component-wise to form the final output.

Leonardo da Vinci's Sketch Dataset:

We propose a new dataset to train the two-part network, based on Leonardo da Vinci’s sketches, and complement the dataset with synthetically generated line drawings composed by simple primitives. We collect 71 of Leonard da Vinci’s old sketch scans, and manually provide annotations of the underlying line drawing. The annotations are added by drawing on top of the old sketches using a pen tablet. Here, we choose those sketches with limited amount of shading that can be considered line drawings, avoiding artistic drawings, and the most heavily deteriorated sketches. We do not annotate dirt nor text as ground truth lines. Out of the 71 images, we reserve 10 for evaluation, leaving 61 images to be used for training.

Download Leonardo da Vinci Dataset (126MB)


Input [Sasaki et al. 2017] Ours


Kazuma Sasaki, Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa.
"Learning to Restore Deteriorated Line Drawing".
The Visual Computer (Proc. of Computer Graphics International 2018).
  author = {Kazuma Sasaki and Satoshi Iizuka and Edgar Simo-Serra and Hiroshi Ishikawa},
  title = {{Learning to Restore Deteriorated Line Drawing}},
  journal = {The Visual Computer (Proc. of Computer Graphics International 2018)},
  year = {2018},
  volume = {34},
  number = {6-8},
  pages = {1077-1085},