Learning to Restore Deteriorated Line Drawing
Kazuma Sasaki Satoshi
Iizuka Edgar Simo-Serra Hiroshi
Ishikawa
Computer Graphics International 2018
Abstract:
We propose a fully automatic approach to restore
aged old line drawings. We decompose the task into two subtasks:
the line extraction subtask, which aims to extract line
fragments and remove the paper texture background, and the
restoration subtask, which fills in possible gaps and deterioration
of the lines to produce a clean line drawing. Our approach
is based on a convolutional neural network that consists
of two sub-networks corresponding to the two subtasks.
They are trained as part of a single framework in an end-to-end
fashion. We also introduce a new dataset consisting of
manually annotated sketches by Leonardo da Vinci which,
in combination with a synthetic data generation approach,
allows training the network to restore deteriorated line drawings.
We evaluate our method on challenging 500-year-old
sketches and compare with existing approaches with a user
study, in which it is found that our approach is preferred
72.7% of the time.
Paper (14.7MB) Dataset (126MB) BibTex
Line Restoration Architecture:
Our model consists of two submodels: a line extraction network, and a
restoration network. The output of the line extraction network is used
as an input to the restoration network, and the output of both the line
extraction network and restoration network is added component-wise
to form the final output.
Leonardo da Vinci's Sketch Dataset:
We propose a new dataset to train the two-part network,
based on Leonardo da Vinci’s sketches, and complement the
dataset with synthetically generated line drawings composed by simple primitives.
We collect 71 of Leonard da Vinci’s old sketch scans, and
manually provide annotations of the underlying line drawing.
The annotations are added by drawing on top of the old
sketches using a pen tablet. Here, we choose those sketches
with limited amount of shading that can be considered line
drawings, avoiding artistic drawings, and the most heavily deteriorated
sketches. We do not annotate dirt nor text as ground truth lines.
Out of the 71 images, we reserve 10 for evaluation, leaving 61 images
to be used for training.
Download Leonardo da Vinci Dataset (126MB)
