One of The Earliest Papers for Neural Image Style Transfer

Sik-Ho Tsang

·

Follow

3 min read

·

Aug 27, 2023

--

Listen

Share

A Neural Algorithm of Artistic Style
Artistic Style Transfer, University of Tübingen, Bernstein Center for Computational Neuroscience, Graduate School for Neural Information Processing, Max Planck Institute for Biological Cybernetics, and Baylor College of Medicine
2016 Journal of Vision, Over 2800 Citations (Sik-Ho Tsang @ Medium)

Style Transfer
2016 [Image Style Transfer] [Perceptual Loss] [GAN-CLS, GAN-INT, GAN-CLS-INT] ==== My Other Paper Readings Are Also Over Here ====

• An artificial system is introduced based on a Deep Neural Network that creates artistic images of high perceptual quality.
• The system uses neural representations to separate and recombine content and style of arbitrary images.
• This paper is the same as Image Style Transfer in 2016 CVPR.

Outline

1. Artistic Style Transfer
2. Results

1. Artistic Style Transfer

1.1. Model

• VGG-19 is used where only fully connected layers are not used.
• A given input image ~x is encoded in each layer of the CNN.
• The responses in a layer l can be stored in a matrix Fl.

1.2. Content Representation

• Let ~p and ~x be the original image and the image that is generated and Pl and Fl their respective feature representation in layer l.
• The squared-error loss between the two feature representations is the content loss:

• where ith filter at position j in layer l.

1.3. Style Representation

• On top of the CNN responses in each layer of the network, a style representation is built that computes the correlations between the different filter responses. These feature correlations are given by the Gram matrix Gl:

• Let ~a and ~x be the original image and the image that is generated and Al and Gl their respective style representations in layer l. The contribution of that layer to the total loss is then:

• The total style loss is:

1.4. Total Loss

• The total loss is the weighted sum of content loss and style loss:

2. Results

• The images were created by finding an image that simultaneously matches the content representation of the photograph and the style representation of the artwork.