Based on: Leon A. Gatys, Alexander S. Ecker, Matthias Bethge, "A Neural Algorithm of Artistic Style", arXiv:1508.06576
See also: https://github.com/fchollet/keras/blob/master/examples/neural_style_transfer.py
See some examples on: https://www.bonaccorso.eu/2016/11/13/neural-artistic-style-transfer-experiments-with-keras/
There are three possibile canvas setup:
- Picture: The canvas is filled with the original picture
- Style: The canvas is filled with the style image (resized to match picture dimensions)
- Random from style: The canvas is filled with a random pattern generated starting from the style image
Some usage examples (both with VGG16 and VGG19):
Picture and style over random:
canvas='random_from_style', alpha_style=1.0, alpha_picture=0.25, picture_layer='block4_conv1'
Style over picture:
canvas='picture', alpha_style=0.0025, alpha_picture=1.0, picture_layer='block4_conv1'
Picture over style:
canvas='style', alpha_style=0.001, alpha_picture=1.0, picture_layer='block5_conv1'
For a mix of style transfer and deepdream generation, see the examples below.
neural_styler = NeuralStyler(picture_image_filepath='img\\GB.jpg',
style_image_filepath='img\\Magritte.jpg',
destination_folder='\\destination_folder',
alpha_picture=0.4,
alpha_style=0.6,
verbose=True)
neural_styler.fit(canvas='picture', optimization_method='L-BFGS-B')
neural_styler = NeuralStyler(picture_image_filepath='img\\GB.jpg',
style_image_filepath='img\\Magritte.jpg',
destination_folder='\\destination_folder',
alpha_picture=0.25,
alpha_style=1.0,
picture_layer='block4_conv1',
style_layers=('block1_conv1',
'block2_conv1',
'block3_conv1',
'block4_conv1',
'block5_conv1'))
neural_styler.fit(canvas='random_from_style', optimization_method='CG')
(With different settings and optimization algorithms)
|
Cezanne |
Magritte |
|
Dalì |
Matisse |
|
Picasso |
Rembrandt |
|
De Chirico |
Mondrian |
|
Van Gogh |
Schiele |
I'm still working on some experiments based on loss function which tries to maximize the L2 norm of the last convolutional block (layers 1 and 2). I've excluded those from the style_layers tuple and tuned the parameters to render a "dream" together with a styled image. You can try the following snippet:
# Dream loss function
dream_loss_function = -5.0*K.sum(K.square(convnet.get_layer('block5_conv1').output)) + \
-2.5*K.sum(K.square(convnet.get_layer('block5_conv2').output))
# Composite loss function
composite_loss_function = (self.alpha_picture * picture_loss_function) + \
(self.alpha_style * style_loss_function) + \
dream_loss_function
The composite loss function isnt't "free" to maximize the norm like in Keras DeepDream, because the MSE with the gramian terms forces the filters to get similar to the style, however, it's possible to obtain interesting results. The following pictures show the famous Tübingen styled with a Braque painting and forced to render "random" elements (they're similar to animal heads and eyes) like in a dream:
|
|
|
This example, instead, has been created using a VGG19 with a Cezanne painting and:
style_layers=('block1_conv1',
'block2_conv1',
'block3_conv1',
'block4_conv1',
'block5_conv1',
'block5_conv2')
# Dream loss function
dream_loss_function = -10.0*K.sum(K.square(convnet.get_layer('block5_conv1').output)) + \
-5.0*K.sum(K.square(convnet.get_layer('block5_conv2').output))
(Original image by Manfred Brueckels - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=6937538)
|
|
|
- Python 2.7-3.5
- Keras
- Theano/Tensorflow
- SciPy
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent