Naive Background Style Transfer

Naive Background Style Transfer implemented using Keras and TensorFlow by Vikram Shenoy.

Overview

Naive background style transfer performs style transfer only on the background of the given content image. This naive approach uses two networks to reproduce this style transfer. The DeepLabv3+ model generates a segmentation map of the image which is processed to create a binary mask highlighting the foreground from the background. The style transfer model uses this segmented mask to guide the stylized pixels only onto the background of the given content image. For more details, check out the report.

The above architecture shows how the algorithm works and creates the resulting output image.

The picture below is of me taken on my trip to Amsterdam.

The picture displayed here is the result of Naive Background Style Transfer in the style of one of my favorite pieces of art, The Starry Night by Vincent Van Gogh.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

For using Naive Background Style Transfer, you need to install Keras, TensorFlow, PIL, and Imageio.

pip install keras
pip install tensorflow
pip install Pillow
pip install imageio

Images

Content Image	Style Image

Run

Run the script main.py in the terminal as follows.

Python main.py

Results

The final output is stored in Output Images.

Intermediate Stages of Style Transfer

Here is the generated image through different intervals of the run.

Transition through epochs

Result of Style Transfer

Ablation Study of Hyperparameters

This section takes a closer look at the hyperparameters of NBST, mainly the content weight and the style weight.

Default Parameters

• Content Weight = 1e3

• Style Weight = 1e-2

High Content Weight

• Content Weight = 1e10

• Style Weight = 1e-2

High Style Weight

• Content Weight = 1e3

• Style Weight = 1e15

Built With

• Keras - Deep Learning Framework
• TensorFlow - Deep Learning Framework
• Google Colab - Cloud Service

Authors

• Vikram Shenoy - Initial work - Vikram Shenoy

Acknowledgments

• Leon A. Gatys's paper, A Neural Algorithm of Artistic Style
• Liang-Chieh Chen's paper, Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Project makes use of the Pre-trained DeepLabv3+ Model by Google
• Content Image: Scraped from Google Image Search, Aila Images
• Style Image: The Scream by Edvard Munch, The Scream
• Style Image: Starry Night by Vincent Van Gogh, The Starry Night