Style Transfer for Videos

Using the Torch implementation by Justin Johnson, an algorithm was developed by Manuel Ruder, Alexey Dosovitskiy and Thomas Brox which facilitates a consistent, low-intensity stylistic transfer of an image to a video sequence. The artistic-videos algorithm requires the full installation of neural-style (Torch, loadcaffe, CUDA backend, cuDNN, neural-style). The painting style is transferred to an image sequence using DeepFlow, an algorithm for calculating the optical flow of images.

Installation is easy to perform. DeepFlow and DeepMatching can be downloaded as CPU versions here. Next, deepmatching-static and deepflow2-static are copied to the neural-style or artistic-videos main directory.

The calculation of the optical flow and the processing of the video clip are launched with the following command in a terminal window:

For details on this, see the corresponding GitHub page by Manuel Ruder. Alternatively, the stylizeVideo.sh file can be edited and launched. The script takes the following form for transferring the Caligari sequence:

Installation of neural-style
Working with neural-style
Style Transfer for Video Clips
Post-production
Alternative Techniques