Image&video processing, programming etc: 2014

Installing theano on windows

0. install Anaconda
1. download Theano sources from git (install it using setup.py)
2. Setup NVIDIA GPU Toolkit. I have installed version 6.5
3. Setup Visual Studio Community Edition 2013
4. Create config file .theanorc in c:\Users\X\:

[global]
floatX = float32
device = gpu

[nvcc]
fastmath = True
compiler_bindir=C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl.exe

Without the line with path to cl.exe I got:

nvcc fatal   : Cannot find compiler 'cl.exe' in PATH
['nvcc', '-shared', '-O3', '-use_fast_math', '-Xlinker', '/DEBUG',

'-D HAVE_ROUND', '-m64', '-Xcompiler',

 '-DCUDA_NDARRAY_CUH=f411a53ee0a470fbaad3b5c4a681ef64,-D

NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,/Zi,/MD',

'-Ic:\\anaconda\\theano\\theano\\sandbox\\cuda',

'-Ic:\\anaconda\\lib\\site-packages\\numpy\\core\\include',

'-Ic:\\anaconda\\include', '-o',

'C:\\Users\\X\\AppData\\Local\\Theano\\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_42_Stepping_7_GenuineIntel-2.7.7-64\\cuda_ndarray\\cuda_ndarray.pyd',

'mod.cu', '-Lc:\\anaconda\\libs',

'-Lc:\\anaconda', '-lpython27', '-lcublas', '-lcudart']

Writing simple optical flow in python. Part 3

Today, my imaginary readers, we improve our optical flow dramatically. Lets see, what our algorithm produces for images that have more than 1-pixel shifts.

I0	I1
OF results:

You can see, that the algorithm cannot deal with too large displacements. Even with multiple warps.
Lets apply multiscale scheme now. We need to construct image pyramid:

Writing simple optical flow in python. Part 2

After fixing some errors, it seems my OF is working.

Image I0	Image I1
first iteration OF	1st iteration of warped I1
Second Iteration OF	Second iteration of warped I1
3 iteration Of	3rd iteration of warped I1
Last iteration

Writing simple optical flow in python

First of all we need a couple of test images:

#
import numpy
from StringIO import StringIO

I0 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0         0    0.5000         0         0
         0         0    1.0000         0         0
         0         0    0.5000         0         0
         0         0         0         0         0""") )


I1 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0       0.5         0         0           0
         0       1.0         0         0           0
         0       0.5         0         0           0
         0         0         0         0         0 """) )

Define initial horiozontal and vertical components of optical flow

u = numpy.zeros_like(I0); 
v = numpy.zeros_like(I0);

Lets write class for making warps. As OF usually deals only with small displacements, we need iterative estimation: estimate, shift image by found vectors, find again.

import matplotlib.pyplot as plt
plt.rcParams['image.cmap'] = 'gray'
plt.rcParams['image.interpolation'] = 'nearest'

Loading numpy array from string

Okay children, today we learn how to convert text to numpy matrix.
Source is here.

# loading modules
import numpy
from StringIO import StringIO

# Using StringIO as a file-like wrapper over text
I0 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0         0    0.5000         0         0
         0         0    1.0000         0         0
         0         0    0.5000         0         0
         0    1.0000         0         0         0
         0    0.5000         0         0         0
         0    0.5000    1.0000         0         0
         0         0         0         0         0""") )


I1 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0    0.5000         0         0         0
         0    1.0000         0         0         0
         0    0.5000         0         0         0
         0         0    1.0000         0         0
         0         0    0.5000         0         0
         0         0    0.5000    1.0000         0
         0         0         0         0         0 """) )

Quiver for optical flow

Standard matlab's quiver function has axis origin in left bottom corner, however, images have origin in top left corner. To display optical flow vector field consistenly i use the following fucntion:

 
function [ output ] = quiver_flow( u, v )
%QUIVER_FLOW Displays quiver for optical flow 
%   SMatyunin2014

output = quiver( u, v, 0);
axis ij;
end

Simple tests of classic OF methods

BA method, simple synthetic images

For simple synthetic images:

I0 =

         0    0.5000         0
         0    1.0000         0
         0    0.5000         0
    0.1000         0         0
    0.0500         0         0
    0.0500    0.1000         0

I1 =

    0.5000         0         0
    1.0000         0         0
    0.5000         0         0
         0    0.1000         0
         0    0.0500         0
         0    0.0500    0.1000

Running code by D.Sun. Disabled texture decomposition, disabled multiscale processing.

uv = estimate_flow_interface(I0, I1, 'classic-c-brightness', [], {'display', 1, 'pyramid_levels', 1, 'gnc_pyramid_levels', 1});

Graduated non convexity scheme (GNC)

Optimization of energy terms can be difficult in OF, because of non-convexity and local optima.
Construct a series of energy functions

EQ is convex, quadratic
alpha changes from 1 to 0, so Energy Ec changes from quadratic to original.
for each alpha they find optimum through setting derivatives of Ec to 0.
Solution on each stage becomes initialization on the next one.

Proposed in: D. Sun, S. Roth, J. Lewis, and M. J. Black. Learning optical flow. In ECCV, volume 3, pages 83–97, 2008. [pdf]