Writing simple optical flow in python. Part 3

Today, my imaginary readers, we improve our optical flow dramatically. Lets see, what our algorithm produces for images that have more than 1-pixel shifts.

I0	I1
OF results:

You can see, that the algorithm cannot deal with too large displacements. Even with multiple warps.
Lets apply multiscale scheme now. We need to construct image pyramid:

Writing simple optical flow in python. Part 2

After fixing some errors, it seems my OF is working.

Image I0	Image I1
first iteration OF	1st iteration of warped I1
Second Iteration OF	Second iteration of warped I1
3 iteration Of	3rd iteration of warped I1
Last iteration

Writing simple optical flow in python

First of all we need a couple of test images:

#
import numpy
from StringIO import StringIO

I0 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0         0    0.5000         0         0
         0         0    1.0000         0         0
         0         0    0.5000         0         0
         0         0         0         0         0""") )


I1 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0       0.5         0         0           0
         0       1.0         0         0           0
         0       0.5         0         0           0
         0         0         0         0         0 """) )

Define initial horiozontal and vertical components of optical flow

u = numpy.zeros_like(I0); 
v = numpy.zeros_like(I0); 

Lets write class for making warps. As OF usually deals only with small displacements, we need iterative estimation: estimate, shift image by found vectors, find again.

Quiver for optical flow

Standard matlab's quiver function has axis origin in left bottom corner, however, images have origin in top left corner. To display optical flow vector field consistenly i use the following fucntion:

 
function [ output ] = quiver_flow( u, v )
%QUIVER_FLOW Displays quiver for optical flow 
%   SMatyunin2014

output = quiver( u, v, 0);
axis ij;
end

Simple tests of classic OF methods

BA method, simple synthetic images

For simple synthetic images:

I0 =

         0    0.5000         0
         0    1.0000         0
         0    0.5000         0
    0.1000         0         0
    0.0500         0         0
    0.0500    0.1000         0

I1 =

    0.5000         0         0
    1.0000         0         0
    0.5000         0         0
         0    0.1000         0
         0    0.0500         0
         0    0.0500    0.1000

Running code by D.Sun. Disabled texture decomposition, disabled multiscale processing.

uv = estimate_flow_interface(I0, I1, 'classic-c-brightness', [], {'display', 1, 'pyramid_levels', 1, 'gnc_pyramid_levels', 1});

Graduated non convexity scheme (GNC)

Optimization of energy terms can be difficult in OF, because of non-convexity and local optima.
Construct a series of energy functions





EQ is convex, quadratic
alpha changes from 1 to 0, so Energy Ec changes from quadratic to original.
for each alpha they find optimum through setting derivatives of Ec to 0.
Solution on each stage becomes initialization on the next one.


Proposed in: D. Sun, S. Roth, J. Lewis, and M. J. Black. Learning optical flow. In ECCV, volume 3, pages 83–97, 2008. [pdf]