Writing simple optical flow in python

First of all we need a couple of test images:

#
import numpy
from StringIO import StringIO

I0 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0         0    0.5000         0         0
         0         0    1.0000         0         0
         0         0    0.5000         0         0
         0         0         0         0         0""") )


I1 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0       0.5         0         0           0
         0       1.0         0         0           0
         0       0.5         0         0           0
         0         0         0         0         0 """) )

Define initial horiozontal and vertical components of optical flow

u = numpy.zeros_like(I0); 
v = numpy.zeros_like(I0); 

Lets write class for making warps. As OF usually deals only with small displacements, we need iterative estimation: estimate, shift image by found vectors, find again.

Setting default parameters from imshow in pyplot

import matplotlib.pyplot as plt
plt.rcParams['image.cmap'] = 'gray'
plt.rcParams['image.interpolation'] = 'nearest'

Loading numpy array from string

Okay children, today we learn how to convert text to numpy matrix.
Source is here.

# loading modules
import numpy
from StringIO import StringIO

# Using StringIO as a file-like wrapper over text
I0 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0         0    0.5000         0         0
         0         0    1.0000         0         0
         0         0    0.5000         0         0
         0    1.0000         0         0         0
         0    0.5000         0         0         0
         0    0.5000    1.0000         0         0
         0         0         0         0         0""") )


I1 = numpy.loadtxt(StringIO("""
         0         0         0         0         0
         0    0.5000         0         0         0
         0    1.0000         0         0         0
         0    0.5000         0         0         0
         0         0    1.0000         0         0
         0         0    0.5000         0         0
         0         0    0.5000    1.0000         0
         0         0         0         0         0 """) )

Quiver for optical flow

Standard matlab's quiver function has axis origin in left bottom corner, however, images have origin in top left corner. To display optical flow vector field consistenly i use the following fucntion:

 
function [ output ] = quiver_flow( u, v )
%QUIVER_FLOW Displays quiver for optical flow 
%   SMatyunin2014

output = quiver( u, v, 0);
axis ij;
end

Simple tests of classic OF methods

BA method, simple synthetic images

For simple synthetic images:

I0 =

         0    0.5000         0
         0    1.0000         0
         0    0.5000         0
    0.1000         0         0
    0.0500         0         0
    0.0500    0.1000         0

I1 =

    0.5000         0         0
    1.0000         0         0
    0.5000         0         0
         0    0.1000         0
         0    0.0500         0
         0    0.0500    0.1000

Running code by D.Sun. Disabled texture decomposition, disabled multiscale processing.

uv = estimate_flow_interface(I0, I1, 'classic-c-brightness', [], {'display', 1, 'pyramid_levels', 1, 'gnc_pyramid_levels', 1});

Graduated non convexity scheme (GNC)

Optimization of energy terms can be difficult in OF, because of non-convexity and local optima.
Construct a series of energy functions





EQ is convex, quadratic
alpha changes from 1 to 0, so Energy Ec changes from quadratic to original.
for each alpha they find optimum through setting derivatives of Ec to 0.
Solution on each stage becomes initialization on the next one.


Proposed in: D. Sun, S. Roth, J. Lewis, and M. J. Black. Learning optical flow. In ECCV, volume 3, pages 83–97, 2008. [pdf]

Investigating optical flow by D. Sun (Secrets of optical flow)

Personal page of the author: http://cs.brown.edu/~dqsun/research/index.html
Original paper: http://cs.brown.edu/~dqsun/pubs/cvpr_2010_flow.pdf
Newer paper: Deqing Sun, Stefan Roth, and Michael J. Black. "A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them". International Journal of Computer Vision (IJCV), 2013 [pdf] [Source code]

Look inside the sources:

Code highlight in blogger

Source

place following code before your <head> tag in the HTML of your blogger template: