how to apply inverse optical flow vector on an image? - matlab

Question

I have a small project on Moving object detection in moving camera in which i have to use negative optical flow vector to minimize ego motion compensation. I have a video and some particular consecutive frames in which average of negative optical flow vector has to be computed. I have already calculated Optical flow between say, (k-1)th and kth frame. Also, I have calculated average of optical flow vector V=[u,v], where v is the average of horizontal optical flow and u is the average of vertical flow. Now, I have to apply inverse of optical flow vector i.e., -V to the (k-1)th frame. I'm new to matlab and i don't know much about it. Please help
I have tried this code segment to do so but the results aren't as expected
function I1=reverseOF(I,V)
R=I(:,:,1);
G=I(:,:,2);
B=I(:,:,3);
[m,n]=size(rgb2gray(I));
for i=1:m
for j=1:n
v1=[j i];
v2=-V;
v3=v1.*v2;
R(floor(1+abs(v3(1,2))),floor(1+abs(v3(1,2))))=R(i,j);
G(floor(1+abs(v3(1,2))),floor(1+abs(v3(1,2))))=G(i,j);
B(floor(1+abs(v3(1,2))),floor(1+abs(v3(1,2))))=B(i,j);
I1(floor(1+abs(v3(1,2))),floor(1+abs(v3(1,2))))=I(i,j);
end
end
I1=cat(3,R,G,B);
enter code here
I have used abs() function because otherwise some error was occuring like "attempted to access negative location; index must be a positive or logical".
Image A and Image B are the images that i have used to estimate the optical flow.enter image description here
This is the result that i am obtaining after applying the above function.
enter image description here

Answer 1

Unfortunately, you cant do this easily. This is a quite advanced research problem, because obtaining the inverse of a vector field on a mesh grid is not an easy problem, actually its quite hard.
Notice that your vector field (optical flow) start in a mesh grid, but it doesnt end in a mesh grid, it ends in random subpixel positions. If you just invert this field, doing -V is not enough! The result wont be the inverse!
This is a open research problem, look for example at this 2010 paper that addresses exactly this issue, and proposes a method to create "pseudoinverses".
Suppose you have that inverse, because you computed it somehow. Your code is quite bad for it, and the solutions (abs!) are showing (no offense) that you are not really understanding what you are doing. For a known vector field {Vx,Vy}, size equals to the image size (if its not, you can figure out easily how to interpolate it unsig interp2 ) the code would look something like:
newimg=zeros(size(I));
[ix,iy]=meshgrid(1:size(I,1),1:size(I,2));
newimg(:,:,1)=interp2(I(:,:,1),ix+Vx,iy+Vy); % this is your whole loop.
newimg(:,:,2)=interp2(I(:,:,3),ix+Vx,iy+Vy); % this is your whole loop.
newimg(:,:,3)=interp2(I(:,:,2),ix+Vx,iy+Vy); % this is your whole loop.