Motion vectors between pairs of points - matlab

Question

I have a sequence of nFrames images from a video, and I have computed the SIFT keypoints and descriptors from all of them and stored them in two cell arrays. What I need to do now is to compute the vector between the 1st keypoint on image 1 and the corresponding keypoint on image 2 (the corresponding keypoint will be the one with the same -or most similar- descriptors), and do so for all other keypoints in order to plot a graph showing how the keypoints move on the scene, which reflects the movement of the objects on the scene.
The keypoints are stored as a 4-by-n array (the first row are the x components, the second row are the y components -the other 2 rows are the scale and angle, which I don't need-, and each column corresponds to a different keypoint), so I thought of subtracting the corresponding points from the first image to the second image, but wouldn't I end up having just another point in the middle of those two? How can I store the difference as a vector in order to plot all of them later on on the same graph?
At the moment all I have is this:
% Clear all and add all folders to the current path
clear; close all; clc;
addpath(genpath('.'));
% Set path, nFrames and threshold values
path = 'img/record_tennis';
d = dir([ path,'\*.png' ]);
nFrames = length( d( not([ d.isdir ]) ) );
th = 0.01;
step = 20;
keypts = cell(1,nFrames);
desc = cell(1,nFrames);
% Main loop
for i = 1:step:nFrames
disp([ 'Processing frame number ',num2str(i),' of ',num2str(nFrames),'...' ]);
% Read the current image
imgRGB = imread([ path,'/',d(i).name ]);
img = sum( double(imgRGB),3 ) / 3 / 255;
% Perform SIFT on the current image and plot the keypoints
[ keypts{1,i},desc{1,i} ] = sift( img,'Threshold',th );
imshow( img ), hold on
plot( keypts{1,i}(1,:),keypts{1,i}(2,:),'.' );
pause(1); clc;
end
% Remove empty cells
keypts = keypts( ~cellfun( #isempty,keypts ) );
desc = desc( ~cellfun( #isempty,desc ) );
With this I compute the SIFT keypoints and descriptors from each frame (I have added a step value because the movement is very small and this way it is more noticeable between consecutive images). Then I remove the empty cells due to the step factor, and now I have to compute the vector between corresponding keypoints, but I am stuck with that.

Answer 1

In general, if you want the vector that goes from one set of points to another, you can simply subtract the two (as you mentioned). This will provide you a vector that is [dx, dy]. In order to display that vector field, you could then use quiver which will draw each of these vectors starting from the initial point. The inputs to quiver are:
quiver(initial_x, initial_y, dx, dy, scale)
So if we look at your variables, we can construct each of these variables from your keypts variable.
initial_x = keypts{1}(1,:);
initial_y = keypts{1}(2,:);
dx = keypts{current_frame}(1,:) - initial_x;
dy = keypts{current_frame}(2,:) - initial_y;
Now you should be able to plot this on top of your image to visualize the motion field.
hax = axes();
imshow(img, 'Parent', hax);
hold(hax, 'on')
q = quiver(initial_x, initial_y, dx, dy, 1);
Now, this requires that the points within keypts actually have correspondence between different frames (i.e. keypts{1}(:,1) is the same physical point as keypts{2}(:,1)). This may or may not be true depending on the function that you are calling.