Related
I am trying to achieve 3d reconstruction from 2 images. Steps I followed are,
1. Found corresponding points between 2 images using SURF.
2. Implemented eight point algo to find "Fundamental matrix"
3. Then, I implemented triangulation.
I have got Fundamental matrix and results of triangulation till now. How do i proceed further to get 3d reconstruction? I'm confused reading all the material available on internet.
Also, This is code. Let me know if this is correct or not.
Ia=imread('1.jpg');
Ib=imread('2.jpg');
Ia=rgb2gray(Ia);
Ib=rgb2gray(Ib);
% My surf addition
% collect Interest Points from Each Image
blobs1 = detectSURFFeatures(Ia);
blobs2 = detectSURFFeatures(Ib);
figure;
imshow(Ia);
hold on;
plot(selectStrongest(blobs1, 36));
figure;
imshow(Ib);
hold on;
plot(selectStrongest(blobs2, 36));
title('Thirty strongest SURF features in I2');
[features1, validBlobs1] = extractFeatures(Ia, blobs1);
[features2, validBlobs2] = extractFeatures(Ib, blobs2);
indexPairs = matchFeatures(features1, features2);
matchedPoints1 = validBlobs1(indexPairs(:,1),:);
matchedPoints2 = validBlobs2(indexPairs(:,2),:);
figure;
showMatchedFeatures(Ia, Ib, matchedPoints1, matchedPoints2);
legend('Putatively matched points in I1', 'Putatively matched points in I2');
for i=1:matchedPoints1.Count
xa(i,:)=matchedPoints1.Location(i);
ya(i,:)=matchedPoints1.Location(i,2);
xb(i,:)=matchedPoints2.Location(i);
yb(i,:)=matchedPoints2.Location(i,2);
end
matchedPoints1.Count
figure(1) ; clf ;
imshow(cat(2, Ia, Ib)) ;
axis image off ;
hold on ;
xbb=xb+size(Ia,2);
set=[1:matchedPoints1.Count];
h = line([xa(set)' ; xbb(set)'], [ya(set)' ; yb(set)']) ;
pts1=[xa,ya];
pts2=[xb,yb];
pts11=pts1;pts11(:,3)=1;
pts11=pts11';
pts22=pts2;pts22(:,3)=1;pts22=pts22';
width=size(Ia,2);
height=size(Ib,1);
F=eightpoint(pts1,pts2,width,height);
[P1new,P2new]=compute2Pmatrix(F);
XP = triangulate(pts11, pts22,P2new);
eightpoint()
function [ F ] = eightpoint( pts1, pts2,width,height)
X = 1:width;
Y = 1:height;
[X, Y] = meshgrid(X, Y);
x0 = [mean(X(:)); mean(Y(:))];
X = X - x0(1);
Y = Y - x0(2);
denom = sqrt(mean(mean(X.^2+Y.^2)));
N = size(pts1, 1);
%Normalized data
T = sqrt(2)/denom*[1 0 -x0(1); 0 1 -x0(2); 0 0 denom/sqrt(2)];
norm_x = T*[pts1(:,1)'; pts1(:,2)'; ones(1, N)];
norm_x_ = T*[pts2(:,1)';pts2(:,2)'; ones(1, N)];
x1 = norm_x(1, :)';
y1= norm_x(2, :)';
x2 = norm_x_(1, :)';
y2 = norm_x_(2, :)';
A = [x1.*x2, y1.*x2, x2, ...
x1.*y2, y1.*y2, y2, ...
x1, y1, ones(N,1)];
% compute the SVD
[~, ~, V] = svd(A);
F = reshape(V(:,9), 3, 3)';
[FU, FS, FV] = svd(F);
FS(3,3) = 0; %rank 2 constrains
F = FU*FS*FV';
% rescale fundamental matrix
F = T' * F * T;
end
triangulate()
function [ XP ] = triangulate( pts1,pts2,P2 )
n=size(pts1,2);
X=zeros(4,n);
for i=1:n
A=[-1,0,pts1(1,i),0;
0,-1,pts1(2,i),0;
pts2(1,i)*P2(3,:)-P2(1,:);
pts2(2,i)*P2(3,:)-P2(2,:)];
[~,~,va] = svd(A);
X(:,i) = va(:,4);
end
XP(:,:,1) = [X(1,:)./X(4,:);X(2,:)./X(4,:);X(3,:)./X(4,:); X(4,:)./X(4,:)];
end
function [ P1,P2 ] = compute2Pmatrix( F )
P1=[1,0,0,0;0,1,0,0;0,0,1,0];
[~, ~, V] = svd(F');
ep = V(:,3)/V(3,3);
P2 = [skew(ep)*F,ep];
end
From a quick look, it looks correct. Some notes are as follows:
You normalized code in eightpoint() is no ideal.
It is best done on the points involved. Each set of points will have its scaling matrix. That is:
[pts1_n, T1] = normalize_pts(pts1);
[pts2_n, T2] = normalize-pts(pts2);
% ... code
% solution
F = T2' * F * T
As a side note (for efficiency) you should do
[~,~,V] = svd(A, 0);
You also want to enforce the constraint that the fundamental matrix has rank-2. After you compute F, you can do:
[U,D,v] = svd(F);
F = U * diag([D(1,1),D(2,2), 0]) * V';
In either case, normalization is not the only key to make the algorithm work. You'll want to wrap the estimation of the fundamental matrix in a robust estimation scheme like RANSAC.
Estimation problems like this are very sensitive to non Gaussian noise and outliers. If you have a small number of wrong correspondence, or points with high error, the algorithm will break.
Finally, In 'triangulate' you want to make sure that the points are not at infinity prior to the homogeneous division.
I'd recommend testing the code with 'synthetic' data. That is, generate your own camera matrices and correspondences. Feed them to the estimate routine with varying levels of noise. With zero noise, you should get an exact solution up to floating point accuracy. As you increase the noise, your estimation error increases.
In its current form, running this on real data will probably not do well unless you 'robustify' the algorithm with RANSAC, or some other robust estimator.
Good luck.
Good luck.
Which version of MATLAB do you have?
There is a function called estimateFundamentalMatrix in the Computer Vision System Toolbox, which will give you the fundamental matrix. It may give you better results than your code, because it is using RANSAC under the hood, which makes it robust to spurious matches. There is also a triangulate function, as of version R2014b.
What you are getting is sparse 3D reconstruction. You can plot the resulting 3D points, and you can map the color of the corresponding pixel to each one. However, for what you want, you would have to fit a surface or a triangular mesh to the points. Unfortunately, I can't help you there.
If what you're asking is how to I proceed from fundamental Matrix + corresponding points to a dense model then you still have a lot of work ahead of you.
relative camera locations (R,T) can be calculated from a fundamental matrix assuming you know the internal camera params (up to scale, rotation, translation). To get a full dense matrix there are a few ways to go. you can try using an existing library (PMVS for example). I'd look into OpenMVG but I'm not sure about matlab interface.
Another way to go, you can compute a dense optical flow (many available for matlab). Look for a epipolar OF (It takes a fundamental matrix and restricts the solution to lie on the epipolar lines). Then you can triangulate every pixel to get a depthmap.
Finally you will have to play with format conversions to get from a depthmap to VRML (You can look at meshlab)
Sorry my answer isn't more Matlab oriented.
Can you please help me document this Matlab code that is supposed to produce random shapes?? The wiggliness of the shapes is supposed to be controlled by the variable degree...
But how the rho (radius values) are produced... I can't really get it....
degree = 5;
numPoints = 1000;
blobWidth = 5;
theta = 0:(2*pi)/(numPoints-1):2*pi;
coeffs = rand(degree,1);
rho = zeros(size(theta));
for i = 1:degree
rho = rho + coeffs(i)*sin(i*theta);
end
phase = rand*2*pi;
[x,y] = pol2cart(theta+phase, rho+blobWidth);
plot(x,y)
axis equal
set(gca,'Visible','off')
theta = 0:(2*pi)/(numPoints-1):2*pi;
So this is just a vector of angles in a revolution, if you plot this theta against a constant rho (after calling pol2cart) you will get a circle:
r = ones(size(theta));
[x,y] = pol2cart(theta, r);
plot(x,y)
axis equal
This should be obvious if you understand what pol2cart does because you have a series of all the angles in a circle and a constant radius for all of them. If you don't understand that (i.e. polar coordinates) then that's a very basic mathematical concept you need to go and read up on your own before trying to understand this code.
OK so now a circle in cartesian coords is just a line in polar coords (i.e. plot(theta, r) noting that the horizontal axis now represents angle and the vertical represents radius). So if we want to randomly mess up our circle, we could randomly mess up our line. Using sin does this in a nice smooth way. Adding random frequencies of many sin waves adds less and less predictable "jitter". I think it would help you to understand if you add the following line to your code:
rho = zeros(size(theta));
hold all
for i = 1:degree
rho = rho + coeffs(i)*sin(i*theta);
plot(theta, rho)
end
and contrast this to (be sure to close your figure window before running this)
rho = zeros(size(theta));
hold all
for i = 1:degree
rho = rho + coeffs(i)*sin(i*theta);
plot(theta, coeffs(i)*sin(i*theta))
end
The second one shows you the different frequencies of sin waves used and the first shows how these sum to create unpredictable wavy lines. Now think of the pol2rect function as bending these lines around to make a "circle". If the line is dead straight you get a perfect circle, if it's wavy you get a "wavy" circle.
degree in your code just controls how many sin waves to add up.
finally phase = rand*2*pi; just randomly rotates your shape after it has been created.
Well, this was an amazing piece of code! But regarding rho. What is done is that you have a circle with base radius of 5 (blobwidth) and then you have a random offset coeffs. Then the offset is added to rho in rho = rho + coeffs(i)*sin(i*theta);. This means that the first loop an offset is added to the circle with frequency 1Hz. This then yields a constant offset. The next loop the frequency increases to 2Hz. Then the offset will be added to every second point and the offset may be negative as well. Then it goes on like this. Finally the coordinate is transformed to polar.
A few comments though. The most readable and the easiest way to create theta is to use linspace. And also, since rho is overwritten in the loop, you may as well define it just as rho = 0;
my question is quite trivial, but I'm looking for the vectorized form of it.
My code is:
HubHt = 110; % Hub Height
GridWidth = 150; % Grid length along Y axis
GridHeight = 150; % Grid length along Z axis
RotorDiameter = min(GridWidth,GridHeight); % Turbine Diameter
Ny = 31;
Nz = 45;
%% GRID DEFINITION
dy = GridWidth/(Ny-1);
dz = GridHeight/(Nz-1);
if isequal(mod(Ny,2),0)
iky = [(-Ny/2:-1) (1:Ny/2)];
else
iky = -floor(Ny/2):ceil(Ny/2-1);
end
if isequal(mod(Nz,2),0)
ikz = [(-Nz/2:-1) (1:Nz/2)];
else
ikz = -floor(Nz/2):ceil(Nz/2-1);
end
[Y Z] = ndgrid(iky*dy,ikz*dz + HubHt);
EDIT
Currently I am using this solution, which has reasonable performances:
coord(:,1) = reshape(Y,[numel(Y),1]);
coord(:,2) = reshape(Z,[numel(Z),1]);
dist_y = bsxfun(#minus,coord(:,1),coord(:,1)');
dist_z = bsxfun(#minus,coord(:,2),coord(:,2)');
dist = sqrt(dist_y.^2 + dist_z.^2);
I disagree with Dan and Tal.
I believe you should use pdist rather than pdist2.
D = pdist( [Y(:) Z(:)] ); % a compact form
D = squareform( D ); % square m*n x m*n distances.
I agree with Tal Darom, pdist2 is exactly the function you need. It finds the distance for each pair of coordinates specified in two vectors and NOT the distance between two matrices.
So I'm pretty sure in your case you want this:
pdist2([Y(:), Z(:)], [Y(:), Z(:)])
The matrix [Y(:), Z(:)] is a list of every possible coordinate combination over the 2D space defined by Y-Z. If you want a matrix containing the distance from each point to each other point then you must call pdist2 on this matrix with itself. The result is a 2D matrix with dimensions numel(Y) x numel(Y) and although you haven't defined it I'm pretty sure that both Y and Z are n*m matrices meaning numel(Y) == n*m
EDIT:
A more correct solution suggested by #Shai is just to use pdist since we are comparing points within the same matrix:
pdist([Y(:), Z(:)])
You can use the matlab function pdist2 (I think it is in the statistics toolbox) or you can search online for open source good implementations of this function.
Also,
look at this unswer: pdist2 equivalent in MATLAB version 7
I'm going to start off by stating that, yes, this is homework (my first homework question on stackoverflow!). But I don't want you to solve it for me, I just want some guidance!
The equation in question is this:
I'm told to take N = 50, phi1 = 300, phi2 = 400, 0<=x<=1, and 0<=y<=1, and to let x and y be vectors of 100 equally spaced points, including the end points.
So the first thing I did was set those variables, and used x = linspace(0,1) and y = linspace(0,1) to make the correct vectors.
The question is Write a MATLAB script file called potential.m which calculates phi(x,y) and makes a filled contour plot versus x and y using the built-in function contourf (see the help command in MATLAB for examples). Make sure the figure is labeled properly. (Hint: the top and bottom portions of your domain should be hotter at about 400 degrees versus the left and right sides which should be at 300 degrees).
However, previously, I've calculated phi using either x or y as a constant. How am I supposed to calculate it where both are variables? Do I hold x steady, while running through every number in the vector of y, assigning that to a matrix, incrementing x to the next number in its vector after running through every value of y again and again? And then doing the same process, but slowly incrementing y instead?
If so, I've been using a loop that increments to the next row every time it loops through all 100 values. If I did it that way, I would end up with a massive matrix that has 200 rows and 100 columns. How would I use that in the linspace function?
If that's correct, this is how I'm finding my matrix:
clear
clc
format compact
x = linspace(0,1);
y = linspace(0,1);
N = 50;
phi1 = 300;
phi2 = 400;
phi = 0;
sum = 0;
for j = 1:100
for i = 1:100
for n = 1:N
sum = sum + ((2/(n*pi))*(((phi2-phi1)*(cos(n*pi)-1))/((exp(n*pi))-(exp(-n*pi))))*((1-(exp(-n*pi)))*(exp(n*pi*y(i)))+((exp(n*pi))-1)*(exp(-n*pi*y(i))))*sin(n*pi*x(j)));
end
phi(j,i) = phi1 - sum;
end
end
for j = 1:100
for i = 1:100
for n = 1:N
sum = sum + ((2/(n*pi))*(((phi2-phi1)*(cos(n*pi)-1))/((exp(n*pi))-(exp(-n*pi))))*((1-(exp(-n*pi)))*(exp(n*pi*y(j)))+((exp(n*pi))-1)*(exp(-n*pi*y(j))))*sin(n*pi*x(i)));
end
phi(j+100,i) = phi1 - sum;
end
end
This is the definition of contourf. I think I have to use contourf(X,Y,Z):
contourf(X,Y,Z), contourf(X,Y,Z,n), and contourf(X,Y,Z,v) draw filled contour plots of Z using X and Y to determine the x- and y-axis limits. When X and Y are matrices, they must be the same size as Z and must be monotonically increasing.
Here is the new code:
N = 50;
phi1 = 300;
phi2 = 400;
[x, y, n] = meshgrid(linspace(0,1),linspace(0,1),1:N)
f = phi1-((2./(n.*pi)).*(((phi2-phi1).*(cos(n.*pi)-1))./((exp(n.*pi))-(exp(-n.*pi)))).*((1-(exp(-1.*n.*pi))).*(exp(n.*pi.*y))+((exp(n.*pi))-1).*(exp(-1.*n.*pi.*y))).*sin(n.*pi.*x));
g = sum(f,3);
[x1,y1] = meshgrid(linspace(0,1),linspace(0,1));
contourf(x1,y1,g)
Vectorize the code. For example you can write f(x,y,n) with:
[x y n] = meshgrid(-1:0.1:1,-1:0.1:1,1:10);
f=exp(x.^2-y.^2).*n ;
f is a 3D matrix now just sum over the right dimension...
g=sum(f,3);
in order to use contourf, we'll take only the 2D part of x,y:
[x1 y1] = meshgrid(-1:0.1:1,-1:0.1:1);
contourf(x1,y1,g)
The reason your code takes so long to calculate the phi matrix is that you didn't pre-allocate the array. The error about size happens because phi is not 100x100. But instead of fixing those things, there's an even better way...
MATLAB is a MATrix LABoratory so this type of equation is pretty easy to compute using matrix operations. Hints:
Instead of looping over the values, rows, or columns of x and y, construct matrices to represent all the possible input combinations. Check out meshgrid for this.
You're still going to need a loop to sum over n = 1:N. But for each value of n, you can evaluate your equation for all x's and y's at once (using the matrices from hint 1). The key to making this work is using element-by-element operators, such as .* and ./.
Using matrix operations like this is The Matlab Way. Learn it and love it. (And get frustrated when using most other languages that don't have them.)
Good luck with your homework!
I have a simple loglog curve as above. Is there some function in Matlab which can fit this curve by segmented lines and show the starting and end points of these line segments ? I have checked the curve fitting toolbox in matlab. They seems to do curve fitting by either one line or some functions. I do not want to curve fitting by one line only.
If there is no direct function, any alternative to achieve the same goal is fine with me. My goal is to fit the curve by segmented lines and get locations of the end points of these segments .
First of all, your problem is not called curve fitting. Curve fitting is when you have data, and you find the best function that describes it, in some sense. You, on the other hand, want to create a piecewise linear approximation of your function.
I suggest the following strategy:
Split manually into sections. The section size should depend on the derivative, large derivative -> small section
Sample the function at the nodes between the sections
Find a linear interpolation that passes through the points mentioned above.
Here is an example of a code that does that. You can see that the red line (interpolation) is very close to the original function, despite the small amount of sections. This happens due to the adaptive section size.
function fitLogLog()
x = 2:1000;
y = log(log(x));
%# Find section sizes, by using an inverse of the approximation of the derivative
numOfSections = 20;
indexes = round(linspace(1,numel(y),numOfSections));
derivativeApprox = diff(y(indexes));
inverseDerivative = 1./derivativeApprox;
weightOfSection = inverseDerivative/sum(inverseDerivative);
totalRange = max(x(:))-min(x(:));
sectionSize = weightOfSection.* totalRange;
%# The relevant nodes
xNodes = x(1) + [ 0 cumsum(sectionSize)];
yNodes = log(log(xNodes));
figure;plot(x,y);
hold on;
plot (xNodes,yNodes,'r');
scatter (xNodes,yNodes,'r');
legend('log(log(x))','adaptive linear interpolation');
end
Andrey's adaptive solution provides a more accurate overall fit. If what you want is segments of a fixed length, however, then here is something that should work, using a method that also returns a complete set of all the fitted values. Could be vectorized if speed is needed.
Nsamp = 1000; %number of data samples on x-axis
x = [1:Nsamp]; %this is your x-axis
Nlines = 5; %number of lines to fit
fx = exp(-10*x/Nsamp); %generate something like your current data, f(x)
gx = NaN(size(fx)); %this will hold your fitted lines, g(x)
joins = round(linspace(1, Nsamp, Nlines+1)); %define equally spaced breaks along the x-axis
dx = diff(x(joins)); %x-change
df = diff(fx(joins)); %f(x)-change
m = df./dx; %gradient for each section
for i = 1:Nlines
x1 = joins(i); %start point
x2 = joins(i+1); %end point
gx(x1:x2) = fx(x1) + m(i)*(0:dx(i)); %compute line segment
end
subplot(2,1,1)
h(1,:) = plot(x, fx, 'b', x, gx, 'k', joins, gx(joins), 'ro');
title('Normal Plot')
subplot(2,1,2)
h(2,:) = loglog(x, fx, 'b', x, gx, 'k', joins, gx(joins), 'ro');
title('Log Log Plot')
for ip = 1:2
subplot(2,1,ip)
set(h(ip,:), 'LineWidth', 2)
legend('Data', 'Piecewise Linear', 'Location', 'NorthEastOutside')
legend boxoff
end
This is not an exact answer to this question, but since I arrived here based on a search, I'd like to answer the related question of how to create (not fit) a piecewise linear function that is intended to represent the mean (or median, or some other other function) of interval data in a scatter plot.
First, a related but more sophisticated alternative using regression, which apparently has some MATLAB code listed on the wikipedia page, is Multivariate adaptive regression splines.
The solution here is to just calculate the mean on overlapping intervals to get points
function [x, y] = intervalAggregate(Xdata, Ydata, aggFun, intStep, intOverlap)
% intOverlap in [0, 1); 0 for no overlap of intervals, etc.
% intStep this is the size of the interval being aggregated.
minX = min(Xdata);
maxX = max(Xdata);
minY = min(Ydata);
maxY = max(Ydata);
intInc = intOverlap*intStep; %How far we advance each iteraction.
if intOverlap <= 0
intInc = intStep;
end
nInt = ceil((maxX-minX)/intInc); %Number of aggregations
parfor i = 1:nInt
xStart = minX + (i-1)*intInc;
xEnd = xStart + intStep;
intervalIndices = find((Xdata >= xStart) & (Xdata <= xEnd));
x(i) = aggFun(Xdata(intervalIndices));
y(i) = aggFun(Ydata(intervalIndices));
end
For instance, to calculate the mean over some paired X and Y data I had handy with intervals of length 0.1 having roughly 1/3 overlap with each other (see scatter image):
[x,y] = intervalAggregate(Xdat, Ydat, #mean, 0.1, 0.333)
x =
Columns 1 through 8
0.0552 0.0868 0.1170 0.1475 0.1844 0.2173 0.2498 0.2834
Columns 9 through 15
0.3182 0.3561 0.3875 0.4178 0.4494 0.4671 0.4822
y =
Columns 1 through 8
0.9992 0.9983 0.9971 0.9955 0.9927 0.9905 0.9876 0.9846
Columns 9 through 15
0.9803 0.9750 0.9707 0.9653 0.9598 0.9560 0.9537
We see that as x increases, y tends to decrease slightly. From there, it is easy enough to draw line segments and/or perform some other kind of smoothing.
(Note that I did not attempt to vectorize this solution; a much faster version could be assumed if Xdata is sorted.)