I'm trying to use reconstructScene to project my disparity map into a 3D point cloud. In the online documentation, this function takes a 4x4 reprojection_matrix as input, but
When I try to use a 4x4 matrix as input, I get the following error
Error using reconstructScene
Expected stereoParams to be one of these types:
stereoParameters
Instead its type was double.
Error in reconstructScene (line 74)
validateattributes(stereoParams, {'stereoParameters'}, {}, ...
Looking deeper, in my installation of MATLAB R2021b, the function has the following header.
% reconstructScene Reconstructs a 3-D scene from a disparity map.
% xyzPoints = reconstructScene(disparityMap, stereoParams) returns an
% M-by-N-by-3 array of [X,Y,Z] coordinates of world points corresponding
% to pixels in disparityMap, an M-by-N array of disparity values.
% stereoParams is a stereoParameters object. The 3-D world coordinates
% are relative to the optical center of camera 1 of the stereo system
% represented by stereoParams.
Okay, well then, I'll construct a stereoParameters object. To do so, I first need a cameraParameters. My attempt follows:
intrinsics = cameraIntrinsics([3999, 3988], [959.5, 599], [1920, 1199]); #focal length, camera center, and imager size
left_camera = cameraParameters('Intrinsics', intrinsics, 'TranslationVectors', [0, 0, 0], 'RotationMatrices', eye(3)); # My left camera coordinates are the same as the world coordinate system so my translation and rotation are unchanged.
The cameraParameters line produces the error
Error using cameraIntrinsics/cameraParameters
Cannot access method 'cameraParameters' in class 'cameraIntrinsics'.
TLDR; How can I run reconstructScene given that I have a disparity map and the camera intrinsics and extrinsics? How do I need to format my input?
Related
I am trying to find the threshold of a 3D image that is 258 x 318 x 801 double. I first reshaped the image into 1D array and then used graythresh
ROI = reshape(postImg,[],1);
thresh = graythresh(ROI);
But I was trying to find the actually intensity threshold instead of a value between 0 and 1. Is there a way to convert this other than using multithresh?
From MATLAB documentation:
The graythresh function converts multidimensional arrays to 2-D
arrays, using reshape, and ignores any nonzero imaginary part of I.
So, your reshape is probably redundant. I think this would do:
thresh = graythresh(postImg); % postIm can be 3D
BinIm = imbinarize(postIm,thresh); % creates a binary mask of your image
I am trying to implement brain tumor segmentation on 3D brain MRI(.mha data type).
After preliminary segmentation, I am applying 26-neighbor connected component algorithm(using bwconncomp) to obtain the largest connected component by obtaining the component with the largest volume, following which I need to calculate the centroid of the resultant component.
I am not sure if my method of calculating the largest connected component and the centroid is correct, because the centroid obtained and its nearby voxels all have value 0.
Also I am having confusion with the representation of 3D voxel coordinates. For eg. if centroid=(x,y,z), does it correspond to x=row,y=column and z=2D slice?
Any help would be appreciated. Below is my code with the relevant part.
CC=bwconncomp(Ibin,26); %Input Black & White 3D data of size 240x240x155
Pixelid=regionprops(CC,'PixelIdxList');
[prow pcol]=size(Pixelid);
maxval=numel(Pixelid(1).PixelIdxList);
index=1;
for i=1:prow
number=numel([Pixelid(i).PixelIdxList]);
if (number>maxval) %calculating the component with max number of voxels
maxval=number;
index=i;
end
end
for i=1:prow
if i~=index
Ibin(Pixelid(i).PixelIdxList)=0;
end
end
CC1=bwconncomp(Ibin,26);
Cent=regionprops(CC1,'Centroid');
I changed your code to the following:
CC=bwconncomp(Ibin,26);
PixelIdxList = CC.PixelIdxList;
maxval = numel(PixelIdxList{1});
index = 1;
for ii = 1:length(PixelIdxList)
number = numel(PixelIdxList{ii});
if number > maxval
maxval = number;
index = ii;
end
end
[y,x,z] = ind2sub(size(Ibin),PixelIdxList{index})
centroid = [mean(x), mean(y), mean(z)];
bwconncomp already gives you a PixelIdxList so you don't have to use regionprops. The PixelIdxList lists pixels by their linear indices, so you have to convert them into subscripts to get x, y, and z coordinates. The first dimension in MATLAB matrix represents y coordinates, and second dimension represents x, while the third dimension represents z. Centroid is calculated by taking the mean x, y, and z coordinates of all the pixels contained in the object.
This is the first time I do the image processing. So I have a lot of questions:
I have two pictures which are taken from different position, one from the left and the other one from the right like the picture below.[![enter image description here][1]][1]
Step 1: Read images by using imread function
I1 = imread('DSC01063.jpg');
I2 = imread('DSC01064.jpg');
Step 2: Using camera calibrator app in matlab to get the cameraParameters
load cameraParams.mat
Step 3: Remove Lens Distortion by using undistortImage function
[I1, newOrigin1] = undistortImage(I1, cameraParams, 'OutputView', 'same');
[I2, newOrigin2] = undistortImage(I2, cameraParams, 'OutputView', 'same');
Step 4: Detect feature points by using detectSURFFeatures function
imagePoints1 = detectSURFFeatures(rgb2gray(I1), 'MetricThreshold', 600);
imagePoints2 = detectSURFFeatures(rgb2gray(I2), 'MetricThreshold', 600);
Step 5: Extract feature descriptors by using extractFeatures function
features1 = extractFeatures(rgb2gray(I1), imagePoints1);
features2 = extractFeatures(rgb2gray(I2), imagePoints2);
Step 6: Match Features by using matchFeatures function
indexPairs = matchFeatures(features1, features2, 'MaxRatio', 1);
matchedPoints1 = imagePoints1(indexPairs(:, 1));
matchedPoints2 = imagePoints2(indexPairs(:, 2));
From there, how can I construct the 3D point cloud ??? In step 2, I used the checkerboard as in the picture attach to calibrate the camera[![enter image description here][2]][2]
The square size is 23 mm and from the cameraParams.mat I know the intrinsic matrix (or camera calibration matrix K) which has the form K=[alphax 0 x0; 0 alphay y0; 0 0 1].
I need to compute the Fundamental matrix F, Essential matrix E in order to calculate the camera matrices P1 and P2, right ???
After that when I have the camera matrices P1 and P2, I use the linear triangulation methods to estimate 3D point cloud. Is it the correct way??
I appreciate if you have any suggestion for me?
Thanks!
To triangulate the points you need the so called "camera matrices" and the points in 2D in each of the images (that you already have).
In Matlab you have the function triangulate, that does the job for you.
If you have calibrated the cameras, you shoudl have this information already. Anyways, you have here an example of how to create the "stereoParams" object needed for the triangulation.
Yes, that is the correct way. Now that you have matched points, you can use estimateFundamentalMatrix to compute the fundamental matrix F. Then you get the essential matrix E by multiplying F by extrinsics. Be careful about the order of multiplication, because the intrinsic matrix in cameraParameters is transposed relative to what you see in most textbooks.
Now, you have to decompose E into a rotation and a translation, from which you can construct the camera matrix for the second camera using cameraMatrix. You also need the camera matrix for the first camera, for which the rotation would be a 3x3 identity matrix, and translation will be a 3-element 0 vector.
Edit: there is now a cameraPose function in MATLAB, which computes an up-to-scale relative pose ('R' and 't') given the Fundamental matrix and the camera parameters.
I have two 3D arrays, one contains the 3D data of an electric field and the other is a 3D mask of the object that I am interested in, taking the product of these two arrays gives me an array with the electric field of just the mask. I have successfully created an isosurface of the mask but when I try to make an isosurface of the product of the mask and the electric field array I get the same graph as when I plot just the mask (which is just an array of zeroes and 1's). Is there a way to represent the data on the isosurface?
For the moment I simply put my arrays in to the isosurface function:
isosurface(mask), which gives me:
I then try to plot the product of my mask and data array:
isosurface(mask.*EArr) and I get the following:
But if I look at just a single slice using the code
imagesc(mask(:,:,35).*EArr(:,:,35)) I get:
What I'm looking to do is to get view the slices of the last picture as a 3D object similar to the way that I can view my mask as a 3D object in the first picture.
If I understand well, you want to plot your field on the surface of your mask.
To do so, get the patch returned by isosurface, then intrepolate your field on vertices of this patch. Finaly, plot it using the resulting interpolated data as 'FaceVertexCData'.
Here is an example with dummy data :
% PREPARATION
% ===========
% GENERATE A GRID
[X Y Z] = meshgrid([0:1/100:1],[0:1/100:1],[0:1/100:1]);
% GENERATE RANDOM DATA
DATA = zeros(101,101,101);
DATA(:) = interp3([0:1/10:1],[0:1/10:1],[0:1/10:1],rand(11,11,11),X(:),Y(:),Z(:),'cubic');
% GENERATE A RANDOM MASK
MASK = zeros(101,101,101);
MASK(:) = sqrt(sum([X(:)-0.5 Y(:)-0.5 Z(:)-0.5].^2,2)) - 0.3 - interp3([0:1/10:1],[0:1/10:1],[0:1/10:1],0.1*rand(11,11,11),X(:),Y(:),Z(:),'cubic');
%
% ACTUAL PROBLEM
% ==============
% EXTRACT THE MASK SURFACE
SURF = isosurface(X,Y,Z,MASK,0);
% INTERPOLATE DATA ON MASK SURFACE
DATA_SURF = interp3(X,Y,Z,DATA,SURF.vertices(:,1),SURF.vertices(:,2),SURF.vertices(:,3));
% PLOT THE MASK SURFACE AND DATA
hold on; axis square; axis([0 1 0 1 0 1]); view(3); camlight
patch('Faces',SURF.faces,'Vertices',SURF.vertices,'EdgeColor','none','FaceColor','interp','FaceVertexCData',DATA_SURF);
This gives things like that :
Try using isosurface(mask,th) where th is the value where the isosurface will be created. As I dont know the magnitude of your data I can not suggest you a value. Try different values, e.g. 0.01
I have a stereo camera system and I am trying this MATLAB's Computer Vision toolbox example (http://www.mathworks.com/help/vision/ug/sparse-3-d-reconstruction-from-multiple-views.html) with my own images and camera calibration files. I used Caltech's camera calibration toolbox (http://www.vision.caltech.edu/bouguetj/calib_doc/).
First I tried each camera separately based on first example and found the intrinsic camera calibration matrices for each camera and saved them. I also undistorted the left and right images using Caltech toolbox. Therefore I commented out the code for that from MATLAB example.
Here are the instrinsic camera matrices:
K1=[1050 0 630;0 1048 460;0 0 1];
K2=[1048 0 662;0 1047 468;0 0 1];
BTW, these are the right and center lenses from bumblebee XB3 cameras.
Question: aren't they supposed to be the same?
Then I did stereo calibration based on fifth example. I saved the rotation matrix (R) and translation matrix (T) from that. Therefore I commented out the code for that from MATLAB example.
Here are the rotation and translation matrices:
R=[0.9999 -0.0080 -0.0086;0.0080 1 0.0048;0.0086 -0.0049 1];
T=[120.14 0.55 1.04];
Then I fed all these images and calibration files and camera matrices to the MATLAB example and tried to find the 3-D point cloud but the results are not promising. I am attaching the code here. I think here are two problems:
1- My epipolar constraint value is too large!(to the power of 16)
2- I am not sure about the camera matrices and how I calculated them from R, and T from Caltech toolbox!
P.S. as far as feature extraction goes that is fine.
would be great if someone can help.
clear
close all
clc
files = {'Left1.tif';'Right1.tif'};
for i = 1:numel(files)
files{i}=fullfile('...\sparse_matlab', files{i});
images(i).image = imread(files{i});
end
figure;
montage(files); title('Pair of Original Images')
% Intrinsic camera parameters
load('Calib_Results_Left.mat')
K1 = KK;
load('Calib_Results_Right.mat')
K2 = KK;
%Extrinsics using stereo calibration
load('Calib_Results_stereo.mat')
Rotation=R;
Translation=T';
images(1).CameraMatrix=[Rotation; Translation] * K1;
images(2).CameraMatrix=[Rotation; Translation] * K2;
% Detect feature points and extract SURF descriptors in images
for i = 1:numel(images)
%detect SURF feature points
images(i).points = detectSURFFeatures(rgb2gray(images(i).image),...
'MetricThreshold',600);
%extract SURF descriptors
[images(i).featureVectors,images(i).points] = ...
extractFeatures(rgb2gray(images(i).image),images(i).points);
end
% Visualize several extracted SURF features from the Left image
figure; imshow(images(1).image);
title('1500 Strongest Feature Points from Globe01');
hold on;
plot(images(1).points.selectStrongest(1500));
indexPairs = ...
matchFeatures(images(1).featureVectors, images(2).featureVectors,...
'Prenormalized', true,'MaxRatio',0.4) ;
matchedPoints1 = images(1).points(indexPairs(:, 1));
matchedPoints2 = images(2).points(indexPairs(:, 2));
figure;
% Visualize correspondences
showMatchedFeatures(images(1).image,images(2).image,matchedPoints1,matchedPoints2,'montage' );
title('Original Matched Features from Globe01 and Globe02');
% Set a value near zero, It will be used to eliminate matches that
% correspond to points that do not lie on an epipolar line.
epipolarThreshold = .05;
for k = 1:length(matchedPoints1)
% Compute the fundamental matrix using the example helper function
% Evaluate the epipolar constraint
epipolarConstraint =[matchedPoints1.Location(k,:),1]...
*helperCameraMatricesToFMatrix(images(1).CameraMatrix,images(2).CameraMatrix)...
*[matchedPoints2.Location(k,:),1]';
%%%% here my epipolarConstraint results are bad %%%%%%%%%%%%%
% Only consider feature matches where the absolute value of the
% constraint expression is less than the threshold.
valid(k) = abs(epipolarConstraint) < epipolarThreshold;
end
validpts1 = images(1).points(indexPairs(valid, 1));
validpts2 = images(2).points(indexPairs(valid, 2));
figure;
showMatchedFeatures(images(1).image,images(2).image,validpts1,validpts2,'montage');
title('Matched Features After Applying Epipolar Constraint');
% convert image to double format for plotting
doubleimage = im2double(images(1).image);
points3D = ones(length(validpts1),4); % store homogeneous world coordinates
color = ones(length(validpts1),3); % store color information
% For all point correspondences
for i = 1:length(validpts1)
% For all image locations from a list of correspondences build an A
pointInImage1 = validpts1(i).Location;
pointInImage2 = validpts2(i).Location;
P1 = images(1).CameraMatrix'; % Transpose to match the convention in
P2 = images(2).CameraMatrix'; % in [1]
A = [
pointInImage1(1)*P1(3,:) - P1(1,:);...
pointInImage1(2)*P1(3,:) - P1(2,:);...
pointInImage2(1)*P2(3,:) - P2(1,:);...
pointInImage2(2)*P2(3,:) - P2(2,:)];
% Compute the 3-D location using the smallest singular value from the
% singular value decomposition of the matrix A
[~,~,V]=svd(A);
X = V(:,end);
X = X/X(end);
% Store location
points3D(i,:) = X';
% Store pixel color for visualization
y = round(pointInImage1(1));
x = round(pointInImage1(2));
color(i,:) = squeeze(doubleimage(x,y,:))';
end
% add green point representing the origin
points3D(end+1,:) = [0,0,0,1];
color(end+1,:) = [0,1,0];
% show images
figure('units','normalized','outerposition',[0 0 .5 .5])
subplot(1,2,1); montage(files,'Size',[1,2]); title('Original Images')
% plot point-cloud
hAxes = subplot(1,2,2); hold on; grid on;
scatter3(points3D(:,1),points3D(:,2),points3D(:,3),50,color,'fill')
xlabel('x-axis (mm)');ylabel('y-axis (mm)');zlabel('z-axis (mm)')
view(20,24);axis equal;axis vis3d
set(hAxes,'XAxisLocation','top','YAxisLocation','left',...
'ZDir','reverse','Ydir','reverse');
grid on
title('Reconstructed Point Cloud');
First of all, the Computer Vision System Toolbox now includes a Camera Calibrator App for calibrating a single camera, and also support for programmatic stereo camera calibration. It would be easier for you to use those tools, because the example you are using and the Caltech Calibration Toolbox use somewhat different conventions.
The example uses the pre-multiply convention, i.e. row vector * matrix, while the Caltech toolbox uses the post-multiply convention (matrix * column vector). That means that if you do use the camera parameters from Caltech, you would have to transpose the intrinsic matrix and the rotation matrices. That could be the main cause of your problems.
As far as the intrinsics being different between your two cameras, that is perfectly normal. All cameras are slightly different.
It would also help to see the matched features that you've used for triangulation. Given that you are reconstructing an elongated object, it doesn't seem too surprising to see the reconstructed points form a line in 3D...
You could also try rectifying the images and doing a dense reconstruction, as in the example I've linked to above.