I wanted to rectify a stereo image pair coming from two different modalities (visual and thermal). I calibrated both cameras using
[cameraParams,imagesUsed,estimationErrors] = estimateCameraParameters(imagePoints,worldPoints);
giving me a stereoParameter object, since imagePoints contains matching checkerboard points for both modalities.
To rectify, I used the following call:
[J1,J2] = rectifyStereoImages(I1,I2, cameraParamsStereo);
where I1 is a visual image and I2 a thermal one. Unfortunately, that gives me the error:
Error using rectifyStereoImages>parseInputs (line 106) Inputs must be
of the same size and data types.
Error in rectifyStereoImages (line 96) [I1, I2, interp, outputView,
fillValues] = parseInputs(...
The resolutions of both images are quite different (2048x1088 for visual, 384x288 for thermal). From my undertanding however, rectification in principle should still work, since it is done similarly in this paper. Honestly however, I am not sure how...
Question:
Is there a way in MATLAB to rectify images of different sizes? If not, is there an algorithm to do so that can be easily implemented?
Unfortunately, the rectifyStereoImages function requires the two images to have the same size. This is a limitation of the implementation, not the algorithm.
One thing you can do is undistort the images using the undistortImage function, find matching points (e.g. using matchFeatures), and then use the triangulate function to get a sparse 3-D reconstruction.
You can also try making the two images the same size, by padding the smaller image with zeros. You would need to pad the calibration images before calibration, so that everything is consistent.
Related
Problem statement : Image stitching/mosaic to be done with set of images (taken from same camera with certain level of overlap) which are captured either vertically or horizontally (any one of them)
Definitions :
Train Image : The Image to be transformed.
Query Image : The image to be stitched after transformation with train Image.
Approaches Taken :
The following points describe the approach:
Image registration : The key points based ORB (Orient fast and rotate brief) feature is used for registration of images.The opencv function such as cv2.ORB_create(nfeatures=10000,WTA_K=4) is used. The detect and describe will give features for train and query images.
Calibration : Calibrate the pixels to be stitched. i.e the above feature based detection is used to find the absolute position for the calibration to happen. This is where we search the key points of both images and find equivalent ones to find position for calibration to happen. Used Brute force method to match between train and query images. By taking features mathed, we get the Homography matrix by using cv2.findHomography function. Used the RANSAC (Random sample consensus) method , which is very sensitive to reprojThresh value. I used a pretty high value of 90, which worked for about 10 images stitching (covering the next step of blending). But then later images started working for a low value of 3 or so. Which made me dynamically find this reprojthresh. Which is difficult to set , as our evaluation of stitching is subjective, hence not getting quantification to say this Homography matrix is perfect for stitching. I have used inlier ratio and determinant of homography to check on the quality of transformation. But not able to generalize it for all images.
Image blending : This is the final stage where the position found above is mapped on to perspective projection. This will map the calibrated positions into transformed images which can be blended with other images. Here I am using a very simple blending mechanism after the homography matrix transforms it. which just finds locations of the transformed image and pad it with another image (assumed to be properly aligned).This stage is assumed to be correct, as I keep the homography transformed image as a checking mechanism to confirm this.
Questions :
Does this approach look good ? if not, please suggest alternatives.
If so, how to check homography matrix quality in terms of quantification, so that it can be automatically used for further blending.
Homography matrix generation using RANSAC is used multiple times to fine tune the homography, but the transformed image is misaligned. is this advisable ?
I'm trying to perform downsampling so I can create a gaussian pyramid in opencv. Right now I'm trying to translate this MATLAB code, and I'm not sure if my approach is right.
R = R(1:2:r, 1:2:c, :);
So this code only retrieves the odd-numbered pixels of an image. I read the documents of opencv and thought that the resize() method would be the one I'm looking for.
https://docs.opencv.org/3.0-beta/modules/imgproc/doc/geometric_transformations.html#resize
Imgproc.resize(src,dest,new Size(), 0.5,0.5, INTER_NEAREST);
But I'm not sure if this code will retreat only the odd numbered pixels though... Do they show the exact same behavior?
I am trying to compute the 3D coordinates from several pair of two view points.
First, I used the matlab function estimateFundamentalMatrix() to get the F of the matched points (Number > 8) which is:
F1 =[-0.000000221102386 0.000000127212463 -0.003908602702784
-0.000000703461004 -0.000000008125894 -0.010618266198273
0.003811584026121 0.012887141181108 0.999845683961494]
And my camera - taken these two pictures - was pre-calibrated with the intrinsic matrix:
K = [12636.6659110566, 0, 2541.60550098958
0, 12643.3249022486, 1952.06628069233
0, 0, 1]
From this information I then computed the essential matrix using:
E = K'*F*K
With the method of SVD, I finally got the projective transformation matrices:
P1 = K*[ I | 0 ]
and
P2 = K*[ R | t ]
Where R and t are:
R = [ 0.657061402787646 -0.419110137500056 -0.626591577992727
-0.352566614260743 -0.905543541110692 0.235982367268031
-0.666308558758964 0.0658603659069099 -0.742761951588233]
t = [-0.940150699101422
0.320030970080146
0.117033504470591]
I know there should be 4 possible solutions, however, my computed 3D coordinates seemed to be not correct.
I used the camera to take pictures of a FLAT object with marked points. I matched the points by hand (which means there should not be obvious mistake exists about the raw material). But the result turned out to be a surface with a little bit banding.
I guess this might be due to the reason pictures did not processed with distortions (but actually I remember I did).
I just want to know whether this method to solve the 3D reconstruction issue right? Especially when we already know the camera intrinsic matrix.
Edit by JCraft at Aug.4: I have redone the process and got some pictures showing the problem, I will write another question with detail then post the link.
Edit by JCraft at Aug.4: I have posted a new question: Calibrated camera get matched points for 3D reconstruction, ideal test failed. And #Schorsch really appreciate your help formatting my question. I will try to learn how to do inputs in SO and also try to improve my gramma. Thanks!
If you only have the fundamental matrix and the intrinsics, you can only get a reconstruction up to scale. That is your translation vector t is in some unknown units. You can get the 3D points in real units in several ways:
You need to have some reference points in the world with known distances between them. This way you can compute their coordinates in your unknown units and calculate the scale factor to convert your unknown units into real units.
You need to know the extrinsics of each camera relative to a common coordinate system. For example, you can have a checkerboard calibration pattern somewhere in your scene that you can detect and compute extrinsics from. See this example. By the way, if you know the extrinsics, you can compute the Fundamental matrix and the camera projection matrices directly, without having to match points.
You can do stereo calibration to estimate the R and the t between the cameras, which would also give you the Fundamental and the Essential matrices. See this example.
Flat objects are critical surfaces, not possible to achive your goal from them. try adding two (or more) points off the plane (see Hartley and Zisserman or other text on the matter if still interested)
I have an image that shows depth of the image using colors where warmer colors represent the closer parts of the image and cooler colors represent objects further away. I want to represent this image as a surf plot showing the depth. I have to do this in java but I think its easier to understand the process in Matlab first before moving on. I tried using the size of the image and plotting that but it kept giving me errors. Any help would be much appreciated.
I tried the surf function:
`img = imread('sample.png');
grayImage = rgb2gray(img);
surf(double(img))`
and got this error:
>> surf
Attempt to execute SCRIPT surf as a function:
C:\Users\kuchin\Documents\MATLAB\surf.m
Error in surf (line 3)
surf(double(img))
EDIT
As seen in your comment in this own post, your problem is that you are overriding the surf function by another surf.m file you have. Dont name your matlab files with the same name that Matlab built functions. Go to C:\Users\kuchin\Documents\MATLAB\surf.m and name it mysurf.m. It will resolve your problem.
ORIGINAL POST
If you have a depth image (MxN) of doubles just
surf(img);
if they are uint8
surf(double(img));
Will do the trick.
Test code:
img=imread( 'coins.png' );
surf(double(img))
Outputs:
I am trying to detect corners (x/y coordinates) in 2D scatter vectors of data.
The data is from a laser rangefinder and our current platform uses Matlab (though standalone programs/libs are an option, but the Nav/Control code is on Matlab so it must have an interface).
Corner detection is part of a SLAM algorithm and the corners will serve as the landmarks.
I am also looking to achieve something close to 100Hz in terms of speed if possible (I know its Matlab, but my data set is pretty small.)
Sample Data:
[Blue is the raw data, red is what I need to detect. (This view is effectively top down.)]
[Actual vector data from above shots]
Thus far I've tried many different approaches, some more successful than others.
I've never formally studied machine vision of any kind.
My first approach was a homebrew least squares line fitter, that would split lines in half resurivly until they met some r^2 value and then try to merge ones with similar slope/intercepts. It would then calculate the intersections of these lines. It wasn't very good, but did work around 70% of the time with decent accuracy, though it had some bad issues with missing certain features completely.
My current approach uses the clusterdata function to segment my data based on mahalanobis distance, and then does basically the same thing (least squares line fitting / merging). It works ok, but I'm assuming there are better methods.
[Source Code to Current Method] [cnrs, dat, ~, ~] = CornerDetect(data, 4, 1) using the above data will produce the locations I am getting.
I do not need to write this from scratch, it just seemed like most of the higher-class methods are meant for 2D images or 3D point clouds, not 2D scatter data. I've read a lot about Hough transforms and all sorts of data clustering methods (k-Means etc). I also tried a few canned line detectors without much success. I tried to play around with Line Segment Detector but it needs a greyscale image as an input and I figured it would be prohibitivly slow to convert my vector into a full 2D image to feed it into something like LSD.
Any help is greatly appreciated!
I'd approach it as a problem of finding extrema of curvature that are stable at multiple scales - and the split-and-merge method you have tried with lines hints at that.
You could use harris corner detector for detecting corners.