I am trying to obtain a disparity map from a homemade stereo camera setup. The baseline is 125mm and both cameras are fixed to a 3D-printed support. I've previously calibrated the cameras with 15 images of a checkboard pattern of 80mm square size using MatLab's calibration tool.
Using the intrinsics an extrinsics given by MatLab calibration tool, I rectified the images and build the disparity map on a MatLab script. However, the disparity is not good enough for my application. Do you think the calibration is not good or could be due to other problems?
Here are the results:
As you see through the lines I draw, the rectification of the images is not well done, since the epipolar constraint doesn't apply.
As you can see I used one of the calibration images to check. However, it happens the same on other images. I'm particularly concerned about the ground, as it contains a lot of noise and invalid points, which is not good enough for my algorithms, so I need to improve it.
Related
Problem statement : Image stitching/mosaic to be done with set of images (taken from same camera with certain level of overlap) which are captured either vertically or horizontally (any one of them)
Definitions :
Train Image : The Image to be transformed.
Query Image : The image to be stitched after transformation with train Image.
Approaches Taken :
The following points describe the approach:
Image registration : The key points based ORB (Orient fast and rotate brief) feature is used for registration of images.The opencv function such as cv2.ORB_create(nfeatures=10000,WTA_K=4) is used. The detect and describe will give features for train and query images.
Calibration : Calibrate the pixels to be stitched. i.e the above feature based detection is used to find the absolute position for the calibration to happen. This is where we search the key points of both images and find equivalent ones to find position for calibration to happen. Used Brute force method to match between train and query images. By taking features mathed, we get the Homography matrix by using cv2.findHomography function. Used the RANSAC (Random sample consensus) method , which is very sensitive to reprojThresh value. I used a pretty high value of 90, which worked for about 10 images stitching (covering the next step of blending). But then later images started working for a low value of 3 or so. Which made me dynamically find this reprojthresh. Which is difficult to set , as our evaluation of stitching is subjective, hence not getting quantification to say this Homography matrix is perfect for stitching. I have used inlier ratio and determinant of homography to check on the quality of transformation. But not able to generalize it for all images.
Image blending : This is the final stage where the position found above is mapped on to perspective projection. This will map the calibrated positions into transformed images which can be blended with other images. Here I am using a very simple blending mechanism after the homography matrix transforms it. which just finds locations of the transformed image and pad it with another image (assumed to be properly aligned).This stage is assumed to be correct, as I keep the homography transformed image as a checking mechanism to confirm this.
Questions :
Does this approach look good ? if not, please suggest alternatives.
If so, how to check homography matrix quality in terms of quantification, so that it can be automatically used for further blending.
Homography matrix generation using RANSAC is used multiple times to fine tune the homography, but the transformed image is misaligned. is this advisable ?
I'd like to find a transformation that projects the image from the Left camera onto the image from the Right camera so that the two become aligned. I already managed to do that with two similar cameras (IGB and RGB), by using the disparity map and shifting each pixel by the corresponding disparity value. My problem is that this doesn't work for other cameras that I'm using (for example multispectral and infrared sensors), because the calculated disparity maps have very little detail. I am currently using the Matlab Computer Vision Tool Box, and I suspect that the problem is the poor correlation of information in the images (little correspondences found by the disparity algorithm).
I would like to know if there is another way of doing this transformation, for example just by using the Extrinsic and Intrinsic Parameters of the cameras (the are already calibrated).
The disparity IS the transformation you are looking for. Any sensible left-right mapping depends on 3D information, which the disparity provides. Anything else is just hallucinating some values based on assumptions that may or may not make sense.
This paper describes nicely the geometry of a stereo image system. I am trying to figure out, if the cameras tilted towards each other with a certain angle, how the calculation would change? I looked around but couldn't find any reference to tilted camera systems.
Unfortunately, the calculation changes significantly. The rectified case (where both cameras are well-aligned to each other) has the advantage that you can calculate the disparity and the depth is proportional to the disparity. This is not the case in the general case.
When you introduce tilts, you end up with something called epipolar geometry. Here is a paper about this I just googled. In order to calculate the depth from a pixel-pair you need the fundamental matrix or the essential matrix. Both are not easy to obtain from the image pair. If, however, you have the geometric relation of both cameras (translation and rotation), calculating these matrices is a lot easier.
There are several ways to calculate the depth of a pixel-pair. One way is to use the fundamental matrix to rectify both images (although rectifying is not easy either, or even unique) and run a simple disparity check.
I am developing a project of detecting vehicles' headlights in night scene. First I am working on a demo on MATLAB. My detection method is edge detection using Difference of Gaussian (DoG): I take the convolution of the image with Gaussian blur with 2 difference sigma then minus 2 filtered images to find edge. My result is shown below:
Now my problem is to find a method in MATLAB to circle the round edge such as car's headlights and even street lights and ignore other edge. If you guys got any suggestion, please tell me.
I think you may be able to get a better segmentation using a slightly different approach.
There is already strong contrast between the lights and the background, so you can take advantage of this to segment out the bright spots using a simple threshold, then you can apply some blob detection to filter out any small blobs (e.g. streetlights). Then you can proceed from there with contour detection, Hough circles, etc. until you find the objects of interest.
As an example, I took your source image and did the following:
Convert to 8-bit greyscale
Apply Gaussian blur
Threshold
This is a section of the source image:
And this is the thresholded overlay:
Perhaps this type of approach is worth exploring further. Please comment to let me know what you think.
I have a video of moving parts taken using a static camera. I wish to track & analyze the co-ordinates of various parts in the video. But the co-ordinates values are affected by camera movement. How do I calibrate the camera shake? I don't have any static point in the video (except for the top&bottom edges of video).
All I wish to get is the co-ordinates of (centroids, may be) moving parts adjusted for camera shake. I use MATLAB's computer vision toolbox to process the video.
I've worked on super-resolution algorithms in the past, and as a side affect, I got image stabilization using phase correlation. It's very resilient to noise, and it's quite fast. You should be able to achieve sub-pixel accuracy using a weighted centroid around the peak location, or some kind of peak-fitting routine. Running phase correlation on successive frames will tell you the translational shift that occurs frame-to-frame. You can use an affine warp to remove the shift.
A similar, but slower, approach is here this example is using Normalized Cross Correlation.
If you're using Matlab 2013a or later then video stabilization can be done using point matching Point Matching or by Template Matching. I guess they're available in Matlab 2012b but I haven't tested it out.