Problem statement : Image stitching/mosaic to be done with set of images (taken from same camera with certain level of overlap) which are captured either vertically or horizontally (any one of them)
Definitions :
Train Image : The Image to be transformed.
Query Image : The image to be stitched after transformation with train Image.
Approaches Taken :
The following points describe the approach:
Image registration : The key points based ORB (Orient fast and rotate brief) feature is used for registration of images.The opencv function such as cv2.ORB_create(nfeatures=10000,WTA_K=4) is used. The detect and describe will give features for train and query images.
Calibration : Calibrate the pixels to be stitched. i.e the above feature based detection is used to find the absolute position for the calibration to happen. This is where we search the key points of both images and find equivalent ones to find position for calibration to happen. Used Brute force method to match between train and query images. By taking features mathed, we get the Homography matrix by using cv2.findHomography function. Used the RANSAC (Random sample consensus) method , which is very sensitive to reprojThresh value. I used a pretty high value of 90, which worked for about 10 images stitching (covering the next step of blending). But then later images started working for a low value of 3 or so. Which made me dynamically find this reprojthresh. Which is difficult to set , as our evaluation of stitching is subjective, hence not getting quantification to say this Homography matrix is perfect for stitching. I have used inlier ratio and determinant of homography to check on the quality of transformation. But not able to generalize it for all images.
Image blending : This is the final stage where the position found above is mapped on to perspective projection. This will map the calibrated positions into transformed images which can be blended with other images. Here I am using a very simple blending mechanism after the homography matrix transforms it. which just finds locations of the transformed image and pad it with another image (assumed to be properly aligned).This stage is assumed to be correct, as I keep the homography transformed image as a checking mechanism to confirm this.
Questions :
Does this approach look good ? if not, please suggest alternatives.
If so, how to check homography matrix quality in terms of quantification, so that it can be automatically used for further blending.
Homography matrix generation using RANSAC is used multiple times to fine tune the homography, but the transformed image is misaligned. is this advisable ?
Related
I am unable to find a method to detect the text area in the document and apply a filter to the rest of the image to clear it from any noise. Please refer to this image. If I do apply a filter to the image anyway, the text doesn't remain visible anymore.
Is there an algorithm in MATLAB that can help me find the textarea and treat it separately?
You can greatly reduce the amount of noise by applying erosion, followed by a gaussian blur (σ=2; the image will be converted to grayscale) followed by conversion back to binary.
Erosion alone will reduce the amount of noise significantly:
After application of gaussian blur and re-conversion into binary, the noise will be further reduced. Note that very small text (like the sub-headline) will be also degraded:
All three needed operations seem to be already implemented in MatLab: See Erosion; Gaussian Blur and Thresholding. However, note that for the example automatic (histogram based) thresholding was used to determine the optimal threshold.
I am trying to obtain a disparity map from a homemade stereo camera setup. The baseline is 125mm and both cameras are fixed to a 3D-printed support. I've previously calibrated the cameras with 15 images of a checkboard pattern of 80mm square size using MatLab's calibration tool.
Using the intrinsics an extrinsics given by MatLab calibration tool, I rectified the images and build the disparity map on a MatLab script. However, the disparity is not good enough for my application. Do you think the calibration is not good or could be due to other problems?
Here are the results:
As you see through the lines I draw, the rectification of the images is not well done, since the epipolar constraint doesn't apply.
As you can see I used one of the calibration images to check. However, it happens the same on other images. I'm particularly concerned about the ground, as it contains a lot of noise and invalid points, which is not good enough for my algorithms, so I need to improve it.
Currently I hope to use scale space representation to filter one image. Features in one image can be filtered using an Gaussian smooth filter with one optimal sigma. It means different features in one image can be expressed best in different scale under scale space representation.
For example, I have one image with one tree in it. In the scale space representation, three sigma values are used and they are represented as sigma0, sigma1 and sigma2. The ground is best expressed in the smoothed image with sigma0 because it contains textures mainly. The branches are best expressed in the smoother image with sigma1 and the trunk is with the smoother image with sigma2. If I hope to filter the image, I hope that the filtered pixels for the group is from the smoothed image with sigma0.
The filtered pixels for the branches are from the smoothed image with sigma1. The filtered pixels for the trunk are from the smoothed image with sigma2.
It requires that I need to determine in which smoothed image one pixel is expressed best. Is this idea plausible?
I am trying to use differece-of-Gaussian of two successive smoothed images to perform the above task. Is there any other way to combine the three smoothed image?
I use Matlab to implement the idea. The values of the three sigmas is 1.0, 2.0 and 3.0. The corresponding size of Gaussian kernel is 3, 5 and 7. I use the function fspecial to generate the kernel. Are the parameter reasonable? Please share your experience with the scale space representation to help me. You can provide some links to useful papers.
your idea is very much plausible! You are just one step away from it. I did something very similar once and it looked like this:
After smoothing your images and extracting the edges for each smoothing step (I used a weighted [to compensate for maxima supression after Gauss filtering] Sobel filter for this since DOG was not quite stable for my aplication), you can proyect (and normalize) your whole stack of edge images into a single image ("cummulative edges") which will contain the characteristic edges. You can then compare the cummulative edges image (using cross-correlation or whatever you wish) with every single image in your edge stack, the biggest value of this comparation is then the smooth-scale in which the pixel is expressed the best.
Hope that makes sense for you after reading it a couple of times.
Also don't be afraid of using much bigger kernel sizes, while it all depends on your application, I ended up using things of 51 and bigger!!! (was working with 40MP images though...)
T. Lindeberg has literally dozens of papers related to this problem. I found this one the most useful, but since you are already in the right track, I don't think reading the 50 pages will make you that much smarter. The most important part of it is maybe this one:
Principle for scale selection:
In the absence of other evidence, assume that a scale level, at which some
(possibly non-linear) combination of normalized derivatives assumes a
local maximum over scales, can be treated as reflecting a characteristic
length of a corresponding structure in the data.
I am developing a project of detecting vehicles' headlights in night scene. First I am working on a demo on MATLAB. My detection method is edge detection using Difference of Gaussian (DoG): I take the convolution of the image with Gaussian blur with 2 difference sigma then minus 2 filtered images to find edge. My result is shown below:
Now my problem is to find a method in MATLAB to circle the round edge such as car's headlights and even street lights and ignore other edge. If you guys got any suggestion, please tell me.
I think you may be able to get a better segmentation using a slightly different approach.
There is already strong contrast between the lights and the background, so you can take advantage of this to segment out the bright spots using a simple threshold, then you can apply some blob detection to filter out any small blobs (e.g. streetlights). Then you can proceed from there with contour detection, Hough circles, etc. until you find the objects of interest.
As an example, I took your source image and did the following:
Convert to 8-bit greyscale
Apply Gaussian blur
Threshold
This is a section of the source image:
And this is the thresholded overlay:
Perhaps this type of approach is worth exploring further. Please comment to let me know what you think.
I've scanned an old photo with paper texture pattern and I would like to remove the texture as much as possible without lowering the image quality. Is there a way, probably using Image Processing toolbox in MATLAB?
I've tried to apply FFT transformation (using Photoshop plugin), but I couldn't find any clear white spots to be paint over. Probably the pattern is not so regular for this method?
You can see the sample below. If you need the full image I can upload it somewhere.
Unfortunately, you're pretty much stuck in the spatial domain, as the pattern isn't really repetitive enough for Fourier analysis to be of use.
As #Jonas and #michid have pointed out, filtering will help you with a problem like this. With filtering, you face a trade-off between the amount of detail you want to keep and the amount of noise (or unwanted image components) you want to remove. For example, the median filter used by #Jonas removes the paper texture completely (even the round scratch near the bottom edge of the image) but it also removes all texture within the eyes, hair, face and background (although we don't really care about the background so much, it's the foreground that matters). You'll also see a slight decrease in image contrast, which is usually undesirable. This gives the image an artificial look.
Here's how I would handle this problem:
Detect the paper texture pattern:
Apply Gaussian blur to the image (use a large kernel to make sure that all the paper texture information is destroyed
Calculate the image difference between the blurred and original images
EDIT 2 Apply Gaussian blur to the difference image (use a small 3x3 kernel)
Threshold the above pattern using an empirically-determined threshold. This yields a binary image that can be used as a mask.
Use median filtering (as mentioned by #Jonas) to replace only the parts of the image that correspond to the paper pattern.
Paper texture pattern (before thresholding):
You want as little actual image information to be present in the above image. You'll see that you can very faintly make out the edge of the face (this isn't good, but it's the best I have time for). You also want this paper texture image to be as even as possible (so that thresholding gives equal results across the image). Again, the right hand side of the image above is slightly darker, meaning that thresholding it well will be difficult.
Final image:
The result isn't perfect, but it has completely removed the highly-visible paper texture pattern while preserving more high-frequency content than the simpler filtering approaches.
EDIT
The filled-in areas are typically plain-colored and thus stand out a bit if you look at the image very closely. You could also try adding some low-strength zero-mean Gaussian noise to the filled-in areas to make them look more realistic. You'd have to pick the noise variance to match the background. Determining it empirically may be good enough.
Here's the processed image with the noise added:
Note that the parts where the paper pattern was removed are more difficult to see because the added Gaussian noise is masking them. I used the same Gaussian distribution for the entire image but if you want to be more sophisticated you can use different distributions for the face, background, etc.
A median filter can help you a bit:
img = imread('http://i.stack.imgur.com/JzJMS.jpg');
%# convert rgb to grayscale
img = rgb2gray(img);
%# apply median filter
fimg = medfilt2(img,[15 15]);
%# show
imshow(fimg,[])
Note that you may want to pad the image first to avoid edge effects.
EDIT: A smaller filter kernel than [15 15] will preserve image texture better, but will leave more visible traces of the filtering.
Well i have tried out a different approach using Anisotropc diffusion using the 2nd coefficient that operates on wider areas
Here is the output i got:
From what i can See from the Picture, the Noise has a relatively high Frequency Compared to the image itself. So applying a low Pass filter should work. Have a look at the Power spectrum abs(fft(...)) to determine the cutoff Frequency.