I am trying to write a program that uses computer vision techniques to detect (and track) tiny blobs in a stream of very noisy images. The image stream comes from an dual X ray imaging setup, which outputs left and right views (different sizes because of collimating differently). My data is of two types: one set of images are not so noisy, which I am just using to try different techniques with, and the other set are noisier, and this is where the detection needs to work at the end. The image stream is at 60 Hz. This is an example of a raw image from the X ray imager:
Here are some cropped out samples of the regions of interest. The blobs that need to be detected are the small black spots near the center of the image.
Initially I started off with a simple contour/blob detection techniques in OpenCV, which were not very helpful. Eventually I moved on to techniques such as "opening" the image using morphological operators, and subsequently performing a Laplacian of Gaussian blob detection to detect areas of interest. This gave me better results for the low-noise versions of the images, but fails when it comes to the high-noise ones: gives me too many false positives. Here is a result from a low-noise image (please note input image was inverted).
The code for my current LoG based approach in MATLAB goes as below:
while ~isDone(videoReader)
frame = step(videoReader);
roi_frame = imcrop(frame, [660 410 120 110]);
I_roi = rgb2gray(roi_frame);
I_roi = imcomplement(I_roi);
I_roi = wiener2(I_roi, [5 5]);
background = imopen(I_roi,strel('disk',3));
I2 = imadjust(I_roi - background);
K = imgaussfilt(I2, 5);
level = graythresh(K);
bw = im2bw(I2);
sigma = 3;
% Filter image with LoG
I = double(bw);
h = fspecial('log',sigma*30,sigma);
Ifilt = -imfilter(I,h);
% Threshold for points of interest
Ifilt(Ifilt < 0.001) = 0;
% Dilate to obtain local maxima
Idil = imdilate(Ifilt,strel('disk',50));
% This is the final image
P = (Ifilt == Idil) .* Ifilt;
Is there any way I can improve my current detection technique to make it work for images with a lot of background noise? Or are there techniques better suited for images like this?
The approach I would take:
-Average background subtraction
-Aggressive Gaussian smoothing (this filter should be shaped based on your target object, off the top of my head I think you want the sigma about half the smallest cross section of your object, but you may want to fiddle with this) Basically the goal is blurring the noise as much as possible without completely losing your target objects (based on shape and size)
-Edge detection. Try to be specific to the object if possible (basically, look at what the object's edge looks like after Gaussian smoothing and set your edge detection to look for that width and contrast shift)
-May consider running a closing operation here.
-Search the whole image for islands (fully enclosed regions) filter based on size and then on shape.
I am taking a hunch that despite the incredibly low signal to noise ratio, your granularity of noise is hopefully significantly smaller than your object size. (if your noise is both equivalent contrast and same ballpark size as your object... you are sunk and need to re-evaluate your acquisition imo)
Another note based on your speed needs. Extreme amounts of processing savings can be made through knowing last known positions and searching locally and also knowing where new targets can enter the image from.
Related
I'm using the function regionprops to detect the number of trees on a image taked by drone.
First I removed the ground using Blue NDVI:
Image with threshold:
Then I used the function regionprops to detect the number of trees on image:
But there are a problem on region 15, because all trees on that region are connected and it detects as one tree.
I tried to separate the trees on that region using Watershed Segmentation, but its not working:
Am I doing this the wrong way?
Is there a better method to separate the trees?
If anyone can help me with this problem I will appreciate. Here is the region 15 without the ground:
If it helps, here is the Gradient Magnitude image:
It has been some time since this question was asked. I hope it is not too late for an answer. I see a general problem of using watershed segmentation in similar questions. Sometimes the objects are apart, not touching each other like in this example . In such cases, only blurring the image is enough to use watershed segmentation. Sometimes the objects are located closely and touch each other, thus the boundaries of objects are not clear like in this example. In such cases, using distance transform-->blur-->watershed helps. In this question, the logical approach should be using distance transform. However, this time the boundaries are not clear due to shadows on and nearby the trees. In such cases, it is good to use any information that helps to separate the objects as in here or emphasise objects itself.
In this question, I suggest using colour information to emphasise tree pixels.
Here are the MATLAB codes and results.
im=imread('https://i.stack.imgur.com/aBHUL.jpg');
im=im(58:500,86:585,:);
imOrig=im;
%% Emphasize trees
im=double(im);
r=im(:,:,1);
g=im(:,:,2);
b=im(:,:,3);
tmp=((g-r)./(r-b));
figure
subplot(121);imagesc(tmp),axis image;colorbar
subplot(122);imagesc(tmp>0),axis image;colorbar
%% Transforms
% Distance transform
im_dist=bwdist(tmp<0);
% Blur
sigma=10;
kernel = fspecial('gaussian',4*sigma+1,sigma);
im_blured=imfilter(im_dist,kernel,'symmetric');
figure
subplot(121);imagesc(im_dist),axis image;colorbar
subplot(122);imagesc(im_blured),axis image;colorbar
% Watershed
L = watershed(max(im_blured(:))-im_blured);
[x,y]=find(L==0);
figure
subplot(121);
imagesc(imOrig),axis image
hold on, plot(y,x,'r.','MarkerSize',3)
%% Local thresholding
trees=zeros(size(im_dist));
centers= [];
for i=1:max(L(:))
ind=find(L==i & im_blured>1);
mask=L==i;
[thr,metric] =multithresh(g(ind),1);
trees(ind)=g(ind)>thr*1;
trees_individual=trees*0;
trees_individual(ind)=g(ind)>thr*1;
s=regionprops(trees_individual,'Centroid');
centers=[centers; cat(1,[],s.Centroid)];
end
subplot(122);
imagesc(trees),axis image
hold on, plot(y,x,'r.','MarkerSize',3)
subplot(121);
hold on, plot(centers(:,1),centers(:,2),'k^','MarkerFaceColor','r','MarkerSize',8)
You could try out a marker-based watershed. Vanilla watershed transforms never work out of the box in my experience. One way to perform one would be to first create a distance map of the segmented area by using imdist(). Then you could suppress local maxima by calling imhmax(). Then calling watershed() will usually perform noticeably better.
Here's a sample script on how to do it:
bwTrees = imopen(bwTrees, strel('disk', 10));
%stabilize the borders to lessen oversegmentation
distTrees = -bwDist(~bwTrees); %Distance transform
distTrees(~bwTrees) = -Inf; %set background to -Inf
distTrees = imhmin(distTrees, 3); %suppress local minima
basins = watershed(distTrees);
ridges = basins == 0;
segmentedTrees = bwTrees & ~ridges; %segment
segmentedTrees = imopen(segmentedTrees, strel('disk', 2));
%remove 'segmentation trash' caused by oversegmentation near the borders.
I fiddled around with the parameters for ~10min but got fairly poor results:
You'd need to pour work into this. Mostly in pre- and post-processing via the morphology. More curvature would help the segmentation, if you could lower the sensitivity of the segmentation in the first part. The size of the h-minima transform is also an paramater of interest. You can probably get adequate results this way.
Probably a better approach would come from the world of clustering techniques. If you have or can find a way to estimate the number of trees in the forest you should be able to use traditional clustering methods to segment out the trees. A Gaussian mixture model or a k-means with k-trees would probably work much better than a marker based watershed if you get even nearly the right amount of trees. Normally I'd estimate the number of trees based on the number of suppressed maxima on a h-maxima transform, but your labels might be a bit too sausagey for that. It's worth a try though.
I have an image which consists of a black and white document against a heterogeneous colored background. I need to automatically detect the document in my image.
I have tried Otsu's method and a Local Thresholding method, but neither were successful. Also, edge detectors like Canny and Sobel didn't work.
Can anyone suggest some method to automatically detect the document?
Here's an example starting image:
After using various threshold methods, I was able to get the following output:
What follows is an automated global method for isolating an area of low color saturation (e.g. a b/w page) against a colored background. This may work well as an alternative approach when other approaches based on adaptive thresholding of grayscale-converted images fail.
First, we load an RGB image I, covert from RGB to HSV, and isolate the saturation channel:
I = imread('/path/to/image.jpg');
Ihsv = rgb2hsv(I); % convert image to HSV
Isat = Ihsv(:,:,2); % keep only saturation channel
In general, a good first step when deciding how to proceed with any object detection task is to examine the distribution of pixel values. In this case, these values represent the color saturation levels at each point in our image:
% Visualize saturation value distribution
imhist(Isat); box off
From this histogram, we can see that there appear to be at least 3 distinct peaks. Given that our target is a black and white sheet of paper, we’re looking to isolate saturation values at the lower end of the spectrum. This means we want to find a threshold that separates the lower 1-2 peaks from the higher values.
One way to do this in an automated way is through Gaussian Mixture Modeling (GMM). GMM can be slow, but since you’re processing images offline I assume this is not an issue. We’ll use Matlab’s fitgmdist function here and attempt to fit 3 Gaussians to the saturation image:
% Find threshold for calling ROI using GMM
n_gauss = 3; % number of Gaussians to fit
gmm_opt = statset('MaxIter', 1e3); % max iterations to converge
gmmf = fitgmdist(Isat(:), n_gauss, 'Options', gmm_opt);
Next, we use the GMM fit to classify each pixel and visualize the results of our GMM classification:
% Classify pixels using GMM
gmm_class = cluster(gmmf, Isat(:));
% Plot histogram, colored by class
hold on
bin_edges = linspace(0,1,256);
for j=1:n_gauss, histogram(Isat(gmm_class==j), bin_edges); end
In this example, we can see that the GMM ended up grouping the 2 far left peaks together (blue class) and split the higher values into two classes (yellow and red). Note: your colors might be different, since GMM is sensitive to random initial conditions. For our use here, this is probably fine, but we can check that the blue class does in fact capture the object we’d like to isolate by visualizing the image, with pixels colored by class:
% Visualize classes as image
im_class = reshape(gmm_class ,size(Isat));
imagesc(im_class); axis image off
So it seems like our GMM segmentation on saturation values gets us in the right ballpark - grouping the document pixels (blue) together. But notice that we still have two problems to fix. First, the big bar across the bottom is also included in the same class with the document. Second, the text printed on the page is not being included in the document class. But don't worry, we can fix these problems by applying some filters on the GMM-grouped image.
First, we’ll isolate the class we want, then do some morphological operations to low-pass filter and fill gaps in the objects.
Isat_bw = im_class == find(gmmf.mu == min(gmmf.mu)); %isolate desired class
opened = imopen(Isat_bw, strel('disk',3)); % morph open
closed = imclose(Isat_bw, strel('disk',50)); % morph close
imshow(closed)
Next, we’ll use a size filter to isolate the document ROI from the big object at the bottom. I’ll assume that your document will never fill the entire width of the image and that any solid objects bigger than the sheet of paper are not wanted. We can use the regionprops function to give us statistics about the objects we detect and, in this case, we’ll just return the objects’ major axis length and corresponding pixels:
% Size filtering
props = regionprops(closed,'PixelIdxList','MajorAxisLength');
[~,ridx] = min([props.MajorAxisLength]);
output_im = zeros(numel(closed),1);
output_im(props(ridx).PixelIdxList) = 1;
output_im = reshape(output_im, size(closed));
% Display final mask
imshow(output_im)
Finally, we are left with output_im - a binary mask for a single solid object corresponding to the document. If this particular size filtering rule doesn’t work well on your other images, it should be possible to find a set of values for other features reported by regionprops (e.g. total area, minor axis length, etc.) that give reliable results.
A side-by-side comparison of the original and the final masked image shows that this approach produces pretty good results for your sample image, but some of the parameters (like the size exclusion rules) may need to be tuned if results for other images aren't quite as nice.
% Display final image
image([I I.*uint8(output_im)]); axis image; axis off
One final note: be aware that the GMM algorithm is sensitive to random initial conditions, and therefore might randomly fail, or produce undesirable results. Because of this, it's important to have some kind of quality control measures in place to ensure that these random failures are detected. One possibility is to use the posterior probabilities of the GMM model to form some kind of criteria for rejecting a certain fit, but that’s beyond the scope of this answer.
I have two images – mannequin with and without garment.
Please refer sample images below. Ignore the jewels, footwear on the mannequin, imagine the second mannequin has only dress.
I want to extract only the garment from the two images for further processing.
The complexity is that there is slight displacement in the position of camera when taking the two pictures. Due to this simple subtraction to generate the garment mask will not work.
Can anyone tell me how to handle it?
I think I need to do registration between the two images so that I can extract only the garment from the image?
Any references to blogs, articles and codes is highly appreciated.
--
Thanks
Idea
This is an idea of how you could do it, I haven't tested it but my gut tells me it might work. I'm assuming that there will be slight differences in the pose of the manequin as well as the camera attitude.
Let the original image be A, and the clothed image be B.
Take the difference D = |A - B|, apply a median filter that is proportional to the largest deviation you expect from pose and camera attitude error: Dmedian = Median(D, kernelsize).
Quantize Dmedian into a binary mask Dmask = Q(Dmedian, threshold) using appropriate threshold values to obtain an approximate mask for the garment (this will be smaller than the garment itself due to the median filter). Reject any shapes in Dmedian that have too small area by setting their pixels to 0.
Expand the shape(s) in Dmask proportionally to the size of the median kernel into Emask=expand(Dmask, k*kernelsize). Then construct the difference in the masks Fmask=|Dmask - Emask| which now contains areas of pixels where the garment edge is expected to be. For every pixel in Fmask which is in this area, find the correlation Cxy between A and B using a small neighbourhood, store the correlations into an image C=1.0 - Corr(A,B, Fmask, n).
Your final garment mask will be M=C+Dmask.
Explanation
Since your image has nice and continuous swatches of colour, the difference between the two similar images will be thin lines and small gradients where the pose and camera attitude is different. When taking a median filter of the difference image over a sufficiently large kernel, these lines will be removed because they are in a minority of the pixels.
The garment on the other hand will (hopefully) have a significant difference from the colors in the unclothed version. And will generate a bigger difference. Thresholding the difference after the median filter should give you a rough mask of the garment that is undersized dues to some of the pixels on the edge being rejected due to their median values being too low. You could stop here if the approximation is good enough for you.
By expanding the mask we obtained above we get a probable region for the "true" edge. The above process has served to narrow our search region for the true edge considerably and we can apply a more costly correlation search between the images along this edge to find where the garment is. High correlation means no carment and low correlation means garment.
We use the inverted correlation as an alpha value together with the initially smaller mask to obtain a alpha valued mask of the garment that can be used for extracting it.
Clarification
Expand: What I mean by "expanding the mask" is to find the contour of the mask region and outsetting/growing/enlarging it to make it larger.
Corr(A,B,Fmask,n): Is just an arbitrarily chosen correlation function that gives correlation between pixels in A and B that are selected by the mask Fmask using a region of size n. The function returns 1.0 for perfect match and 0.0 for anti-match for each pixel tested. A good function is this pseudocode:
foreach px_pos in Fmask where Fmask[px_pos] == 1
Ap = subregion(A, px_pos, size) - mean(mean(A));
Bp = subregion(B, px_pos, size) - mean(mean(B))
Cxy = sum(sum(Ap .* Bp))*sum(sum(Ap .* Bp)) / (sum(sum(Ap.*Ap))*sum(sum(Bp.*Bp)))
C[px_pos] = 1.0 - Cxy;
end
where subregion selects a region of size size around the pixel with position px_pos.
You can see that if Ap == Bp then Cxy=1
Here with i have attached two consecutive frames captured by a cmos camera with IR Filter.The object checker board was stationary at the time of capturing images.But the difference between two images are nearly 31000 pixels.This could be affect my result.can u tell me What kind of noise is this?How can i remove it.please suggest me any algorithms or any function possible to remove those noises.
Thank you.Sorry for my poor English.
Image1 : [1]: http://i45.tinypic.com/2wptqxl.jpg
Image2: [2]: http://i45.tinypic.com/v8knjn.jpg
That noise appears to result from camera sensor (Bayer to RGB conversion). There's the checkerboard pattern still left.
Also lossy jpg contributes a lot to the process. You should first have an access to raw images.
From those particular images I'd first try to use edge detection filters (Sobel Horizontal and Vertical) to make a mask that selects between some median/local histogram equalization for the flat areas and to apply some checker board reducing filter to the edges. The point is that probably no single filter is able to do good for both jpeg ringing artifacts and to the jagged edges. Then the real question is: what other kind of images should be processed?
From the comments: if corner points are to be made exact, then the solution more likely is to search for features (corner points with subpixel resolution) and make a mapping from one set of points to the other images set of corners, and search for the best affine transformation matrix that converts these sets to each other. With this matrix one can then perform resampling of the other image.
One can fortunately estimate motion vectors with subpixel resolution without brute force searching all possible subpixel locations: when calculating a matched filter, one gets local maximums for potential candidates of exact matches. But this is not all there is. One can try to calculate a more precise approximation of the peak location by studying the matched filter outputs in the nearby pixels. For exact match the output should be symmetric. Otherwise the 'energies' of the matched filter are biased towards the second best location. (A 2nd degree polynomial fit + finding maximum can work.)
Looking closely at these images, I must agree with #Aki Suihkonen.
In my view, the main noise comes from the jpeg compression, that causes sharp edges to "ring". I'd try a "de-speckle" type of filter on the images, and see if this makes a difference. Some info that can help you implement this can be found in this link.
In a more quick and dirty fashion, you apply one of the many standard tools, for example, given the images are a and b:
(i) just smooth the image with a Gaussian filter, this can reduce noise differences between the images by an order of magnitude. For example:
h=fspecial('gaussian',15,2);
a=conv2(a,h,'same');
b=conv2(b,h,'same');
(ii) Reduce Noise By Adaptive Filtering
a = wiener2(a,[5 5]);
b = wiener2(b,[5 5]);
(iii) Adjust ntensity Values Using Histogram Equalization
a = histeq(a);
b = histeq(b);
(iv) Adjust Intensity Values to a Specified Range
a = imadjust(a,[0 0.2],[0.5 1]);
b = imadjust(b,[0 0.2],[0.5 1]);
If your images are supposed to be black and white but you have captured them in gray scale there could be difference due to noise.
You can convert the images to black and white by defining a threshold, any pixel with a value less than that threshold should be assigned 0 and anything larger than that threshold should be assigned 1, or whatever your gray scale range is (maybe 255).
Assume your image is I, to make it black and white assuming your gray scale image level is from 0 to 255, assume you choose a threshold of 100:
ind = find(I < 100);
I(ind) = 0;
ind = find(I >= 100);
I(ind) = 255;
Now you have a black and white image, do the same thing for the other image and you should get very small difference if the camera and the subject have note moved.
I need to know how to clean noise from an image with Matlab.
lets look at this example:
as you see the numbers is not look clearly.
so how can I clean the noise and the pixels that are not the numbers so the identification will be easier.
thanks.
Let's do it step by step in Mathematica:
(*first separate the image in HSB channels*)
i1 = ColorSeparate[ColorNegate#yourColorImage, "HSB"]
(*Let's keep the B Channel*)
i2 = i1[[3]]
(*And Binarize it *)
i3 = Binarize[i2, 0.92]
(*Perform a Thinning to get the skeleton*)
i4 = Thinning[i3]
(*Now we cut those hairs*)
i5 = Pruning[i4, 10]
(*Remove the small lines*)
i6 = DeleteSmallComponents[i5, 30]
(*And finally dilate*)
i7 = Dilation[i6, 3]
(*Now we can perform an OCR*)
TextRecognize#i7
-->"93 269 23"
Done!
Since this question is tagged MATLAB, I translated #belisarius's solution as such (which I think is superior to the currently accepted answer):
%# read image
I = imread('http://i.stack.imgur.com/nGNGf.png');
%# complement it, and convert to HSV colorspace
hsv = rgb2hsv(imcomplement(I));
I1 = hsv(:,:,3); %# work with V channel
%# Binarize/threshold image
I2 = im2bw(I1, 0.92);
%# Perform morphological thinning to get the skeleton
I3 = bwmorph(I2, 'thin',Inf);
%# prune the skeleton (remove small branches at the endpoints)
I4 = bwmorph(I3, 'spur', 7);
%# Remove small components
I5 = bwareaopen(I4, 30);
%# dilate image
I6 = imdilate(I5, strel('square',2*3+1));
%# show step-by-step results
figure('Position',[200 150 700 700])
subplot(711), imshow(I)
subplot(712), imshow(I1)
subplot(713), imshow(I2)
subplot(714), imshow(I3)
subplot(715), imshow(I4)
subplot(716), imshow(I5)
subplot(717), imshow(I6)
Finally you can apply some form of OCR to recognize the numbers. Unfortunately, MATLAB has no built-in function equivalent to TextRecognize[] in Mathematica... In the meanwhile, look in the File Exchange, I'm sure you will find dozens of submissions filling the gap :)
Did you start with a bilevel (two color, black and white)? Or did you threshold it yourself?
If it's the latter, you may find it easier to perform noise reduction before you threshold. In this case, please upload the image you have before thresholding.
If it's the former, then you'll have a tough time as traditional noise reduction is concerned. The reason is that a lot of noise reduction approaches take advantage of the distinction in statistical properties between the noise and the actual natural image. By thresholding, that distinction is essentially destroyed.
EDIT
OK, technically, your image isn't really noisy -- it's blurry (letters are running into each other) and has background interference.
But anyway, here is how I would deal with it:
Pick a color channel to work with (RGB is three channels, typically one is enough). I chose green because it looked the easiest to manipulate.
Blur the image (I used a 5x5 Gaussian kernel in GIMP)
Threshold using an empirically determined threshold (basically, try each threshold until you get a decent result). It's OK if some of the numbers have gaps -- we can close them in the next step
Morphological image processing (erosion and dilation)
Green channel:
Blur (5x5 Gaussian):
Thresholded image (I used a threshold of ~93 in GIMP):
Final result:
You can see that the gaps in the middle 6 and 9 have dissapeared. Unfortunately, I couldn't get the gap in the left 3 to go away -- it's simply too large. Here's what the problems causing this are:
The line along the top of the image is much darker than some parts of the 3. If you use a threshold to remove the line, then a gap will be created. If you were to somehow remove that line (e.g. by more zealous cropping), the thresholding result would be much better as far as the 3 is concerned.
Also, the middle 2 and 6 are running together. Heavy thresholding is required to prevent them from both forming the same blob after thresholding.
I think there are two things you could aim to do to make them more detectable:
Remove patches smaller than a certain number of pixels (this would remove the spots between the sets of digits)
Numbers should be "closed" forms, so you need an algorithm to detect the pixels (at the top of each number) that should be changed to black in order to "close" the number "shapes".
You also have linear features that are a part of the noise signal which could be detected through edge / line detection.
Detecting contiguous "zones" and calculating characteristics such as compactness or length / height might also help in identifying which structures to keep...