I'm trying to find a starting point, but I can't seem to find the right answer. I'd be very grateful for some guidance. I also don't know the proper terminology, hence the title.
I took an image of a bag with a black background behind it.
And I want to extract the bag, similar to this.
And if possible, find the center, like this.
Essentially, I want to be able to extract the blob of pixels and then find the center point.
I know these are two separate questions, but I figured if someone can do the latter, then they can do the first. I am using MATLAB, but would like to write my own code and not use their image processing functions, like edge(). What methods/algorithms can I use? Any papers/links would be nice (:
Well, assuming that your image only consists of a black background and a bag inside it, a very common way to perform what you're asking is to threshold the image, then find the centroid of all of the white pixels.
I did a Google search and the closest thing that I can think of that matches what you want looks like this:
http://ak.picdn.net/shutterstock/videos/3455555/preview/stock-footage-single-blank-gray-shopping-bag-loop-rotate-on-black-background.jpg
This image is RGB for some reason, even though it's grayscale so we're going to convert this to grayscale. I'm assuming you can't use any built-in MATLAB functions and so rgb2gray is out. You can still implement it yourself though as rgb2gray implements the SMPTE Rec. 709 standard.
Once we read in the image, you can threshold the image and then find the centroid of all of the white pixels. That can be done using find to determine the non-zero row and column locations and then you'd just find the mean of both of them separately. Once we do that, we can show the image and plot a red circle where the centroid is located. As such:
im = imread('http://ak.picdn.net/shutterstock/videos/3455555/preview/stock-footage-single-blank-gray-shopping-bag-loop-rotate-on-black-background.jpg');
%// Convert colour image to grayscale
im = double(im);
im = 0.299*im(:,:,1) + 0.587*im(:,:,2) + 0.114*im(:,:,3);
im = uint8(im);
thresh = 30; %// Choose threshold here
%// Threshold image
im_thresh = im > thresh;
%// Find non-zero locations
[rows,cols] = find(im_thresh);
%// Find the centroid
mean_row = mean(rows);
mean_col = mean(cols);
%// Show the image and the centroid
imshow(im); hold on;
plot(mean_col, mean_row, 'r.', 'MarkerSize', 18);
When I run the above code, this is what we get:
Not bad! Now your next concern is the case of handling multiple objects. As you have intelligently determined, this code only detects one object. For the case of multiple objects, we're going to have to do something different. What you need to do is identify all of the objects in the image by an ID. This means that we need to create a matrix of IDs where each pixel in this matrix denotes which object the object belongs to. After, we iterate through each object ID and find each centroid. This is performed by creating a mask for each ID, finding the centroid of that mask and saving this result. This is what is known as finding the connected components.
regionprops is the most common way to do this in MATLAB, but as you want to implement this yourself, I will defer you to my post I wrote a while ago on how to find the connected components of a binary image:
How to find all connected components in a binary image in Matlab?
Mind you, that algorithm is not the most efficient one so it may take a few seconds, but I'm sure you don't mind the wait :) So let's deal with the case of multiple objects now. I also found this image on Google:
We'd threshold the image as normal, then what's going to be different is performing a connected components analysis, then we iterate through each label and find the centroid. However, an additional constraint that I'm going to enforce is that we're going to check the area of each object found in the connected components result. If it's less than some number, this means that the object is probably attributed to quantization noise and we should skip this result.
Therefore, assuming that you took the code in the above linked post and placed it into a function called conncomptest which has the following prototype:
B = conncomptest(A);
So, take the code in the referenced post, and place it into a function called conncomptest.m with a function header such that:
function B = conncomptest(A)
where A is the input binary image and B is the matrix of IDs, you would do something like this:
im = imread('http://cdn.c.photoshelter.com/img-get2/I0000dqEHPhmGs.w/fit=1000x750/84483552.jpg');
im = double(im);
im = 0.299*im(:,:,1) + 0.587*im(:,:,2) + 0.114*im(:,:,3);
im = uint8(im);
thresh = 30; %// Choose threshold here
%// Threshold image
im_thresh = im > thresh;
%// Perform connected components analysis
labels = conncomptest(im_thresh);
%// Find the total number of objects in the image
num_labels = max(labels(:));
%// Find centroids of each object and show the image
figure;
imshow(im);
hold on;
for idx = 1 : num_labels
%// Find the ith object mask
mask = labels == idx;
%// Find the area
arr = sum(mask(:));
%// If area is less than a threshold
%// don't process this object
if arr < 50
continue;
end
%// Else, find the centroid normally
%// Find non-zero locations
[rows,cols] = find(mask);
%// Find the centroid
mean_row = mean(rows);
mean_col = mean(cols);
%// Show the image and the centroid
plot(mean_col, mean_row, 'r.', 'MarkerSize', 18);
end
We get:
I have no intention of detracting from Ray's (#rayryeng) excellent advice and, as usual, beautifully crafted, reasoned and illustrated answer, however I note that you are interested in solutions other than Matlab and actually want to develop your own code, so I though I would provide some additional options for you.
You could look to the excellent ImageMagick, which is installed in most Linux distros and available for OS X, other good operating systems and Windows. It includes a "Connected Components" method and if you apply it to this image:
like this at the command-line:
convert bags.png -threshold 20% \
-define connected-components:verbose=true \
-define connected-components:area-threshold=600 \
-connected-components 8 -auto-level output.png
The output will be:
Objects (id: bounding-box centroid area mean-color):
2: 630x473+0+0 309.0,252.9 195140 srgb(0,0,0)
1: 248x220+0+0 131.8,105.5 40559 srgb(249,249,249)
7: 299x231+328+186 507.5,304.8 36620 srgb(254,254,254)
3: 140x171+403+0 458.0,80.2 13671 srgb(253,253,253)
12: 125x150+206+323 259.8,382.4 10940 srgb(253,253,253)
8: 40x50+339+221 357.0,248.0 1060 srgb(0,0,0)
which shows 6 objects, one per line, and gives the bounding boxes, centroids and mean-colour of each. So, the 3rd line means a box 299 pixels wide by 231 pixels tall, with its top-left corner at 328 across from the top-left of the image and 186 pixels down from the top-left corner.
If I draw in the bounding boxes, you can see them here:
The centroids are also listed for you.
The outout image from the command above is like this, showing each shape shaded in a different shade of grey. Note that the darkest one has come up black so is VERY hard to see - nearly impossible :-)
If, as you say, you wish to look at writing your own connected component code, you could look at my code in another answer of mine... here
Anyway, I hope this helps and you see it just as an addition to the excellent answer Ray has already provided.
Related
I'm trying to detect seams in welding images for an autonomous welding process.
I want to find pixel positions of the detected line (the red line in the desired image) in the original image.
I used the following code and finally removed noise from the image to reach the result below.
clc,clear,clf;
im = imread('https://i.stack.imgur.com/UJcKA.png');
imshow(im);title('Original image'); pause(0.5);
sim = edge(im, 'sobel');
imshow(sim);title('after Sobel'); pause(0.5);
mask = im > 5;
se = strel('square', 5);
mask_s = imerode(mask, se);
mask(mask_s) = false;
mask = imdilate(mask, se);
sim(mask) = false;
imshow(sim);title('after mask');pause(0.5);
sim= medfilt2(sim);
imshow(sim);title('after noise removal')
Unfortunately there is nothing remaining in the image to find the seam perfectly.
Any help would be appreciated.
Download Original image.
You need to make your filter more robust to noise. This can be done by giving it a larger support:
filter = [ones(2,9);zeros(1,9);-ones(2,9)];
msk = imerode(im > 0, ones(11)); % only object pixels, discarding BG
fim =imfilter(im,filter);
robust = bwmorph((fim>0.75).*msk,'skel',inf); % get only strong pixels
The robust mask looks like:
As you can see, the seam line is well detected, we just need to pick it as the largest connected component:
st = regionprops(bwlabel(robust,8), 'Area', 'PixelList');
[ma mxi] = max([st.Area]); % select the region with the largest area
Now we can fit a polygon (2nd degree) to the seem:
pp=polyfit(st(mxi).PixelList(:,1), st(mxi).PixelList(:,2), 2);
And here it is over the image:
imshow(im, 'border','tight');hold on;
xx=1:size(im,2);plot(xx,polyval(pp,xx)+2,'r');
Note the +2 Y offset due to filter width.
PS,
You might find this thread relevant.
Shai gives a great answer, but I wanted to add a bit more context about why your noise filtering doesn't work.
Why median filtering doesn't work
Wikipedia suggests that median filtering removes noise while preserving edges, which is why you might have chosen to use it. However, in your case it will almost certainly not work, here's why:
Median filtering slides a window across the image. In each area, it replaces the central pixel with the median value from the surrounding window. medfilt2 uses a 3x3 window by default. Let's look at a 3x3 block near your line,
A 3x3 block around [212 157] looks like this
[0 0 0
1 1 1
0 0 0]
The median value is 0! So even though we're in the middle of a line segment, the pixel will be filtered out.
The alternative to median filtering
Shai's method for removing noise instead finds the largest connected group of pixels and ignores smaller groups of pixels. If you also wanted to remove these small groups from your image, Matlab provides a filter bwareaopen which removes small objects from binary images.
For example, if you replace your line
sim= medfilt2(sim);
with
sim= bwareaopen(sim, 4);
The result is much better
Alternative edge detectors
One last note, Shai uses a horizontal gradient filter to find horizontal edges in your image. It works great because your edge is horizontal. If you edge will not always be horizontal, you might want to use another edge detection method. In your original code, you use Sobel, but Matlab provides many options, all of which perform better if you tune their thresholds. As an example, in the following image, I've highlighted the pixels selected by your code (with bwareaopen modification) using four different edge detectors.
I am trying to detect the edge from black horizontal line to the gray-smeared foreground.
The desired edge/result is slightly marked red.
What have I tried so far :
My approach was to use the standard Chan-Vese Segmentation combined with several preprocseeing methods like gaussian blurring, maximum filter or morpholigocal operator like erosion. However, when I am initializing the level set function at the lower part of the image, the contour gets stuck right before the dersired edge.
Due to the noise I can't get rid off without destroying important information of the image, simple methods like the sobel or prewitt filtering might fail.
Another apporoach of me was to search for the maximum/minimum intensity columnwise of the image and mark the darkest pixel per column.
As you can assume, this will fail too because the edge I am looking for is not the only part that has dark pixels, that's why this method is very error-prone.
Edit
Snakes does not help either.
The active contour, marked as blue, simply goes over the edge and at the left and right the contour gets stuck. The Code I tried wasthe function Snake2D(I,P,Options) taken from here.
Here is the original image if you would like to help me.
I think your approach using the rows and finding the maximum is probably easiest.
The one problem you have is distinguishing the two main maxima. For this, you can apply a rough smoothing, to find the middle between the two maxima (blue line in image below). You can then only take the lower bit, which is the one you are interested in and find the maximum of this bit.
In a final step, just add up the two indices.
Result:
Could go like this:
ib = imread('LHkm2.png'); %Read image
sz = size(ib); %get dimensions
for i = 1:sz(2)
[~, ind_mid(i)] = max(smooth(-double(ib(:, i)), 130));%First round
line_to_smooth = ib(ind_mid(i):end, i);%Get line with one maximum
[~, ind(i)] = min(smooth(double(line_to_smooth), 10));%Second round
ind(i) = ind(i) + ind_mid(i);%Add indices to get final position
end
imshow(ib,[]);
hold on;
plot(ind_mid, 'LineWidth', 3);
plot(ind, 'LineWidth', 3);
Note: You can of course smooth the final line just like any other graphs to get rid of bumps like this:
ind = smooth(ind, 10)
where 10 is your smoothing window (the higher the broader, see here.
I am struggling to find a good contour detection function that would count the number of contour in bw images that I have processed using some previous tools. As you can see, my profile picture is an example of such images,
,
In this image, ideally, I wish to have a function which counts four closed contour.
I don't mind if it also detects the really tiny contours in between, or the entire shape itself as extra contours. As long as it counts the medium sized ones, I can fix the rest by applying area threshold. My problem is that any function I have tried detects only one contour - the entire shape, as it cannot separate it to the su-conours which are connected to one another.
Any suggestions?
Here is my shot at this, although your question might get closed because it's off-topic, too broad or a possible duplicate. Anyhow I propose another way to count the number of contours. You could also do it using bwboundaries as was demonstrated in the link provided by #knedlsepp in the possible duplicate. Just for the sake of it here is another way.
The idea is to apply a morphological closure of your image and actually count the number of enclosed surfaces instead instead of contours. That might end up being the same thing but I think it's easier to visualize surfaces.
Since the shapes in your image look like circle (kind of...) the structuring element used to close the image is a disk. The size (here 5) is up to you but for the image you provided its fine. After that, use regionprops to locate image regions (here the blobs) and count them, which comes back to counting contours I guess. You can provide the Area parameter to filter out shapes based on their area. Here I ask the function to provide centroids to plot them.
Whole code:
clear
clc
close all
%// Read, threshold and clean up the image
Im = im2bw(imread('ImContour.png'));
Im = imclearborder(Im);
%// Apply disk structuring element to morphologically close the image.
%// Play around with the size to alter the output.
se = strel('disk',5);
Im_closed = imclose(Im,se);
%// Find centroids of circle-ish shapes. Youcan also get the area to filter
%// out those you don't want.
S = regionprops(~Im_closed,'Centroid','Area');
%// remove the outer border of the image (1st output of regioprops).
S(1) = [];
%// Make array with centroids and show them.
Centro = vertcat(S.Centroid);
imshow(Im)
hold on
scatter(Centro(:,1),Centro(:,2),40,'filled')
And the output:
So as you see the algorithm detected 5 regions, but try playing a bit with the parameters and you will see which ones to change to get the desired output of 4.
Have fun!
I was working on my image processing problem with detecting coins.
I have some images like this one here:
and wanted to separate the falsely connected coins.
We already tried the watershed method as stated on the MATLAB-Homepage:
the-watershed-transform-strategies-for-image-segmentation.html
especially since the first example is exactly our problem.
But instead we get a somehow very messed up separation as you can see here:
We already extracted the area of the coin using the regionprops Extrema parameter and casting the watershed only on the needed area.
I'd appreciate any help with the problem or even another method of getting it separated.
If you have the Image Processing Toolbox, I can also suggest the Circular Hough Transform through imfindcircles. However, this requires at least version R2012a, so if you don't have it, this won't work.
For the sake of completeness, I'll assume you have it. This is a good method if you want to leave the image untouched. If you don't know what the Hough Transform is, it is a method for finding straight lines in an image. The circular Hough Transform is a special case that aims to find circles in the image.
The added advantage of the circular Hough Transform is that it is able to detect partial circles in an image. This means that those regions in your image that are connected, we can detect them as separate circles. How you'd call imfindcircles is in the following fashion:
[centers,radii] = imfindcircles(A, radiusRange);
A would be your binary image of objects, and radiusRange is a two-element array that specifies the minimum and maximum radii of the circles you want to detect in your image. The outputs are:
centers: A N x 2 array that tells you the (x,y) co-ordinates of each centre of a circle that is detected in the image - x being the column and y being the row.
radii: For each corresponding centre detected, this also gives the radius of each circle detected. This is a N x 1 array.
There are additional parameters to imfindcircles that you may find useful, such as the Sensitivity. A higher sensitivity means that it is able to detect circular shapes that are more non-uniform, such as what you are showing in your image. They aren't perfect circles, but they are round shapes. The default sensitivity is 0.85. I set it to 0.9 to get good results. Also, playing around with your image, I found that the radii ranged from 50 pixels to 150 pixels. Therefore, I did this:
im = im2bw(imread('http://dennlinger.bplaced.net/t06-4.jpg'));
[centers,radii] = imfindcircles(im, [50 150], 'Sensitivity', 0.9);
The first line of code reads in your image directly from StackOverflow. I also convert this to logical or true black and white as the image you uploaded is of type uint8. This image is stored in im. Next, we call imfindcircles in the method that we described.
Now, if we want to visualize the detected circles, simply use imshow to show your image, then use the viscircles to draw the circles in the image.
imshow(im);
viscircles(centers, radii, 'DrawBackgroundCircle', false);
viscircles by default draws the circles with a white background over the contour. I want to disable this because your image has white circles and I don't want to show false contouring. This is what I get with the above code:
Therefore, what you can take away from this is the centers and radii variables. centers will give you the centre of each detected circle while radii will tell you what the radii is for each circle.
Now, if you want to simulate what regionprops is doing, we can iterate through all of the detected circles and physically draw them onto a 2D map where each circle would be labeled by an ID number. As such, we can do something like this:
[X,Y] = meshgrid(1:size(im,2), 1:size(im,1));
IDs = zeros(size(im));
for idx = 1 : numel(radii)
r = radii(idx);
cen = centers(idx,:);
loc = (X - cen(1)).^2 + (Y - cen(2)).^2 <= r^2;
IDs(loc) = idx;
end
We first define a rectangular grid of points using meshgrid and initialize an IDs array of all zeroes that is the same size as the image. Next, for each pair of radii and centres for each circle, we define a circle that is centered at this point that extends out for the given radius. We then use these as locations into the IDs array and set it to a unique ID for that particular circle. The result of IDs will be that which resembles the output of bwlabel. As such, if you want to extract the locations of where the idx circle is, you would do:
cir = IDs == idx;
For demonstration purposes, this is what the IDs array looks like once we scale the IDs such that it fits within a [0-255] range for visibility:
imshow(IDs, []);
Therefore, each shaded circle of a different shade of gray denotes a unique circle that was detected with imfindcircles.
However, the shades of gray are probably a bit ambiguous for certain coins as this blends into the background. Another way that we could visualize this is to apply a different colour map to the IDs array. We can try using the cool colour map, with the total number of colours to be the number of unique circles + 1 for the background. Therefore, we can do something like this:
cmap = cool(numel(radii) + 1);
RGB = ind2rgb(IDs, cmap);
imshow(RGB);
The above code will create a colour map such that each circle gets mapped to a unique colour in the cool colour map. The next line applies a mapping where each ID gets associated with a colour with ind2rgb and we finally show the image.
This is what we get:
Edit: the following solution is more adequate to scenarios where one does not require fitting the exact circumferences, although simple heuristics could be used to approximate the radii of the coins in the original image based on the centers found in the eroded one.
Assuming you have access to the Image Processing toolbox, try imerode on your original black and white image. It will apply an erosion morphological operator to your image. In fact, the Matlab webpage with the documentation of that function has an example strikingly similar to your problem/image and they use a disk structure.
Run the following code (based on the example linked above) assuming the image you submitted is called ima.jpg and is local to the code:
ima=imread('ima.jpg');
se = strel('disk',50);
eroded = imerode(ima,se);
imshow(eroded)
and you will see the image that follows as output. After you do this, you can use bwlabel to label the connected components and compute whatever properties you may want, for example, count the number of coins or detect their centers.
I have a (naïve probably) question, I just wanted to clarify this part. So, when I take a dsift on one image I generally get an 128xn matrix. Thing is, that n value, is not always the same across different images. Say image 1 gets an 128x10 matrix, while image 2 gets a 128x18 matrix. I am not quite sure why is this happening.
I think that each column of 128dimension represents a single image feature or a single patch detected from the image. So in the case of 128x18, we have extracted 18 patches and described them with 128 values each. If this is correct, why cant we have a fixed numbers of patches per image, say 20 patches, so every time our matrixes would be 128x20.
Cheers!
This is because the number of reliable features that are detected per image change. Just because you detect 10 features in one image does not mean that you will be able to detect the same number of features in the other image. What does matter is how close one feature from one image matches with another.
What you can do (if you like) is extract the, say, 10 most reliable features that are matched the best between the two images, if you want to have something constant. Choose a number that is less than or equal to the minimum of the number of patches detected between the two. For example, supposing you detect 50 features in one image, and 35 features in another image. After, when you try and match the features together, this results in... say... 20 best matched points. You can choose the best 10, or 15, or even all of the points (20) and proceed from there.
I'm going to show you some example code to illustrate my point above, but bear in mind that I will be using vl_sift and not vl_dsift. The reason why is because I want to show you visual results with minimal pre- and post-processing. Should you choose to use vl_dsift, you'll need to do a bit of work before and after you compute the features by dsift if you want to visualize the same results. If you want to see the code to do that, you can check out the vl_dsift help page here: http://www.vlfeat.org/matlab/vl_dsift.html. Either way, the idea about choosing the most reliable features applies to both sift and dsift.
For example, supposing that Ia and Ib are uint8 grayscale images of the same object or scene. You can first detect features via SIFT, then match the keypoints.
[fa, da] = vl_sift(im2single(Ia));
[fb, db] = vl_sift(im2single(Ib));
[matches, scores] = vl_ubcmatch(da, db);
matches contains a 2 x N matrix, where the first row and second row of each column denotes which feature index in the first image (first row) matched best with the second image (second row).
Once you do this, sort the scores in ascending order. Lower scores mean better matches as the default matching method between two features is the Euclidean / L2 norm. As such:
numBestPoints = 10;
[~,indices] = sort(scores);
%// Get the numBestPoints best matched features
bestMatches = matches(:,indices(1:numBestPoints));
This should then return the 10 best matches between the two images. FWIW, your understanding about how the features are represented in vl_feat is spot on. These are stored in da and db. Each column represents a descriptor of a particular patch in the image, and it is a histogram of 128 entries, so there are 128 rows per feature.
Now, as an added bonus, if you want to display how each feature from one image matches to another image, you can do the following:
%// Spawn a new figure and show the two images side by side
figure;
imagesc(cat(2, Ia, Ib));
%// Extract the (x,y) co-ordinates of each best matched feature
xa = fa(1,bestMatches(1,:));
%// CAUTION - Note that we offset the x co-ordinates of the
%// second image by the width of the first image, as the second
%// image is now beside the first image.
xb = fb(1,bestMatches(2,:)) + size(Ia,2);
ya = fa(2,bestMatches(1,:));
yb = fb(2,bestMatches(2,:));
%// Draw lines between each feature
hold on;
h = line([xa; xb], [ya; yb]);
set(h,'linewidth', 1, 'color', 'b');
%// Use VL_FEAT method to show the actual features
%// themselves on top of the lines
vl_plotframe(fa(:,bestMatches(1,:)));
fb2 = fb; %// Make a copy so we don't mutate the original
fb2(1,:) = fb2(1,:) + size(Ia,2); %// Remember to offset like we did before
vl_plotframe(fb2(:,bestMatches(2,:)));
axis image off; %// Take out the axes for better display