Related
I want to extract the rectangular-like shapes (some may have triangular extensions on one side) with an image like this;
What I have done in MATLAB is;
BW=imread('Capture2.JPG');
BW=~im2bw(BW);
SE = strel('rectangle',[1 6]);
BW = ~imclose(BW,SE);
BW2 = imfill(~BW,'holes');
figure
imshow(~BW)
figure
imshow(BW2);
s = regionprops(BW,'BoundingBox','Area','PixelIdxList');
s = s(2:end);
[lab, numberOfClosedRegions] = bwlabel(BW);
figure
imshow(BW)
for i=1:numel(s)
rec = s(i);
ratio = rec.BoundingBox(4)/rec.BoundingBox(3);%height/width
% ratio > 0.3 && ratio < 51.6 && rec.Area > 1100 && rec.Area < 22500
% if ratio > 0.16
rectangle('Position',s(i).BoundingBox,'EdgeColor','r','LineWidth',2);
text(rec.BoundingBox(1),rec.BoundingBox(2),num2str(i),'fontsize',16);
% end
end
What I have come up is;
As it is seen, there are regions find as part of texts, shape inside of a block(index = 3) and non-block region(index = 11). I need to discard the inside regions and non-block areas.
The other issue is since regions are defined by white areas I need to get the black borders of the blocks so that I can capture the block itself, not the inner white region. How can I solve these issues?
I both tried inverting the image and using the methods but no success.
EDIT: Code improvement & additional image
One of the images can be like this including non-rectangular but object of interest shapes (leftmost).
Another issue if image not as good as it should be some, line considered as open especially diagonal and 1px wide ones which causes regionprops misses them.
Code improvement;
close all;
image=imread('Capture1.JPG');
BW = image;
BW = ~im2bw(BW);
SE = strel('rectangle',[3 6]);
BW = ~imclose(BW,SE); % closes some caps to be closed region
r = regionprops(BW,'PixelIdxList');
BW(r(1).PixelIdxList) = 0; %removes outermost white space allowing to connection lines disapear
se = strel('rectangle',[6 1]);
BW = imclose(BW,se);% closes some caps to be closed region
BW = imfill(BW,'holes');
s = regionprops(BW,{'Area', 'ConvexArea', 'BoundingBox','ConvexHull','PixelIdxList'});
%mostly the area and convex area are similar but if convex area is much greater than the area itself it is a complex shape like concave intermediate sections then remove
noidx = [];
for i=1:numel(s)
rec = s(i);
if rec.Area*1.5 < rec.ConvexArea
BW(rec.PixelIdxList) = 0;
noidx(end+1)=i;
end
end
s(noidx)=[];
%no condition for remaining regions
figure
imshow(BW)
for i=1:numel(s)
rec = s(i);
ratio = rec.BoundingBox(4)/rec.BoundingBox(3);%height/width
% ratio > 0.3 && ratio < 51.6 && rec.Area > 1100 && rec.Area < 22500
% if ratio > 0.16
rectangle('Position',s(i).BoundingBox,'EdgeColor','r','LineWidth',2);
text(rec.BoundingBox(1),rec.BoundingBox(2),num2str(i),'fontsize',16,'color','red');
% end
end
Result is;
Advantage is all the remaining regions are region if interest no exception and no condition for area constraint etc. because image size can be different thus the area.
But even this doesn't work on second image. Because of the text below the blocks (which is always the case -> first image was cleared to be uploaded) and the diagonal tips of the leftmost blocks considered open lines.
By adding two conditions I got some good results:
The rectangle need to be fully closed
The area need to be bigger than x pixels (1100 in this case)
In order to check if the rectangle is closed or not, I created an index for each polygon. Those index have the same shape as the rectangles. So if sum(~BW(index)) == sum(index(:)) it mean that the polygon is closed.
The updated code:
warning off
BW=imread('test.jpg');
BW=~im2bw(BW);
SE = strel('rectangle',[1 6]);
BW = ~imclose(BW,SE);
BW2 = imfill(~BW,'holes');
s = regionprops(BW,'BoundingBox','Area','PixelIdxList');
s = s(2:end);
[lab, numberOfClosedRegions] = bwlabel(BW);
figure
imshow(imread('test.jpg'))
inc = 1;
for i=1:numel(s)
rec = s(i);
s(i).BoundingBox = floor(s(i).BoundingBox + [-1,-1,2,2]);
%Creation of the index
clear ind
ind = zeros(size(BW));
y = s(i).BoundingBox(1);
x = s(i).BoundingBox(2);
h = s(i).BoundingBox(3);
w = s(i).BoundingBox(4);
ind(x:x+w,[y,y+h]) = 1;
ind([x,x+w],y:y+h) = 1;
ind = logical(ind);
if sum(~BW(ind)) == sum(ind(:)) && rec.Area > 1100
rectangle('Position',s(i).BoundingBox,'EdgeColor','r','LineWidth',1);
text(rec.BoundingBox(1),rec.BoundingBox(2),num2str(inc),'fontsize',16);
inc = inc + 1;
end
end
RESULT
How can you discard non-rectangular regions? Well I'm sure you can come up with some mathematical properties which are pretty unique to rectangles.
The area is the product of width and heigth.
The perimeter is the twice the sum of width and height.
It obviously has 4 rectangular corners.
I guess the first property would be sufficient and that you can come up with more rules if you need to.
You can get rid of unwanted rectangles and other small stuff with a minimum size constraint or check if they are enclosed by a rectangle.
This should be pretty straight forward.
I am trying to create a dynamic random dots stereogram by updating the position of the dots, so that I can get an inward motion of the dots (optic flow).
I first created a disk filled with dots whose position is randomised:
se = strel('disk',4);
x = linspace(-1,1,768); %generates 768 points in a vector with a [-1:1] interval
[X,Y] = meshgrid(x,x); %generates a rectangular grid in replicating the x vector.
[T,R] = cart2pol(X,Y);
threshold = 1 - (dot_nb / 768^2); %dot_nb=5;
Rd = double(rand(768,768)>threshold);
Rd(R>0.95 | R<0.06)=0; %remove some of the dots in order to get a circle and no dots in the middle of it
I am now trying to update the location of the dots at a specific refresh rate (20Hz) but I am having troubles with it.
I thought the following lines of code (which are in a bigger loop controlling the frame rate) could work but obviously something is missing and I do not know how else I could do it:
update = 0.1;
for i = 1:size(R,1) %R is the matrix with the radius values of the dots
for j = 1:size(R,2)
if R(i,j)<0.95 && R(i,j)>0.06
R(i,j) = R(i,j)-update;
else
R(i,j)=0;
end
end
end
Has anyone any idea how I could do to get what I'm looking for?
Thank you in advance!
I'm making an image processing project and I have stuck in one the project's steps. Here is the situation;
This is my mask:
and I want to detect the maximum-sized rectangle that can fit into this mask like this.
I'm using MATLAB for my project. Do you know any fast way to accomplish this aim. Any code sample, approach or technique would be great.
EDIT 1 : The two algorithms below are works with lot's of the cases. But both of them give wrong results in some difficult cases. I'am using both of them in my project.
This approach starts with the entire image and shrinks each border in turn pixel-by-pixel until it finds an acceptable rectangle.
It takes ~0.02 seconds to run on the example image, so it's reasonably fast.
EDIT: I should clarify that this isn't meant to be a universal solution. This algorithm relies on the rectangle being centered and having roughly the same aspect ratio as the image itself. However, in the cases where it is appropriate, it is fast. #DanielHsH offered a solution which they claim works in all cases.
The code:
clear; clc;
tic;
%% // read image
imrgb= imread('box.png');
im = im2bw(rgb2gray(imrgb)); %// binarize image
im = 1-im; %// convert "empty" regions to 0 intensity
[rows,cols] = size(im);
%% // set up initial parameters
ULrow = 1; %// upper-left row (param #1)
ULcol = 1; %// upper-left column (param #2)
BRrow = rows; %// bottom-right row (param #3)
BRcol = cols; %// bottom-right column (param #4)
parameters = 1:4; %// parameters left to be updated
pidx = 0; %// index of parameter currently being updated
%% // shrink region until acceptable
while ~isempty(parameters); %// update until all parameters reach bounds
%// 1. update parameter number
pidx = pidx+1;
pidx = mod( pidx-1, length(parameters) ) + 1;
p = parameters(pidx); %// current parameter number
%// 2. update current parameter
if p==1; ULrow = ULrow+1; end;
if p==2; ULcol = ULcol+1; end;
if p==3; BRrow = BRrow-1; end;
if p==4; BRcol = BRcol-1; end;
%// 3. grab newest part of region (row or column)
if p==1; region = im(ULrow,ULcol:BRcol); end;
if p==2; region = im(ULrow:BRrow,ULcol); end;
if p==3; region = im(BRrow,ULcol:BRcol); end;
if p==4; region = im(ULrow:BRrow,BRcol); end;
%// 4. if the new region has only zeros, stop shrinking the current parameter
if isempty(find(region,1))
parameters(pidx) = [];
end
end
toc;
params = [ULrow ULcol BRrow BRcol]
area = (BRrow-ULrow)*(BRcol-ULcol)
The results for this image:
Elapsed time is 0.027032 seconds.
params =
10 25 457 471
area =
199362
Code to visualize results:
imrgb(params(1):params(3),params(2):params(4),1) = 0;
imrgb(params(1):params(3),params(2):params(4),2) = 255;
imrgb(params(1):params(3),params(2):params(4),3) = 255;
imshow(imrgb);
Another example image:
Here is a correct answer.
You must use dynamic programming! Other methods of direct calculation (like cutting 1 pixel from each edge) might produce sub-optimal results. My method guarantees that it selects the largest possible rectangle that fits in the mask. I assume that the mask has 1 big convex white blob of any shape with black background around it.
I wrote 2 methods. findRect() which finds the best possible square (starting on x,y with length l). The second method LargestInscribedImage() is an example of how to find any rectangle (of any aspect ratio). The trick is to resize the mask image, find a square and resize it back.
In my example the method finds the larges rectangle that can be fit in the mask having the same aspect ration as the mask image. For example if the mask image is of size 100x200 pixels than the algorithm will find the largest rectangle having aspect ratio 1:2.
% ----------------------------------------------------------
function LargestInscribedImage()
% ----------------------------------------------------------
close all
im = double(imread('aa.bmp')); % Balck and white image of your mask
im = im(:,:,1); % If it is colored RGB take only one of the channels
b = imresize(im,[size(im,1) size(im,1)]); Make the mask square by resizing it by its aspect ratio.
SC = 1; % Put 2..4 to scale down the image an speed up the algorithm
[x1,y1,l1] = findRect(b,SC); % Lunch the dyn prog algorithm
[x2,y2,l2] = findRect(rot90(b),SC); % rotate the image by 90deg and solve
% Rotate back: x2,y2 according to rot90
tmp = x2;
x2 = size(im,1)/SC-y2-l2;
y2 = tmp;
% Select the best solution of the above (for the original image and for the rotated by 90degrees
if (l1>=l2)
corn = sqCorn(x1,y1,l1);
else
corn = sqCorn(x2,y2,l2);
end
b = imresize(b,1/SC);
figure;imshow(b>0); hold on;
plot(corn(1,:),corn(2,:),'O')
corn = corn*SC;
corn(1,:) = corn(1,:)*size(im,2)/size(im,1);
figure;imshow(im); hold on;
plot(corn(1,:),corn(2,:),'O')
end
function corn = sqCorn(x,y,l)
corn = [x,y;x,y+l;x+l,y;x+l,y+l]';
end
% ----------------------------------------------------------
function [x,y,l] = findRect(b,SC)
b = imresize(b,1/SC);
res = zeros(size(b,1),size(b,2),3);
% initialize first col
for i = 1:1:size(b,1)
if (b(i,1) > 0)
res(i,1,:) = [i,1,0];
end
end
% initialize first row
for i = 1:1:size(b,2)
if (b(1,i) > 0)
res(1,i,:) = [1,i,0];
end
end
% DynProg
for i = 2:1:size(b,1)
for j = 2:1:size(b,2)
isWhite = b(i,j) > 0;
if (~isWhite)
res(i,j,:)=res(i-1,j-1,:); % copy
else
if (b(i-1,j-1)>0) % continuous line
lineBeg = [res(i-1,j-1,1),res(i-1,j-1,2)];
lineLenght = res(i-1,j-1,3);
if ((b(lineBeg(1),j)>0)&&(b(i,lineBeg(2))>0)) % if second diag is good
res(i,j,:) = [lineBeg,lineLenght+1];
else
res(i,j,:)=res(i-1,j-1,:); % copy since line has ended
end
else
res(i,j,:) = [i,j,0]; % Line start
end
end
end
end
% check last col
[maxValCol,WhereCol] = max(res(:,end,3));
% check last row
[maxValRow,WhereRow] = max(res(end,:,3));
% Find max
x= 0; y = 0; l = 0;
if (maxValCol>maxValRow)
y = res(WhereCol,end,1);
x = res(WhereCol,end,2);
l = maxValCol;
else
y = res(end,WhereRow,1);
x = res(end,WhereRow,2);
l = maxValRow;
end
corn = [x,y;x,y+l;x+l,y;x+l,y+l]';
% figure;imshow(b>0); hold on;
% plot(corn(1,:),corn(2,:),'O')
return;
end
The black boundaries in your image are curved and not closed. For example, in the top right corner, the black boundaries won't meet and form a closed contour. Therefore, a simple strategy in one of my comments will not work.
I am now providing you with a skeleton of a code which you can play with and add conditions as per your need. My idea is as follows:
To find left-side x-coordinate of the rectangle, first count the white pixels each column of the image contains:
%I assume that the image has already been converted to binary.
whitePixels=sum(img,1);
Then find the rate of change:
diffWhitePixels=diff(whitePixels);
If you see the bar plot of diffWhitePixels then you will observe various large entries (which indicate that the white region is still not in a straight line, and it is not a proper place to put the rectangles left vertical edge). Small entries (in your image, less than 5) indicate you can put the rectangle edge there.
You can do similar things to determine right, top and bottom edge positions of the rectangles.
Discussion:
First of all, the problem is ill-posed in my opinion. What do you mean by maximum-sized rectangle? Is it maximum area or length of side? In all possible cases, I don't think above method can get the correct answer. I can think of two or three cases right now where above method would fail, but it will at least give you the right answer on images similar to the given image, provided you adjust the values.
You can put some constraints once you know how your images are going to look. For example, if the black boundary curves inside, you can say that you don't want a column such as [0;0;...0;1;1;...0;0;...;0;1;1;...;1] i.e. zeros surrounded by ones. Another constraint could be how many black pixels do you want to allow? You can also crop the image till to remove extra black pixels. In your image, you can crop the image (programmatically) from the left and the bottom edge. Cropping an image is probably necessary, and definitely the better thing to do.
I'm making an image processing project and I have stuck in one the project's steps. Here is the situation;
This is my mask:
and I want to detect the maximum-sized rectangle that can fit into this mask like this.
I'm using MATLAB for my project. Do you know any fast way to accomplish this aim. Any code sample, approach or technique would be great.
EDIT 1 : The two algorithms below are works with lot's of the cases. But both of them give wrong results in some difficult cases. I'am using both of them in my project.
This approach starts with the entire image and shrinks each border in turn pixel-by-pixel until it finds an acceptable rectangle.
It takes ~0.02 seconds to run on the example image, so it's reasonably fast.
EDIT: I should clarify that this isn't meant to be a universal solution. This algorithm relies on the rectangle being centered and having roughly the same aspect ratio as the image itself. However, in the cases where it is appropriate, it is fast. #DanielHsH offered a solution which they claim works in all cases.
The code:
clear; clc;
tic;
%% // read image
imrgb= imread('box.png');
im = im2bw(rgb2gray(imrgb)); %// binarize image
im = 1-im; %// convert "empty" regions to 0 intensity
[rows,cols] = size(im);
%% // set up initial parameters
ULrow = 1; %// upper-left row (param #1)
ULcol = 1; %// upper-left column (param #2)
BRrow = rows; %// bottom-right row (param #3)
BRcol = cols; %// bottom-right column (param #4)
parameters = 1:4; %// parameters left to be updated
pidx = 0; %// index of parameter currently being updated
%% // shrink region until acceptable
while ~isempty(parameters); %// update until all parameters reach bounds
%// 1. update parameter number
pidx = pidx+1;
pidx = mod( pidx-1, length(parameters) ) + 1;
p = parameters(pidx); %// current parameter number
%// 2. update current parameter
if p==1; ULrow = ULrow+1; end;
if p==2; ULcol = ULcol+1; end;
if p==3; BRrow = BRrow-1; end;
if p==4; BRcol = BRcol-1; end;
%// 3. grab newest part of region (row or column)
if p==1; region = im(ULrow,ULcol:BRcol); end;
if p==2; region = im(ULrow:BRrow,ULcol); end;
if p==3; region = im(BRrow,ULcol:BRcol); end;
if p==4; region = im(ULrow:BRrow,BRcol); end;
%// 4. if the new region has only zeros, stop shrinking the current parameter
if isempty(find(region,1))
parameters(pidx) = [];
end
end
toc;
params = [ULrow ULcol BRrow BRcol]
area = (BRrow-ULrow)*(BRcol-ULcol)
The results for this image:
Elapsed time is 0.027032 seconds.
params =
10 25 457 471
area =
199362
Code to visualize results:
imrgb(params(1):params(3),params(2):params(4),1) = 0;
imrgb(params(1):params(3),params(2):params(4),2) = 255;
imrgb(params(1):params(3),params(2):params(4),3) = 255;
imshow(imrgb);
Another example image:
Here is a correct answer.
You must use dynamic programming! Other methods of direct calculation (like cutting 1 pixel from each edge) might produce sub-optimal results. My method guarantees that it selects the largest possible rectangle that fits in the mask. I assume that the mask has 1 big convex white blob of any shape with black background around it.
I wrote 2 methods. findRect() which finds the best possible square (starting on x,y with length l). The second method LargestInscribedImage() is an example of how to find any rectangle (of any aspect ratio). The trick is to resize the mask image, find a square and resize it back.
In my example the method finds the larges rectangle that can be fit in the mask having the same aspect ration as the mask image. For example if the mask image is of size 100x200 pixels than the algorithm will find the largest rectangle having aspect ratio 1:2.
% ----------------------------------------------------------
function LargestInscribedImage()
% ----------------------------------------------------------
close all
im = double(imread('aa.bmp')); % Balck and white image of your mask
im = im(:,:,1); % If it is colored RGB take only one of the channels
b = imresize(im,[size(im,1) size(im,1)]); Make the mask square by resizing it by its aspect ratio.
SC = 1; % Put 2..4 to scale down the image an speed up the algorithm
[x1,y1,l1] = findRect(b,SC); % Lunch the dyn prog algorithm
[x2,y2,l2] = findRect(rot90(b),SC); % rotate the image by 90deg and solve
% Rotate back: x2,y2 according to rot90
tmp = x2;
x2 = size(im,1)/SC-y2-l2;
y2 = tmp;
% Select the best solution of the above (for the original image and for the rotated by 90degrees
if (l1>=l2)
corn = sqCorn(x1,y1,l1);
else
corn = sqCorn(x2,y2,l2);
end
b = imresize(b,1/SC);
figure;imshow(b>0); hold on;
plot(corn(1,:),corn(2,:),'O')
corn = corn*SC;
corn(1,:) = corn(1,:)*size(im,2)/size(im,1);
figure;imshow(im); hold on;
plot(corn(1,:),corn(2,:),'O')
end
function corn = sqCorn(x,y,l)
corn = [x,y;x,y+l;x+l,y;x+l,y+l]';
end
% ----------------------------------------------------------
function [x,y,l] = findRect(b,SC)
b = imresize(b,1/SC);
res = zeros(size(b,1),size(b,2),3);
% initialize first col
for i = 1:1:size(b,1)
if (b(i,1) > 0)
res(i,1,:) = [i,1,0];
end
end
% initialize first row
for i = 1:1:size(b,2)
if (b(1,i) > 0)
res(1,i,:) = [1,i,0];
end
end
% DynProg
for i = 2:1:size(b,1)
for j = 2:1:size(b,2)
isWhite = b(i,j) > 0;
if (~isWhite)
res(i,j,:)=res(i-1,j-1,:); % copy
else
if (b(i-1,j-1)>0) % continuous line
lineBeg = [res(i-1,j-1,1),res(i-1,j-1,2)];
lineLenght = res(i-1,j-1,3);
if ((b(lineBeg(1),j)>0)&&(b(i,lineBeg(2))>0)) % if second diag is good
res(i,j,:) = [lineBeg,lineLenght+1];
else
res(i,j,:)=res(i-1,j-1,:); % copy since line has ended
end
else
res(i,j,:) = [i,j,0]; % Line start
end
end
end
end
% check last col
[maxValCol,WhereCol] = max(res(:,end,3));
% check last row
[maxValRow,WhereRow] = max(res(end,:,3));
% Find max
x= 0; y = 0; l = 0;
if (maxValCol>maxValRow)
y = res(WhereCol,end,1);
x = res(WhereCol,end,2);
l = maxValCol;
else
y = res(end,WhereRow,1);
x = res(end,WhereRow,2);
l = maxValRow;
end
corn = [x,y;x,y+l;x+l,y;x+l,y+l]';
% figure;imshow(b>0); hold on;
% plot(corn(1,:),corn(2,:),'O')
return;
end
The black boundaries in your image are curved and not closed. For example, in the top right corner, the black boundaries won't meet and form a closed contour. Therefore, a simple strategy in one of my comments will not work.
I am now providing you with a skeleton of a code which you can play with and add conditions as per your need. My idea is as follows:
To find left-side x-coordinate of the rectangle, first count the white pixels each column of the image contains:
%I assume that the image has already been converted to binary.
whitePixels=sum(img,1);
Then find the rate of change:
diffWhitePixels=diff(whitePixels);
If you see the bar plot of diffWhitePixels then you will observe various large entries (which indicate that the white region is still not in a straight line, and it is not a proper place to put the rectangles left vertical edge). Small entries (in your image, less than 5) indicate you can put the rectangle edge there.
You can do similar things to determine right, top and bottom edge positions of the rectangles.
Discussion:
First of all, the problem is ill-posed in my opinion. What do you mean by maximum-sized rectangle? Is it maximum area or length of side? In all possible cases, I don't think above method can get the correct answer. I can think of two or three cases right now where above method would fail, but it will at least give you the right answer on images similar to the given image, provided you adjust the values.
You can put some constraints once you know how your images are going to look. For example, if the black boundary curves inside, you can say that you don't want a column such as [0;0;...0;1;1;...0;0;...;0;1;1;...;1] i.e. zeros surrounded by ones. Another constraint could be how many black pixels do you want to allow? You can also crop the image till to remove extra black pixels. In your image, you can crop the image (programmatically) from the left and the bottom edge. Cropping an image is probably necessary, and definitely the better thing to do.
I have various sets of points, which are supposed to be part of different lines (see image below). What I want to do is to group the points which are part of the same line. To do so, I made a Hough transform that turns cartesian coordinates into polar one. What I need to do is to locate the position of the intersection in the Hough plane; do you have any suggestion on how to do it?
This is what I am doing now:
Apply a threshold to the image of the Hough transform, so to
consider only the pixels with at least half of the maximum intensity
max_Hough = max(max(Hough));
for ii = 1:dim_X
for jj = 1:dim_Y
if Hough(ii,jj) > max_Hough*0.5
Hough_bin(ii,jj) = Hough(ii,jj);
end
end
end
Locate the blobs in the corresponding image
se = strel('disk',1);
Hough_final = imclose(Hough_bin,se);
[L,num] = bwlabel(Hough_final);
% Let's exclude the single points
counts = sum(bsxfun(#eq,L(:),1:num));
valid_blobs = find(counts>threshold);
number_valid = length(find(counts>threshold));
if (number_valid~=0)
for blob_num = (1:number_valid)
for x = (1:dim_X)
for y = (1:dim_X)
if (L(x,y) == valid_blobs(blob_num))
hist_bin_clean(x,y) = Hough_final(x,y);
end
end
end
end
end
Find the centroids of the blobs
Ilabel = bwlabel(hist_bin_clean);
stat = regionprops(Ilabel,'centroid');
You can see the result in the second image below, with the centroid pointed by the white arrow. I would expect it to be a few pixels higher and to the left... How would you improve the result?