Related
I'm working on an application to determine from an image the degree of alignment of a fiber network. I've read several papers on this issue and they basically do this:
Find the 2D discrete Fourier transform (DFT = F(u,v)) of the image (gray, range 0-255)
Find the Fourier Spectrum (FS = abs(F(u,v))) and the Power Spectrum (PS = FS^2)
Convert spectrum to polar coordinates and divide it into 1º intervals.
Calculate number-averaged line intensities (FI) for each interval (theta), that is, the average of all the intensities (pixels) forming "theta" degrees with respect to the horizontal axis.
Transform FI(theta) to cartesian coordinates
Cxy(theta) = [FI*cos(theta), FI*sin(theta)]
Find eigenvalues (lambda1 and lambda2) of the matrix Cxy'*Cxy
Find alignment index as alpha = 1 - lamda2/lambda1
I've implemented this in MATLAB (code below), but I'm not sure whether it is ok since point 3 and 4 are not really clear for me (I'm getting similar results to those of the papers, but not in all cases). For instance, in point 3, "spectrum" is referring to FS or to PS?. And in point 4, how should this average be done? are all the pixels considered? (even though there are more pixels in the diagonal).
rgb = imread('network.tif');%513x513 pixels
im = rgb2gray(rgb);
im = imrotate(im,-90);%since FFT space is rotated 90º
FT = fft2(im) ;
FS = abs(FT); %Fourier spectrum
PS = FS.^2; % Power spectrum
FS = fftshift(FS);
PS = fftshift(PS);
xoffset = (513-1)/2;
yoffset = (513-1)/2;
% Avoid low frequency points
x1 = 5;
y1 = 0;
% Maximum high frequency pixels
x2 = 255;
y2 = 0;
for theta = 0:pi/180:pi
% Transposed rotation matrix
Rt = [cos(theta) sin(theta);
-sin(theta) cos(theta)];
% Find radial lines necessary for improfile
xy1_rot = Rt * [x1; y1] + [xoffset; yoffset];
xy2_rot = Rt * [x2; y2] + [xoffset; yoffset];
plot([xy1_rot(1) xy2_rot(1)], ...
[xy1_rot(2) xy2_rot(2)], ...
'linestyle','none', ...
'marker','o', ...
'color','k');
prof = improfile(F,[xy1_rot(1) xy2_rot(1)],[xy1_rot(2) xy2_rot(2)]);
i = i + 1;
FI(i) = sum(prof(:))/length(prof);
Cxy(i,:) = [FI(i)*cos(theta), FI(i)*sin(theta)];
end
C = Cxy'*Cxy;
[V,D] = eig(C)
lambda2 = D(1,1);
lambda1 = D(2,2);
alpha = 1 - lambda2/lambda1
Figure: A) original image, B) plot of log(P+1), C) polar plot of FI.
My main concern is that when I choose an artificial image perfectly aligned (attached figure), I get alpha = 0.91, and it should be exactly 1.
Any help will be greatly appreciated.
PD: those black dots in the middle plot are just the points used by improfile.
I believe that there are a couple sources of potential error here that are leading to you not getting a perfect alpha value.
Discrete Fourier Transform
You have discrete imaging data which forces you to take a discrete Fourier transform which inevitably (depending on the resolution of the input data) have some accuracy issues.
Binning vs. Sampling Along a Line
The way that you have done the binning is that you literally drew a line (rotated by a particular angle) and sampled the image along that line using improfile. Using improfile performs interpolation of your data along that line introducing yet another potential source of error. The default is nearest neighbor interpolation which in the example shown below can cause multiple "profiles" to all pick up the same points.
This was with a rotation of 1-degree off-vertical when technically you'd want those peaks to only appear for a perfectly vertical line. It is clear to see how this sort of interpolation of the Fourier spectrum can lead to a spread around the "correct" answer.
Data Undersampling
Similar to Nyquist sampling in the Fourier domain, sampling in the spatial domain has some requirements as well.
Imagine for a second that you wanted to use 45-degree bin widths instead of the 1-degree. Your approach would still sample along a thin line and use that sample to represent 45-degrees worth or data. Clearly, this is a gross under-sampling of the data and you can imagine that the result wouldn't be very accurate.
It becomes more and more of an issue the further you get from the center of the image since the data in this "bin" is really pie wedge shaped and you're approximating it with a line.
A Potential Solution
A different approach to binning would be to determine the polar coordinates (r, theta) for all pixel centers in the image. Then to bin the theta components into 1-degree bins. Then sum all of the values that fall into that bin.
This has several advantages:
It removes the undersampling that we talked about and draws samples from the entire "pie wedge" regardless of the sampling angle.
It ensures that each pixel belongs to one and only one angular bin
I have implemented this alternate approach in the code below with some false horizontal line data and am able to achieve an alpha value of 0.988 which I'd say is pretty good given the discrete nature of the data.
% Draw a bunch of horizontal lines
data = zeros(101);
data([5:5:end],:) = 1;
fourier = fftshift(fft2(data));
FS = abs(fourier);
PS = FS.^2;
center = fliplr(size(FS)) / 2;
[xx,yy] = meshgrid(1:size(FS,2), 1:size(FS, 1));
coords = [xx(:), yy(:)];
% De-mean coordinates to center at the middle of the image
coords = bsxfun(#minus, coords, center);
[theta, R] = cart2pol(coords(:,1), coords(:,2));
% Convert to degrees and round them to the nearest degree
degrees = mod(round(rad2deg(theta)), 360);
degreeRange = 0:359;
% Band pass to ignore high and low frequency components;
lowfreq = 5;
highfreq = size(FS,1)/2;
% Now average everything with the same degrees (sum over PS and average by the number of pixels)
for k = degreeRange
ps_integral(k+1) = mean(PS(degrees == k & R > lowfreq & R < highfreq));
fs_integral(k+1) = mean(FS(degrees == k & R > lowfreq & R < highfreq));
end
thetas = deg2rad(degreeRange);
Cxy = [ps_integral.*cos(thetas);
ps_integral.*sin(thetas)]';
C = Cxy' * Cxy;
[V,D] = eig(C);
lambda2 = D(1,1);
lambda1 = D(2,2);
alpha = 1 - lambda2/lambda1;
This is the processed image and I can't increase the bwareaopen() as it won't work for my other image.
Anyway I'm trying to find the shortest points in the centre points of the barcode, to get the straight line across the centre points in the barcode.
Example:
After doing a centroid command, the points in the barcode are near to each other. Therefore, I just wanted to get the shortest points(which is the barcode) and draw a straight line across.
All the points need not be join, best fit points will do.
Step 1
Step 2
Step 3
If you dont have the x,y elements Andrey uses, you can find them by segmenting the image and using a naive threshold value on the area to avoid including the number below the bar code.
I've hacked out a solution in MATLAB doing the following:
Loading the image and making it binary
Extracting all connected components using bwlabel().
Getting useful information about each of them via regionprops() [.centroid will be a good approximation to the middel point for the lines].
Thresholded out small regions (noise and numbers)
Extracted x,y coordinates
Used Andreys linear fit solution
Code:
set(0,'DefaultFigureWindowStyle','docked');
close all;clear all;clc;
Im = imread('29ekeap.jpg');
Im=rgb2gray(Im);
%%
%Make binary
temp = zeros(size(Im));
temp(Im > mean(Im(:)))=1;
Im = temp;
%Visualize
f1 = figure(1);
imagesc(Im);colormap(gray);
%Find connected components
LabelIm = bwlabel(Im);
RegionInfo = regionprops(LabelIm);
%Remove background region
RegionInfo(1) = [];
%Get average area of regions
AvgArea = mean([RegionInfo(1:end).Area]);
%Vector to keep track of likely "bar elements"
Bar = zeros(length(RegionInfo),1);
%Iterate over regions, plot centroids if area is big enough
for i=1:length(RegionInfo)
if RegionInfo(i).Area > AvgArea
hold on;
plot(RegionInfo(i).Centroid(1),RegionInfo(i).Centroid(2),'r*')
Bar(i) = 1;
end
end
%Extract x,y points for interpolation
X = [RegionInfo(Bar==1).Centroid];
X = reshape(X,2,length(X)/2);
x = X(1,:);
y = X(2,:);
%Plot line according to Andrey
p = polyfit(x,y,1);
xMin = min(x(:));
xMax = max(x(:));
xRange = xMin:0.01:xMax;
yRange = p(1).*xRange + p(2);
plot(xRange,yRange,'LineWidth',2,'Color',[0.9 0.2 0.2]);
The result is a pretty good fitted line. You should be able to extend it to the ends by using the 'p' polynomal and evaluate when you dont encounter any more '1's if needed.
Result:
If you already found the x,y of the centers, you should use polyfit function:
You will then find the polynomial coefficients of the best line. In order to draw a segment, you can take the minimal and maximal x
p = polyfit(x,y,1);
xMin = min(x(:));
xMax = max(x(:));
xRange = xMin:0.01:xMax;
yRange = p(1).*xRange + p(2);
plot(xRange,yRange);
If your ultimate goal is to generate a line perpendicular to the bars in the bar code and passing roughly through the centroids of the bars, then I have another option for you to consider...
A simple solution would be to perform a Hough transform to detect the primary orientation of lines in the bar code. Once you find the angle of the lines in the bar code, all you have to do is rotate that by 90 degrees to get the slope of a perpendicular line. The centroid of the entire bar code can then be used as an intercept for this line. Using the functions HOUGH and HOUGHPEAKS from the Image Processing Toolbox, here's the code starting with a cropped version of your image from step 1:
img = imread('bar_code.jpg'); %# Load the image
img = im2bw(img); %# Convert from RGB to BW
[H, theta, rho] = hough(img); %# Perform the Hough transform
peak = houghpeaks(H); %# Find the peak pt in the Hough transform
barAngle = theta(peak(2)); %# Find the angle of the bars
slope = -tan(pi*(barAngle + 90)/180); %# Compute the perpendicular line slope
[y, x] = find(img); %# Find the coordinates of all the white image points
xMean = mean(x); %# Find the x centroid of the bar code
yMean = mean(y); %# Find the y centroid of the bar code
xLine = 1:size(img,2); %# X points of perpendicular line
yLine = slope.*(xLine - xMean) + yMean; %# Y points of perpendicular line
imshow(img); %# Plot bar code image
hold on; %# Add to the plot
plot(xMean, yMean, 'r*'); %# Plot the bar code centroid
plot(xLine, yLine, 'r'); %# Plot the perpendicular line
And here's the resulting image:
I posted another question about the Roberts operator, but I decided to post a new one since my code has changed significantly since that time.
My code runs, but it does not generate the correct image, instead the image becomes slightly brighter.
I have not found a mistake in the algorithm, but I know this is not the correct output. If I compare this program's output to edge(<image matrix>,'roberts',<threshold>);, or to images on wikipedia, it looks nothing like the effect of the roberts operator shown there.
code:
function [] = Robertize(filename)
Img = imread(filename);
NewImg = Img;
SI = size(Img);
I_W = SI(2)
I_H = SI(1)
Robertsx = [1,0;0,-1];
Robertsy = [0,-1;1,0];
M_W = 2; % do not need the + 1, I assume the for loop means while <less than or equal to>
% x and y are reversed...
for y=1 : I_H
for x=1 : I_W
S = 0;
for M_Y = 1 : M_W
for M_X = 1 : M_W
if (x + M_X - 1 < 1) || (x + M_X - 1 > I_W)
S = 0;
%disp('out of range, x');
continue
end
if (y + M_Y - 1 < 1) || (y + M_Y - 1 > I_H)
S = 0;
%disp('out of range, y');
continue
end
S = S + Img(y + M_Y - 1 , x + M_X - 1) * Robertsx(M_Y,M_X);
S = S + Img(y + M_Y - 1, x + M_X - 1) * Robertsy(M_Y,M_X);
% It is y + M_Y - 1 because you multiply Robertsx(1,1) *
% Img(y,x).
end
end
NewImg(y,x) = S;
end
end
imwrite(NewImg,'Roberts.bmp');
end
I think you may be misinterpreting how the Roberts Cross operator works. Use this page as a guide. Notice that it states that you convolve the original image separately with the X and Y operator. Then, you may calculate the final gradient (i.e. "total edge content") value by taking the square root of the sum of squares of the two (x and y) gradient values for a particular pixel. You're presently summing the x and y values into a single image, which will not give the correct results.
EDIT
I'll try to explain a bit better. The problem with summation instead of squaring/square root is that you can end up with negative values. Negative values are natural using this operator depending on the edge orientation. That may be why you think the image 'lightens' -- because when you display the image in MATLAB the negative values go to black, the zero values go to grey, and the positive values go to white. Here's the image I get when I run your code (with a few changes -- mostly setting NewImg to be zeros(size(Img)) so it's a double type instead of uint8. uint8 types don't allow negative values... Here's the image I get:.
You have to be very careful when trying to save files as well. Instead of calling imwrite, call imshow(NewImg,[]). That will automatically rescale the values in the double-valued image to show them correctly, with the most negative number being equal to black and most positive equal to white. Thus, in areas with little edge content (like the sky), we would expect grey and that's what we get!
I ran your code and got the effect you described. See how everything looks lighter:
Figure 1 - Original on the left, original roberts transformation on the right
The image on my system was actually saturated. My image was uint8 and the operations were pushing the image past 255 or under 0 (for the negative side) and everything became lighter.
By changing the line of code in the imread to convert to double as in
Img = double(rgb2gray( imread(filename)));
(note my image was color so I did an rgb conversion, too. You might use
Img = double(( imread(filename)));
I got the improved image:
Original on left, corrected code on right.
Note that I could also produce this result using 2d convolution rather than your loop:
Robertsx = [1,0;0,-1];
Robertsy = [0,-1;1,0];
dataR = conv2(data, Robertsx) + conv2(data, Robertsy);
figure(2);
imagesc(dataR);
colormap gray
axis image
For the following result:
Here is an example implementation. You could easily replace CONV2/IMFILTER with your own 2D convolution/correlation function:
%# convolve image with Roberts kernels
I = im2double(imread('lena512_gray.jpg')); %# double image, range [0,1]
hx = [+1 0;0 -1]; hy = [0 +1;-1 0];
%#Gx = conv2(I,hx);
%#Gy = conv2(I,hy);
Gx = imfilter(I,hx,'conv','same','replicate');
Gy = imfilter(I,hy,'conv','same','replicate');
%# gradient approximation
G = sqrt(Gx.^2+Gy.^2);
figure, imshow(G), colormap(gray), title('Gradient magnitude [0,1]')
%# direction of the gradient
Gdir = atan2(Gy,Gx);
figure, imshow(Gdir,[]), title('Gradient direction [-\pi,\pi]')
colormap(hot), colorbar%, caxis([-pi pi])
%# quiver plot
ySteps = 1:8:size(I,1);
xSteps = 1:8:size(I,2);
[X,Y] = meshgrid(xSteps,ySteps);
figure, imshow(G,[]), hold on
quiver(X, Y, Gx(ySteps,xSteps), Gy(ySteps,xSteps), 3)
axis image, hold off
%# binarize gradient, and compare against MATLAB EDGE function
BW = im2bw(G.^2, 6*mean(G(:).^2));
figure
subplot(121), imshow(BW)
subplot(122), imshow(edge(I,'roberts')) %# performs additional thinning step
After detecting the lines in an image using Hough lines, how can I use it to calculate the change in angle (rotation) of the lines of a reference image?
Note to readers: This is a follow-up question, refer to these for background:
How to select maximum intensity in Hough transform in MATLAB?
Calculating displacement moved in MATLAB
The process is similar to what I showed before. Below I am using the images from your previous question (since you provided only one, I created the other by rotating the first by 10 degrees).
We start by detecting the lines for the two images. We do this with the help of the Hough transform functions. This what it looks like applied to both images:
Next, we want to perform image registration using the line endpoints as control-points. First, we make sure the points correspond to each other in the two images. I do this by computing the convex hull using convhull which automatically sorts them in counterclockwise-order (or is it in the opposite direction!). The numbers shown above indicate the order.
Finally, we use the function cp2tform to get the transformation matrix, which we use to align the images and extract the translation, rotation, and scaling.
The following is the complete code:
%% # Step 1: read and prepare images
%# (since you provided only one, I created the other by rotating the first).
I1 = imread('http://i.stack.imgur.com/Se6zX.jpg');
I1 = rgb2gray( imcrop(I1, [85 35 445 345]) ); %# Get rid of white border
I2 = imrotate(I1, -10, 'bilinear', 'crop'); %# Create 2nd by rotating 10 degrees
%% # Step 2: detect the cross sign endpoints (sorted in same order)
p1 = getCross(I1);
p2 = getCross(I2);
%% # Step 3: perform Image Registration
%# Find transformation that maps I2 to I1 using the 4 control points for each
t = cp2tform(p2,p1,'affine');
%# Transform I2 to be aligned with I1
II2 = imtransform(I2, t, 'XData',[1 size(I1,2)], 'YData',[1 size(I1,1)]);
%# Plot
figure('menu','none')
subplot(131), imshow(I1), title('I1')
subplot(132), imshow(I2), title('I2')
subplot(133), imshow(II2), title('I2 (aligned)')
%# Recover affine transformation params (translation, rotation, scale)
ss = t.tdata.Tinv(2,1);
sc = t.tdata.Tinv(1,1);
tx = t.tdata.Tinv(3,1);
ty = t.tdata.Tinv(3,2);
scale = sqrt(ss*ss + sc*sc)
rotation = atan2(ss,sc)*180/pi
translation = [tx ty]
And here's the function that extract the lines endpoints:
function points = getCross(I)
%# Get edges (simply by thresholding)
I = imfilter(I, fspecial('gaussian', [7 7], 1), 'symmetric');
BW = imclearborder(~im2bw(I, 0.5));
%# Hough transform
[H,T,R] = hough(BW);
%# Detect peaks
P = houghpeaks(H, 2);
%# Detect lines
lines = houghlines(BW, T, R, P);
%# Sort 2D points in counterclockwise order
points = [vertcat(lines.point1); vertcat(lines.point2)];
idx = convhull(points(:,1), points(:,2));
points = points(idx(1:end-1),:);
end
with the result:
scale =
1.0025
rotation =
-9.7041
translation =
32.5270 -38.5021
The rotation is recovered as almost 10 degrees (with some inevitable error), and scaling is effectively 1 (meaning there was no zooming). Note that there was a translation component in the above example, because rotation was not performed around the center of the cross sign).
I am not sure what the MATLAB implementation of the Hough transform is, but the orientation of the line will be simply be at a right angle (90 degrees or pi/2 radians) to the angle you've used to identify the line in the first place.
I hope that helps. There's decent coverage of Hough transforms on the web and Wikipedia is a good place to start.
BW = poly2mask(x, y, m, n) computes a
binary region of interest (ROI) mask,
BW, from an ROI polygon, represented
by the vectors x and y. The size of BW
is m-by-n.
poly2mask sets pixels in BW
that are inside the polygon (X,Y) to 1
and sets pixels outside the polygon to
0.
Problem:
Given such a binary mask BW of a convex quadrilateral, what would be the most efficient way to determine the four corners?
E.g.,
Best Solution so far:
Use edge to find the bounding lines, the Hough transform to find the 4 lines in the edge image and then find the intersection points of those 4 lines or use a corner detector on the edge image. Seems complicated, and I can't help feeling there's a simpler solution out there.
Btw, convhull doesn't always return 4 points (maybe someone can suggest qhull options to prevent that) : it returns a few points along the edges as well.
EDIT:
Amro's answer seems quite elegant and efficient. But there could be multiple "corners" at each real corner since the peaks aren't unique. I could cluster them based on θ and average the "corners" around a real corner but the main problem is the use of order(1:10).
Is 10 enough to account for all the corners or will this exclude a "corner" at a real corner?
This is somewhat similar to what #AndyL suggested. However I'm using the boundary signature in polar coordinates instead of the tangent.
Note that I start by extracting the edges, getting the boundary, then converting it to signature. Finally we find the points on the boundary that are furthest from the centroid, those points constitute the corners found. (Alternatively we can also detect peaks in the signature for corners).
The following is a complete implementation:
I = imread('oxyjj.png');
if ndims(I)==3
I = rgb2gray(I);
end
subplot(221), imshow(I), title('org')
%%# Process Image
%# edge detection
BW = edge(I, 'sobel');
subplot(222), imshow(BW), title('edge')
%# dilation-erosion
se = strel('disk', 2);
BW = imdilate(BW,se);
BW = imerode(BW,se);
subplot(223), imshow(BW), title('dilation-erosion')
%# fill holes
BW = imfill(BW, 'holes');
subplot(224), imshow(BW), title('fill')
%# get boundary
B = bwboundaries(BW, 8, 'noholes');
B = B{1};
%%# boudary signature
%# convert boundary from cartesian to ploar coordinates
objB = bsxfun(#minus, B, mean(B));
[theta, rho] = cart2pol(objB(:,2), objB(:,1));
%# find corners
%#corners = find( diff(diff(rho)>0) < 0 ); %# find peaks
[~,order] = sort(rho, 'descend');
corners = order(1:10);
%# plot boundary signature + corners
figure, plot(theta, rho, '.'), hold on
plot(theta(corners), rho(corners), 'ro'), hold off
xlim([-pi pi]), title('Boundary Signature'), xlabel('\theta'), ylabel('\rho')
%# plot image + corners
figure, imshow(BW), hold on
plot(B(corners,2), B(corners,1), 's', 'MarkerSize',10, 'MarkerFaceColor','r')
hold off, title('Corners')
EDIT:
In response to Jacob's comment, I should explain that I first tried to find the peaks in the signature using first/second derivatives, but ended up taking the furthest N-points. 10 was just an ad-hoc value, and would be difficult to generalize (I tried taking 4 same as number of corners, but it didn't cover all of them). I think the idea of clustering them to remove duplicates is worth looking into.
As far as I see it, the problem with the 1st approach was that if you plot rho without taking θ into account, you will get a different shape (not the same peaks), since the speed by which we trace the boundary is different and depends on the curvature. If we could figure out how to normalize that effect, we can get more accurate results using derivatives.
If you have the Image Processing Toolbox, there is a function called cornermetric which can implement a Harris corner detector or Shi and Tomasi's minimum eigenvalue method. This function has been present since version 6.2 of the Image Processing Toolbox (MATLAB version R2008b).
Using this function, I came up with a slightly different approach from the other answers. The solution below is based on the idea that a circular area centered at each "true" corner point will overlap the polygon by a smaller amount than a circular area centered over an erroneous corner point that is actually on the edge. This solution can also handle cases where multiple points are detected at the same corner...
The first step is to load the data:
rawImage = imread('oxyjj.png');
rawImage = rgb2gray(rawImage(7:473, 9:688, :)); % Remove the gray border
subplot(2, 2, 1);
imshow(rawImage);
title('Raw image');
Next, compute the corner metric using cornermetric. Note that I am masking the corner metric by the original polygon, so that we are looking for corner points that are inside the polygon (i.e. trying to find the corner pixels of the polygon). imregionalmax is then used to find the local maxima. Since you can have clusters of greater than 1 pixel with the same corner metric, I then add noise to the maxima and recompute so that I only get 1 pixel in each maximal region. Each maximal region is then labeled using bwlabel:
cornerImage = cornermetric(rawImage).*(rawImage > 0);
maxImage = imregionalmax(cornerImage);
noise = rand(nnz(maxImage), 1);
cornerImage(maxImage) = cornerImage(maxImage)+noise;
maxImage = imregionalmax(cornerImage);
labeledImage = bwlabel(maxImage);
The labeled regions are then dilated (using imdilate) with a disk-shaped structuring element (created using strel):
diskSize = 5;
dilatedImage = imdilate(labeledImage, strel('disk', diskSize));
subplot(2, 2, 2);
imshow(dilatedImage);
title('Dilated corner points');
Now that the labeled corner regions have been dilated, they will partially overlap the original polygon. Regions on an edge of the polygon will have about 50% overlap, while regions that are on a corner will have about 25% overlap. The function regionprops can be used to find the areas of overlap for each labeled region, and the 4 regions that have the least amount of overlap can thus be considered as the true corners:
maskImage = dilatedImage.*(rawImage > 0); % Overlap with the polygon
stats = regionprops(maskImage, 'Area'); % Compute the areas
[sortedValues, index] = sort([stats.Area]); % Sort in ascending order
cornerLabels = index(1:4); % The 4 smallest region labels
maskImage = ismember(maskImage, cornerLabels); % Mask of the 4 smallest regions
subplot(2, 2, 3);
imshow(maskImage);
title('Regions of minimal overlap');
And we can now get the pixel coordinates of the corners using find and ismember:
[r, c] = find(ismember(labeledImage, cornerLabels));
subplot(2, 2, 4);
imshow(rawImage);
hold on;
plot(c, r, 'r+', 'MarkerSize', 16, 'LineWidth', 2);
title('Corner points');
And here's a test with a diamond shaped region:
I like to solve this problem by working with a boundary, because it reduces this from a 2D problem to a 1D problem.
Use bwtraceboundary() from the image processing toolkit to extract a list of points on the boundary. Then convert the boundary into a series of tangent vectors (there are a number of ways to do this, one way would be to subrtact the
ith point along the boundary from the i+deltath point.) Once you have a list of vectors, take the dot product of adjacent vectors. The four points with the smallest dot products are your corners!
If you want your algorithm to work on polygons with an abritrary number of vertices, then simply search for dot products that are a certain number of standard deviations below the median dot product.
I decided to use a Harris corner detector (here's a more formal description) to obtain the corners. This can be implemented as follows:
%% Constants
Window = 3;
Sigma = 2;
K = 0.05;
nCorners = 4;
%% Derivative masks
dx = [-1 0 1; -1 0 1; -1 0 1];
dy = dx'; %SO code color fix '
%% Find the image gradient
% Mask is the binary image of the quadrilateral
Ix = conv2(double(Mask),dx,'same');
Iy = conv2(double(Mask),dy,'same');
%% Use a gaussian windowing function and compute the rest
Gaussian = fspecial('gaussian',Window,Sigma);
Ix2 = conv2(Ix.^2, Gaussian, 'same');
Iy2 = conv2(Iy.^2, Gaussian, 'same');
Ixy = conv2(Ix.*Iy, Gaussian, 'same');
%% Find the corners
CornerStrength = (Ix2.*Iy2 - Ixy.^2) - K*(Ix2 + Iy2).^2;
[val ind] = sort(CornerStrength(:),'descend');
[Ci Cj] = ind2sub(size(CornerStrength),ind(1:nCorners));
%% Display
imshow(Mask,[]);
hold on;
plot(Cj,Ci,'r*');
Here, the problem with multiple corners thanks to Gaussian windowing function which smooths the intensity change. Below, is a zoomed version of a corner with the hot colormap.
Here's an example using Ruby and HornetsEye. Basically the program creates a histogram of the quantised Sobel gradient orientation to find dominant orientations. If four dominant orientations are found, lines are fitted and the intersections between neighbouring lines are assumed to be the corners of the projected rectangle.
#!/usr/bin/env ruby
require 'hornetseye'
include Hornetseye
Q = 36
img = MultiArray.load_ubyte 'http://imgur.com/oxyjj.png'
dx, dy = 8, 6
box = [ dx ... 688, dy ... 473 ]
crop = img[ *box ]
crop.show
s0, s1 = crop.sobel( 0 ), crop.sobel( 1 )
mag = Math.sqrt s0 ** 2 + s1 ** 2
mag.normalise.show
arg = Math.atan2 s1, s0
msk = mag >= 500
arg_q = ( ( arg.mask( msk ) / Math::PI + 1 ) * Q / 2 ).to_int % Q
hist = arg_q.hist_weighted Q, mag.mask( msk )
segments = ( hist >= hist.max / 4 ).components
lines = arg_q.map segments
lines.unmask( msk ).normalise.show
if segments.max == 4
pos = MultiArray.scomplex *crop.shape
pos.real = MultiArray.int( *crop.shape ).indgen! % crop.shape[0]
pos.imag = MultiArray.int( *crop.shape ).indgen! / crop.shape[0]
weights = lines.hist( 5 ).major 1.0
centre = lines.hist_weighted( 5, pos.mask( msk ) ) / weights
vector = pos.mask( msk ) - lines.map( centre )
orientation = lines.hist_weighted( 5, vector ** 2 ) ** 0.5
corner = Sequence[ *( 0 ... 4 ).collect do |i|
i1, i2 = i + 1, ( i + 1 ) % 4 + 1
l1, a1, l2, a2 = centre[i1], orientation[i1], centre[i2], orientation[i2]
( l1 * a1.conj * a2 - l2 * a1 * a2.conj -
l1.conj * a1 * a2 + l2.conj * a1 * a2 ) /
( a1.conj * a2 - a1 * a2.conj )
end ]
result = MultiArray.ubytergb( *img.shape ).fill! 128
result[ *box ] = crop
corner.to_a.each do |c|
result[ c.real.to_i + dx - 1 .. c.real.to_i + dx + 1,
c.imag.to_i + dy - 1 .. c.imag.to_i + dy + 1 ] = RGB 255, 0, 0
end
result.show
end