Two-dimensional matched filter - matlab

I want to implement two dimensional matched filter for blood vessel extraction according to the paper "Detection of Blood Vessels in Retinal Images Using Two-Dimensional Matched Filters" by Chaudhuri et al., IEEE Trans. on Medical Imaging, 1989 (there's a PDF on the author's web site).
A brief discription is that blood vessel's cross-section has a gaussian distribution and therefore I want to use gaussian matched filter to increase SNR. Such a kernel may be mathematically expressed as:
K(x,y) = -exp(-x^2/2*sigma^2) for |x|<3*sigma, |y|<L/2
L here is the length of vessel with fixed orientation. Experimentally sigma=1.5 and L = 7.
My MATLAB code for this part is:
s = 1.5; %sigma
t = -3*s:3*s;
theta=0:15:165; %different rotations
%one dimensional kernel
x = 1/sqrt(6*s)*exp(-t.^2/(2*s.^2));
L=7;
%two dimensional gaussian kernel
x2 = repmat(x,L,1);
Consider the response of this filter for a pixel belonging to the background retina. Assuming the background to have constant intensity with zero mean additive Gaussian white noise, the expected value of the filter output should ideally be zero. The convolution kernel is, therefore, modified by subtracting the mean value of s(t) from the function itself. The mean value of the kernel is determined as: m = Sum(K(x,y))/(number of points).
Thus, the convolutional mask used in this algorithm is given by: K(x, y) = K(x,y) - m.
My MATLAB code:
m = sum(x2(:))/(size(x2,1)*size(x2,2));
x2 = x2-m;
A vessel may be oriented at any angle 0<theta<180 and the matched filter response is maximum when when it is aligned at theta+- 90 (cross-section distribution is gaussian not the vessel itself).
Thus we need to rotate the matched filter 12 times with 15 degree increment.
My MATLAB code is attached here but I don't get a desirable result. Any help is appreciated.
%apply rotated matched filter on image
r = {};
for k = 1:12
x3=imrotate(x2,theta(k),'crop');%figure;imagesc(x3);colormap gray;
r{k}=conv2(img,x3);
end
w=[];h = zeros(584,565);
for i = 1:565
for j = 1:584
for k = 1:32
w= [w ,r{k}(j,i)];
end
h(j,i)=max(abs(w));
w = [];
end
end
%show result
figure('Name','after matched filter');imagesc(h);colormap gray
For rotation I used imrotate which seems more sensible to me but in the paper it is different: suppose p=[x,y] be a discrete point in the kernel. To compute coefficients in the rotated kernel we have [u,v] = p*Rotation_Matrix.
Rotation_Matrix=[cos(theta),sin(theta);-sin(theta),cos(theta)]
And the kernel is:
K(x,y) = -exp(-u^2/2*s^2)
But the new kernel doesn't have a gaussian shape anymore. Using imrotate preserves gaussian shape. So what is the benefit of using Rotation matrix?
Input image is:
Output:
Matched filtering helps increase SNR but background noise is amplified too.
Am I right to use imrotate to rotate the kernel? My main problem is with rotation matrix that why and what is the right code to implement it.

The reason to build the filter from its analytic expression for each rotation, rather than using imrotate, is that the filter extent is not circular, and therefore rotating brings in "new" pixel values and pushes some other pixels out of the kernel. Furthermore, rotating a kernel constructed as here (smooth transition along one direction, step edge along the other dimension) requires different interpolation methods along each dimension, which imrotate cannot do. The resulting rotated kernel will always be wrong.
Both these issues can be easily seen when displaying the kernel you make together with two rotated versions:
This display brings an additional issues to the front: the kernel is not centered on a pixel, causing it to shift the output by half a pixel.
Note also that, when subtracting the mean, it is important that this mean be computed only over the original domain of the filter, and that any zeros used to pad this domain to a rectangular shape remain zero (these should not become negative).
The rotated kernels can be constructed as follows:
m = max(ceil(3*s),(L-1)/2);
[x,y] = meshgrid(-m:m,-m:m); % non-rotated coordinate system, contains (0,0)
t = pi/6; % angle in radian
u = cos(t)*x - sin(t)*y; % rotated coordinate system
v = sin(t)*x + cos(t)*y; % rotated coordinate system
N = (abs(u) <= 3*s) & (abs(v) <= L/2); % domain
k = exp(-u.^2/(2*s.^2)); % kernel
k = k - mean(k(N));
k(~N) = 0; % set kernel outside of domain to 0
This is the result for the three rotations used in the example above (the grey around the edges of the kernel corresponds to the value 0, the black pixels have a negative value):
Another issue is that you use conv2 with the default 'full' output shape, you should be using 'same' here, so that the output of the filter matches the input.
Note that, instead of computing all filter responses, and computing the max afterwards, it is much easier to compute the max as you compute each filter response. All of the above leads to the following code:
img = im2double(rgb2gray(img));
s = 1.5; %sigma
L = 7;
theta = 0:15:165; %different rotations
out = zeros(size(img));
m = max(ceil(3*s),(L-1)/2);
[x,y] = meshgrid(-m:m,-m:m); % non-rotated coordinate system, contains (0,0)
for t = theta
t = t / 180 * pi; % angle in radian
u = cos(t)*x - sin(t)*y; % rotated coordinate system
v = sin(t)*x + cos(t)*y; % rotated coordinate system
N = (abs(u) <= 3*s) & (abs(v) <= L/2); % domain
k = exp(-u.^2/(2*s.^2)); % kernel
k = k - mean(k(N));
k(~N) = 0; % set kernel outside of domain to 0
res = conv2(img,k,'same');
out = max(out,res);
end
out = out/max(out(:)); % force output to be in [0,1] interval that MATLAB likes
imwrite(out,'so_result.png')
I get the following output:

Related

Fitting largest circle in free area in image with distributed particle

I am working on images to detect and fit the largest possible circle in any of the free areas of an image containing distributed particles:
(able to detect the location of particle).
One direction is to define a circle touching any 3-point combination, checking if the circle is empty, then finding the largest circle among all empty circles. However, it leads to a huge number of combination i.e. C(n,3), where n is the total number of particles in the image.
I would appreciate if anyone can provide me any hint or alternate method that I can explore.
Lets do some maths my friend, as maths will always get to the end!
Wikipedia:
In mathematics, a Voronoi diagram is a partitioning of a plane into
regions based on distance to points in a specific subset of the plane.
For example:
rng(1)
x=rand(1,100)*5;
y=rand(1,100)*5;
voronoi(x,y);
The nice thing about this diagram is that if you notice, all the edges/vertices of those blue areas are all to equal distance to the points around them. Thus, if we know the location of the vertices, and compute the distances to the closest points, then we can choose the vertex with highest distance as our center of the circle.
Interestingly, the edges of a Voronoi regions are also defined as the circumcenters of the triangles generated by a Delaunay triangulation.
So if we compute the Delaunay triangulation of the area, and their circumcenters
dt=delaunayTriangulation([x;y].');
cc=circumcenter(dt); %voronoi edges
And compute the distances between the circumcenters and any of the points that define each triangle:
for ii=1:size(cc,1)
if cc(ii,1)>0 && cc(ii,1)<5 && cc(ii,2)>0 && cc(ii,2)<5
point=dt.Points(dt.ConnectivityList(ii,1),:); %the first one, or any other (they are the same distance)
distance(ii)=sqrt((cc(ii,1)-point(1)).^2+(cc(ii,2)-point(2)).^2);
end
end
Then we have the center (cc) and radius (distance) of all possible circles that have no point inside them. We just need the biggest one!
[r,ind]=max(distance); %Tada!
Now lets plot
hold on
ang=0:0.01:2*pi;
xp=r*cos(ang);
yp=r*sin(ang);
point=cc(ind,:);
voronoi(x,y)
triplot(dt,'color','r','linestyle',':')
plot(point(1)+xp,point(2)+yp,'k');
plot(point(1),point(2),'g.','markersize',20);
Notice how the center of the circle is on one vertex of the Voronoi diagram.
NOTE: this will find the center inside [0-5],[0-5]. you can easily modify it to change this constrain. You can also try to find the circle that fits on its entirety inside the interested area (as opposed to just the center). This would require a small addition in the end where the maximum is obtained.
I'd like to propose another solution based on a grid search with refinement. It's not as advanced as Ander's or as short as rahnema1's, but it should be very easy to follow and understand. Also, it runs quite fast.
The algorithm contains several stages:
We generate an evenly-spaced grid.
We find the minimal distances of points in the grid to all provided points.
We discard all points whose distances are below a certain percentile (e.g. 95th).
We choose the region which contains the largest distance (this should contain the correct center if my initial grid is fine enough).
We create a new meshgrid around the chosen region and find distances again (this part is clearly sub-optimal, because the distances are computed to all points, including far and irrelevant ones).
We iterate the refinement within the region, while keeping an eye on the variance of the top 5% of values -> if it drops below some preset threshold we break.
Several notes:
I have made the assumption that circles cannot go beyond the scattered points' extent (i.e. the bounding square of the scatter acts as an "invisible wall").
The appropriate percentile depends on how fine the initial grid is. This will also affect the amount of while iterations, and the optimal initial value for cnt.
function [xBest,yBest,R] = q42806059
rng(1)
x=rand(1,100)*5;
y=rand(1,100)*5;
%% Find the approximate region(s) where there exists a point farthest from all the rest:
xExtent = linspace(min(x),max(x),numel(x));
yExtent = linspace(min(y),max(y),numel(y)).';
% Create a grid:
[XX,YY] = meshgrid(xExtent,yExtent);
% Compute pairwise distance from grid points to free points:
D = reshape(min(pdist2([XX(:),YY(:)],[x(:),y(:)]),[],2),size(XX));
% Intermediate plot:
% figure(); plot(x,y,'.k'); hold on; contour(XX,YY,D); axis square; grid on;
% Remove irrelevant candidates:
D(D<prctile(D(:),95)) = NaN;
D(D > xExtent | D > yExtent | D > yExtent(end)-yExtent | D > xExtent(end)-xExtent) = NaN;
%% Keep only the region with the largest distance
L = bwlabel(~isnan(D));
[~,I] = max(table2array(regionprops('table',L,D,'MaxIntensity')));
D(L~=I) = NaN;
% surf(XX,YY,D,'EdgeColor','interp','FaceColor','interp');
%% Iterate until sufficient precision:
xExtent = xExtent(~isnan(min(D,[],1,'omitnan')));
yExtent = yExtent(~isnan(min(D,[],2,'omitnan')));
cnt = 1; % increase or decrease according to the nature of the problem
while true
% Same ideas as above, so no explanations:
xExtent = linspace(xExtent(1),xExtent(end),20);
yExtent = linspace(yExtent(1),yExtent(end),20).';
[XX,YY] = meshgrid(xExtent,yExtent);
D = reshape(min(pdist2([XX(:),YY(:)],[x(:),y(:)]),[],2),size(XX));
D(D<prctile(D(:),95)) = NaN;
I = find(D == max(D(:)));
xBest = XX(I);
yBest = YY(I);
if nanvar(D(:)) < 1E-10 || cnt == 10
R = D(I);
break
end
xExtent = (1+[-1 +1]*10^-cnt)*xBest;
yExtent = (1+[-1 +1]*10^-cnt)*yBest;
cnt = cnt+1;
end
% Finally:
% rectangle('Position',[xBest-R,yBest-R,2*R,2*R],'Curvature',[1 1],'EdgeColor','r');
The result I'm getting for Ander's example data is [x,y,r] = [0.7832, 2.0694, 0.7815] (which is the same). The execution time is about half of Ander's solution.
Here are the intermediate plots:
Contour of the largest (clear) distance from a point to the set of all provided points:
After considering distance from the boundary, keeping only the top 5% of distant points, and considering only the region which contains the largest distance (the piece of surface represents the kept values):
And finally:
You can use bwdist from Image Processing Toolbox to compute the distance transform of the image. This can be regarded as a method to create voronoi diagram that well explained in #AnderBiguri's answer.
img = imread('AbmxL.jpg');
%convert the image to a binary image
points = img(:,:,3)<200;
%compute the distance transform of the binary image
dist = bwdist(points);
%find the circle that has maximum radius
radius = max(dist(:));
%find position of the circle
[x y] = find(dist == radius);
imshow(dist,[]);
hold on
plot(y,x,'ro');
The fact that this problem can be solved using a "direct search" (as can be seen in another answer) means one can look at this as a global optimization problem. There exist various ways to solve such problems, each appropriate for certain scenarios. Out of my personal curiosity I have decided to solve this using a genetic algorithm.
Generally speaking, such an algorithm requires us to think of the solution as a set of "genes" subject to "evolution" under a certain "fitness function". As it happens, it's quite easy to identify the genes and the fitness function in this problem:
Genes: x , y, r.
Fitness function: technically, maximum area of circle, but this is equivalent to the maximum r (or minimum -r, since the algorithm requires a function to minimize).
Special constraint - if r is larger than the euclidean distance to the closest of the provided points (that is, the circle contains a point), the organism "dies".
Below is a basic implementation of such an algorithm ("basic" because it's completely unoptimized, and there is lot of room for optimizationno pun intended in this problem).
function [x,y,r] = q42806059b(cloudOfPoints)
% Problem setup
if nargin == 0
rng(1)
cloudOfPoints = rand(100,2)*5; % equivalent to Ander's initialization.
end
%{
figure(); plot(cloudOfPoints(:,1),cloudOfPoints(:,2),'.w'); hold on; axis square;
set(gca,'Color','k'); plot(0.7832,2.0694,'ro'); plot(0.7832,2.0694,'r*');
%}
nVariables = 3;
options = optimoptions(#ga,'UseVectorized',true,'CreationFcn',#gacreationuniform,...
'PopulationSize',1000);
S = max(cloudOfPoints,[],1); L = min(cloudOfPoints,[],1); % Find geometric bounds:
% In R2017a: use [S,L] = bounds(cloudOfPoints,1);
% Here we also define distance-from-boundary constraints.
g = ga(#(g)vectorized_fitness(g,cloudOfPoints,[L;S]), nVariables,...
[],[], [],[], [L 0],[S min(S-L)], [], options);
x = g(1); y = g(2); r = g(3);
%{
plot(x,y,'ro'); plot(x,y,'r*');
rectangle('Position',[x-r,y-r,2*r,2*r],'Curvature',[1 1],'EdgeColor','r');
%}
function f = vectorized_fitness(genes,pts,extent)
% genes = [x,y,r]
% extent = [Xmin Ymin; Xmax Ymax]
% f, the fitness, is the largest radius.
f = min(pdist2(genes(:,1:2), pts, 'euclidean'), [], 2);
% Instant death if circle contains a point:
f( f < genes(:,3) ) = Inf;
% Instant death if circle is too close to boundary:
f( any( genes(:,3) > genes(:,1:2) - extent(1,:) | ...
genes(:,3) > extent(2,:) - genes(:,1:2), 2) ) = Inf;
% Note: this condition may possibly be specified using the A,b inputs of ga().
f(isfinite(f)) = -genes(isfinite(f),3);
%DEBUG:
%{
scatter(genes(:,1),genes(:,2),10 ,[0, .447, .741] ,'o'); % All
z = ~isfinite(f); scatter(genes(z,1),genes(z,2),30,'r','x'); % Killed
z = isfinite(f); scatter(genes(z,1),genes(z,2),30,'g','h'); % Surviving
[~,I] = sort(f); scatter(genes(I(1:5),1),genes(I(1:5),2),30,'y','p'); % Elite
%}
And here's a "time-lapse" plot of 47 generations of a typical run:
(Where blue points are the current generation, red crosses are "insta-killed" organisms, green hexagrams are the "non-insta-killed" organisms, and the red circle marks the destination).
I'm not used to image processing, so it's just an Idea:
Implement something like a gaussian filter (blur) which transforms each particle (pixels) to a round gradiant with r=image_size (all of them overlapping). This way, you should get a picture where the most white pixels should be the best results. Unfortunately, the demonstration in gimp failed because the extreme blurring made the dots disappearing.
Alternatively, you could incrementelly extend all existing pixels by marking all neighbour pixels in an area (example: r=4), the pixels left would be the same result (those with the biggest distance to any pixel)

Fourier transform for fiber alignment

I'm working on an application to determine from an image the degree of alignment of a fiber network. I've read several papers on this issue and they basically do this:
Find the 2D discrete Fourier transform (DFT = F(u,v)) of the image (gray, range 0-255)
Find the Fourier Spectrum (FS = abs(F(u,v))) and the Power Spectrum (PS = FS^2)
Convert spectrum to polar coordinates and divide it into 1º intervals.
Calculate number-averaged line intensities (FI) for each interval (theta), that is, the average of all the intensities (pixels) forming "theta" degrees with respect to the horizontal axis.
Transform FI(theta) to cartesian coordinates
Cxy(theta) = [FI*cos(theta), FI*sin(theta)]
Find eigenvalues (lambda1 and lambda2) of the matrix Cxy'*Cxy
Find alignment index as alpha = 1 - lamda2/lambda1
I've implemented this in MATLAB (code below), but I'm not sure whether it is ok since point 3 and 4 are not really clear for me (I'm getting similar results to those of the papers, but not in all cases). For instance, in point 3, "spectrum" is referring to FS or to PS?. And in point 4, how should this average be done? are all the pixels considered? (even though there are more pixels in the diagonal).
rgb = imread('network.tif');%513x513 pixels
im = rgb2gray(rgb);
im = imrotate(im,-90);%since FFT space is rotated 90º
FT = fft2(im) ;
FS = abs(FT); %Fourier spectrum
PS = FS.^2; % Power spectrum
FS = fftshift(FS);
PS = fftshift(PS);
xoffset = (513-1)/2;
yoffset = (513-1)/2;
% Avoid low frequency points
x1 = 5;
y1 = 0;
% Maximum high frequency pixels
x2 = 255;
y2 = 0;
for theta = 0:pi/180:pi
% Transposed rotation matrix
Rt = [cos(theta) sin(theta);
-sin(theta) cos(theta)];
% Find radial lines necessary for improfile
xy1_rot = Rt * [x1; y1] + [xoffset; yoffset];
xy2_rot = Rt * [x2; y2] + [xoffset; yoffset];
plot([xy1_rot(1) xy2_rot(1)], ...
[xy1_rot(2) xy2_rot(2)], ...
'linestyle','none', ...
'marker','o', ...
'color','k');
prof = improfile(F,[xy1_rot(1) xy2_rot(1)],[xy1_rot(2) xy2_rot(2)]);
i = i + 1;
FI(i) = sum(prof(:))/length(prof);
Cxy(i,:) = [FI(i)*cos(theta), FI(i)*sin(theta)];
end
C = Cxy'*Cxy;
[V,D] = eig(C)
lambda2 = D(1,1);
lambda1 = D(2,2);
alpha = 1 - lambda2/lambda1
Figure: A) original image, B) plot of log(P+1), C) polar plot of FI.
My main concern is that when I choose an artificial image perfectly aligned (attached figure), I get alpha = 0.91, and it should be exactly 1.
Any help will be greatly appreciated.
PD: those black dots in the middle plot are just the points used by improfile.
I believe that there are a couple sources of potential error here that are leading to you not getting a perfect alpha value.
Discrete Fourier Transform
You have discrete imaging data which forces you to take a discrete Fourier transform which inevitably (depending on the resolution of the input data) have some accuracy issues.
Binning vs. Sampling Along a Line
The way that you have done the binning is that you literally drew a line (rotated by a particular angle) and sampled the image along that line using improfile. Using improfile performs interpolation of your data along that line introducing yet another potential source of error. The default is nearest neighbor interpolation which in the example shown below can cause multiple "profiles" to all pick up the same points.
This was with a rotation of 1-degree off-vertical when technically you'd want those peaks to only appear for a perfectly vertical line. It is clear to see how this sort of interpolation of the Fourier spectrum can lead to a spread around the "correct" answer.
Data Undersampling
Similar to Nyquist sampling in the Fourier domain, sampling in the spatial domain has some requirements as well.
Imagine for a second that you wanted to use 45-degree bin widths instead of the 1-degree. Your approach would still sample along a thin line and use that sample to represent 45-degrees worth or data. Clearly, this is a gross under-sampling of the data and you can imagine that the result wouldn't be very accurate.
It becomes more and more of an issue the further you get from the center of the image since the data in this "bin" is really pie wedge shaped and you're approximating it with a line.
A Potential Solution
A different approach to binning would be to determine the polar coordinates (r, theta) for all pixel centers in the image. Then to bin the theta components into 1-degree bins. Then sum all of the values that fall into that bin.
This has several advantages:
It removes the undersampling that we talked about and draws samples from the entire "pie wedge" regardless of the sampling angle.
It ensures that each pixel belongs to one and only one angular bin
I have implemented this alternate approach in the code below with some false horizontal line data and am able to achieve an alpha value of 0.988 which I'd say is pretty good given the discrete nature of the data.
% Draw a bunch of horizontal lines
data = zeros(101);
data([5:5:end],:) = 1;
fourier = fftshift(fft2(data));
FS = abs(fourier);
PS = FS.^2;
center = fliplr(size(FS)) / 2;
[xx,yy] = meshgrid(1:size(FS,2), 1:size(FS, 1));
coords = [xx(:), yy(:)];
% De-mean coordinates to center at the middle of the image
coords = bsxfun(#minus, coords, center);
[theta, R] = cart2pol(coords(:,1), coords(:,2));
% Convert to degrees and round them to the nearest degree
degrees = mod(round(rad2deg(theta)), 360);
degreeRange = 0:359;
% Band pass to ignore high and low frequency components;
lowfreq = 5;
highfreq = size(FS,1)/2;
% Now average everything with the same degrees (sum over PS and average by the number of pixels)
for k = degreeRange
ps_integral(k+1) = mean(PS(degrees == k & R > lowfreq & R < highfreq));
fs_integral(k+1) = mean(FS(degrees == k & R > lowfreq & R < highfreq));
end
thetas = deg2rad(degreeRange);
Cxy = [ps_integral.*cos(thetas);
ps_integral.*sin(thetas)]';
C = Cxy' * Cxy;
[V,D] = eig(C);
lambda2 = D(1,1);
lambda1 = D(2,2);
alpha = 1 - lambda2/lambda1;

Image Processing - Using Radon Transform for Pattern Recognition in MATLAB

I am attempting to extract the Radon Signature in order to recognize patterns of clothing (striped,plaid, irregular and patternless) as done in 1.
Algorithm to be implemented :
1. Use sobel operator to compute the gradient map as f(x,y).
2. Perform Radon transform based on maximum disk area.
3. Compute the variance of r under all theta directions.
4. Employ L2-norm to normalize the feature vector.
5. Plot Radon Signature as a bar chart of var(r) for all theta values.
I have done the following :
img = imread('plaid.jpg');
grey = rgb2gray(img);
img2 = edge(grey, 'sobel');
vararray=zeros(1,size(theta,2));
theta = -89:90;
for j = 1: size(theta,2)
[R3,xp3] = radon (img2,theta(j));
vararray(j) = var(R3);
end
vararray = vararray/norm(vararray);
figure(1), bar(theta,vararray),title('Radon Signature');
I believe that my error lies in the first 2 steps. I am unsure how to perform Radon only on the maximum disk area.
My results are shown on the right, while from the article (referenced below) is shown on the left.
However, my results should at least show 2 distinct peaks as shown in the acticle's results, but they do not.
Any assistance is appreciated.
Source of Algorithm : "Assistive Clothing Pattern Recognition for Visually Impaired People" by Xiaodong Yang, Student Member, IEEE, Shuai Yuan, and YingLi Tian, Senior Member, IEEE
Maximum disk area is, as #beaker thought, defined by the maximum filled circle that fits inside the bounding box of the image. That you can observe from the Fig.3 b) of the article.
Another thing you did wrong, is using edge detector edge(grey, 'sobel') while you should use gradient map or more formally gradient magnitude. Here's a code which produces a curve close to what is shown in Fig 3d. How to quantify it to six peaks, remains a question.
A = imread( 'Layer-5.png' ); % image from the article
A = double(rgb2gray( A ));
% gradient magnitude
dx = imfilter(A,fspecial('sobel') ); % x, 3x3 kernel
dy = imfilter(A,fspecial('sobel')'); % y
gradmag = sqrt( dx.^2 + dy.^2 );
% mask by disk
R = min( size(A)/2 ); % radius
disk = insertShape(zeros(size(A)),'FilledCircle', [size(A)/2,R] );
mask = double(rgb2gray(disk)~=0);
gradmag = mask.*gradmag;
% radon transform
theta = linspace(0,180,180);
vars = zeros(size(theta));
for u = 1:length(theta)
[rad,xp] =radon( gradmag, theta(u) );
indices = find( abs(xp)<R );
% ignore radii outside the maximum disk area
% so you don't sum up zeroes into variance
vars(u) = var( rad( indices ) );
end
vars = vars/norm(vars);
figure; plot( vars );
Bear in mind, images copied from the article appear with jpg artefacts. After good denoising (a tad too much here), e.g.,
you get much more prominent results.

Matlab Template Matching Using FFT

I am struggling with template matching in the Fourier domain in Matlab. Here are my images (the artist is RamalamaCreatures on DeviantArt):
My aim is to place a bounding box around the ear of the possum, like this example (where I performed template matching using normxcorr2):
Here is the Matlab code I am using:
clear all; close all;
template = rgb2gray(imread('possum_ear.jpg'));
background = rgb2gray(imread('possum.jpg'));
%% calculate padding
bx = size(background, 2);
by = size(background, 1);
tx = size(template, 2); % used for bbox placement
ty = size(template, 1);
%% fft
c = real(ifft2(fft2(background) .* fft2(template, by, bx)));
%% find peak correlation
[max_c, imax] = max(abs(c(:)));
[ypeak, xpeak] = find(c == max(c(:)));
figure; surf(c), shading flat; % plot correlation
%% display best match
hFig = figure;
hAx = axes;
position = [xpeak(1)-tx, ypeak(1)-ty, tx, ty];
imshow(background, 'Parent', hAx);
imrect(hAx, position);
The code is not functioning as intended - it is not identifying the correct region. This is the failed result - the wrong area is boxed:
This is the surface plot of the correlations for the failed match:
Hope you can help! Thanks.
What you're doing in your code is actually not correlation at all. You are using the template and performing convolution with the input image. If you recall from the Fourier Transform, the multiplication of the spectra of two signals is equivalent to the convolution of the two signals in time/spatial domain.
Basically, what you are doing is that you are using the template as a kernel and using that to filter the image. You are then finding the maximum response of this output and that's what is deemed to be where the template is. Where the response is being boxed makes sense because that region is entirely white, and using the template as the kernel with a region that is entirely white will give you a very large response, which is why it most likely identified that area to be the maximum response. Specifically, the region will have a lot of high values (~255 or so), and naturally performing convolution with the template patch and this region will give you a very large output due to the operation being a weighted sum. As such, if you used the template in a dark area of the image, the output would be small - which is false because the template is also consisting of dark pixels.
However, you can certainly use the Fourier Transform to locate where the template is, but I would recommend you use Phase Correlation instead. Basically, instead of computing the multiplication of the two spectra, you compute the cross power spectrum instead. The cross power spectrum R between two signals in the frequency domain is defined as:
Source: Wikipedia
Ga and Gb are the original image and the template in frequency domain, and the * is the conjugate. The o is what is known as the Hadamard product or element-wise product. I'd also like to point out that the division of the numerator and denominator of this fraction is also element-wise. Using the cross power spectrum, if you find the (x,y) location here that produces the absolute maximum response, this is where the template should be located in the background image.
As such, you simply need to change the line of code that computes the "correlation" so that it computes the cross power spectrum instead. However, I'd like to point out something very important. When you perform normxcorr2, the correlation starts right at the top-left corner of the image. The template matching starts at this location and it gets compared with a window that is the size of the template where the top-left corner is the origin. When finding the location of the template match, the location is with respect to the top-left corner of the matched window. Once you compute normxcorr2, you traditionally add the half of the rows and half of the columns of the maximum response to find the centre location.
Because we are more or less doing the same operations for template matching (sliding windows, correlation, etc.) with the FFT / frequency domain, when you finish finding the peak in this correlation array, you must also take this into account. However, your call to imrect to draw a rectangle around where the template matches takes in the top left corner of a bounding box anyway, so there's no need to do the offset here. As such, we're going to modify that code slightly but keep the offset logic in mind when using this code for later if want to find the centre location of the match.
I've modified your code as well to read in the images directly from StackOverflow so that it's reproducible:
clear all; close all;
template = rgb2gray(imread('http://i.stack.imgur.com/6bTzT.jpg'));
background = rgb2gray(imread('http://i.stack.imgur.com/FXEy7.jpg'));
%% calculate padding
bx = size(background, 2);
by = size(background, 1);
tx = size(template, 2); % used for bbox placement
ty = size(template, 1);
%% fft
%c = real(ifft2(fft2(background) .* fft2(template, by, bx)));
%// Change - Compute the cross power spectrum
Ga = fft2(background);
Gb = fft2(template, by, bx);
c = real(ifft2((Ga.*conj(Gb))./abs(Ga.*conj(Gb))));
%% find peak correlation
[max_c, imax] = max(abs(c(:)));
[ypeak, xpeak] = find(c == max(c(:)));
figure; surf(c), shading flat; % plot correlation
%% display best match
hFig = figure;
hAx = axes;
%// New - no need to offset the coordinates anymore
%// xpeak and ypeak are already the top left corner of the matched window
position = [xpeak(1), ypeak(1), tx, ty];
imshow(background, 'Parent', hAx);
imrect(hAx, position);
With that, I get the following image:
I also get the following when showing a surface plot of the cross power spectrum:
There is a clear defined peak where the rest of the output has a very small response. That's actually a property of Phase Correlation and so obviously, the location of the maximum value is clearly defined and this is where the template is located.
Hope this helps!
Just ended up implementing the same with python with similar ideas as #rayryeng's using scipy.fftpack.fftn() / ifftn() functions with the following result on the same target and template images:
import numpy as np
import scipy.fftpack as fp
from skimage.io import imread
from skimage.color import rgb2gray, gray2rgb
import matplotlib.pylab as plt
from skimage.draw import rectangle_perimeter
im = 255*rgb2gray(imread('http://i.stack.imgur.com/FXEy7.jpg')) # target
im_tm = 255*rgb2gray(imread('http://i.stack.imgur.com/6bTzT.jpg')) # template
# FFT
F = fp.fftn(im)
F_tm = fp.fftn(im_tm, shape=im.shape)
# compute the best match location
F_cc = F * np.conj(F_tm)
c = (fp.ifftn(F_cc/np.abs(F_cc))).real
i, j = np.unravel_index(c.argmax(), c.shape)
print(i, j)
# 214 317
# draw rectangle around the best match location
im2 = (gray2rgb(im)).astype(np.uint8)
rr, cc = rectangle_perimeter((i,j), end=(i + im_tm.shape[0], j + im_tm.shape[1]), shape=im.shape)
for x in range(-2,2):
for y in range(-2,2):
im2[rr + x, cc + y] = (255,0,0)
# show the output image
plt.figure(figsize=(10,10))
plt.imshow(im2)
plt.axis('off')
plt.show()
Also, the below animation shows the result obtained while locating a bird's template image inside a set of (target) frames extracted from a video with a flock of birds.
One thing to note: the output is very much dependent on the similarity of the size and shape of the object that is to be matched with the template, if it's quite different from that of the template image, the template may not be matched at all.

Circular Hough transform using edge gradients and orientations

I'm currently busy with implementing two circular Hough transforms. The last one is based on an idea which is described in a paper for iris segmentation (http://www.cse.iitk.ac.in/users/naditya/Aditya_Nigam_icic2012P_C.pdf).
I've tried to implement every step but this kind of circular detection doesn't give any positive and accurate results. Also when running this algorithm on an image, containing a simple circle without noise, the maximum value which appears in the accumulator matrix is mostly around 3 (which means that only 3 edge points where found, voting for this center point...). Based on my outcomes, I guess this has something to do with a mistake within my code? Or maybe I forgot to implement a crucial step?
% create binairy edge image
edgeImage = double(edge(image,'sobel'));
% create sobel masks
horizontalMaskSobel = fspecial('sobel');
verticalMaskSobel = fspecial('sobel')';
% calculate gradients
gradientX = imfilter(edgeImage,horizontalMaskSobel);
gradientY = imfilter(edgeImage,verticalMaskSobel);
% calculate the orientation for each edge within the image
orientationImage = atan2(gradientY,gradientX);
% create accumulator space
for y in searchwindow % height
for x in searchwindow % width
for r in radii;
if edgeImage(y,x) > 0 % bright enough?
cy1 = y + r*sin(orientationImage(y,x)-pi/2);
cx1 = x - r*cos(orientationImage(y,x)-pi/2);
cy2 = y - r*sin(orientationImage(y,x)-pi/2);
cx2 = x + r*cos(orientationImage(y,x)-pi/2);
if calculated coordinates within searchwindow
accumulator(cy1,cx1,r)++;
accumulator(cy2,cx2,r)++;
end
end
end
end
end