I need to know how to align an image in Matlab for further work.
for example I have the next license plate image and I want to recognize all
the digits.
my program works for straight images so, I need to align the image and then
preform the optical recognition system.
The method should be as much as universal that fits for all kinds of plates and in all kinds of angles.
EDIT: I tried to do this with Hough Transform but I didn't Succeed. anybody can help me do to this?
any help will be greatly appreciated.
The solution was first hinted at by #AruniRC in the comments, then implemented by #belisarius in Mathematica. The following is my interpretation in MATLAB.
The idea is basically the same: detect edges using Canny method, find prominent lines using Hough Transform, compute line angles, finally perform a Shearing Transform to align the image.
%# read and crop image
I = imread('http://i.stack.imgur.com/CJHaA.png');
I = I(:,1:end-3,:); %# remove small white band on the side
%# egde detection
BW = edge(rgb2gray(I), 'canny');
%# hough transform
[H T R] = hough(BW);
P = houghpeaks(H, 4, 'threshold',ceil(0.75*max(H(:))));
lines = houghlines(BW, T, R, P);
%# shearing transforma
slopes = vertcat(lines.point2) - vertcat(lines.point1);
slopes = slopes(:,2) ./ slopes(:,1);
TFORM = maketform('affine', [1 -slopes(1) 0 ; 0 1 0 ; 0 0 1]);
II = imtransform(I, TFORM);
Now lets see the results
%# show edges
figure, imshow(BW)
%# show accumlation matrix and peaks
figure, imshow(imadjust(mat2gray(H)), [], 'XData',T, 'YData',R, 'InitialMagnification','fit')
xlabel('\theta (degrees)'), ylabel('\rho'), colormap(hot), colorbar
hold on, plot(T(P(:,2)), R(P(:,1)), 'gs', 'LineWidth',2), hold off
axis on, axis normal
%# show image with lines overlayed, and the aligned/rotated image
figure
subplot(121), imshow(I), hold on
for k = 1:length(lines)
xy = [lines(k).point1; lines(k).point2];
plot(xy(:,1), xy(:,2), 'g.-', 'LineWidth',2);
end, hold off
subplot(122), imshow(II)
In Mathematica, using Edge Detection and Hough Transform:
If you are using some kind of machine learning toolbox for text recognition, try to learn from ALL plates - not only aligned ones. Recognition results should be equally well if you transform the plate or dont, since by transforming, no new informations according to the true number will enhance the image.
If all the images have a dark background like that one, you could binarize the image, fit lines to the top or bottom of the bright area and calculate an affine projection matrix from the line gradient.
Related
I want to convert an image from Cartesian to Polar and to use it for opengl texture.
So I used matlab referring to the two articles below.
Link 1
Link 2
My code is exactly same with Link 2's anwser
% load image
img = imread('my_image.png');
% convert pixel coordinates from cartesian to polar
[h,w,~] = size(img);
[X,Y] = meshgrid((1:w)-floor(w/2), (1:h)-floor(h/2));
[theta,rho] = cart2pol(X, Y);
Z = zeros(size(theta));
% show pixel locations (subsample to get less dense points)
XX = X(1:8:end,1:4:end);
YY = Y(1:8:end,1:4:end);
tt = theta(1:8:end,1:4:end);
rr = rho(1:8:end,1:4:end);
subplot(121), scatter(XX(:),YY(:),3,'filled'), axis ij image
subplot(122), scatter(tt(:),rr(:),3,'filled'), axis ij square tight
% show images
figure
subplot(121), imshow(img), axis on
subplot(122), warp(theta, rho, Z, img), view(2), axis square
The result was exactly what I wanted, and I was very satisfied except for one thing. It's the area (red circled area) in the picture just below. Considering that the opposite side (blue circled area) is not, I think this part should also be filled. Because of this part is empty, so there is a problem when using it as a texture.
And I wonder how I can fill this part. Thank you.
(little difference from Link 2's answer code like degree<->radian and axis values. but i think it is not important.)
Those issues you show in your question happen because your algorithm is wrong.
What you did (push):
throw a grid on the source image
transform those points
try to plot these colored points and let MATLAB do some magic to make it look like a dense picture
Do it the other way around (pull):
throw a grid on the output
transform that backwards
sample the input at those points
The distinction is called "push" (into output) vs "pull" (from input). Only Pull gives proper results.
Very little MATLAB code is necessary. You just need pol2cart and interp2, and a meshgrid.
With interp2 you get to choose the interpolation (linear, cubic, ...). Nearest-neighbor interpolation leaves visible artefacts.
im = im2single(imread("PQFax.jpg"));
% center of polar map, manually picked
cx = 10 + 409/2;
cy = 7 + 413/2;
% output parameters
radius = 212;
dRho = 1;
dTheta = 2*pi / (2*pi * radius);
Thetas = pi/2 - (0:dTheta:2*pi);
Rhos = (0:dRho:radius);
% polar mesh
[Theta, Rho] = meshgrid(Thetas, Rhos);
% transform...
[Xq,Yq] = pol2cart(Theta, Rho);
% translate to sit on the circle's center
Xq = Xq + cx;
Yq = Yq + cy;
% sample image at those points
Ro = interp2(im(:,:,1), Xq,Yq, "cubic");
Go = interp2(im(:,:,2), Xq,Yq, "cubic");
Bo = interp2(im(:,:,3), Xq,Yq, "cubic");
Vo = cat(3, Ro, Go, Bo);
Vo = imrotate(Vo, 180);
imshow(Vo)
The other way around (get a "torus" from a "ribbon") is quite similar. Throw a meshgrid on the torus space, subtract center, transform from cartesian to polar, and use those to sample from the "ribbon" image into the "torus" image.
I'm more familiar with OpenCV than with MATLAB. Perhaps MATLAB has something like OpenCV's warpPolar(), or a generic remap(). In any case, the operation is trivial to do entirely "by hand" but there are enough supporting functions to take the heavy lifting off your hands (interp2, pol2cart, meshgrid).
1.- The white arcs tell that the used translation pol-cart introduces significant errors.
2.- Reversing the following script solves your question.
It's a script that goes from cart-pol without introducing errors or ignoring input data, which is what happens when such wide white arcs show up upon translation apparently correct.
clear all;clc;close all
clc,cla;
format long;
A=imread('shaffen dass.jpg');
[sz1 sz2 sz3]=size(A);
szx=sz2;szy=sz1;
A1=A(:,:,1);A2=A(:,:,2);A3=A(:,:,3); % working with binary maps or grey scale images this wouldn't be necessary
figure(1);imshow(A);
hold all;
Cx=floor(szx/2);Cy=floor(szy/2);
plot(Cx,Cy,'co'); % because observe image centre not centered
Rmin=80;Rmax=400; % radius search range for imfindcircles
[centers, radii]=imfindcircles(A,[Rmin Rmax],... % outer circle
'ObjectPolarity','dark','Sensitivity',0.9);
h=viscircles(centers,radii);
hold all; % inner circle
[centers2, radii2]=imfindcircles(A,[Rmin Rmax],...
'ObjectPolarity','bright');
h=viscircles(centers2,radii2);
% L=floor(.5*(radii+radii2)); % this is NOT the length X that should have the resulting XY morphed graph
L=floor(2*pi*radii); % expected length of the morphed graph
cx=floor(.5*(centers(1)+centers2(1))); % coordinates of averaged circle centres
cy=floor(.5*(centers(2)+centers2(2)));
plot(cx,cy,'r*'); % check avg centre circle is not aligned to figure centre
plot([cx 1],[cy 1],'r-.');
t=[45:360/L:404+1-360/L]; % if step=1 then we only get 360 points but need an amount of L points
% if angle step 1/L over minute waiting for for loop to finish
R=radii+5;x=R*sind(t)+cx;y=R*cosd(t)+cy; % build outer perimeter
hL1=plot(x,y,'m'); % axis equal;grid on;
% hold all;
% plot(hL1.XData,hL1.YData,'ro');
x_ref=hL1.XData;y_ref=hL1.YData;
% Sx=zeros(ceil(R),1);Sy=zeros(ceil(R),1);
Sx={};Sy={};
for k=1:1:numel(hL1.XData)
Lx=floor(linspace(x_ref(k),cx,ceil(R)));
Ly=floor(linspace(y_ref(k),cy,ceil(R)));
% plot(Lx,Ly,'go'); % check
% plot([cx x(k)],[cy y(k)],'r');
% L1=unique([Lx;Ly]','rows');
Sx=[Sx Lx'];Sy=[Sy Ly'];
end
sx=cell2mat(Sx);sy=cell2mat(Sy);
[s1 s2]=size(sx);
B1=uint8(zeros(s1,s2));
B2=uint8(zeros(s1,s2));
B3=uint8(zeros(s1,s2));
for n=1:1:s2
for k=1:1:s1
B1(k,n)=A1(sx(k,n),sy(k,n));
B2(k,n)=A2(sx(k,n),sy(k,n));
B3(k,n)=A3(sx(k,n),sy(k,n));
end
end
C=uint8(zeros(s1,s2,3));
C(:,:,1)=B1;
C(:,:,2)=B2;
C(:,:,3)=B3;
figure(2);imshow(C);
the resulting
3.- let me know if you'd like some assistance writing pol-cart from this script.
Regards
John BG
I'm working on images with overlapping line shapes (left plot). Ultimately I want to segment single objects. I'm working with a Hough transform to achieve this and it works well in finding lines of (significantly) different orientation - e.g. represented by the two maxima in the hough space below (middle plot).
the green and yellow lines (left plot) and crosses (right plot) stem from an approach to do something with the thickness of the line. I couldn't figure out how to extract a broad line though, so I didn't follow up.
I'm aware of the ambiguity of assigning the "overlapping pixels". I will address that later.
Since I don't know, how many line objects one connected region may contain, my idea is to iteratively extract the object corresponding to the hough line with the highest activation (here painted in blue), i.e. remove the line shaped object from the image, so that the next iteration will find only the other line.
But how do I detect, which pixels belong to the line shaped object?
The function hough_bin_pixels(img, theta, rho, P) (from here - shown in the right plot) gives pixels corresponding to the particular line. But that obviously is too thin of a line to represent the object.
Is there a way to segment/detect the whole object that is orientied along the strongest houghline?
The key is knowing that thick lines in the original image translate to wider peaks on the Hough Transform. This image shows the peaks of a thin and a thick line.
You can use any strategy you like to group all the pixels/accumulator bins of each peak together. I would recommend using multithresh and imquantize to convert it to a BW image, and then bwlabel to label the connected components. You could also use any number of other clustering/segmentation strategies. The only potentially tricky part is figuring out the appropriate thresholding levels. If you can't get anything suitable for your application, err on the side of including too much because you can always get rid of erroneous pixels later.
Here are the peaks of the Hough Transform after thresholding (left) and labeling (right)
Once you have the peak regions, you can find out which pixels in the original image contributed to each accumulator bin using hough_bin_pixels. Then, for each peak region, combine the results of hough_bin_pixels for every bin that is part of the region.
Here is the code I threw together to create the sample images. I'm just getting back into matlab after not using it for a while, so please forgive the sloppy code.
% Create an image
image = zeros(100,100);
for i = 10:90
image(100-i,i)=1;
end;
image(10:90, 30:35) = 1;
figure, imshow(image); % Fig. 1 -- Original Image
% Hough Transform
[H, theta_vals, rho_vals] = hough(image);
figure, imshow(mat2gray(H)); % Fig. 2 -- Hough Transform
% Thresholding
thresh = multithresh(H,4);
q_image = imquantize(H, thresh);
q_image(q_image < 4) = 0;
q_image(q_image > 0) = 1;
figure, imshow(q_image) % Fig. 3 -- Thresholded Peaks
% Label connected components
L = bwlabel(q_image);
figure, imshow(label2rgb(L, prism)) % Fig. 4 -- Labeled peaks
% Reconstruct the lines
[r, c] = find(L(:,:)==1);
segmented_im = hough_bin_pixels(image, theta_vals, rho_vals, [r(1) c(1)]);
for i = 1:size(r(:))
seg_part = hough_bin_pixels(image, theta_vals, rho_vals, [r(i) c(i)]);
segmented_im(seg_part==1) = 1;
end
region1 = segmented_im;
[r, c] = find(L(:,:)==2);
segmented_im = hough_bin_pixels(image, theta_vals, rho_vals, [r(1) c(1)]);
for i = 1:size(r(:))
seg_part = hough_bin_pixels(image, theta_vals, rho_vals, [r(i) c(i)]);
segmented_im(seg_part==1) = 1;
end
region2 = segmented_im;
figure, imshow([region1 ones(100, 1) region2]) % Fig. 5 -- Segmented lines
% Overlay and display
out = cat(3, image, region1, region2);
figure, imshow(out); % Fig. 6 -- For fun, both regions overlaid on original image
can any one please help me in filling these black holes by values taken from neighboring non-zero pixels.
thanks
One nice way to do this is to is to solve the linear heat equation. What you do is fix the "temperature" (intensity) of the pixels in the good area and let the heat flow into the bad pixels. A passable, but somewhat slow, was to do this is repeatedly average the image then set the good pixels back to their original value with newImage(~badPixels) = myData(~badPixels);.
I do the following steps:
Find the bad pixels where the image is zero, then dilate to be sure we get everything
Apply a big blur to get us started faster
Average the image, then set the good pixels back to their original
Repeat step 3
Display
You could repeat averaging until the image stops changing, and you could use a smaller averaging kernel for higher precision---but this gives good results:
The code is as follows:
numIterations = 30;
avgPrecisionSize = 16; % smaller is better, but takes longer
% Read in the image grayscale:
originalImage = double(rgb2gray(imread('c:\temp\testimage.jpg')));
% get the bad pixels where = 0 and dilate to make sure they get everything:
badPixels = (originalImage == 0);
badPixels = imdilate(badPixels, ones(12));
%# Create a big gaussian and an averaging kernel to use:
G = fspecial('gaussian',[1 1]*100,50);
H = fspecial('average', [1,1]*avgPrecisionSize);
%# User a big filter to get started:
newImage = imfilter(originalImage,G,'same');
newImage(~badPixels) = originalImage(~badPixels);
% Now average to
for count = 1:numIterations
newImage = imfilter(newImage, H, 'same');
newImage(~badPixels) = originalImage(~badPixels);
end
%% Plot the results
figure(123);
clf;
% Display the mask:
subplot(1,2,1);
imagesc(badPixels);
axis image
title('Region Of the Bad Pixels');
% Display the result:
subplot(1,2,2);
imagesc(newImage);
axis image
set(gca,'clim', [0 255])
title('Infilled Image');
colormap gray
But you can get a similar solution using roifill from the image processing toolbox like so:
newImage2 = roifill(originalImage, badPixels);
figure(44);
clf;
imagesc(newImage2);
colormap gray
notice I'm using the same badPixels defined from before.
There is a file on Matlab file exchange, - inpaint_nans that does exactly what you want. The author explains why and in which cases it is better than Delaunay triangulation.
To fill one black area, do the following:
1) Identify a sub-region containing the black area, the smaller the better. The best case is just the boundary points of the black hole.
2) Create a Delaunay triangulation of the non-black points in inside the sub-region by:
tri = DelaunayTri(x,y); %# x, y (column vectors) are coordinates of the non-black points.
3) Determine the black points in which Delaunay triangle by:
[t, bc] = pointLocation(tri, [x_b, y_b]); %# x_b, y_b (column vectors) are coordinates of the black points
tri = tri(t,:);
4) Interpolate:
v_b = sum(v(tri).*bc,2); %# v contains the pixel values at the non-black points, and v_b are the interpolated values at the black points.
Let's say I have 9 MxN black and white images that are in some way related to one another (i.e. time lapse of some event). What is a way that I can display all of these images on one surface plot?
Assume the MxN matrices only contain 0's and 1's. Assume the images simply contain white lines on a black background (i.e. pixel value == 1 if that pixel is part of a line, 0 otherwise). Assume images are ordered in such a way as to suggest movement progression of line(s) in subsequent images. I want to be able to see a "side-view" (or volumetric representation) of these images which will show the surface that a particular line "carves out" in its movement across the images.
Coding is done in MATLAB. I have looked at plot (but it only does 2D plots) and surf, which does 3D plots but doesn't work for my MxNx9 matrix of images. I have also tried to experiment with contourslice, but not sure what parameters to pass it.
Thanks!
Mariya
Are these images black and white with simple features on a "blank" field, or greyscale, with more dense information?
I can see a couple of approaches.
You can use movie() to display a sequence of images as an animation.
For a static view of sparse, simple data, you could plot each image as a separate layer in a single figure, giving each layer a different color for the foreground, and using AlphaData to make the background transparent so all the steps in the sequenc show through. The gradient of colors corresponds to position in the image sequence. Here's an example.
function plotImageSequence
% Made-up test data
nLayers = 9;
x = zeros(100,100,nLayers);
for i = 1:nLayers
x(20+(3*i),:,i) = 1;
end
% Plot each image as a "layer", indicated by color
figure;
hold on;
for i = 1:nLayers
layerData = x(:,:,i);
alphaMask = layerData == 1;
layerData(logical(layerData)) = i; % So each layer gets its own color
image('CData',layerData,...
'AlphaData',alphaMask,...
'CDataMapping','scaled');
end
hold off
Directly showing the path of movement a "line" carves out is hard with raster data, because Matlab won't know which "moved" pixels in two subsequent images are associated with each other. Don't suppose you have underlying vector data for the geometric features in the images? Plot3() might allow you to show their movement, with time as the z axis. Or you could use the regular plot() and some manual fiddling to plot the paths of all the control points or vertexes in the geometric features.
EDIT: Here's a variation that uses patch() to draw each pixel as a little polygon floating in space at the Z level of its index in the image sequence. I think this will look more like the "surface" style plots you are asking for. You could fiddle with the FaceAlpha property to make dense plots more legible.
function plotImageSequencePatch
% Made-up test data
nLayers = 6;
sz = [50 50];
img = zeros(sz(1),sz(2),nLayers);
for i = 1:nLayers
img(20+(3*i),:,i) = 1;
end
% Plot each image as a "layer", indicated by color
% With each "pixel" as a separate patch
figure;
set(gca, 'XLim', [0 sz(1)]);
set(gca, 'YLim', [0 sz(2)]);
hold on;
for i = 1:nLayers
layerData = img(:,:,i);
[x,y] = find(layerData); % X,Y of all pixels
% Reshape in to patch outline
x = x';
y = y';
patch_x = [x; x+1; x+1; x];
patch_y = [y; y; y+1; y+1];
patch_z = repmat(i, size(patch_x));
patch(patch_x, patch_y, patch_z, i);
end
hold off
After doing the Hough transform in MATLAB, how do I pick the lines so that I can compare between two or more images?
I followed the example given by Amro and actually what I wanted to detect is the two lines in the first picture. However, what I got is the one in the second picture. How can I do this?
I think you meant the goal to be to detect lines in an image, not comparing two images (?).
Anyway, to find the maximum intensities in the Hough transform matrix generated by the hough function, we use the houghpeaks function, and pass it the desired number of peaks to detect.
EDIT1:
I figured I would add an example to show the procedure:
%# Load image, process it, find edges
I = rgb2gray( imread('pillsetc.png') );
I = imcrop(I, [30 30 450 350]);
J = imfilter(I, fspecial('gaussian', [17 17], 5), 'symmetric');
BW = edge(J, 'canny');
%# Perform Hough transform and show matrix
[H,T,R] = hough(BW);
imshow(imadjust(mat2gray(H)), [], 'XData',T, 'YData',R, ...
'InitialMagnification','fit')
xlabel('\theta (degrees)'), ylabel('\rho')
axis on, axis normal, hold on
colormap(hot), colorbar
%# Detect peaks
P = houghpeaks(H, 4);
plot(T(P(:,2)), R(P(:,1)), 'gs', 'LineWidth',2);
%# Detect lines and overlay on top of image
lines = houghlines(BW, T, R, P);
figure, imshow(I), hold on
for k = 1:length(lines)
xy = [lines(k).point1; lines(k).point2];
plot(xy(:,1), xy(:,2), 'g.-', 'LineWidth',2);
end
hold off
EDIT2:
Following your recent update, I managed to detect the lines by only making a few changes to the same above code:
I cropped the region to: [200 70 160 140]
I used an 11x11 Gaussian filter with sigma=3
Note: You will have to add the offset to get the position of the lines in the original image uncropped. Also, if you want more accurate results, you might want to detect four lines and get the lines in the middle as shown below: