How to perform matching by MSER and HOG in Matlab - matlab

I wanted to know if there is any full implementation of image-matching by MSER and HOG in Matlab. Currently I am using VLFeat but found difficulties when performing the image matching. Any help?
Btw, I've tried the below code in VLFeat -Matlab environment but unfortunately the matching can't be performed.
%Matlab code
%
pfx = fullfile(vl_root,'figures','demo') ;
randn('state',0) ;
rand('state',0) ;
figure(1) ; clf ;
Ia = imread(fullfile(vl_root,'data','roofs1.jpg')) ;
Ib = imread(fullfile(vl_root,'data','roofs2.jpg')) ;
Ia = uint8(rgb2gray(Ia)) ;
Ib = uint8(rgb2gray(Ib)) ;
[ra,fa] = vl_mser(I,'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ;
[rb,fb] = vl_mser(I,'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ;
[matches, scores] = vl_ubcmatch(fa, fb);
figure(1) ; clf ;
imagesc(cat(2, Ia, Ib));
axis image off ;
vl_demo_print('mser_match_1', 1);
figure(2) ; clf ;
imagesc(cat(2, Ia, Ib));
xa = ra(1, matches(1,:));
xb = rb(1, matches(2,:)) + size(Ia,2);
ya = ra(2, matches(1,:));
yb = rb(2,matches(2,:));
hold on ;
h = line([xa ; xb], [ya ; yb]);
set(h, 'linewidth', 1, 'color', 'b');
vl_plotframe(ra(:,matches(1,:)));
rb(1,:) = fb(1,:) + size(Ia,2);
vl_plotframe(rb(:,mathces(2,:)));
axis image off ;
vl_demo_print('mser_match_2', 1);
%%%%%%

There are a couple problems. First, the code has several errors and doesn't run as-is. I've pasted my working version below.
More importantly, you're trying to use the SIFT feature-matching function to match the MSER ellipsoids. This won't work at all, since SIFT gives a very high dimensional feature vector based on local image gradients, and the MSER detector is just giving you a bounding ellipsoid.
VLFeat doesn't appear to include an MSER-matching function, so you'll probably have to write your own. Take a look at the original MSER paper to understand how they did matching:
"Robust wide-baseline stereo from maximally stable extremal regions", Matas et al. 2002
% Read the input images
Ia = imread(fullfile(vl_root,'data','roofs1.jpg')) ;
Ib = imread(fullfile(vl_root,'data','roofs2.jpg')) ;
% Convert to grayscale
Ia = uint8(rgb2gray(Ia)) ;
Ib = uint8(rgb2gray(Ib)) ;
% Find MSERs
[ra,fa] = vl_mser(Ia, 'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ;
[rb,fb] = vl_mser(Ib, 'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ;
% Match MSERs
[matches, scores] = vl_ubcmatch(fa, fb);
% Display the original input images
figure(1); clf;
imagesc(cat(2, Ia, Ib));
axis image off;
colormap gray;
% Display a second copy with the matches overlaid
figure(2) ; clf ;
imagesc(cat(2, Ia, Ib));
axis image off;
colormap gray;
xa = fa(1, matches(1,:));
ya = fa(2, matches(1,:));
xb = fb(1, matches(2,:)) + size(Ia,2);
yb = fb(2, matches(2,:));
hold on ;
h = line([xa ; xb], [ya ; yb]);
set(h, 'linewidth', 1, 'color', 'y');

I don't know how, but MSER matching works in Matlab itself.
The code below
file1 = 'roofs1.jpg';
file2 = 'roofs2.jpg';
I1 = imread(file1);
I2 = imread(file2);
I1 = rgb2gray(I1);
I2 = rgb2gray(I2);
% %Find the SURF features.
% points1 = detectSURFFeatures(I1);
% points2 = detectSURFFeatures(I2);
points1 = detectMSERFeatures(I1);
points2 = detectMSERFeatures(I2);
%Extract the features.
[f1, vpts1] = extractFeatures(I1, points1);
[f2, vpts2] = extractFeatures(I2, points2);
%Retrieve the locations of matched points. The SURF featurevectors are already normalized.
indexPairs = matchFeatures(f1, f2, 'Prenormalized', true) ;
matched_pts1 = vpts1(indexPairs(:, 1));
matched_pts2 = vpts2(indexPairs(:, 2));
figure; showMatchedFeatures(I1,I2,matched_pts1,matched_pts2,'montage');
legend('matched points 1','matched points 2');
gives the following picture

Related

Detecting lanes in a video using matlab and image processing.

I am a little bit new to matlab and imageprocessing and I was given a task at my faculty to carry out a project which detects the lanes for a moving car in a video. I tried to use some tutorials on Mathworks and other sites and there were really helpful and I came out with a code that detects lanes in an image and I just want to know how to apply my code on a video as I see it working properly on an image.
and here is my code :
img = imread ('test_image.jpg');
I = rgb2gray (img);
%making a gaussian kernel
sigma = 1 ; %standard deviation of distribution
kernel = zeros (5,5); %for a 5x5 kernel
W = 0 ;
for i = 1:5
for j = 1:5
sq_dist = (i-3)^2 + (j-3)^2 ;
kernel (i,j) = exp (-1*exp(sq_dist)/(2*sigma));
W = W + kernel (i,j) ;
end
end
kernenl = kernel/W ;
%Now we apply the filter to the image
[m,n] = size (I) ;
output = zeros (m,n);
Im = padarray (I , [2 2]);
for i=1:m
for j=1:n
temp = Im (i:i+4 , j:j+4);
temp = double(temp);
conv = temp.*kernel;
output(i,j) = sum(conv(:));
end
end
output = uint8(output);
%--------------Binary image-------------
level = graythresh(output);
c= im2bw (output,level);
%---------------------------------------
output2 = edge (c , 'canny',level);
figure (1);
%Segment out the region of interest
ROI = maskedImage;
CannyROI = edge (ROI , 'canny',.45);
%----------------------------------
set (gcf, 'Position', get (0,'Screensize'));
%subplot (141), imshow (I), title ('original image');
%subplot (142), imshow (c), title ('Binary image');
%subplot (143), imshow (output2), title ('Canny image');
%subplot (144), imshow (CannyROI), title ('ROI image');
[H ,T ,R] = hough(CannyROI);
imshow (H,[],'XData',T,'YData',R,'initialMagnification','fit');
xlabel('\theta'), ylabel('\rho');
axis on , axis normal, hold on ;
P = houghpeaks(H,5,'threshold',ceil (0.3*max(H(:))));
x = T(P(:,2));
y = R(P(:,1));
plot (x,y,'s','color','white');
%Find lines and plot them
lines = houghlines (CannyROI,T,R,P,'FillGap',5,'MinLength',7);
figure, imshow (img), hold on
max_len = 0 ;
for k = 1:length(lines);
xy = [lines(k).point1; lines(k).point2];
plot (xy(:,1), xy(:,2), 'LineWidth', 5 , 'Color', 'blue');
%plot beginnings and ends of the lines
plot (xy(1,1), xy(1,2),'x', 'LineWidth', 2, 'Color', 'yellow');
plot (xy(2,1), xy(2,2),'x', 'LineWidth', 2, 'Color', 'red');
%determine the endpoints of the longest line segment
len = norm(lines(k).point1 - lines(k).point2);
if (len>max_len)
max_len = len;
xy_long = xy;
end
end
and here is the link of the image and the video :
https://github.com/rslim087a/road-video
https://github.com/rslim087a/road-image
Thanks in advance.
Basically video processing happens in such a way that video will be converted to video frames (images). So if you need, you can convert your video to video frames and run the code, looping over the folder having the video frames. Change the imread function to get images from video frames folder...
img = imread(path_to_video_frames_folder/*)

Plot bivariate map in Matlab

I have two maps (matrices), and want to use them to generate the corresponding bivariate map.
The below code illustrates the idea using 'scatter', but because my original matrices are quite large, I need a different solution than to draw each point individually.
red_blue_colormap = brewermap(20,'RdYlBu'); %using ColorBrewer colormaps 'brewermap'
white2red = newmap(10:-1:1,:); %color map going from white to red
white2blue = newmap(11:20,:); %color map going from white to blue
X = rand(13,8); Y = rand(13,8); %sample matrices
figure;
for i=1:size(X,1)
for j=1:size(X,2)
portion_red = white2red(ceil(X(i,j)*10),:); %value between 'white' and 'red' corresponding to value of X(i,j)
portion_blue = white2blue(ceil(Y(i,j)*10),:);
subplot(2,2,1); hold on; title('X');
scatter(i,j,100,portion_red ,'s','filled');
subplot(2,2,2); hold on; title('Y');
scatter(i,j,100,portion_blue ,'s','filled');
subplot(2,2,3); hold on; title('X vs. Y');
w1 = Y(i,j)/(X(i,j) + Y(i,j)); %relative weight of 'Y'
color1 = portion_red - ((portion_red - portion_blue ) * w1);
scatter(i,j,100,color1,'s','filled');
subplot(2,2,4); hold on; title('Color matrix');
portion_red = white2red(ceil(i/size(X,1)*10),:);
portion_blue = white2blue(ceil(j/size(X,2)*10),:);
w2 = (j/size(X,2))/(i/size(X,1) + j/size(X,2));
color2 = portion_red - ((portion_red - portion_blue ) * w2);
scatter(i,j,100,color2,'s','filled');
end
end
This generates the following figure: https://image.ibb.co/kotP7m/bivariate_map.png
I want to plot the bottom left figure in a more efficient way. Any ideas?

how to extract each characters from a image?with using this code

i have to extract each characters from a image here i am uploading the code it is segmenting the horizontal lines but not able to segment the each characters along with the horizontal line segmentation loop. some1 please help to correct the code
this is the previous code:
%%horizontal histogram
H = sum(rotatedImage, 2);
darkPixels = H < 100; % Threshold
% label
[labeledRegions, numberOfRegions] = bwlabel(darkPixels);
fprintf('Number of regions = %d\n', numberOfRegions);
% Find centroids
measurements = regionprops(labeledRegions, 'Centroid');
% Get them into an array
allCentroids = [measurements.Centroid];
xCentroids = int32(allCentroids(1:2:end));
yCentroids = int32(allCentroids(2:2:end));
% Now you can just crop out some line of text you're interested in, into a separate image:
hold off;
plotLocation = 8;
for band = 1 : numberOfRegions-1
row1 = yCentroids(band);
row2 = yCentroids(band+1);
thisLine = rotatedImage(row1 : row2, :);
subplot(7, 2, plotLocation)
imshow(thisLine, [])
%% Let's compute and display the histogram.
verticalProjection = sum(thisLine, 2);
set(gcf, 'NumberTitle', 'Off')
t = verticalProjection;
t(t==0) = inf;
mayukh=min(t);
% 0 where there is background, 1 where there are letters
letterLocations = verticalProjection > mayukh;
% Find Rising and falling edges
d = diff(letterLocations);
startingRows = find(d>0);
endingRows = find(d<0);
% Extract each region
y=1;
for k = 1 : length(startingRows)
% Get sub image of just one character...
subImage = thisLine(:, startingRows(k):endingRows(k));
[L,num] = bwlabel(subImage);
for z= 1 : num
bw= ismember( L, z);
% Construct filename for this particular image.
baseFileName = sprintf('templates %d.png', y);
y=y+1;
% Prepend the folder to make the full file name.
fullFileName = fullfile('C:\Users\Omm\Downloads\', baseFileName);
% Do the write to disk.
imwrite(bw, fullFileName);
pause(2);
imshow(bw);
pause(5)
end;
y=y+2;
end;
plotLocation = plotLocation + 2;
end
but not segmenting the whole lines
Why don't you simply use regionprops with 'Image' property?
img = imread('http://i.stack.imgur.com/zpYa5.png'); %// read the image
bw = img(:,:,1) > 128; %// conver to mask
Use some minor morphological operations to handle spurious pixels
dbw = imdilate(bw, ones(3));
lb = bwlabel(dbw).*bw; %// label each character as a connected component
Now you can use regionprops to get each image
st = regionprops( lb, 'Image' );
Visualize the results
figure;
for ii=1:numel(st),
subplot(4,5,ii);
imshow(st(ii).Image,'border','tight');
title(num2str(ii));
end

Radon Transform Line Detection

I'm trying to detect lines in a grayscale image. For that purpose, I'm using Radon transform in MATLAB. An example of my m-file is like below. I can detect multiple lines using this code. I also draw lines using shift and rotation properties for lines. However, I didn't understand how to get the start and end points of the detecting lines after getting rho and theta values.
It is easy for Hough transform since there is a function called houghlines() that returns the list of the lines for the given peaks. Is there any function that i can use for Radon transform similar to this function?
% Radon transform line detection algorithm
clear all; close all;
% Determine the path of the input image
str_inputimg = '3_lines.png' ;
% Read input image
I = imread(str_inputimg) ;
% If the input image is RGB or indexed color, convert it to grayscale
img_colortype = getfield(imfinfo(str_inputimg), 'ColorType') ;
switch img_colortype
case 'truecolor'
I = rgb2gray(I) ;
case 'indexedcolor'
I = ind2gray(I) ;
end
figure;
subplot(2,2,1) ;
imshow(I) ;
title('Original Image') ;
% Convert image to black white
%BW = edge(I,'Sobel');
BW=im2bw(I,0.25) ;
subplot(2,2,2) ;
imshow(BW);
title('BW Image') ;
% Radon transform
% Angle projections
theta = [0:179]' ;
[R, rho] = radon(BW, theta) ;
subplot(2,2,3) ;
imshow(R, [], 'XData', theta, 'YData', rho, 'InitialMagnification', 'fit');
xlabel('\theta'), ylabel('\rho');
axis on, axis normal, hold on;
% Detect the peaks of transform output
% Threshold value for peak detection
threshold_val = ceil(0.3*max(R(:))) ;
% Maximum nof peaks to identify in the image
max_nofpeaks = 5 ;
max_indexes = find(R(:)>threshold_val) ;
max_values = R(max_indexes) ;
[sorted_max, temp_indexes] = sort(max_values, 'descend') ;
sorted_indexes = max_indexes(temp_indexes) ;
% Get the first highest peaks for the sorted array
if (length(sorted_max) <= max_nofpeaks)
peak_values = sorted_max(1:end) ;
peak_indexes = sorted_indexes(1:end) ;
else
peak_values = sorted_max(1:max_nofpeaks) ;
peak_indexes = sorted_indexes(1:max_nofpeaks) ;
end
[y, x] = ind2sub(size(R), peak_indexes ) ;
peaks = [rho(y) theta(x)] ;
plot(peaks(:,2), peaks(:,1), 's', 'color','white');
title('Radon Transform & Peaks') ;
% Detected lines on the image
subplot(2,2,4), imshow(I), title('Detected lines'), hold on
x_center = floor(size(I, 2)/2) ;
y_center = floor(size(I, 1)/2) ;
for p=1:length(peaks)
x_1 = [-x_center, x_center] ;
y_1 = [0, 0] ;
% Shift at first
x_1_shifted = x_1 ;
y_1_shifted = [y_1(1)-peaks(p,1), y_1(2)-peaks(p,1)] ;
% Rotate
peaks(p,2) = 90 - peaks(p,2) ;
t=peaks(p,2)*pi/180;
rotation_mat = [ cos(t) -sin(t) ; sin(t) cos(t) ] ;
x_y_rotated = rotation_mat*[x_1_shifted; y_1_shifted] ;
x_rotated = x_y_rotated(1,:) ;
y_rotated = x_y_rotated(2,:) ;
plot( x_rotated+x_center, y_rotated+y_center, 'b', 'linewidth', 2 );
end
hold off;
There's a suggestion at math.SE which might help. Then there's a rather complicated-looking research paper "Sharp endpoint estimates for the X-ray transform and the Radon
transform in finite fields", which appears just to show certain bounds on estimation accuracy.
From skimming other papers, it appears that it's a nontrivial problem. I suspect it may be simpler (if less accurate) to use some adaptation of a Sobel-operation to identify high gradient points along the discovered line, and claim those as endpoints.

Segmenting cursive character (Arabic OCR)

I want to segment an Arabic word into single characters. Based on the histogram/profile, I assume that I can do the segmentation process by cut/segment the characters based on it's baseline (it have similar pixel values).
But, unfortunately, I still stuck to build the appropriate code, to make it works.
% Original Code by Soumyadeep Sinha
% Saving each single segmented character as one file
function [segm] = trysegment (a)
myFolder = 'D:\1. Thesis FINISH!!!\Data set\trial';
level = graythresh (a);
bw = im2bw (a, level);
b = imcomplement (bw);
i= padarray(b,[0 10]);
verticalProjection = sum(i, 1);
set(gcf, 'Name', 'Trying Segmentation for Cursive', 'NumberTitle', 'Off')
subplot(2, 2, 1);imshow(i);
subplot(2,2,3);
plot(verticalProjection, 'b-'); %histogram show by this code
% hist(reshape(input,[],3),1:max(input(:)));
grid on;
% % t = verticalProjection;
% % t(t==0) = inf;
% % mayukh = min(t)
% 0 where there is background, 1 where there are letters
letterLocations = verticalProjection > 0;
% Find Rising and falling edges
d = diff(letterLocations);
startingColumns = find(d>0);
endingColumns = find(d<0);
% Extract each region
y=1;
for k = 1 : length(startingColumns)
% Get sub image of just one character...
subImage = i(:, startingColumns(k):endingColumns(k));
% se = strel('rectangle',[2 4]);
% dil = imdilate(subImage, se);
th = bwmorph(subImage,'thin',Inf);
n = imresize (th, [64 NaN], 'bilinear');
figure, imshow (n);
[L,num] = bwlabeln(n);
for z= 1 : num
bw= ismember(L, z);
% Construct filename for this particular image.
baseFileName = sprintf('char %d.png', y);
y=y+1;
% Prepend the folder to make the full file name.
fullFileName = fullfile(myFolder, baseFileName);
% Do the write to disk.
imwrite(bw, fullFileName);
% subplot(2,2,4);
% pause(2);
% imshow(bw);
end
% y=y+1;
end;
segm = (n);
Word image is as follow:
Why the code isn't work?
do you have any recommendation of another codes?
or suggested algorithm to make it works, to do a good segmentation on cursive character?
Thanks before.
Replace this code part from the posted code
% 0 where there is background, 1 where there are letters
letterLocations = verticalProjection > 0;
% Find Rising and falling edges
d = diff(letterLocations);
startingColumns = find(d>0);
endingColumns = find(d<0);
with the new code part
threshold=max(verticalProjection)/3;
thresholdedProjection=verticalProjection > threshold;
count=0;
startingColumnsIndex=0;
for i=1:length(thresholdedProjection)
if thresholdedProjection(i)
if(count>0)
startingColumnsIndex=startingColumnsIndex+1;
startingColumns(startingColumnsIndex)= i-floor(count/2);
count=0;
end
else
count=count+1;
end
end
endingColumns=[startingColumns(2:end)-1 i-floor(count/2)];
No changes needed for the rest of the code.