Detecting lanes in a video using matlab and image processing. - matlab

I am a little bit new to matlab and imageprocessing and I was given a task at my faculty to carry out a project which detects the lanes for a moving car in a video. I tried to use some tutorials on Mathworks and other sites and there were really helpful and I came out with a code that detects lanes in an image and I just want to know how to apply my code on a video as I see it working properly on an image.
and here is my code :
img = imread ('test_image.jpg');
I = rgb2gray (img);
%making a gaussian kernel
sigma = 1 ; %standard deviation of distribution
kernel = zeros (5,5); %for a 5x5 kernel
W = 0 ;
for i = 1:5
for j = 1:5
sq_dist = (i-3)^2 + (j-3)^2 ;
kernel (i,j) = exp (-1*exp(sq_dist)/(2*sigma));
W = W + kernel (i,j) ;
end
end
kernenl = kernel/W ;
%Now we apply the filter to the image
[m,n] = size (I) ;
output = zeros (m,n);
Im = padarray (I , [2 2]);
for i=1:m
for j=1:n
temp = Im (i:i+4 , j:j+4);
temp = double(temp);
conv = temp.*kernel;
output(i,j) = sum(conv(:));
end
end
output = uint8(output);
%--------------Binary image-------------
level = graythresh(output);
c= im2bw (output,level);
%---------------------------------------
output2 = edge (c , 'canny',level);
figure (1);
%Segment out the region of interest
ROI = maskedImage;
CannyROI = edge (ROI , 'canny',.45);
%----------------------------------
set (gcf, 'Position', get (0,'Screensize'));
%subplot (141), imshow (I), title ('original image');
%subplot (142), imshow (c), title ('Binary image');
%subplot (143), imshow (output2), title ('Canny image');
%subplot (144), imshow (CannyROI), title ('ROI image');
[H ,T ,R] = hough(CannyROI);
imshow (H,[],'XData',T,'YData',R,'initialMagnification','fit');
xlabel('\theta'), ylabel('\rho');
axis on , axis normal, hold on ;
P = houghpeaks(H,5,'threshold',ceil (0.3*max(H(:))));
x = T(P(:,2));
y = R(P(:,1));
plot (x,y,'s','color','white');
%Find lines and plot them
lines = houghlines (CannyROI,T,R,P,'FillGap',5,'MinLength',7);
figure, imshow (img), hold on
max_len = 0 ;
for k = 1:length(lines);
xy = [lines(k).point1; lines(k).point2];
plot (xy(:,1), xy(:,2), 'LineWidth', 5 , 'Color', 'blue');
%plot beginnings and ends of the lines
plot (xy(1,1), xy(1,2),'x', 'LineWidth', 2, 'Color', 'yellow');
plot (xy(2,1), xy(2,2),'x', 'LineWidth', 2, 'Color', 'red');
%determine the endpoints of the longest line segment
len = norm(lines(k).point1 - lines(k).point2);
if (len>max_len)
max_len = len;
xy_long = xy;
end
end
and here is the link of the image and the video :
https://github.com/rslim087a/road-video
https://github.com/rslim087a/road-image
Thanks in advance.

Basically video processing happens in such a way that video will be converted to video frames (images). So if you need, you can convert your video to video frames and run the code, looping over the folder having the video frames. Change the imread function to get images from video frames folder...
img = imread(path_to_video_frames_folder/*)

Related

how to extract each characters from a image?with using this code

i have to extract each characters from a image here i am uploading the code it is segmenting the horizontal lines but not able to segment the each characters along with the horizontal line segmentation loop. some1 please help to correct the code
this is the previous code:
%%horizontal histogram
H = sum(rotatedImage, 2);
darkPixels = H < 100; % Threshold
% label
[labeledRegions, numberOfRegions] = bwlabel(darkPixels);
fprintf('Number of regions = %d\n', numberOfRegions);
% Find centroids
measurements = regionprops(labeledRegions, 'Centroid');
% Get them into an array
allCentroids = [measurements.Centroid];
xCentroids = int32(allCentroids(1:2:end));
yCentroids = int32(allCentroids(2:2:end));
% Now you can just crop out some line of text you're interested in, into a separate image:
hold off;
plotLocation = 8;
for band = 1 : numberOfRegions-1
row1 = yCentroids(band);
row2 = yCentroids(band+1);
thisLine = rotatedImage(row1 : row2, :);
subplot(7, 2, plotLocation)
imshow(thisLine, [])
%% Let's compute and display the histogram.
verticalProjection = sum(thisLine, 2);
set(gcf, 'NumberTitle', 'Off')
t = verticalProjection;
t(t==0) = inf;
mayukh=min(t);
% 0 where there is background, 1 where there are letters
letterLocations = verticalProjection > mayukh;
% Find Rising and falling edges
d = diff(letterLocations);
startingRows = find(d>0);
endingRows = find(d<0);
% Extract each region
y=1;
for k = 1 : length(startingRows)
% Get sub image of just one character...
subImage = thisLine(:, startingRows(k):endingRows(k));
[L,num] = bwlabel(subImage);
for z= 1 : num
bw= ismember( L, z);
% Construct filename for this particular image.
baseFileName = sprintf('templates %d.png', y);
y=y+1;
% Prepend the folder to make the full file name.
fullFileName = fullfile('C:\Users\Omm\Downloads\', baseFileName);
% Do the write to disk.
imwrite(bw, fullFileName);
pause(2);
imshow(bw);
pause(5)
end;
y=y+2;
end;
plotLocation = plotLocation + 2;
end
but not segmenting the whole lines
Why don't you simply use regionprops with 'Image' property?
img = imread('http://i.stack.imgur.com/zpYa5.png'); %// read the image
bw = img(:,:,1) > 128; %// conver to mask
Use some minor morphological operations to handle spurious pixels
dbw = imdilate(bw, ones(3));
lb = bwlabel(dbw).*bw; %// label each character as a connected component
Now you can use regionprops to get each image
st = regionprops( lb, 'Image' );
Visualize the results
figure;
for ii=1:numel(st),
subplot(4,5,ii);
imshow(st(ii).Image,'border','tight');
title(num2str(ii));
end

Radon Transform Line Detection

I'm trying to detect lines in a grayscale image. For that purpose, I'm using Radon transform in MATLAB. An example of my m-file is like below. I can detect multiple lines using this code. I also draw lines using shift and rotation properties for lines. However, I didn't understand how to get the start and end points of the detecting lines after getting rho and theta values.
It is easy for Hough transform since there is a function called houghlines() that returns the list of the lines for the given peaks. Is there any function that i can use for Radon transform similar to this function?
% Radon transform line detection algorithm
clear all; close all;
% Determine the path of the input image
str_inputimg = '3_lines.png' ;
% Read input image
I = imread(str_inputimg) ;
% If the input image is RGB or indexed color, convert it to grayscale
img_colortype = getfield(imfinfo(str_inputimg), 'ColorType') ;
switch img_colortype
case 'truecolor'
I = rgb2gray(I) ;
case 'indexedcolor'
I = ind2gray(I) ;
end
figure;
subplot(2,2,1) ;
imshow(I) ;
title('Original Image') ;
% Convert image to black white
%BW = edge(I,'Sobel');
BW=im2bw(I,0.25) ;
subplot(2,2,2) ;
imshow(BW);
title('BW Image') ;
% Radon transform
% Angle projections
theta = [0:179]' ;
[R, rho] = radon(BW, theta) ;
subplot(2,2,3) ;
imshow(R, [], 'XData', theta, 'YData', rho, 'InitialMagnification', 'fit');
xlabel('\theta'), ylabel('\rho');
axis on, axis normal, hold on;
% Detect the peaks of transform output
% Threshold value for peak detection
threshold_val = ceil(0.3*max(R(:))) ;
% Maximum nof peaks to identify in the image
max_nofpeaks = 5 ;
max_indexes = find(R(:)>threshold_val) ;
max_values = R(max_indexes) ;
[sorted_max, temp_indexes] = sort(max_values, 'descend') ;
sorted_indexes = max_indexes(temp_indexes) ;
% Get the first highest peaks for the sorted array
if (length(sorted_max) <= max_nofpeaks)
peak_values = sorted_max(1:end) ;
peak_indexes = sorted_indexes(1:end) ;
else
peak_values = sorted_max(1:max_nofpeaks) ;
peak_indexes = sorted_indexes(1:max_nofpeaks) ;
end
[y, x] = ind2sub(size(R), peak_indexes ) ;
peaks = [rho(y) theta(x)] ;
plot(peaks(:,2), peaks(:,1), 's', 'color','white');
title('Radon Transform & Peaks') ;
% Detected lines on the image
subplot(2,2,4), imshow(I), title('Detected lines'), hold on
x_center = floor(size(I, 2)/2) ;
y_center = floor(size(I, 1)/2) ;
for p=1:length(peaks)
x_1 = [-x_center, x_center] ;
y_1 = [0, 0] ;
% Shift at first
x_1_shifted = x_1 ;
y_1_shifted = [y_1(1)-peaks(p,1), y_1(2)-peaks(p,1)] ;
% Rotate
peaks(p,2) = 90 - peaks(p,2) ;
t=peaks(p,2)*pi/180;
rotation_mat = [ cos(t) -sin(t) ; sin(t) cos(t) ] ;
x_y_rotated = rotation_mat*[x_1_shifted; y_1_shifted] ;
x_rotated = x_y_rotated(1,:) ;
y_rotated = x_y_rotated(2,:) ;
plot( x_rotated+x_center, y_rotated+y_center, 'b', 'linewidth', 2 );
end
hold off;
There's a suggestion at math.SE which might help. Then there's a rather complicated-looking research paper "Sharp endpoint estimates for the X-ray transform and the Radon
transform in finite fields", which appears just to show certain bounds on estimation accuracy.
From skimming other papers, it appears that it's a nontrivial problem. I suspect it may be simpler (if less accurate) to use some adaptation of a Sobel-operation to identify high gradient points along the discovered line, and claim those as endpoints.

To refresh imshow in Matlab?

I want to convert this answer's code to imshow.
It creates a movie in MOVIE2AVI by
%# preallocate
nFrames = 20;
mov(1:nFrames) = struct('cdata',[], 'colormap',[]);
%# create movie
for k=1:nFrames
surf(sin(2*pi*k/20)*Z, Z)
mov(k) = getframe(gca);
end
close(gcf)
movie2avi(mov, 'myPeaks1.avi', 'compression','None', 'fps',10);
My pseudocode
%# preallocate
nFrames = 20;
mov(1:nFrames) = struct('cdata',[], 'colormap',[]);
%# create movie
for k=1:nFrames
imshow(signal(:,k,:),[1 1 1]) % or simply imshow(signal(:,k,:))
drawnow
mov(k) = getframe(gca);
end
close(gcf)
movie2avi(mov, 'myPeaks1.avi', 'compression','None', 'fps',10);
However, this creates the animation in the screen, but it saves only a AVI -file which size is 0 kB. The file myPeaks1.avi is stored properly after running the surf command but not from imshow.
I am not sure about the command drawnow.
Actual case code
%% HSV 3rd version
% https://stackoverflow.com/a/29801499/54964
rgbImage = imread('http://i.stack.imgur.com/cFOSp.png');
% Extract blue using HSV
hsvImage=rgb2hsv(rgbImage);
I=rgbImage;
R=I(:,:,1);
G=I(:,:,2);
B=I(:,:,3);
R((hsvImage(:,:,1)>(280/360))|(hsvImage(:,:,1)<(200/360)))=255;
G((hsvImage(:,:,1)>(280/360))|(hsvImage(:,:,1)<(200/360)))=255;
B((hsvImage(:,:,1)>(280/360))|(hsvImage(:,:,1)<(200/360)))=255;
I2= cat(3, R, G, B);
% Binarize image, getting all the pixels that are "blue"
bw=im2bw(rgb2gray(I2),0.9999);
% The label most repeated will be the signal.
% So we find it and separate the background from the signal using label.
% Label each "blob"
lbl=bwlabel(~bw);
% Find the blob with the highes amount of data. That will be your signal.
r=histc(lbl(:),1:max(lbl(:)));
[~,idxmax]=max(r);
% Profit!
signal=rgbImage;
signal(repmat((lbl~=idxmax),[1 1 3]))=255;
background=rgbImage;
background(repmat((lbl==idxmax),[1 1 3]))=255;
%% Error Testing
comp_image = rgb2gray(abs(double(rgbImage) - double(signal)));
if ( sum(sum(comp_image(32:438, 96:517))) > 0 )
break;
end
%% Video
% 5001 units so 13.90 (= 4.45 + 9.45) seconds.
% In RGB, original size 480x592.
% Resize to 480x491
signal = signal(:, 42:532, :);
% Show 7 seconds (298 units) at a time.
% imshow(signal(:, 1:298, :));
%% Video VideoWriter
% movie2avi deprecated in Matlab
% https://stackoverflow.com/a/11054155/54964
% https://stackoverflow.com/a/29952648/54964
%# figure
hFig = figure('Menubar','none', 'Color','white');
Z = peaks;
h = imshow(Z, [], 'InitialMagnification',1000, 'Border','tight');
colormap parula; axis tight manual off;
set(gca, 'nextplot','replacechildren', 'Visible','off');
% set(gcf,'Renderer','zbuffer'); % on some Windows
%# preallocate
N = 40; % 491;
vidObj = VideoWriter('myPeaks3.avi');
vidObj.Quality = 100;
vidObj.FrameRate = 10;
open(vidObj);
%# create movie
for k=1:N
set(h, 'CData', signal(:,k:k+40,:))
% drawnow
writeVideo(vidObj, getframe(gca));
end
%# save as AVI file
close(vidObj);
How can you substitute the drawing function by imshow or corresponding?
How can you store the animation correctly?
Here is some code to try:
%// plot
hFig = figure('Menubar','none', 'Color','white');
Z = peaks;
%h = surf(Z);
h = imshow(Z, [], 'InitialMagnification',1000, 'Border','tight');
colormap jet
axis tight manual off
%// preallocate movie structure
N = 40;
mov = struct('cdata',cell(1,N), 'colormap',cell(1,N));
%// aninmation
for k=1:N
%set(h, 'ZData',sin(2*pi*k/N)*Z)
set(h, 'CData',sin(2*pi*k/N)*Z)
drawnow
mov(k) = getframe(hFig);
end
close(hFig)
%// save AVI movie, and open video file
movie2avi(mov, 'file.avi', 'Compression','none', 'Fps',10);
winopen('file.avi')
Result (not really the video, just a GIF animation):
Depending on the codecs installed on your machine, you can apply video compression, e.g:
movie2avi(mov, 'file.avi', 'Compression','XVID', 'Quality',100, 'Fps',10);
(assuming you have the Xvid encoder installed).
EDIT:
Here is my implementation of the code you posted:
%%// extract blue ECG signal
%// retrieve picture: http://stackoverflow.com/q/29800089
imgRGB = imread('http://i.stack.imgur.com/cFOSp.png');
%// detect axis lines and labels
imgHSV = rgb2hsv(imgRGB);
BW = (imgHSV(:,:,3) < 1);
BW = imclose(imclose(BW, strel('line',40,0)), strel('line',10,90));
%// clear those masked pixels by setting them to background white color
imgRGB2 = imgRGB;
imgRGB2(repmat(BW,[1 1 3])) = 255;
%%// create sliding-window video
len = 40;
signal = imgRGB2(:,42:532,:);
figure('Menubar','none', 'NumberTitle','off', 'Color','k')
hImg = imshow(signal(:,1:1+len,:), ...
'InitialMagnification',100, 'Border','tight');
vid = VideoWriter('signal.avi');
vid.Quality = 100;
vid.FrameRate = 60;
open(vid);
N = size(signal,2);
for k=1:N-len
set(hImg, 'CData',signal(:,k:k+len,:))
writeVideo(vid, getframe());
end
close(vid);
The result look like this:

How to perform matching by MSER and HOG in Matlab

I wanted to know if there is any full implementation of image-matching by MSER and HOG in Matlab. Currently I am using VLFeat but found difficulties when performing the image matching. Any help?
Btw, I've tried the below code in VLFeat -Matlab environment but unfortunately the matching can't be performed.
%Matlab code
%
pfx = fullfile(vl_root,'figures','demo') ;
randn('state',0) ;
rand('state',0) ;
figure(1) ; clf ;
Ia = imread(fullfile(vl_root,'data','roofs1.jpg')) ;
Ib = imread(fullfile(vl_root,'data','roofs2.jpg')) ;
Ia = uint8(rgb2gray(Ia)) ;
Ib = uint8(rgb2gray(Ib)) ;
[ra,fa] = vl_mser(I,'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ;
[rb,fb] = vl_mser(I,'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ;
[matches, scores] = vl_ubcmatch(fa, fb);
figure(1) ; clf ;
imagesc(cat(2, Ia, Ib));
axis image off ;
vl_demo_print('mser_match_1', 1);
figure(2) ; clf ;
imagesc(cat(2, Ia, Ib));
xa = ra(1, matches(1,:));
xb = rb(1, matches(2,:)) + size(Ia,2);
ya = ra(2, matches(1,:));
yb = rb(2,matches(2,:));
hold on ;
h = line([xa ; xb], [ya ; yb]);
set(h, 'linewidth', 1, 'color', 'b');
vl_plotframe(ra(:,matches(1,:)));
rb(1,:) = fb(1,:) + size(Ia,2);
vl_plotframe(rb(:,mathces(2,:)));
axis image off ;
vl_demo_print('mser_match_2', 1);
%%%%%%
There are a couple problems. First, the code has several errors and doesn't run as-is. I've pasted my working version below.
More importantly, you're trying to use the SIFT feature-matching function to match the MSER ellipsoids. This won't work at all, since SIFT gives a very high dimensional feature vector based on local image gradients, and the MSER detector is just giving you a bounding ellipsoid.
VLFeat doesn't appear to include an MSER-matching function, so you'll probably have to write your own. Take a look at the original MSER paper to understand how they did matching:
"Robust wide-baseline stereo from maximally stable extremal regions", Matas et al. 2002
% Read the input images
Ia = imread(fullfile(vl_root,'data','roofs1.jpg')) ;
Ib = imread(fullfile(vl_root,'data','roofs2.jpg')) ;
% Convert to grayscale
Ia = uint8(rgb2gray(Ia)) ;
Ib = uint8(rgb2gray(Ib)) ;
% Find MSERs
[ra,fa] = vl_mser(Ia, 'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ;
[rb,fb] = vl_mser(Ib, 'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ;
% Match MSERs
[matches, scores] = vl_ubcmatch(fa, fb);
% Display the original input images
figure(1); clf;
imagesc(cat(2, Ia, Ib));
axis image off;
colormap gray;
% Display a second copy with the matches overlaid
figure(2) ; clf ;
imagesc(cat(2, Ia, Ib));
axis image off;
colormap gray;
xa = fa(1, matches(1,:));
ya = fa(2, matches(1,:));
xb = fb(1, matches(2,:)) + size(Ia,2);
yb = fb(2, matches(2,:));
hold on ;
h = line([xa ; xb], [ya ; yb]);
set(h, 'linewidth', 1, 'color', 'y');
I don't know how, but MSER matching works in Matlab itself.
The code below
file1 = 'roofs1.jpg';
file2 = 'roofs2.jpg';
I1 = imread(file1);
I2 = imread(file2);
I1 = rgb2gray(I1);
I2 = rgb2gray(I2);
% %Find the SURF features.
% points1 = detectSURFFeatures(I1);
% points2 = detectSURFFeatures(I2);
points1 = detectMSERFeatures(I1);
points2 = detectMSERFeatures(I2);
%Extract the features.
[f1, vpts1] = extractFeatures(I1, points1);
[f2, vpts2] = extractFeatures(I2, points2);
%Retrieve the locations of matched points. The SURF featurevectors are already normalized.
indexPairs = matchFeatures(f1, f2, 'Prenormalized', true) ;
matched_pts1 = vpts1(indexPairs(:, 1));
matched_pts2 = vpts2(indexPairs(:, 2));
figure; showMatchedFeatures(I1,I2,matched_pts1,matched_pts2,'montage');
legend('matched points 1','matched points 2');
gives the following picture

Matlab for loop animations

I'm trying to animate the graph of a function but I cant get the program to graph the correct points. I want to plot points between time 0 and 10 and animate this graph. How do I get the plot as a function of time?
clear;
w = 2*pi;
t = 0:.01:10;
y = sin(w*t);
x = cos(w*t);
for j=1:10
plot(x(6*j),y(6*j),'*');
axis square;
grid on;
F(j) = getframe;
end
movie(F,1,1);
I refined the code to:
clear;
w = 2*pi;
for j=2:11
t=j-1;
y = sin(w*t);
x = cos(w*t);
plot(x(t),y(t),'*');
axis square;
grid on;
F(j) = getframe;
end
movie(F);
This should do what I'm trying however now I'm getting an "Index exceeds matrix dimension." How can I solve this?
Here is an example that show an animated point along a circular path, while recording an AVI movie.
To learn more about doing animations and recording movies in MATLAB, check out this guide.
%# some parameters
ERASEMODE = 'normal'; %# normal,xor
RENDERER = 'painters'; %# painters,zbuffer,opengl
%# data
t = linspace(0,2*pi,100)'; %'# adjust number of points here
D = [cos(t) -sin(t)];
%# plot circluar path
figure('DoubleBuffer','on', 'Renderer',RENDERER)
plot(D(:,1), D(:,2), 'Color','b', 'LineWidth',2)
grid on, axis([-1.5 1.5 -1.5 1.5]), axis square
xlabel('x'), ylabel('y'), title('Circle Animation')
%#set(gca, 'DrawMode','fast')
%# moving point
hPoint = line('XData',D(1,1), 'YData',D(1,2), 'EraseMode',ERASEMODE, ...
'Color','r', 'Marker','.', 'MarkerSize',30);
%# moving coordinates text
hTxtCoords = text(D(1,1), D(1,2), sprintf('(%.2f,%.2f)',D(1,:)), ...
'Color',[0.2 0.2 0.2], 'FontSize',8, 'EraseMode',ERASEMODE, ...
'HorizontalAlignment','left', 'VerticalAlignment','top');
%# angle text
hTxtAngle = text(0, 0, num2str(t(1),'%.02f'), ...
'FontSize',15, 'EraseMode',ERASEMODE, ...
'HorizontalAlignment','center', 'VerticalAlignment','middle');
%# prepare video output
useVideoWriter = ~verLessThan('matlab','7.11');
if useVideoWriter
vid = VideoWriter('vid.avi');
vidObj.Quality = 100;
vid.FrameRate = 30;
open(vid);
else
vid = avifile('vid.avi', 'fps',30, 'quality',100);
end
%# loop
for i=1:numel(t)
set(hPoint, 'XData',D(i,1), 'YData',D(i,2)) %# update point location
set(hTxtAngle, 'String',num2str(t(i),'%.02f')) %# update angle text
set(hTxtCoords, 'Position',D(i,:), ... %# update angle text
'String',sprintf('(%.3f,%.3f)',D(i,:)))
drawnow %# force refresh
if ~ishandle(hPoint), break; end %# if you close the figure
%# capture frame
if useVideoWriter
writeVideo(vid,getframe);
else
vid = addframe(vid, getframe(gcf));
end
end
%# close and save video output
if useVideoWriter
close(vid);
else
vid = close(vid);
end
%# open AVI file using system default player
winopen('vid.avi')
It's doing exactly what you ask it to do. You're subsampling the x and y, so it looks kind of funny.
Try
plot(x,y);
axis square;
ax = axis;
for jx = 1 : length(t),
plot(x(ix), y(ix), '*');
axis(ax); grid on;
F(ix) = getframe;
end
movie(F, 1, 1/(t(2)-t(1)))
I would also use t = 1 : 0.1 : 10; so that it plots at 10 FPS instead of 100. Slowing the frequency down to, say, w = pi; will be smoother as well.
At the end of the day, Matlab is just not a great animation solution.
Answer to refined code question
You'd need to use plot(x,y);, but this will reveal another error - your frame index does not start at 1. It will choke on F(j) in the first iteration, where j = 2. Why not just loop over t = 1 : 10?