moving shadow elimination in video - matlab

Question

I wish to remove moving shadow in video. I followed the steps as mentioned in this article, but getting the same result before and after applying threshold operation on the swt output. didn't get the expected output...Can anyone suggest me what I am doing wrong?
Steps to do for shadow removal:
(i) Read the video.
(ii) After the color conversion split the h s v components
(iii) applying the stationary wavelet transform on on s and v components of frame
(iv) calculate skew value for respective swt output of s and v component
(v) Assign the value 1 and 0 to 's and v' pixel if swt of v is greater than skewness value of v likewise for s too.
(vi) Do inverse swt over the s and v
(vii)Combine the h s and v
clc; % Clear the command window.
close all; % Close all figures (except those of imtool.)
imtool close all; % Close all imtool figures.
clear; % Erase all existing variables.
workspace; % Make sure the workspace panel is showing.
fontSize = 22;
movieFullFileName = fullfile('cam4v.mp4');
% Check to see that it exists.
if ~exist(movieFullFileName, 'file')
strErrorMessage = sprintf('File not found:\n%s\nYou can choose a new one,
or cancel', movieFullFileName);
response = questdlg(strErrorMessage, 'File not found', 'OK - choose a new
movie.', 'Cancel', 'OK - choose a new movie.');
if strcmpi(response, 'OK - choose a new movie.')
[baseFileName, folderName, FilterIndex] = uigetfile('*.avi');
if ~isequal(baseFileName, 0)
movieFullFileName = fullfile(folderName, baseFileName);
else
return;
end
else
return;
end
end
try
videoObject = VideoReader(movieFullFileName);
% Determine how many frames there are.
numberOfFrames = videoObject.NumberOfFrames;
vidHeight = videoObject.Height;
vidWidth = videoObject.Width;
numberOfFramesWritten = 0;
% Prepare a figure to show the images in the upper half of the screen.
figure;
% screenSize = get(0, 'ScreenSize');
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Ask user if they want to write the individual frames out to disk.
promptMessage = sprintf('Do you want to save the individual frames out to
individual disk files?');
button = questdlg(promptMessage, 'Save individual frames?', 'Yes', 'No',
'Yes');
if strcmp(button, 'Yes')
writeToDisk = true;
% Extract out the various parts of the filename.
[folder, baseFileName, extentions] = fileparts(movieFullFileName);
% Make up a special new output subfolder for all the separate
% movie frames that we're going to extract and save to disk.
% (Don't worry - windows can handle forward slashes in the folder
name.)
folder = pwd; % Make it a subfolder of the folder where this m-file
lives.
outputFolder = sprintf('%s/Movie Frames from %s', folder,
baseFileName);
% Create the folder if it doesn't exist already.
if ~exist(outputFolder, 'dir')
mkdir(outputFolder);
end
else
writeToDisk = false;
end
% Loop through the movie, writing all frames out.
% Each frame will be in a separate file with unique name.
for frame = 1 : numberOfFrames
% Extract the frame from the movie structure.
thisFrame = read(videoObject, frame);
% Display it
hImage = subplot(2, 2, 1);
image(thisFrame);
caption = sprintf('Frame %4d of %d.', frame, numberOfFrames);
title(caption, 'FontSize', fontSize);
drawnow; % Force it to refresh the window.
% Write the image array to the output file, if requested.
if writeToDisk
% Construct an output image file name.
outputBaseFileName = sprintf('Frame %4.4d.png', frame);
outputFullFileName = fullfile(outputFolder, outputBaseFileName);
% Stamp the name and frame number onto the image.
% At this point it's just going into the overlay,not actually
getting written into the pixel values.
text(5, 15, outputBaseFileName, 'FontSize', 20);
% Extract the image with the text "burned into" it.
frameWithText = getframe(gca);
% frameWithText.cdata is the image with the text
% actually written into the pixel values.
% Write it out to disk.
imwrite(frameWithText.cdata, outputFullFileName, 'png');
end
if frame == 1
xlabel('Frame Number');
ylabel('Gray Level');
% Get size data later for preallocation if we read
% the movie back in from disk.
[rows, columns, numberOfColorChannels] = size(thisFrame);
end
% Update user with the progress. Display in the command window.
if writeToDisk
progressIndication = sprintf('Wrote frame %4d of %d.', frame,
numberOfFrames);
else
progressIndication = sprintf('Processed frame %4d of %d.', frame,
numberOfFrames);
end
disp(progressIndication);
% Increment frame count (should eventually = numberOfFrames
% unless an error happens).
numberOfFramesWritten = numberOfFramesWritten + 1;
% Now let's do the differencing
alpha = 0.5;
if frame == 1
Background = thisFrame;
else
% Change background slightly at each frame
% Background(t+1)=(1-alpha)*I+alpha*Background
Background = (1-alpha)* thisFrame + alpha * Background;
end
% Display the changing/adapting background.
subplot(2, 2, 3);
imshow(Background);
title('Adaptive Background', 'FontSize', fontSize);
% Do color conversion from rgb to hsv
x=rgb2hsv(thisFrame);
y=rgb2hsv(Background);
% Split the hsv component to h,s,v value
Hx = x(:,:,1); Hy = y(:,:,1);
Sx = x(:,:,2); Sy = y(:,:,2);
Vx = x(:,:,3); Vy = y(:,:,3);
%Find the absolute diffrence between h s v value of current and
previous frame
dh=(abs(double(Hx) - double(Hy)));
ds1=(abs(double(Sx) - double(Sy)));
dv1=(abs(double(Vx) - double(Vy)));
%Perform the swt2 transformation on difference of s and v value
[as,hs,vs,ds] = swt2(ds1,1,'haar');
[av,hv,vv,dv] = swt2(dv1,1,'haar');
%Compute the skewness value of 'swt of v'
sav=skewness(av(:));
shv=skewness(hv(:));
svv=skewness(vv(:));
sdv=skewness(dv(:));
%Compute the skewness value of 'swt of s'
sas=skewness(as(:));
shs=skewness(hs(:));
svs=skewness(vs(:));
sds=skewness(ds(:));
% Do the shadow detection based on the output of swt and skew of 'v'
value
% Compare swt v output with its skew value if the av >=sav then av is
assigned to one else it becomes zero.This operation continues till variable i
b=(av>=sav); f= (as>=sas);
c=(hv>=shv); g=(hs>=shs);
d=(vv>=svv); h=(vs>=svs);
e=(dv>=sdv); i=(ds>=sds);
% Remove the shadows based on 'and operation
j=(b&f); l=(d&h);
k=(c&g); m=(e&i);
% Do inverse swt operation
recv = iswt2(b,c,d,e,'haar');
recs= iswt2(j,k,l,m,'haar');
%Combine the value of h,s and v
de_shadow=cat(3,dh,recs,recv);
rgb=hsv2rgb(de_shadow);
% Plot the image.
subplot(2, 2, 4);
imshow(rgb);
title('output Image', 'FontSize', fontSize);
end
% Alert user that we're done.
if writeToDisk
finishedMessage = sprintf('Done! It wrote %d frames to
folder\n"%s"', numberOfFramesWritten, outputFolder);
else
finishedMessage = sprintf('Done! It processed %d frames of\n"%s"',
numberOfFramesWritten, movieFullFileName);
end
disp(finishedMessage); % Write to command window.
uiwait(msgbox(finishedMessage)); % Also pop up a message box.
% Exit if they didn't write any individual frames out to disk.
if ~writeToDisk
return;
end
% Ask user if they want to read the individual frames from the disk,
% that they just wrote out, back into a movie and display it.
promptMessage = sprintf('Do you want to recall the individualframes\nback
from disk into a movie?\n(This will take several seconds.)');
button = questdlg(promptMessage, 'Recall Movie?', 'Yes', 'No', 'Yes');
if strcmp(button, 'No')
return;
end
% Create a VideoWriter object to write the video out to a new, different
file.
writerObj = VideoWriter('Newcam4v.mp4');
open(writerObj);
% Read the frames back in from disk, and convert them to a movie.
% Preallocate recalledMovie, which will be an array of structures.
% First get a cell array with all the frames.
allTheFrames = cell(numberOfFrames,1);
allTheFrames(:) = {zeros(vidHeight, vidWidth, 3, 'uint8')};
% Next get a cell array with all the colormaps.
allTheColorMaps = cell(numberOfFrames,1);
allTheColorMaps(:) = {zeros(256, 3)};
% Now combine these to make the array of structures.
recalledMovie = struct('cdata', allTheFrames, 'colormap',allTheColorMaps)
for frame = 1 : numberOfFrames
% Construct an output image file name.
outputBaseFileName = sprintf('Frame %4.4d.png', frame);
outputFullFileName = fullfile(outputFolder, outputBaseFileName);
% Read the image in from disk.
thisFrame = imread(outputFullFileName);
% Convert the image into a "movie frame" structure.
thisFrame=imresize(thisFrame,[452, 231]);
recalledMovie(frame) = im2frame(thisFrame);
% Write this frame out to a new video file.
writeVideo(writerObj, thisFrame);
end
close(writerObj);
% Get rid of old image and plot.
delete(hImage);
% Create new axes for our movie.
subplot(1, 3, 2);
axis off; % Turn off axes numbers.
title('Movie recalled from disk', 'FontSize', fontSize);
% Play the movie in the axes.
movie(recalledMovie);
% Note: if you want to display graphics or text in the overlay
% as the movie plays back then you need to do it like I did at first
% (at the top of this file where you extract and imshow a frame at a
time.)
msgbox('Done this experiment!');
catch ME
% Some error happened if you get here.
strErrorMessage = sprintf('Error extracting movie frames
from:\n\n%s\n\nError: %s\n\n)', movieFullFileName, ME.message);
uiwait(msgbox(strErrorMessage));
end