moving shadow removal in MATLAB - matlab

I am trying to remove moving shadows in video using the 'stationary wavelet transform technique' as mentioned in the reference paper. I coded in MATLAB but not getting expected output. I am begineer in MATLAB so Can anyone please review my code to check whether it followed all the steps as mentioned in the paper.
Recap of the algorithm followed in the reference paper:
(i) Convert the rgb video frame to hsv
(ii)split the hsv into individual component
(iii)Find the absolute difference between h,s and v component of current and background frame
(iv)Apply the SWT transformation on difference 's' and 'v'
(V)Compute the skewness value for the swt output
(vi)For shadow detection: If the swt output of 'v' greater than its skewness, assign those pixel value to '1' else'0'
(vii) For shadow removal: thresholding operation is applied based on swt output of's' reference paper
function shadowremoval()
obj = setupSystemObjects();
while ~isDone(obj.reader)
frame = readFrame();
mask1 = shadow(frame);
displayTrackingResults();
end
%% Create System Objects
function obj = setupSystemObjects()
% Create a video file reader.
obj.reader = vision.VideoFileReader('visiontraffic.avi');
% Create two video players, one to display the video,
% and one to display the foreground mask.
obj.videoPlayer = vision.VideoPlayer('Position', [10, 250, 700, 400]);
obj.maskPlayer = vision.VideoPlayer('Position', [720, 250, 700, 400]);
obj.detector = vision.ForegroundDetector('NumGaussians', 3, ...
'NumTrainingFrames', 40, 'MinimumBackgroundRatio', 0.7);
end
%% Read a Video Frame
% Read the next video frame from the video file.
function frame = readFrame()
frame = obj.reader.step();
end
%% Perform the operation to remove shadows
function mask1 = shadow(frame)
Background=0.0;
% Detect foreground.
mask1 = obj.detector.step(frame);
mask1 = uint8(repmat(mask1, [1, 1, 3])) .* 255;
% Apply morphological operations to remove noise and fill in holes.
% mask1 = imerode(mask1, strel('rectangle', [3,3]));
% mask1 = imclose(mask1, strel('rectangle', [15, 15]));
mask1 = imopen(mask1, strel('rectangle', [15,15]));
mask1 = imfill(mask1, 'holes');
% Now let's do the differencing
alpha = 0.5;
if frame == 1
Background = frame;
else
% Change background slightly at each frame
% Background(t+1)=(1-alpha)*I+alpha*Background
Background = (1-alpha)* frame + alpha * Background;
end
% Do color conversion from rgb to hsv
x=rgb2hsv(mask1);
y=rgb2hsv(Background);
% Split the hsv component to h,s,v value
Hx = x(:,:,1);
Sx = x(:,:,2);
Vx = x(:,:,3);
Hy = y(:,:,1);
Sy = y(:,:,2);
Vy = y(:,:,3);
% Calculate a difference between this frame and the background.
dh=(abs(double(Hx) - double(Hy)));
ds1=(abs(double(Sx) - double(Sy)));
dv1=(abs(double(Vx) - double(Vy)));
% Perform the 'swt'
[as,hs,vs,ds] = swt2(ds1,1,'haar');
[av,hv,vv,dv] = swt2(dv1,1,'haar');
%Compute the skewness value of 'swt of v'
sav=skewness(av(:));
shv=skewness(hv(:));
svv=skewness(vv(:));
sdv=skewness(dv(:));
%Compute the skewness value of 'swt of s'
sas=skewness(as(:));
shs=skewness(hs(:));
svs=skewness(vs(:));
sds=skewness(ds(:));
%Perform the thresholding operation
b=(av>=sav);
c=(hv>=shv);
d=(vv>=svv);
e=(dv>=sdv);
f=(as>=sas);
g=(hs>=shs);
h=(vs>=svs);
i=(ds>=sds);
j=(b&f);
k=(c&g);
l=(d&h);
m=(e&i);
%Perform the inverse 'swt'operation
recv = iswt2(b,c,d,e,'haar');
recs= iswt2(j,k,l,m,'haar');
de_shadow=cat(3,dh,recs,recv);
mask1=hsv2rgb(de_shadow);
mask1=rgb2gray(mask1);
end
function displayTrackingResults()
% Convert the frame and the mask to uint8 RGB.
frame = im2uint8(frame);
mask1 = uint8(repmat(mask1, [1, 1, 3])) .* 255;
% Display the mask and the frame.
obj.maskPlayer.step(mask1);
obj.videoPlayer.step(frame);
end
end

Related

Video Stabilization Using Point Feature Matching WITHOUT LOSING RGB COLORS on frames on MATLAB

I'd like to stabilize a 13 min video captured by a quadcopter over a traffic crossroads without losing its 3 color channels (RGB). Matlab's own function leads to a gray scale video which is an unwanted case for the main and future objective, vehicle tracking. New thoughts are appreciated.
Below you can find my own code (works and converts the video to gray scale) edited over the Matlab's own script written on the following page:
Matlab's related Webpage : Video Stabilization Using Point Feature Matching
clc; clear all; close all;
filename = 'Quad_video_erst';
hVideoSrc = vision.VideoFileReader('Quad_video_erst.mp4', 'ImageColorSpace', 'Intensity');
% Create and open video file
myVideo = VideoWriter('vivi.avi');
open(myVideo);
hVPlayer = vision.VideoPlayer;
%% Step 1: Read Frames from a Movie File
for i=1:10 % testing for a short run
imgA = step(hVideoSrc); % Read first frame into imgA
imgB = step(hVideoSrc); % Read second frame into imgB
%% Step 2: SURF DETECTION
pointsA=surf_function_CAN(imgA);
pointsB=surf_function_CAN(imgB);
%% Step 3. Select Correspondences Between Points
% Extract FREAK descriptors for the corners
[featuresA, pointsA] = extractFeatures(imgA, pointsA);
[featuresB, pointsB] = extractFeatures(imgB, pointsB);
indexPairs = matchFeatures(featuresA, featuresB);
pointsA = pointsA(indexPairs(:, 1), :);
pointsB = pointsB(indexPairs(:, 2), :);
%% Step 4: Estimating Transform from Noisy Correspondences
[tform, pointsBm, pointsAm] = estimateGeometricTransform(...
pointsB, pointsA, 'affine');
imgBp = imwarp(imgB, tform, 'OutputView', imref2d(size(imgB)));
pointsBmp = transformPointsForward(tform, pointsBm.Location);
%% Step 5: Step 5. Transform Approximation and Smoothing
% Extract scale and rotation part sub-matrix.
H = tform.T;
R = H(1:2,1:2);
% Compute theta from mean of two possible arctangents
theta = mean([atan2(R(2),R(1)) atan2(-R(3),R(4))]);
% Compute scale from mean of two stable mean calculations
scale = mean(R([1 4])/cos(theta));
% Translation remains the same:
translation = H(3, 1:2);
% Reconstitute new s-R-t transform:
HsRt = [[scale*[cos(theta) -sin(theta); sin(theta) cos(theta)];...
translation], [0 0 1]'];
tformsRT = affine2d(HsRt);
imgBold = imwarp(imgB, tform, 'OutputView', imref2d(size(imgB)));
imgBsRt = imwarp(imgB, tformsRT, 'OutputView', imref2d(size(imgB)));
%% Write the Video
writeVideo(myVideo,imfuse(imgBold,imgBsRt,'ColorChannels','red-cyan'));
end
And the function:
function [ surf_points ] = surf_function_CAN(img)
surfpoints_raw= detectSURFFeatures(img);
[featuresOriginal, validPtsOriginal] = extractFeatures(img, surfpoints_raw);
strongestPoints = validPtsOriginal.selectStrongest(1600);
array=strongestPoints.Location;
% New - Get X and Y coordinates
X = array(:,1);
Y = array(:,2);
% New - Determine a mask to grab the points we want
ind = (((X>156-9-70 & X<156+9+70) & (Y>406-9-70 & Y<406+9+70)) | ...
((X>684-11-70 & X<684+11+70) & (Y>274-11-70 & Y<274+11+70)) | ...
((X>1066-15-70 & X<1066+15+70) & (Y>67-15-70 & Y<67+15+70)) | ...
((X>1559-15-70 & X<1559+15+70) & (Y>867-15-70 & Y<867+15+70)) | ...
((X>1082-18-70 & X<1082+18+70) & (Y>740-18-100 & Y<740+18+100))) ;
% New - Create new SURFPoints structure that contains all information
% from the points we need
array_filtered =strongestPoints(ind);
surf_points= array_filtered;
end
Firstly, if you look through their example you should use the part where they perform the loop, not the part where they show how to implement it between 2 frames as they are not exactly compatible. Other than that the only thing you need to do is perform the analysis on the a grayscale image, but implement the transformation on the color image:
%% Load Video and Open Save File
filename = 'shaky_car.avi';
hVideoSrc = vision.VideoFileReader(filename);
myVideo = VideoWriter('vivi.avi');
open(myVideo);
% Get next Image
colorImg = step(hVideoSrc);
% Try to Convert to Grayscale
try
imgB = rgb2gray(colorImg);
RGB = true;
catch % Image is not RGB
imgB = colorImg;
RGB = false;
end
Hcumulative = eye(3);
ptThresh = 0.1;
% Loop Through Video
while ~isDone(hVideoSrc)
imgA = imgB;
% Get Next Image
colorImg = step(hVideoSrc);
% Convert to Grayscale
if RGB
imgB = rgb2gray(colorImg);
else
imgB = colorImg;
end
%% Calculate Transformation
% Generate Prospective Points
pointsA = detectFASTFeatures(imgA, 'MinContrast', ptThresh);
pointsB = detectFASTFeatures(imgB, 'MinContrast', ptThresh);
% Extract Features for the Corners
[featuresA, pointsA] = extractFeatures(imgA, pointsA);
[featuresB, pointsB] = extractFeatures(imgB, pointsB);
indexPairs = matchFeatures(featuresA, featuresB);
pointsA = pointsA(indexPairs(:, 1), :);
pointsB = pointsB(indexPairs(:, 2), :);
[tform] = estimateGeometricTransform(pointsB, pointsA, 'affine');
% Extract Rotation & Translations
H = tform.T;
R = H(1:2,1:2);
theta = mean([atan2(R(2),R(1)) atan2(-R(3),R(4))]);
scale = mean(R([1 4])/cos(theta));
translation = H(3, 1:2);
% Reconstitute Trnasform
HsRt = [[scale*[cos(theta) -sin(theta); sin(theta) cos(theta)]; ...
translation], [0 0 1]'];
Hcumulative = HsRt*Hcumulative;
% Perform Transformation on Color Image
img = imwarp(colorImg, affine2d(Hcumulative),'OutputView',imref2d(size(imgB)));
% Save Transformed Color Image to Video File
writeVideo(myVideo,img)
end
close(myVideo)

Can anyone suggest how do I obtain all the values of the variable bc?

The variable bc is getting overwritten and I am Unable to plot all the values of the variable from the start.
I tried exporting the variable to a csv file but that didn't work.
I'm trying to detect a red, green, blue object and plot its coordinates versus time in matlab.
a = imaqhwinfo;
[camera_name, camera_id, format] = getCameraInfo(a);
% Capture the video frames using the videoinput function
% You have to replace the resolution & your installed adaptor name.
vid = videoinput(camera_name, camera_id, format);
% Set the properties of the video object
set(vid, 'FramesPerTrigger', Inf);
set(vid, 'ReturnedColorspace', 'rgb')
vid.FrameGrabInterval = 5;
%start the video aquisition here
start(vid)
% Set a loop that stop after 100 frames of aquisition
while(vid.FramesAcquired<=100)
% Get the snapshot of the current frame
data = getsnapshot(vid);
% Now to track red objects in real time
% we have to subtract the red component
% from the grayscale image to extract the red components in the image.
diff_im = imsubtract(data(:,:,1), rgb2gray(data));
%Use a median filter to filter out noise
diff_im = medfilt2(diff_im, [3 3]);
% Convert the resulting grayscale image into a binary image.
diff_im = im2bw(diff_im,0.18);
% Remove all those pixels less than 300px
diff_im = bwareaopen(diff_im,300);
% Label all the connected components in the image.
bw = bwlabel(diff_im, 8);
% Here we do the image blob analysis.
% We get a set of properties for each labeled region.
stats = regionprops(bw, 'BoundingBox', 'Centroid');
% Display the image
imshow(data)
hold on
%
% %This is a loop to bound the red objects in a rectangular box.
for object = 1:length(stats)
bb = stats(object).BoundingBox;
bc = stats(object).Centroid;
rectangle('Position',bb,'EdgeColor','r','LineWidth',2)
plot(bc(1),bc(2), '-m+')
x = bc(1);
y = bc(2);
csvwrite('bcx.dat', bc(1));
csvwrite('bcy.dat', bc(2));
a=text(bc(1)+15,bc(2), strcat('X: ', num2str(round(bc(1))), ' Y: ', num2str(round(bc(2)))));
set(a, 'FontName', 'Arial', 'FontWeight', 'bold', 'FontSize', 12, 'Color', 'yellow');
end
hold off
end
% Both the loops end here.
% Stop the video aquisition.
stop(vid);
% Flush all the image data stored in the memory buffer.
flushdata(vid);
% Clear all variables
clear all
%sprintf('%s','That was all about Image tracking, Guess that was pretty easy :) ')

I want to make panorama image but it is showing the error message Undefined function 'imageSet' for input arguments of type 'char'

Undefined function 'imageSet' for input arguments of type 'char'.
Error in build (line 3)
buildingScene = imageSet(buildingDir);
% Load images.
buildingDir = fullfile(toolboxdir('vision'), 'visiondata', 'building');
buildingScene = imageSet(buildingDir);
% Display images to be stitched
montage(buildingScene.ImageLocation)
% Read the first image from the image set.
I = read(buildingScene, 1);
% Initialize features for I(1)
grayImage = rgb2gray(I);
points = detectSURFFeatures(grayImage);
[features, points] = extractFeatures(grayImage, points);
% Initialize all the transforms to the identity matrix. Note that the
% projective transform is used here because the building images are fairly
% close to the camera. Had the scene been captured from a further distance,
% an affine transform would suffice.
tforms(buildingScene.Count) = projective2d(eye(3));
% Iterate over remaining image pairs
for n = 2:buildingScene.Count
% Store points and features for I(n-1).
pointsPrevious = points;
featuresPrevious = features;
% Read I(n).
I = read(buildingScene, n);
% Detect and extract SURF features for I(n).
grayImage = rgb2gray(I);
points = detectSURFFeatures(grayImage);
[features, points] = extractFeatures(grayImage, points);
% Find correspondences between I(n) and I(n-1).
indexPairs = matchFeatures(features, featuresPrevious, 'Unique', true);
matchedPoints = points(indexPairs(:,1), :);
matchedPointsPrev = pointsPrevious(indexPairs(:,2), :);
% Estimate the transformation between I(n) and I(n-1).
tforms(n) = estimateGeometricTransform(matchedPoints, matchedPointsPrev,...
'projective', 'Confidence', 99.9, 'MaxNumTrials', 2000);
% Compute T(1) * ... * T(n-1) * T(n)
tforms(n).T = tforms(n-1).T * tforms(n).T;
end
avgXLim = mean(xlim, 2);
[~, idx] = sort(avgXLim);
centerIdx = floor((numel(tforms)+1)/2);
centerImageIdx = idx(centerIdx);
Tinv = invert(tforms(centerImageIdx));
for i = 1:numel(tforms)
tforms(i).T = Tinv.T * tforms(i).T;
end
for i = 1:numel(tforms)
[xlim(i,:), ylim(i,:)] = outputLimits(tforms(i), [1 imageSize(2)], [1 imageSize(1)]);
end
% Find the minimum and maximum output limits
xMin = min([1; xlim(:)]);
xMax = max([imageSize(2); xlim(:)]);
yMin = min([1; ylim(:)]);
yMax = max([imageSize(1); ylim(:)]);
% Width and height of panorama.
width = round(xMax - xMin);
height = round(yMax - yMin);
% Initialize the "empty" panorama.
panorama = zeros([height width 3], 'like', I);
Step 4 - Create the Panorama
Use imwarp to map images into the panorama and use vision.AlphaBlender to overlay the images together.
blender = vision.AlphaBlender('Operation', 'Binary mask', ...
'MaskSource', 'Input port');
% Create a 2-D spatial reference object defining the size of the panorama.
xLimits = [xMin xMax];
yLimits = [yMin yMax];
panoramaView = imref2d([height width], xLimits, yLimits);
% Create the panorama.
for i = 1:buildingScene.Count
I = read(buildingScene, i);
% Transform I into the panorama.
warpedImage = imwarp(I, tforms(i), 'OutputView', panoramaView);
% Create an mask for the overlay operation.
warpedMask = imwarp(ones(size(I(:,:,1))), tforms(i), 'OutputView', panoramaView);
% Clean up edge artifacts in the mask and convert to a binary image.
warpedMask = warpedMask >= 1;
% Overlay the warpedImage onto the panorama.
panorama = step(blender, panorama, warpedImage, warpedMask);
end
figure
imshow(panorama)
imageSet requires the Computer Vision Toolbox from MATLAB R2014b or higher. See the release notes from the Computer Vision Toolbox here: http://www.mathworks.com/help/vision/release-notes.html#R2014b
If you have R2014a or lower, imageSet does not come with your distribution. The only option you have is to upgrade your MATLAB distribution. Sorry if this isn't what you wanted to hear!

Color and Texture-Based Shadow Detection

I'm doing a project on tracking and Object classification in video surveillance,and i have some difficulties with removing the shadows from the foreground objects.
The problem is that i can't just use any method that i want, i have to use the same method described in this article Shadow removal with blob based morphological reconstruction .
In the article they use a hybrid shadow removal method- RGB color based detection and texture based detection. Both the colour and texture based procedures are used in parallel, followed by an assertion process that combines the results of the two.
So i would like to get some help with the shadow removal Matlab code.
my code so far
function Tracking_Objects()
% Create System objects used for reading video, detecting moving objects,
% and displaying the results.
obj = setupSystemObjects();
tracks = initializeTracks(); % Create an empty array of tracks.
nextId = 1; % ID of the next track
% Detect moving objects, and track them across video frames.
while ~isDone(obj.reader)
frame = readFrame();
[centroids, bboxes, mask] = detectObjects(frame);
predictNewLocationsOfTracks();
[assignments, unassignedTracks, unassignedDetections] = ...
detectionToTrackAssignment();
updateAssignedTracks();
updateUnassignedTracks();
deleteLostTracks();
createNewTracks();
displayTrackingResults();
end
%% Create System Objects
function obj = setupSystemObjects()
% Initialize Video I/O
% Create objects for reading a video from a file, drawing the tracked
% objects in each frame, and playing the video.
% Create a video file reader.
obj.reader = vision.VideoFileReader('input.avi');
% Create two video players, one to display the video,
% and one to display the foreground mask.
obj.videoPlayer = vision.VideoPlayer('Position', [10, 250, 700, 400]);
obj.maskPlayer = vision.VideoPlayer('Position', [720, 250, 700, 400]);
obj.detector = vision.ForegroundDetector('NumGaussians', 3, ...
'NumTrainingFrames', 40, 'MinimumBackgroundRatio', 0.7);
obj.blobAnalyser = vision.BlobAnalysis('BoundingBoxOutputPort', true, ...
'AreaOutputPort', true, 'CentroidOutputPort', true, ...
'MinimumBlobArea', 75);
end
%% Initialize Tracks
function tracks = initializeTracks()
% create an empty array of tracks
tracks = struct(...
'id', {}, ...
'bbox', {}, ...
'kalmanFilter', {}, ...
'age', {}, ...
'totalVisibleCount', {}, ...
'consecutiveInvisibleCount', {});
end
%% Read a Video Frame
% Read the next video frame from the video file.
function frame = readFrame()
frame = obj.reader.step();
end
%% Detect Objects
function [centroids, bboxes, mask] = detectObjects(frame)
% Detect foreground.
mask = obj.detector.step(frame);
% Apply morphological operations to remove noise and fill in holes.
mask = imopen(mask, strel('rectangle', [3,3]));
mask = imclose(mask, strel('rectangle', [15, 15]));
mask = imfill(mask, 'holes');
% Perform blob analysis to find connected components.
[~, centroids, bboxes] = obj.blobAnalyser.step(mask);
end
%% Predict New Locations of Existing Tracks
function predictNewLocationsOfTracks()
for i = 1:length(tracks)
bbox = tracks(i).bbox;
% Predict the current location of the track.
predictedCentroid = predict(tracks(i).kalmanFilter);
% Shift the bounding box so that its center is at
% the predicted location.
predictedCentroid = int32(predictedCentroid) - bbox(3:4) / 2;
tracks(i).bbox = [predictedCentroid, bbox(3:4)];
end
end
%% Assign Detections to Tracks
function [assignments, unassignedTracks, unassignedDetections] = ...
detectionToTrackAssignment()
nTracks = length(tracks);
nDetections = size(centroids, 1);
% Compute the cost of assigning each detection to each track.
cost = zeros(nTracks, nDetections);
for i = 1:nTracks
cost(i, :) = distance(tracks(i).kalmanFilter, centroids);
end
% Solve the assignment problem.
costOfNonAssignment = 20;
[assignments, unassignedTracks, unassignedDetections] = ...
assignDetectionsToTracks(cost, costOfNonAssignment);
end
%% Update Assigned Tracks
function updateAssignedTracks()
numAssignedTracks = size(assignments, 1);
for i = 1:numAssignedTracks
trackIdx = assignments(i, 1);
detectionIdx = assignments(i, 2);
centroid = centroids(detectionIdx, :);
bbox = bboxes(detectionIdx, :);
% Correct the estimate of the object's location
% using the new detection.
correct(tracks(trackIdx).kalmanFilter, centroid);
% Replace predicted bounding box with detected
% bounding box.
tracks(trackIdx).bbox = bbox;
% Update track's age.
tracks(trackIdx).age = tracks(trackIdx).age + 1;
% Update visibility.
tracks(trackIdx).totalVisibleCount = ...
tracks(trackIdx).totalVisibleCount + 1;
tracks(trackIdx).consecutiveInvisibleCount = 0;
end
end
%% Update Unassigned Tracks
% Mark each unassigned track as invisible, and increase its age by 1.
function updateUnassignedTracks()
for i = 1:length(unassignedTracks)
ind = unassignedTracks(i);
tracks(ind).age = tracks(ind).age + 1;
tracks(ind).consecutiveInvisibleCount = ...
tracks(ind).consecutiveInvisibleCount + 1;
end
end
%% Delete Lost Tracks
function deleteLostTracks()
if isempty(tracks)
return;
end
invisibleForTooLong = 20;
ageThreshold = 8;
% Compute the fraction of the track's age for which it was visible.
ages = [tracks(:).age];
totalVisibleCounts = [tracks(:).totalVisibleCount];
visibility = totalVisibleCounts ./ ages;
% Find the indices of 'lost' tracks.
lostInds = (ages < ageThreshold & visibility < 0.6) | ...
[tracks(:).consecutiveInvisibleCount] >= invisibleForTooLong;
% Delete lost tracks.
tracks = tracks(~lostInds);
end
%% Create New Tracks
function createNewTracks()
centroids = centroids(unassignedDetections, :);
bboxes = bboxes(unassignedDetections, :);
for i = 1:size(centroids, 1)
centroid = centroids(i,:);
bbox = bboxes(i, :);
% Create a Kalman filter object.
kalmanFilter = configureKalmanFilter('ConstantVelocity', ...
centroid, [200, 50], [100, 25], 100);
% Create a new track.
newTrack = struct(...
'id', nextId, ...
'bbox', bbox, ...
'kalmanFilter', kalmanFilter, ...
'age', 1, ...
'totalVisibleCount', 1, ...
'consecutiveInvisibleCount', 0);
% Add it to the array of tracks.
tracks(end + 1) = newTrack;
% Increment the next id.
nextId = nextId + 1;
end
end
%% Display Tracking Results
% The |displayTrackingResults| function draws a bounding box and label ID
% for each track on the video frame and the foreground mask. It then
% displays the frame and the mask in their respective video players.
function displayTrackingResults()
% Convert the frame and the mask to uint8 RGB.
frame = im2uint8(frame);
mask = uint8(repmat(mask, [1, 1, 3])) .* 255;
minVisibleCount = 8;
if ~isempty(tracks)
% Noisy detections tend to result in short-lived tracks.
% Only display tracks that have been visible for more than
% a minimum number of frames.
reliableTrackInds = ...
[tracks(:).totalVisibleCount] > minVisibleCount;
reliableTracks = tracks(reliableTrackInds);
% Display the objects. If an object has not been detected
% in this frame, display its predicted bounding box.
if ~isempty(reliableTracks)
% Get bounding boxes.
bboxes = cat(1, reliableTracks.bbox);
% Get ids.
ids = int32([reliableTracks(:).id]);
% Create labels for objects indicating the ones for
% which we display the predicted rather than the actual
% location.
labels = cellstr(int2str(ids'));
predictedTrackInds = ...
[reliableTracks(:).consecutiveInvisibleCount] > 0;
isPredicted = cell(size(labels));
isPredicted(predictedTrackInds) = {' predicted'};
labels = strcat(labels, isPredicted);
% Draw the objects on the frame.
frame = insertObjectAnnotation(frame, 'rectangle', ...
bboxes, labels);
% Draw the objects on the mask.
mask = insertObjectAnnotation(mask, 'rectangle', ...
bboxes, labels);
end
end
% Display the mask and the frame.
obj.maskPlayer.step(mask);
obj.videoPlayer.step(frame);
end
end
It would appear that you would need to modify the detectObjects function with the code to eliminate shadows. Currently, all it does is it takes a foreground mask from background subtraction (vision.ForegroundDetector), removes some noise and fills in small gaps using morphology, and then it finds connected components, a.k.a. "blobs". Typically, the blobs do include shadows.
If your shadow removal algorithm operates on a single frame, than this is where the code for it would go.
I think your problem is in your object detection. Your code is using an imopen and and imclose. But I think you want to use imdilate and imerode
%% Detect Objects
function [centroids, bboxes, mask] = detectObjects(frame)
% Detect foreground.
mask = obj.detector.step(frame);
% Apply morphological operations to remove noise and fill in holes.
%this makes the white space smaller to remove small dots/noise
mask = imerode(mask, strel('rectangle', [3,3]));
%this makes the white regions larger to restore the original size
mask = imclose(mask, strel('rectangle', [15, 15]));
mask = imfill(mask, 'holes');
% Perform blob analysis to find connected components.
[~, centroids, bboxes] = obj.blobAnalyser.step(mask);
end
I changed your imopen and imclose you may need to change your structuring elements as well. If you look at the matlab documentation imopen is the same as erosion followed by dilation, or imopened = imdilate(imerode(im,se)));
Imclose is dilation followed by erosion or imcloseed = imerode(imdilate(im,se))
the paper is using an erosion followed by a dilation (an imopen operation). So that is what I put in the code. If you want to use the same structuring element for both operations you can change
%this makes the white space smaller to remove small dots/noise
mask = imerode(mask, strel('rectangle', [3,3]));
%this makes the white regions larger to restore the original size
mask = imclose(mask, strel('rectangle', [15, 15]));
to
mask = imopen(mask,strel('rectangle',[15 15]));
since you didn't include your pictures I can't try it out, but I think it should work after you play with your strel function

How to draw a filled circle on a video frame using matlab

I have an "inverted-pendulum" video which I try to find the mid point of moving part. I am using Computer Vision Toolbox
I change the mid point's color using detected coordinates. Assume that X is the frame's row number for the detected mid point and the Y is the col number.
while ~isDone(hVideoFileReader)
frame = step(hVideoFileReader);
...
frame(X-3:X+3, Y-3:Y+3, 1) = 1; % # R=1 make the defined region red
frame(X-3:X+3, Y-3:Y+3, 2) = 0; % # G=0
frame(X-3:X+3, Y-3:Y+3, 3) = 0; % # B=0
step(hVideoPlayer, frame);
end
Then I easily have a red square. But I want to add a red filled circle on the detected point, instead of a square. How can I do that?
You can use the insertShape function. Example:
img = imread('peppers.png');
img = insertShape(img, 'FilledCircle', [150 280 35], ...
'LineWidth',5, 'Color','blue');
imshow(img)
The position parameter is specified as [x y radius]
EDIT:
Here is an alternative where we manually draw the circular shape (with transparency):
% some RGB image
img = imread('peppers.png');
[imgH,imgW,~] = size(img);
% circle parameters
r = 35; % radius
c = [150 280]; % center
t = linspace(0, 2*pi, 50); % approximate circle with 50 points
% create a circular mask
BW = poly2mask(r*cos(t)+c(1), r*sin(t)+c(2), imgH, imgW);
% overlay filled circular shape by using the mask
% to fill the image with the desired color (for all three channels R,G,B)
clr = [0 0 255]; % blue color
a = 0.5; % blending factor
z = false(size(BW));
mask = cat(3,BW,z,z); img(mask) = a*clr(1) + (1-a)*img(mask);
mask = cat(3,z,BW,z); img(mask) = a*clr(2) + (1-a)*img(mask);
mask = cat(3,z,z,BW); img(mask) = a*clr(3) + (1-a)*img(mask);
% show result
imshow(img)
I'm using the poly2mask function from Image Processing Toolbox to create the circle mask (idea from this post). If you don't have access to this function, here is an alternative:
[X,Y] = ndgrid((1:imgH)-c(2), (1:imgW)-c(1));
BW = (X.^2 + Y.^2) < r^2;
That way you get a solution using core MATLAB functions only (no toolboxes!)
If you have an older version of MATLAB with the Computer Vision System Toolbox installed, you can use vision.ShapeInserter system object.
Thanks #Dima, I have created a shapeInserter object.
greenColor = uint8([0 255 0]);
hFilledCircle = vision.ShapeInserter('Shape','Circles',...
'BorderColor','Custom',...
'CustomBorderColor', greenColor ,...
'Fill', true, ...
'FillColor', 'Custom',...
'CustomFillColor', greenColor );
...
fc = int32([Y X 7;]);
frame = step(hFilledCircle, frame, fc);
I then applied it to detected point.