Detecting frames for which a face appears in a video - matlab

I need to detect the number of frames for which a face is appearing in a video. I looked into the sample code using CAMShift algorithm provided in the MathWorks site(http://www.mathworks.in/help/vision/examples/face-detection-and-tracking-using-camshift.html). Is there a way of knowing whether a face has appeared in a particular frame?
I'm new to MatLab. I'm assuming the step function will return a false value if no face is detected (condition fails - similar to C). Is there a possible solution? I think using MinSize is also a possible solution.
I am not concerned about the computational burden - although a faster approach for the same would be appreciated. My current code is given below:
clc;
clear all;
videoFileReader = vision.VideoFileReader('Teapot.mp4', 'VideoOutputDataType', 'uint8', 'ImageColorSpace', 'Intensity');
video = VideoReader('Teapot.mp4');
numOfFrames = video.NumberOfFrames;
faceDetector = vision.CascadeObjectDetector();
opFolder = fullfile(cd, 'Face Detected Frames');
frameCount = 0;
shotCount = 0;
while ~isDone(videoFileReader)
videoFrame = step(videoFileReader);
bbox = step(faceDetector, videoFrame);
framCount = frameCount + 1;
for i = 1:size(bbox,1)
shotCount = shotCount + 1;
rectangle('Position',bbox(i,:),'LineWidth', 2, 'EdgeColor', [1 1 0]);
videoOut = insertObjectAnnotation(videoFrame,'rectangle',bbox,'Face');
progIndication = sprintf('Face has been detected in frame %d of %d frames', shotCount, numOfFrames);
figure, imshow(videoOut), title(progIndication);
end
end
release(videoFileReader);

You can use the vision.CascadeObjectDetector object to detect faces in any particular frame. If it does not detect any faces, its step method will return an empty array. The problem is that the face detection algorithm is not perfect. Sometimes it detects false positives, i. e. detects faces where there are none. You can try to mitigate that my setting the MinSize and MaxSize properties, assuming you know what size faces you expect to find.

Related

Most efficient way to track multiple small objects in MATLAB?

I am relatively new to image processing, and have never attempted to do anything with images in MATLAB, so forgive me if i am making some very rookie errors.
I am attempting to make a program that will track ants in a video. The video is taken from a stationary camera, and records the ants from a birds-eye perspective. I am having issues making reliable tracks of the ants however. Initially, i used the ForegroundDetection function, however there were multiple issues:
1.) Stationary ants were not detected
2.) There was too much overlap between objects (high levels of occlusion)
A friend of mine recommended having a larger gap between compared frames, so instead of subtracting frame 1 from frame 2, do frame 1 from frame 30 instead (1 second apart), as this will make the ants that do not move as much more likely to appear on the subtracted image.
Below is the code i have so far. It is a bit of a shot-in-the-dark attempt to solve the problem, as i am running out of ideas:
i = 1;
k = 1;
n = 1;
Video = {};
SubtractedVideo = {};
FilteredVideo = {};
videoFReader = vision.VideoFileReader('001.mp4',...
'ImageColorSpace', 'Intensity', 'VideoOutputDataType', 'uint8');
videoPlayer = vision.VideoPlayer;
blobby = vision.BlobAnalysis('BoundingBoxOutputPort', true, ...
'AreaOutputPort', true, 'CentroidOutputPort', true, ...
'MinimumBlobArea', 1);
shapeInserter = vision.ShapeInserter('BorderColor','White');
while ~isDone(videoFReader) %Read all frame of video
frame = step(videoFReader);
Video{1, i} = frame;
i = i+1;
end
%Perform subtraction
for j=1: 1: numel(Video)-60
CurrentFrame = Video{1,j};
FutureFrame = Video{1,j+60};
SubtractedImage = imsubtract(CurrentFrame, FutureFrame);
SubtractedVideo{1,j} = SubtractedImage;
ImFiltered = imgaussfilt(SubtractedImage, 2);
BWIm = im2bw(ImFiltered, 0.25);
FilteredVideo{1,j} = BWIm;
end
for a = n:numel(FilteredVideo)
frame = Video{1, n};
bbox = step(blobby, FilteredVideo{1, k});
out = step(shapeInserter, frame, bbox);
step(videoPlayer, out);
k = k+1;
end
currently, when i run the code, i get the following error on the line out = step(shapeInserter, frame, bbox):
'The Points input must be a 4-by-N matrix. Each column specifies a different rectangle and is of the form [row column height width].'
My questions is:
1.) Is this the best way to try and solve the problem I'm having? Is there potentially an easier solution?
2.) What does this error mean? How do i solve the issue?
I appreciate any help anyone can give, thank you!

Resize Frame for Optical Flow

I have problem with optical flow if the frame size have been manipulated in any way this gives me error. There are two options either change the resolution of the video at the beginning or somehow how change the frame size in a way that optical flow will work. I will want to add a cascade object to detect nose, mouth and eyes in further development therefore I need solution that will work for individual regions without necessary setting optical flow individually for those regions especially that a bounding box does not have a fixed size and it will displace itself slightly from frame to frame. Here is my code so far, the error is that it is exceeding matrix dimensions.
faceDetector = vision.CascadeObjectDetector();
vidObj = vision.VideoFileReader('MEXTest.mp4','ImageColorSpace','Intensity','VideoOutputDataType','uint8');
converter = vision.ImageDataTypeConverter;
opticalFlow = vision.OpticalFlow('ReferenceFrameDelay', 1);
opticalFlow.OutputValue = 'Horizontal and vertical components in complex form';
shapeInserter = vision.ShapeInserter('Shape','Lines','BorderColor','Custom','CustomBorderColor', 255);
vidPlayer = vision.VideoPlayer('Name','Motion Vector');
while ~isDone(vidObj);
frame = step(vidObj);
fraRes = imresize(frame,0.5);
fbbox = step(faceDetector,fraRes);
I = imcrop(fraRes,fbbox);
im = step(converter,I);
of = step(opticalFlow,im);
lines = videooptflowlines(of, 20);
if ~isempty(lines)
out = step(shapeInserter,im,lines);
step(vidPlayer,out);
end
end
release(vidPlayer);
release(VidObj);
UPDATE: I went and edited the function for optical flow which creates lines and this sorts out the some size issues however it is necessary to to input this manually for each object (so if there is any other way let me know). I think the best solution would be set a fixed size to cascadeObjectDetector, does anyone know how to do this? Or have any other idea?
faceDetector = vision.CascadeObjectDetector(); %I need fixed size for this
faceDetector.MinSize = [150 150];
vidRead = vision.VideoFileReader('MEXTest.mp4','ImageColorSpace','Intensity','VideoOutputDataType','uint8');
convert = vision.ImageDataTypeConverter;
optFlo = vision.OpticalFlow('ReferenceFrameDelay', 1);
optFlo.OutputValue = 'Horizontal and vertical components in complex form';
shapeInserter = vision.ShapeInserter('Shape','Lines','BorderColor','Custom', 'CustomBorderColor', 255);
while ~isDone(vidRead)
frame = step(vidRead);
fraRes = imresize(frame,0.3);
fraSin = im2single(fraRes);
bbox = step(faceDetector,fraSin);
I = imcrop(fraSin, bbox);
im = step(convert, I);
release(optFlo);
of = step(optFlo, im);
lines = optfloo(of, 50); %use videooptflowlines instead of (optfloo)
out = step(shapeInserter, im, lines);
imshow(out);
end

Find a nearly circular band of bright pixels in this image

This is the problem I have: I have an image as shown below. I want to detect the circular region which I have marked with a red line for display here (that particular bright ring).
Initially, this is what I do for now: (MATLAB)
binaryImage = imdilate(binaryImage,strel('disk',5));
binaryImage = imfill(binaryImage, 'holes'); % Fill holes.
binaryImage = bwareaopen(binaryImage, 20000); % Remove small blobs.
binaryImage = imerode(binaryImage,strel('disk',300));
out = binaryImage;
img_display = immultiply(binaryImage,rgb2gray(J1));
figure, imshow(img_display);
The output seems to be cut on one of the parts of the object (for a different image as input, not the one displayed above). I want an output in such a way that it is symmetric (its not always a perfect circle, when it is rotated).
I want to strictly avoid im2bw since as soon as I binarize, I lose a lot of information about the shape.
This is what I was thinking of:
I can detect the outer most circular (almost circular) contour of the image (shown in yellow). From this, I can find out the centroid and maybe find a circle which has a radius of 50% (to locate the region shown in red). But this won't be exactly symmetric since the object is slightly tilted. How can I tackle this issue?
I have attached another image where object is slightly tilted here
I'd try messing around with the 'log' filter. The region you want is essentially low values of the 2nd order derivative (i.e. where the slope is decreasing), and you can detect these regions by using a log filter and finding negative values. Here's a very basic outline of what you can do, and then tweak it to your needs.
img = im2double(rgb2gray(imread('wheel.png')));
img = imresize(img, 0.25, 'bicubic');
filt_img = imfilter(img, fspecial('log',31,5));
bin_img = filt_img < 0;
subplot(2,2,1);
imshow(filt_img,[]);
% Get regionprops
rp = regionprops(bin_img,'EulerNumber','Eccentricity','Area','PixelIdxList','PixelList');
rp = rp([rp.EulerNumber] == 0 & [rp.Eccentricity] < 0.5 & [rp.Area] > 2000);
bin_img(:) = false;
bin_img(vertcat(rp.PixelIdxList)) = true;
subplot(2,2,2);
imshow(bin_img,[]);
bin_img(:) = false;
bin_img(rp(1).PixelIdxList) = true;
bin_img = imfill(bin_img,'holes');
img_new = img;
img_new(~bin_img) = 0;
subplot(2,2,3);
imshow(img_new,[]);
bin_img(:) = false;
bin_img(rp(2).PixelIdxList) = true;
bin_img = imfill(bin_img,'holes');
img_new = img;
img_new(~bin_img) = 0;
subplot(2,2,4);
imshow(img_new,[]);
Output:

How to track a face that is 5m away from source in facial detection for matlab?

I am currently trying to figure out how to be able to detect a face that is 5m away from the source and will have its facial features clear enough for the user to see. The code i am working on is as shown.
faceDetector = vision.CascadeObjectDetector();
%Get the input device using image acquisition toolbox,resolution = 640x480 to improve performance
obj =imaq.VideoDevice('winvideo', 1, 'YUY2_640x480','ROI', [1 1 640 480]);
set(obj,'ReturnedColorSpace', 'rgb');
figure('menubar','none','tag','webcam');
while (true)
frame=step(obj);
bbox=step(faceDetector,frame);
boxInserter = vision.ShapeInserter('BorderColor','Custom',...
'CustomBorderColor',[255 255 0]);
videoOut = step(boxInserter, frame,bbox);
imshow(videoOut,'border','tight');
f=findobj('tag','webcam');
if (isempty(f));
[hueChannel,~,~] = rgb2hsv(frame);
% Display the Hue Channel data and draw the bounding box around the face.
figure, imshow(hueChannel), title('Hue channel data');
rectangle('Position',bbox,'EdgeColor','r','LineWidth',1)
hold off
noseDetector = vision.CascadeObjectDetector('Nose');
faceImage = imcrop(frame,bbox);
imshow(faceImage)
noseBBox = step(noseDetector,faceImage);
noseBBox(1:1) = noseBBox(1:1) + bbox(1:1);
videoInfo = info(obj);
ROI=get(obj,'ROI');
VideoSize = [ROI(3) ROI(4)];
videoPlayer = vision.VideoPlayer('Position',[300 300 VideoSize+30]);
tracker = vision.HistogramBasedTracker;
initializeObject(tracker, hueChannel, bbox);
while (1)
% Extract the next video frame
frame = step(obj);
% RGB -> HSV
[hueChannel,~,~] = rgb2hsv(frame);
% Track using the Hue channel data
bbox = step(tracker, hueChannel);
% Insert a bounding box around the object being tracked
videoOut = step(boxInserter, frame, bbox);
%Insert text coordinates
% Display the annotated video frame using the video player object
step(videoPlayer, videoOut);
pause (.2)
end
% Release resources
release(obj);
release(videoPlayer);
close(gcf)
break
end
pause(0.05)
end
release(obj)
% remove video object from memory
delete(handles.vid);
I am trying to work on this code to figure out the distance it can cover when tracking a face. I couldnt figure out which one handles that. Thanks!
Not sure what your question is, but try this example. It uses the KLT algorithm, which, IMHO, is more robust for face tracking than CAMShift. It also uses the webcam interface in base MATLAB, which is very easy.

Moving point along a curve (3D-Animation plots)

I am trying to make an animation of the trajectory (circular orbit of 7000 km altitude) of a satellite orbiting the Earth. The following vectors x,y,z represents the coordinates of it (obtained integrating the acceleration due to the nonspherical gravitational potential) in the reference system.
fh = figure('DoubleBuffer','on');
ah = axes('Parent',fh,'Units','normalized','Position',[0 0 1 1],...
'DataAspectRatio',[1 1 1],'DrawMode','fast');
x = 1.0e+003 * [ 1.293687086462776 1.355010603320554 ...
1.416226136451621 1.477328806662750 1.538313743926646...
1.841302933101510 2.140623861743577 2.435680048370655...
2.725883985836056 3.830393161542639 4.812047393962632...
5.639553477924236 6.285935904692739 6.778445814703028...
6.981534839226300 6.886918327688911 6.496619397538814...
5.886899070860056 5.061708852126299 4.051251943168882...
2.891621923700204 1.551975259009857 0.148687346809817...
-1.259946709379085 -2.614876359324573 -3.789635985368149...
-4.822735075152957 -5.675398819678173 -6.314344260262741...
-6.725008970265510 -6.860046738669579 -6.714044347581475...
-6.291232549137548 -5.646225528669501 -4.790489239458692...
-3.756316068441812 -2.581710448683235 -1.257064527234605...
0.118190083177733 1.488198207705392 2.797262268588749...
3.943218990855596 4.943060241667732 5.760107224604901...
6.363435161221018 6.741208871652011 6.844507242544970...
6.669637491855506 6.222229021788314 5.549112743364572...
4.665587166679964 3.605338508383659 2.407805301565781...
1.076891826523990 -0.297413079432155 -1.658804233546807...
-2.950960371016551 -4.105336427038419 -5.093651475630134...
-5.875676956725480 -6.417825276834068 -6.694317613708315...
-6.702354075060146 -6.441476385534835 -5.920328191821120...
-5.149356931765655 -4.165756794143557 -3.010476122311884...
-1.730623521107957 -0.547981318845428 0.651933236927557...
1.830754553013015 2.950797411065132];
y = 1.0e+003 *[ -6.879416537989226 -6.867600717396513...
-6.855237614338527 -6.842328214064634 -6.828873545169439...
-6.753459997528374 -6.664593892931937 -6.562452270514113...
-6.447238135027323 -5.857768973060929 -5.080802144227667...
-4.141502963266585 -3.069449548231363 -1.712593819793112...
-0.283073212084787 1.157789207734001 2.547934226666446...
3.733185664633135 4.781256997101091 5.653507474532885...
6.316540958291930 6.760480121739906 6.924451844039825...
6.801366712306432 6.393950562012035 5.763652137956600...
4.918852380803697 3.890903548710424 2.717191733101876...
1.385839187748386 -0.001786735280855 -1.388680800030854...
-2.717513794724399 -3.877348086956174 -4.892062889940518...
-5.723943344458780 -6.341064412332522 -6.729295147896739...
-6.844976271597333 -6.684181367561298 -6.252308741323985...
-5.600523241569850 -4.741636145151388 -3.707934368103928...
-2.537101251915556 -1.208445066639178 0.169057351189467...
1.539102816836380 2.845512534980855 3.993289528709769...
4.989150886098799 5.795183343929699 6.379362665363127...
6.723976759736427 6.794165677259719 6.586864956951024...
6.108394444576384 5.387403581100790 4.449452017586583...
3.332306147336086 2.080126804848620 0.757432563194591...
-0.595089763589023 -1.923045482863719 -3.172486599444496...
-4.302442851663575 -5.254127434062967 -5.988250483410006...
-6.472859710456819 -6.675113607083117 -6.664054266658221...
-6.440275312105615 -6.010308893159839];
z = [ -1.348762314964606 -1.416465504571016 -1.484053975854905...
-1.551522350691171 -1.618865254528658 -1.953510294130345...
-2.284215283426580 -2.610320163346533 -2.931177500785390...
-4.153679292291825 -5.242464339076090 -6.162825517200489...
-6.884797354552217 -7.440577139596716 -7.680358197465111...
-7.594616346122523 -7.183952381870657 -6.529293328494871...
-5.637062917332294 -4.540678277777376 -3.279180600545935...
-1.817413221203883 -0.280548741687378 1.268253040429052...
2.764251377698321 4.066975661566477 5.218214283582148...
6.174673504642019 6.899157495671121 7.375688520371054...
7.548875108319217 7.410793523141250 6.965068314483629...
6.271309946313485 5.343254095742233 4.215431448848456...
2.928028129903598 1.469574073877195 -0.048649548535536...
-1.563638474934283 -3.013536101911645 -4.285161526803897...
-5.397128342069014 -6.308837263463213 -6.985946890567337...
-7.415475222950275 -7.542406523585701 -7.363021555333582...
-6.884639818710263 -6.158276823110702 -5.199186592259776...
-4.043958234344444 -2.736923814690622 -1.283388986878655...
0.219908617803070 1.712828428793243 3.135072606759898...
4.411790351254605 5.510842969067953 6.387336537361380...
7.004133661144990 7.332163450286972 7.366696289243980...
7.105258174916579 6.555393588532904 5.727091807637045...
4.660073989309112 3.399622357708514 1.999243120787114...
0.701744421660999 -0.620073499615723 -1.923270654698332...
-3.164705887374677 ];
load('topo.mat','topo','topomap1');
[x1,y1,z1] = sphere(50);
x1 = 6678.14*x1;
y1 = 6678.14*y1;
z1 = 6678.14*z1;
props.AmbientStrength = 0.1;
props.DiffuseStrength = 1;
props.SpecularColorReflectance = .5;
props.SpecularExponent = 20;
props.SpecularStrength = 1;
props.FaceColor= 'texture';
props.EdgeColor = 'none';
props.FaceLighting = 'phong';
props.Cdata = topo;
surface(x1,y1,z1,props);
light('position',[-1 0 1]);
light('position',[-1.5 0.5 -0.5], 'color', [.6 .2 .2]);
view(3);
handles.p1 = line('parent',ah,'XData',x(1),'YData',y(1),'ZData',...
z(1),'Color','red','LineWidth',2);
handles.p2 = line('parent',ah,'XData',x(end),'YData',y(end),...
'ZData',z(end),'Marker','o','MarkerSize',6,'MarkerFaceColor','b');
oaxes([0 0 0],'Arrow','extend','AxisLabelLocation','side',...
'Xcolor','green','Ycolor','green','Zcolor','green');
axis vis3d equal;
handles.XLim = get(gca,'XLim');
handles.YLim = get(gca,'YLim');
handles.ZLim = get(gca,'ZLim');
set([handles.p1,handles.p2],'Visible','off');
xmin = handles.XLim(1);
ymin = handles.YLim(1);
zmin = handles.ZLim(1);
xmax = handles.XLim(2);
ymax = handles.YLim(2);
zmax = handles.ZLim(2);
set(ah, 'XLim', [xmin xmax],'YLim', [ymin ymax],'Zlim',[zmin zmax]);
view(3);
handles.hsat = line('parent',ah,'XData',x(1), 'YData',y(1),...
'ZData',z(1),'Marker','o', 'MarkerSize',6,'MarkerFaceColor','b');
k = uint8(2);
u2 = uint8(length(x));
while k<u2
handles.htray(k) = line([x(k-1) x(k)],[y(k-1) y(k)],[z(k-1) z(k)],...
'Color','red','LineWidth',3);
set(handles.hsat,'XData',x(k),'YData',y(k),'ZData',z(k));
drawnow;
k = k + 1;
end
where oaxes is a FEX application that allows getting an axes located (in this case) at the origin (0,0,0) of the PlotBox.
I have read the User Guide's Graphics section in the Matlab Help Browser. It recommends to use low-level functions for speeding the graphics output (this is the reason for which I use the line function instead of plot3) and the renderer painters for line graphics. In my case, I can not use it because I have a surface (the Earth) which is not well drawn by it. I want to get something similar to this (I have tried to get in touch with the author but I have not got response). The final result is a slow (it takes 11.4 seconds in my computer with microprocessor intel core i5) and discontinuous animation (perhaps I need more points to get the blue point's movement looks like continuous but the integrator's output points are invariable). I would like to know what I should make to improve it. Thank you for your attention. Cheers.
A couple of things here.
DrawMode=fast probably doesn't do what you think it does. It's turning off depthsorting. I think that you really want depthsorting here.
You're creating line objects in the inner loop. You really want create a small number of graphics objects and reuse them. Could you create a single line object and set the XData, YData, & ZData, in the loop?
You can use hgtransform to avoid modifying the coordinates of hsat (as described here), but that would only make a difference if hsat was much more complex. I don't think it would buy you anything in this case.
You could reduce the resolution of your surface.
You probably want to set the figure's Renderer property to OpenGL.
In this case, but I'm getting almost 20 frames per second on my system with your code. After making those changes, I'm getting about 100 frames per second. What sort of framerate are you shooting for here?
I believe the main reason your animation is slow is because you are using the Phong lighting algorithm which is computationally expensive. To see the effect it has on performance, try specifying Gouraud shading instead:
%#lighting('gouraud');
props.FaceLighting = 'gouraud'; %# faster interpolating method