Real time face detection in MATLAB - matlab

I'm trying to make a real time face detector using MATLAB. I found a sample code on the Mathworks' page, but it uses a sample video. What I'm having a problem with it that this code only can track the one it chooses to even with a few faces in the opening frame. I need it to track several faces at once. Is that possible with a change in this code that is not drastic.
I found the following code on MathWorks' web page:
% Create a cascade detector object.
faceDetector = vision.CascadeObjectDetector();
% Read a video frame and run the detector.
videoFileReader = vision.VideoFileReader('visionface.avi');
videoFrame = step(videoFileReader);
bbox = step(faceDetector, videoFrame);
% Draw the returned bounding box around the detected face.
videoOut = insertObjectAnnotation(videoFrame,'rectangle',bbox,'Face');
figure, imshow(videoOut), title('Detected face');
% Get the skin tone information by extracting the Hue from the video frame
% converted to the HSV color space.
[hueChannel,~,~] = rgb2hsv(videoFrame);
% Display the Hue Channel data and draw the bounding box around the face.
figure, imshow(hueChannel), title('Hue channel data');
rectangle('Position',bbox(1,:),'LineWidth',2,'EdgeColor',[1 1 0])
% Detect the nose within the face region. The nose provides a more accurate
% measure of the skin tone because it does not contain any background
% pixels.
noseDetector = vision.CascadeObjectDetector('Nose');
faceImage = imcrop(videoFrame,bbox(1,:));
noseBBox = step(noseDetector,faceImage);
% The nose bounding box is defined relative to the cropped face image.
% Adjust the nose bounding box so that it is relative to the original video
% frame.
noseBBox(1,1:2) = noseBBox(1,1:2) + bbox(1,1:2);
% Create a tracker object.
tracker = vision.HistogramBasedTracker;
% Initialize the tracker histogram using the Hue channel pixels from the
% nose.
initializeObject(tracker, hueChannel, noseBBox(1,:));
% Create a video player object for displaying video frames.
videoInfo = info(videoFileReader);
videoPlayer = vision.VideoPlayer('Position',[300 300 videoInfo.VideoSize+30]);
% Track the face over successive video frames until the video is finished.
while ~isDone(videoFileReader)
% Extract the next video frame
videoFrame = step(videoFileReader);
% RGB -> HSV
[hueChannel,~,~] = rgb2hsv(videoFrame);
% Track using the Hue channel data
bbox = step(tracker, hueChannel);
% Insert a bounding box around the object being tracked
videoOut = insertObjectAnnotation(videoFrame,'rectangle',bbox,'Face');
% Display the annotated video frame using the video player object
step(videoPlayer, videoOut);
end
% Release resources
release(videoFileReader);
release(videoPlayer);
Thanks in advance!

That example is designed to only track a single face. For tracking multiple objects please take a look at this example, that uses vision.KalmanFilter objects for tracking. You can replace the detection part in this example with the code to detect faces.
Alternatively, take a look at this example that uses the KLT algorithm (vision.PointTracker) to track points. You can modify that to track multiple faces too, but that is considerably more work. You would have to do a lot of bookkeeping to keep track of which points belong to which face.
Edit:
Here is an example of how to use vision.PointTracker to track multiple faces.

Related

Matlab: Extrinsics function on two checkerboards in one image

I am using the extrinsics function for a project that aims to determine distances of objects from the camera that have checkerboards attached to them. This is done for every frame in a video. My workflow is fairly simple:
Attach an asymmetric checkerboard to each object
Calibrate the camera using the Image Processing Toolbox on ~ 20 images
For each frame
while checkerboardsFound < numCheckerboards
% Undistort the image
data = undistortImage(data, cameraParams);
% Get the checkerboard points in the frame
[imagePoints, boardSize] = detectCheckerboardPoints(data);
worldPoints = generateCheckerboardPoints(boardSize, squareSize);
% Get the rotation and translation
[rot, trans] = extrinsics(imagePoints,worldPoints, cameraParams);
% <Draw a black rectangle over the area where the checkerboard points are
% using the imagePoints matrix. This "hides" the checkerboard so we can
% run the detectCheckerboardPoints(data) function again and find a different
% one.>
data = hideCheckerboard(data,...
[imagePoints(1,:);...
imagePoints(end,:)]);
checkerboardsFound = checkerboardsFound + 1;
end;
My problem is that when I find, say, two particular checkerboards that are about 5cm in depth apart and look at the Z value of their respective translation vectors as returned by the extrinsics function, I expect to see a difference of about 50 units (5cm). However, I see a considerably greater difference - e.g. about 400 units (40cm). I am certain the square sizes for the checkerboards are correctly defined (5mm each). The frames come from a Samsung smart phone capturing in video mode (which applies magnification).
Could anyone please help determine what could be causing this discrepancy in the Z values? Is it too ambitious to use the extrinsics function in such a manner?

Creating heat map of video content for specific pixels in each frame as data input

I am trying to make heatmaps for video content. For each frame in the video, I have data of specific pixels that are the focus points, and I would like to reproduce the video in a heatmap form according to the pixel coordinates(30 per frame) I have for each frame. Would that be possible to produce in Matlab, even if its done only for some specific frames, rather than the entire video? Thanks.
If you know the coordinates of the pixels, you can mark them in a 2D binary matrix for each frame. Then add all the matrices, and plot the result using the imagesc command. Here's an example with random pixel locations.
X = zeros(10,10,50); % Create 50 empty frames
for i = 1:50 % Loop through each frame
% Generate 30 random pixel locations
pixel_x = randi(10,[30,1]);
pixel_y = randi(10,[30,1]);
% Mark pixel locations in frame with 1's
for j = 1:30
X(pixel_y(j),pixel_x(j),i) = 1;
end
end
% Sum frames
X_sum = sum(X,3)
% Plot heatmap
figure
imagesc(X_sum)
colormap hot
colorbar

Taking images of 3D model using STLRead

I'm trying to take "images" of a 3D model. I am able to load an STL file through this code. What I need to do is be able rotate the object, and take "images" of it as it rotates.
stlread outputs a face and vertex structure that is compatible with patch(), so I can display the object, but I'm not sure how to actually store that image into a matrix.
I think what you are after is the function getframe, which captures the content of an axes and store the data as a regular "image" matrix.
I made a simple GUI to illustrate the concept. Basically I took the code from the link you provided (stldemo.m) and added a few features to take snapshots. In this case I added the command rotate3d on to rotate the femur inside the axes and a pushbutton whose callback calls getframe and captures the content of the axes. Basically you rotate the femur//patch object as you wish and you press the button to get a snapshot. In the code I make a new figure pop up to convince you it works, but you can of course do whatever you want with the data.
Note that using getframe outputs a structure with a field called cdata...that's what you are after in order to get the content of the axis as a matrix.
So here is the code and a few snapshots to illustrate:
function SnapShotSTL(~)
%// Taken from the stldemo.m file.
%% 3D Model Demo
% This is short demo that loads and renders a 3D model of a human femur. It
% showcases some of MATLAB's advanced graphics features, including lighting and
% specular reflectance.
% Copyright 2011 The MathWorks, Inc.
%% Load STL mesh
% Stereolithography (STL) files are a common format for storing mesh data. STL
% meshes are simply a collection of triangular faces. This type of model is very
% suitable for use with MATLAB's PATCH graphics object.
% Import an STL mesh, returning a PATCH-compatible face-vertex structure
handles.fv = stlread('femur.stl');
%% Render
% The model is rendered with a PATCH graphics object. We also add some dynamic
% lighting, and adjust the material properties to change the specular
% highlighting.
%// Create figure, axes and a pushbutton.
hFigure = figure('Units','Pixels','Position',[200 200 500 600]);
hAxes1 = axes('Units','pixels','Position',[50 50 450 400]);
hSnapShot = uicontrol('Style','push','Position',[50 500 60 30],'String','SnapShot','Callback',#(s,e) GetSnapshot);
patch(handles.fv,'FaceColor', [0.8 0.8 1.0], ...
'EdgeColor', 'none', ...
'FaceLighting', 'gouraud', ...
'AmbientStrength', 0.15);
% Add a camera light, and tone down the specular highlighting
camlight('headlight');
material('dull');
% Fix the axes scaling, and set a nice view angle
axis('image');
view([-135 35]);
rotate3d on %// Enable to rotate the object in the axes
guidata(hFigure,handles);
function GetSnapshot
CurrentImage = getframe(gca);
ImageData = CurrentImage.cdata; %// Access the data. I create a variable but that's not strictly necessary.
%// Here a new figure is created and the image displayed in it... you can store it and do as you like with it.
figure;
imshow(ImageData);
guidata(hFigure,handles);
end
end
So the GUI looks like this when opened:
Then after a rotation of the object/femur:
and finally after pressing the pushbutton, a snapshot is taken (i.e. getframe is executed) and the resulting image matrix displayed in a new figure:
Note that I used imshow to actually display the image but the data can be manipulated as well.
Moreover, you can also use the following calling syntax for getframe in case you want to retrieve the colormap associated with your data:
F = getframe;
[X,map] = frame2im(F);
In this case, X would be equivalent to G when using this code:
G = F.cdata;
Finally, you could set up a loop in which the actual patch object is rotated by itself and frames are taken automatically. That should not be too hard to implement :)
Hope that helps!

MATLAB Image Processing - Find Edge and Area of Image

As a preface: this is my first question - I've tried my best to make it as clear as possible, but I apologise if it doesn't meet the required standards.
As part of a summer project, I am taking time-lapse images of an internal melt figure growing inside a crystal of ice. For each of these images I would like to measure the perimeter of, and area enclosed by the figure formed. Linked below is an example of one of my images:
The method that I'm trying to use is the following:
Load image, crop, and convert to grayscale
Process to reduce noise
Find edge/perimeter
Attempt to join edges
Fill perimeter with white
Measure Area and Perimeter using regionprops
This is the code that I am using:
clear; close all;
% load image and convert to grayscale
tyrgb = imread('TyndallTest.jpg');
ty = rgb2gray(tyrgb);
figure; imshow(ty)
% apply a weiner filter to remove noise.
% N is a measure of the window size for detecting coherent features
N=20;
tywf = wiener2(ty,[N,N]);
tywf = tywf(N:end-N,N:end-N);
% rescale the image adaptively to enhance contrast without enhancing noise
tywfb = adapthisteq(tywf);
% apply a canny edge detection
tyedb = edge(tywfb,'canny');
%join edges
diskEnt1 = strel('disk',8); % radius of 4
tyjoin1 = imclose(tyedb,diskEnt1);
figure; imshow(tyjoin1)
It is at this stage that I am struggling. The edges do not quite join, no matter how much I play around with the morphological structuring element. Perhaps there is a better way to complete the edges? Linked is an example of the figure this code outputs:
The reason that I am trying to join the edges is so that I can fill the perimeter with white pixels and then use regionprops to output the area. I have tried using the imfill command, but cannot seem to fill the outline as there are a large number of dark regions to be filled within the perimeter.
Is there a better way to get the area of one of these melt figures that is more appropriate in this case?
As background research: I can make this method work for a simple image consisting of a black circle on a white background using the below code. However I don't know how edit it to handle more complex images with edges that are less well defined.
clear all
close all
clc
%% Read in RGB image from directory
RGB1 = imread('1.jpg') ;
%% Convert RPG image to grayscale image
I1 = rgb2gray(RGB1) ;
%% Transform Image
%CROP
IC1 = imcrop(I1,[74 43 278 285]);
%BINARY IMAGE
BW1 = im2bw(IC1); %Convert to binary image so the boundary can be traced
%FIND PERIMETER
BWP1 = bwperim(BW1);
%Traces perimeters of objects & colours them white (1).
%Sets all other pixels to black (0)
%Doing the same job as an edge detection algorithm?
%FILL PERIMETER WITH WHITE IN ORDER TO MEASURE AREA AND PERIMETER
BWF1 = imfill(BWP1); %This opens figure and allows you to select the areas to fill with white.
%MEASURE PERIMETER
D1 = regionprops(BWF1, 'area', 'perimeter');
%Returns an array containing the properties area and perimeter.
%D1(1) returns the perimeter of the box and an area value identical to that
%perimeter? The box must be bounded by a perimeter.
%D1(2) returns the perimeter and area of the section filled in BWF1
%% Display Area and Perimeter data
D1(2)
I think you might have room to improve the effect of edge detection in addition to the morphological transformations, for instance the following resulted in what appeared to me a relatively satisfactory perimeter.
tyedb = edge(tywfb,'sobel',0.012);
%join edges
diskEnt1 = strel('disk',7); % radius of 4
tyjoin1 = imclose(tyedb,diskEnt1);
In addition I used bwfill interactively to fill in most of the interior. It should be possible to fill the interior programatically but I did not pursue this.
% interactively fill internal regions
[ny nx] = size(tyjoin1);
figure; imshow(tyjoin1)
tyjoin2=tyjoin1;
titl = sprintf('click on a region to fill\nclick outside window to stop...')
while 1
pts=ginput(1)
tyjoin2 = bwfill(tyjoin2,pts(1,1),pts(1,2),8);
imshow(tyjoin2)
title(titl)
if (pts(1,1)<1 | pts(1,1)>nx | pts(1,2)<1 | pts(1,2)>ny), break, end
end
This was the result I obtained
The "fractal" properties of the perimeter may be of importance to you however. Perhaps you want to retain the folds in your shape.
You might want to consider Active Contours. This will give you a continous boundary of the object rather than patchy edges.
Below are links to
A book:
http://www.amazon.co.uk/Active-Contours-Application-Techniques-Statistics/dp/1447115570/ref=sr_1_fkmr2_1?ie=UTF8&qid=1377248739&sr=8-1-fkmr2&keywords=Active+shape+models+Andrew+Blake%2C+Michael+Isard
A demo:
http://users.ecs.soton.ac.uk/msn/book/new_demo/Snakes/
and some Matlab code on the File Exchange:
http://www.mathworks.co.uk/matlabcentral/fileexchange/28149-snake-active-contour
and a link to a description on how to implement it: http://www.cb.uu.se/~cris/blog/index.php/archives/217
Using the implementation on the File Exchange, you can get something like this:
%% Load the image
% You could use the segmented image obtained previously
% and then apply the snake on that (although I use the original image).
% This will probably make the snake work better and the edges
% in your image is not that well defined.
% Make sure the original and the segmented image
% have the same size. They don't at the moment
I = imread('33kew0g.jpg');
% Convert the image to double data type
I = im2double(I);
% Show the image and select some points with the mouse (at least 4)
% figure, imshow(I); [y,x] = getpts;
% I have pre-selected the coordinates already
x = [ 525.8445 473.3837 413.4284 318.9989 212.5783 140.6320 62.6902 32.7125 55.1957 98.6633 164.6141 217.0749 317.5000 428.4172 494.3680 527.3434 561.8177 545.3300];
y = [ 435.9251 510.8691 570.8244 561.8311 570.8244 554.3367 476.3949 390.9586 311.5179 190.1085 113.6655 91.1823 98.6767 106.1711 142.1443 218.5872 296.5291 375.9698];
% Make an array with the selected coordinates
P=[x(:) y(:)];
%% Start Snake Process
% You probably have to fiddle with the parameters
% a bit more that I have
Options=struct;
Options.Verbose=true;
Options.Iterations=1000;
Options.Delta = 0.02;
Options.Alpha = 0.5;
Options.Beta = 0.2;
figure(1);
[O,J]=Snake2D(I,P,Options);
If the end result is an area/diameter estimate, then why not try to find maximal and minimal shapes that fit in the outline and then use the shapes' area to estimate the total area. For instance, compute a minimal circle around the edge set then a maximal circle inside the edges. Then you could use these to estimate diameter and area of the actual shape.
The advantage is that your bounding shapes can be fit in a way that minimizes error (unbounded edges) while optimizing size either up or down for the inner and outer shape, respectively.

black on black motion segmentation in Matlab

I have a video that I am using background subtraction and motion segmentation on. The floor in the video is black, so when I get the silhouette the feet and parts of the legs are cut off. Is there a fix around this? This is what it looks like.
This is the background image.
This is a piece of my code....
clear all
close all
clc
% Read the video in the video object Mov
MM = mmreader('kassie_test_video.wmv');
% Read in all video frames.
Mov = read(MM);
% Get the number of frames.
FrameNum = MM.NumberOfFrames;
% load 'object_data.mat'
BackgroundImage = (Mov(:,:,:,98)); % background image
% set the sampling rate as well as the threshold for binary image.
downSamplingRate = MM.FrameRate;
%%
index = 1;
clear IM
clear Images
sf=10;ef=sf+30;
for ii =sf:ef
% Extract next frame
Im = im2double(Mov(:,:,:,ii));
% Background subtraction
Ib = rgb2gray(abs(im2double((Mov(:,:,:,ii)))-im2double(BackgroundImage)));
% conversion to binary image.
Thresh = graythresh(Ib);
Ib = im2bw(Ib, Thresh);
se = strel('square',1);
Ib = imerode(Ib,se); % Erode the image
Ib = medfilt2(Ib); % median filtering
Ib = imfill(Ib,'holes'); % fill the holes in the image
imshow(Ib,[])
end
there is a limit to what can be achieved in computer vision utilizing only pixel-processing without incorporating higher-level semantic information. it appears as if he only thing that makes you think the legs are missing is your high-level knowledge of how a body should look like. The real question here: is there any real information in the pixels? if it just so happen that the legs are exactly the same color as the background there's nothing much you can do unless you incorporate high level semantic information.