Implementing a neural network for smile detection in images - matlab

Question

Let me explain the background to the problem before I explain the problem. My task is to take in an image labelled with a smile of not. The files that have smiles are labelled, for example 100a.jpg and 100b.jpg. Where 'a' is used to represent an image without a smile and 'b' is used to represent an image with a smile. As such I'm looking to make a 3 layered network i.e. Layer 1 = input nodes, Layer 2 = hidden layer and layer 3 = output node.
The general algorithm is to:
Take in an image and re-size it to the size of x 24x20.
Apply a forward propagation from the input nodes to the hidden layer.
Apply a forward propagation from the hidden layer to the output node.
Then apply a backward propagation from the output node to the hidden layer. (Formula1)
Then apply a backward propagation from the hidden layer to the input nodes. (Formula2)
Formula 1:
Formula 2:
Now the problem quite simply is... my code never converges and as such I dont have weight vectors that can be used to test the network. Problem is I HAVE NO CLUE WHY THIS IS HAPPENING... Here is the error I display, clearly not converging:
Training done full cycle
0.5015
Training done full cycle
0.5015
Training done full cycle
0.5015
Training done full cycle
0.5038
Training done full cycle
0.5038
Training done full cycle
0.5038
Training done full cycle
0.5038
Training done full cycle
0.5038
Here is my matlab code:
function [thetaLayer12,thetaLayer23]=trainSystem()
%This is just the directory where I read the images from
files = dir('train1/*jpg');
filelength = length(files);
%Here I create my weights between input layer and hidden layer and then
%from the hidden layer to the output node. The reason the value 481 is used
%is because there will be 480 input nodes + 1 bias node. The reason 200 is
%used is for the number of hidden layer nodes
thetaLayer12 = unifrnd (-1, 1 ,[481,200]);
thetaLayer23 = unifrnd (-1, 1 ,[201,1]);
%Learning Rate value
alpha = 0.00125;
%Initalize Convergence Error
globalError = 100;
while(globalError > 0.001)
globalError = 0;
%Run through all the files in my training set. 400 Files to be exact.
for i = 1 : filelength
%Here we find out if the image has a smile in it or not. If there
%Images are labled 1a.jpg, 1b.jpg where images with an 'a' in them
%have no smile and images with a 'b' in them have a smile.
y = isempty(strfind(files(i).name,'a'));
%We read in the image
imageBig = imread(strcat('train1/',files(i).name));
%We resize the image to 24x20
image = imresize(imageBig,[24 20]);
%I then take the 2D image and map it to a 1D vector
inputNodes = reshape(image,480,1);
%A bias value of 1 is added to the top of the vector
inputNodes = [1;inputNodes];
%Forward Propogation is applied the input layer and the hidden
%layer
outputLayer2 = logsig(double(inputNodes')* thetaLayer12);
%Here we then add a bias value to hidden layer nodes
inputNodes2 = [1;outputLayer2'];
%Here we then do a forward propagation from the hidden layer to the
%output node to obtain a single value.
finalResult = logsig(double(inputNodes2')* thetaLayer23);
%Backward Propogation is then applied to the weights between the
%output node and the hidden layer.
thetaLayer23 = thetaLayer23 - alpha*(finalResult - y)*inputNodes2;
%Backward Propogation is then applied to the weights between the
%hidden layer and the input nodes.
thetaLayer12 = thetaLayer12 - (((alpha*(finalResult-y)*thetaLayer23(2:end))'*inputNodes2(2:end))*(1-inputNodes2(2:end))*double(inputNodes'))';
%I sum the error across each iteration over all the images in the
%folder
globalError = globalError + abs(finalResult-y);
if(i == 400)
disp('Training done full cycle');
end
end
%I take the average error
globalError = globalError / filelength;
disp(globalError);
end
end
Any help would seriously be appreciated!!!!

Answer 1

The success of training any machine learning algorithm is heavily dependent on the amount of training examples you use to train your algorithm. You never said exactly how many training examples you have, but in the case of face detection a huge number of examples would probably be needed (if it would work at all).
Think of it this way, a computer scientist shows you two arrays of pixel intensity values. He tells you which one has a simile in it and which does not. Than he shows you two more and asks you to tell him which one has a simile in it.
Fortunately we can work around this to some extent. You can use an autoencoder or a dictionary learner like sparse coding to find higher level structures in the data. Instead of the computer scientist showing your pixels intensities, he could show you edges or even body parts. You could then use this as input to your neural network, but a significant number of training examples would probably still be needed (but lesser than before).
Than analogy was inspired by a talk given by professor Ng of Stanford on unsupervised feature learning.