Single layer neural network [closed] - matlab

Question

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
For the implementation of single layer neural network, I have two data files.
In:
0.832 64.643
0.818 78.843
Out:
0 0 1
0 0 1
The above is the format of 2 data files.
The target output is "1" for a particular class that the corresponding input belongs to and "0" for the remaining 2 outputs.
The problem is as follows:
Your single layer neural network will
find A (3 by 2 matrix) and b (3 by 1
vector) in Y = A*X + b where Y is [C1,
C2, C3]' and X is [x1, x2]'.
To solve the problem above with a
neural network, we can re-write the
equation as follow: Y = A' * X' where
A' = [A b] (3 by 3 matrix) and X' is
[x1, x2, 1]'
Now you can use a neural network with
three input nodes (one for x1, x2, and
1 respectively) and three outputs (C1,
C2, C3).
The resulting 9 (since we have 9
connections between 3 inputs and 3
outputs) weights will be equivalent to
elements of A' matrix.
Basicaly, I am trying to do something like this, but it is not working:
function neuralNetwork
load X_Q2.data
load T_Q2.data
x = X_Q2(:,1);
y = X_Q2(:,2);
learningrate = 0.2;
max_iteration = 50;
% initialize parameters
count = length(x);
weights = rand(1,3); % creates a 1-by-3 array with random weights
globalerror = 0;
iter = 0;
while globalerror ~= 0 && iter <= max_iteration
iter = iter + 1;
globalerror = 0;
for p = 1:count
output = calculateOutput(weights,x(p),y(p));
localerror = T_Q2(p) - output
weights(1)= weights(1) + learningrate *localerror*x(p);
weights(2)= weights(1) + learningrate *localerror*y(p);
weights(3)= weights(1) + learningrate *localerror;
globalerror = globalerror + (localerror*localerror);
end
end
I write this function in some other file and calling it in my previous code.
function result = calculateOutput (weights, x, y)
s = x * weights(1) + y * weights(2) + weights(3);
if s >= 0
result = 1;
else
result = -1;
end

Answer 1

I can spot a few problems with the code. The main issue is that the target is multi-class (not binary), so you need to either use 3 output nodes one for each class (called 1-of-N encoding), or use a single output node with a different activation function (something capable of more than just binary output -1/1 or 0/1)
In the solution below, the perceptron has the following structure:
%# load your data
input = [
0.832 64.643
0.818 78.843
1.776 45.049
0.597 88.302
1.412 63.458
];
target = [
0 0 1
0 0 1
0 1 0
0 0 1
0 0 1
];
%# parameters of the learning algorithm
LEARNING_RATE = 0.1;
MAX_ITERATIONS = 100;
MIN_ERROR = 1e-4;
[numInst numDims] = size(input);
numClasses = size(target,2);
%# three output nodes connected to two-dimensional input nodes + biases
weights = randn(numClasses, numDims+1);
isDone = false; %# termination flag
iter = 0; %# iterations counter
while ~isDone
iter = iter + 1;
%# for each instance
err = zeros(numInst,numClasses);
for i=1:numInst
%# compute output: Y = W*X + b, then apply threshold activation
output = ( weights * [input(i,:)';1] >= 0 ); %#'
%# error: err = T - Y
err(i,:) = target(i,:)' - output; %#'
%# update weights (delta rule): delta(W) = alpha*(T-Y)*X
weights = weights + LEARNING_RATE * err(i,:)' * [input(i,:) 1]; %#'
end
%# Root mean squared error
rmse = sqrt(sum(err.^2,1)/numInst);
fprintf(['Iteration %d: ' repmat('%f ',1,numClasses) '\n'], iter, rmse);
%# termination criteria
if ( iter >= MAX_ITERATIONS || all(rmse < MIN_ERROR) )
isDone = true;
end
end
%# plot points and one-against-all decision boundaries
[~,group] = max(target,[],2); %# actual class of instances
gscatter(input(:,1), input(:,2), group), hold on
xLimits = get(gca,'xlim'); yLimits = get(gca,'ylim');
for i=1:numClasses
ezplot(sprintf('%f*x + %f*y + %f', weights(i,:)), xLimits, yLimits)
end
title('Perceptron decision boundaries')
hold off
The results of training over the five sample you provided:
Iteration 1: 0.447214 0.632456 0.632456
Iteration 2: 0.000000 0.447214 0.447214
...
Iteration 49: 0.000000 0.447214 0.447214
Iteration 50: 0.000000 0.632456 0.000000
Iteration 51: 0.000000 0.447214 0.000000
Iteration 52: 0.000000 0.000000 0.000000
Note that the data used in the example above only contains 5 samples. You would get more meaningful results if you had more training instances in each class.