Matlab Neural Network gives unexpected results - neural-network

Question

I was toying around with the matlab neural network toolbox and I encountered some things I did not expect. My problem is especially with a classification network with no hidden layers, only 1 input and the tansig transfer function. So I expect this classifier to divide a 1D dataset at some point that is defined by the learned input weight and bias.
First of all, I thought that the formula for computing the output y for a given input x is:
y = f(x*w + b)
with w the input weight and b the bias. What is the correct formula for calculating the output of the network?
I also expected that translating the whole dataset by a certain value (+77) would have a big effect on the bias and/or weight. But this doesn't seem to be the case. Why does the translation of the dataset has not much effect on the bias and weight?
This is my code:
% Generate data
p1 = randn(1, 1000);
t1(1:1000) = 1;
p2 = 3 + randn(1, 1000);
t2(1:1000) = -1;
% view data
figure, hist([p1', p2'], 100)
P = [p1 p2];
T = [t1 t2];
% train network without hidden layer
net1 = newff(P, T, [], {'tansig'}, 'trainlm');
[net1,tr] = train(net1, P, T);
% display weight and bias
w = net1.IW{1,1};
b = net1.b{1,1};
disp(w) % -6.8971
disp(b) % -0.2280
% make label decision
class_correct = 0;
outputs = net1(P);
for i = 1:length(outputs)
% choose between -1 and 1
if outputs(i) > 0
outputs(i) = 1;
else
outputs(i) = -1;
end
% compare
if T(i) == outputs(i)
class_correct = class_correct + 1;
end
end
% Calculate the correct classification rate (CCR)
CCR = (class_correct * 100) / length(outputs);
fprintf('CCR: %f \n', CCR);
% plot the errors
errors = gsubtract(T, outputs);
figure, plot(errors)
% I expect these to be equal and near 1
net1(0) % 0.9521
tansig(0*w + b) % -0.4680
% I expect these to be equal and near -1
net1(4) % -0.9991
tansig(4*w + b) % -1
% translate the dataset by +77
P2 = P + 77;
% train network without hidden layer
net2 = newff(P2, T, [], {'tansig'}, 'trainlm');
[net2,tr] = train(net2, P2, T);
% display weight and bias
w2 = net2.IW{1,1};
b2 = net2.b{1,1};
disp(w2) % -5.1132
disp(b2) % -0.1556
I generated an artificial dataset that is made of 2 populations with a normal distribution with a different mean. I plotted these populations in a histogram, and trained the network with it.
I calculate the Correct classification rate, which is the percentage of the correct classified instances of the whole dataset. This is somewhere around 92% so I know the classifier works.
But, I expected net1(x) and tansig(x*w + b) to give the same output, this is not the case. What is the correct formula for calculating the output of my trained network?
And I expected net1 and net2 to have different weights and/or bias because net2 is trained on a translated version (+77) of the dataset where net1 is trained on. Why does the translation of the dataset has not much effect on the bias and weight?

Answer 1

First off, your code leaves the default MATLAB input pre-processing intact. You can check this with:
net2.inputs{1}
When I put your code in I got this:
Neural Network Input
name: 'Input'
feedbackOutput: []
processFcns: {'fixunknowns', removeconstantrows,
mapminmax}
processParams: {1x3 cell array of 2 params}
processSettings: {1x3 cell array of 3 settings}
processedRange: [1x2 double]
processedSize: 1
range: [1x2 double]
size: 1
userdata: (your custom info)
The important part being processFncs set to mapminmax. According to the docs, mapminmax will "Process matrices by mapping row minimum and maximum values to [-1 1]". That's why your "shifting" (re-sampling) your inputs arbitrarily had no effect.
I assume that by "calculating the output" you mean checking the performance of your network. You can do that by first simulating your network on a dataset (see doc nnet/sim) and then checking the "correctness" with the perform function. It will use the same cost function as you did when training:
% get predictions
[Y] = sim(net2,T);
% see how well we did
perf = perform(net2,T,Y);
Boomshuckalucka. Hope that helps.