I create and then train a network with following lines:
% here input is 9x543, target is 2x543 and hidden is 6 and output has 2 neurons
net = newff(input,target,{hidden},{'logsig','logsig'},'trainlm');
[net,tr] = train(net,input,target);
After training phase, I simulate my network with following line:
out1 = sim(net,input);
Then I get weights of network with following lines:
iwNet = net.IW{1,1};
lwNet = net.LW{2,1};
b1Net = net.b{1,1};
b2Net = net.b{2,1};
I implement my own network to get predicted output of input values:
% here input is 543x9 and target is 543x2
out2=logsig(logsig(input*iw'+repmat(b1',size(input,1),1))*lw'+repmat(b2',size(input,1),1));
I was expecting to be out1==out2, since I am using same activation functions, weights, and bias. But out2 has very interesting results. (still 1st column is 1 and second column of output is zero)
Did I miss something?
The problem might be pre-processing functions. Check if you included any:
net.inputs{1}.processFcns
Related
I am trying to learn how to use neural networks in MATLAB and I am starting with a simple example that uses four data points that I split into two row vectors. One of them is Input and the other is Temp. The input vector is a vector from 1 to 4.
Next I run some neural network coding I found from examples. Now I would like for the neural network to predict the outcome of a sample input vector which is a row vector [5 6].
clear all
clc
Input = [1,2,3,4];
Temp = [.25,.15,.1,.07];
Smpl = [5,6]
net = newff(minmax(Input),[20,1],{'logsig','purelin','trainlm'})
net.trainparam.epochs = 500;
net.trainparam.goal = 1e-25;
net.trainparam.lr = .01;
net = train(net,Input,Temp)
TempPr = net(Input)
error = TempPr - Temp
TempPrSmpl = net(Smpl)
The row vector, TempPr, generated by the neural network exactly matches with the target vector, Temp. However, it seems that I am unable to predict values properly. For example I try to predict temperature values for inputs 5 and 6 which I expect them to be less than .07.
But instead the matlab code is returning:
TempPrSmpl =
0.3560 0.3560
Two questions:
Why is the value being returned from MATLAB greater than .07?
Why are there not two different values being returned from MATLAB (one for 5 and one for 6)?
Keeping all parameters constant, I get different Mean Average Percentage Errors on my test data on retraining the neural network. Why is this so? Aren't all components of the neural network training process deterministic? Sometimes, I see a difference of up to 1% on successive trainings.
The training code is below
netFeb = newfit(trainX', trainY', networkConfigFeb);
netFeb.performFcn = 'mae';
netFeb = trainlm(netFeb, trainX', trainY');
% Index for the testing Data
startingInd = find(trainInd == 0, 1, 'first');
endingInd = startingInd + daysInMonth('Feb') - 1 ;
% Testing Data
testX = X(startingInd:endingInd,:);
testY = dailyPeakLoad(startingInd:endingInd,:);
actualLoadFeb = testY;
% Calculate the Forcast Load and the Mean Absolute Percentage Error
forecastLoadFeb = sim(netFeb, testX'';
errFeb = testY - forecastLoadFeb;
errpct = abs(errFeb)./testY*100;
MAPEFeb = mean(errpct(~isinf(errpct)));
As A. Donda hinted, since neural networks initialize their weights randomly, they will generate different networks after training. Thus it will give you different performance. While the training process is deterministic, the initial values are not! You may end up in different local minimums as a result or stop in different places.
If you wish to see why, take a look at Why should weights of Neural Networks be initialized to random numbers?
Edit 1:
Notes
Since the user is defining the testing/training data manually, there is no randomization of the training data sets selected
Say I create a neural network to separate classes:
X1; %Some data in Class 1 100x2
X2; %Some data in Class 2 100x2
classInput = [X1;X2];
negative = zeros(N, 1);
positive = ones(N,1);
classTarget = [positive negative; negative positive];
net = feedforwardnet(20);
net = configure(net, classInput, classTarget);
net = train(net, classInput, classTarget);
%output of training data
output = net(classInput);
I can plot the classes and they are correctly separated:
figure();
hold on
style = {'ro' 'bx'};
for i=1:(2*N)
plot(classInput(i,1),classInput(i,2), style{round(output(i,1))+1});
end
However, how can I apply the network that's just been trained to unseen data? There must be a model which is generated by the network that can be applied to new data?
EDIT: Using sim:
Once the network is trained, if I use sim on the training data:
[Z,Xf,Af] = sim(net,classInput);
The result is as expected. But this only works if the input is of the same size. If for example I want to evalute an individual data point:
[Z1,Xf,Af] = sim(net,[1,2]);
size(Z) == size(Z1), but this clearly doesn't make sense? Surely I can evaluate a single data point?
I'm the OP,
I had assumed that the rows of the input matrices were the data samples and the columns were the "categories", this is the other way around. Transposing the matrices before inputting them to the train() function fixes this.
I am trying to do a 5 fold cross validation with libsvm (matlab) using a precomputed kernel, but, I get the following error message :
Undefined function 'ge' for input arguments of type 'struct'.
this is because the Libsvm return a structure instead of a value in cross validation, How can I solve this problem, this is my code:
load('iris.dat')
data=iris(:,1:4);
class=iris(:,5);
% normalize the data
range=repmat((max(data)-min(data)),size(data,1),1);
data=(data-repmat(min(data),size(data,1),1))./range;
% train
tr_data=[data(1:5,:);data(52:56,:);data(101:105,:)];
tr_lbl=[ones(5,1);2*ones(5,1);3*ones(5,1)];
% kernel computation
sigma=.8
rbfKernel = #(X,Y,sigma) exp((-pdist2(X,Y,'euclidean').^2)./(2*sigma^2));
Ktr=[(1:15)',rbfKernel(tr_data,tr_data,sigma)];
kts=[ (1:150)',rbfKernel(data,tr_data,sigma)];
% svmptrain
bestcv = 0;
for log2c = -1:3
cmd = ['Ktr -t 4 -v 5 -c ', num2str(2^log2c)];
cv = svmtrain2(tr_lbl,tr_data, cmd);
if (cv >= bestcv)
bestcv = cv;
bestc = 2^log2c;
end
end
cmd=['-s 0 -c ', num2str(bestc), 'Ktr -t 4']
model=svmtrain2(tr_lbl,tr_data,cmd)
% svm predict
labels=svmpredict(class,data,model,kts)
The function svmtrain2 you are using is not part of standard MATLAB and also the output of the function is not a structure. But if you insist to use that, you can calculate an score for data using the other existing function:
[f,K] = svmeval(X_eval,varargin)
that evaluates the trained svm using the outputs from svmtrain2. But I prefer to use first the standard functions embedded in MATLAB. In standard MATLAB library there is:
SVMStruct = svmtrain(Training,Group)
that returns a structure, SVMStruct, containing information about the trained support vector machine (SVM) classifier. or
SVMModel = fitcsvm(X,Y)
that returns a support vector machine classifier SVMModel, trained by predictors X and class labels Y for one- or two-class classification. and then you can get some score for each prediction using:
[label,Score] = predict(SVMModel,X)
that returns class likelihood measures, i.e., either scores or posterior probabilities.
You get that error because you are trying to compare a struct and a number.
If what you want is to find the best performance in the training set (as it seems from you comparison), I don't think you can get it directly from the structure returned from svmtrain. You should first use svmpredict with the training set and the trained model, and you can get the accuracy from the resulting structure.
I have a input file containing (17x127) data (Force) and a target file of (3x127)(True Stress).
I have written the following code for training the neural networks:
p=Force;
t=T_Stress;
net =newff(minmax(p),[10,1],{'tansig','purelin'},'trainlm');
net.trainParam.lr = .05; %Learning Rate
net.trainParam.epochs = 300; %Max Ephocs
net.trainParam.goal = 1e-5; %Training Goal in Mean Sqared Error
net.trainParam.show = 50; %# of ephocs in display
[net,tr1] = train(net,p,t);
o1 = sim(net,p)
However I get the following errors:
??? Error using ==> trainlm at 109 Output data size does not match
net.outputs{2}.size.
Error in ==> network.train at 107
[net,tr] = feval(net.trainFcn,net,X,T,Xi,Ai,EW,net.trainParam);
How can I resolve this error?
Try to give the training parameters associated with the "trainlm" training function by looking for "trainlm" in Matlab help
By calling newff with the parameter [10,1] you created a neural network with 10 neurons on the hidden layer and 1 on the output layer.
This does not match the number of outputs you are using. (3x127)
So either change the the newff call or use a different training dataset.
As above commented "
By calling newff with the parameter [10,1] you created a neural
network with 10 neurons on the hidden layer and 1 on the output layer
" you keep the value of output layer as 3 as you have three targets. The command will be as follows:
net =newff(minmax(p),[10,3],{'tansig','purelin'},'trainlm');