Keeping all parameters constant, I get different Mean Average Percentage Errors on my test data on retraining the neural network. Why is this so? Aren't all components of the neural network training process deterministic? Sometimes, I see a difference of up to 1% on successive trainings.
The training code is below
netFeb = newfit(trainX', trainY', networkConfigFeb);
netFeb.performFcn = 'mae';
netFeb = trainlm(netFeb, trainX', trainY');
% Index for the testing Data
startingInd = find(trainInd == 0, 1, 'first');
endingInd = startingInd + daysInMonth('Feb') - 1 ;
% Testing Data
testX = X(startingInd:endingInd,:);
testY = dailyPeakLoad(startingInd:endingInd,:);
actualLoadFeb = testY;
% Calculate the Forcast Load and the Mean Absolute Percentage Error
forecastLoadFeb = sim(netFeb, testX'';
errFeb = testY - forecastLoadFeb;
errpct = abs(errFeb)./testY*100;
MAPEFeb = mean(errpct(~isinf(errpct)));
As A. Donda hinted, since neural networks initialize their weights randomly, they will generate different networks after training. Thus it will give you different performance. While the training process is deterministic, the initial values are not! You may end up in different local minimums as a result or stop in different places.
If you wish to see why, take a look at Why should weights of Neural Networks be initialized to random numbers?
Edit 1:
Notes
Since the user is defining the testing/training data manually, there is no randomization of the training data sets selected
Related
I am trying to learn how to use neural networks in MATLAB and I am starting with a simple example that uses four data points that I split into two row vectors. One of them is Input and the other is Temp. The input vector is a vector from 1 to 4.
Next I run some neural network coding I found from examples. Now I would like for the neural network to predict the outcome of a sample input vector which is a row vector [5 6].
clear all
clc
Input = [1,2,3,4];
Temp = [.25,.15,.1,.07];
Smpl = [5,6]
net = newff(minmax(Input),[20,1],{'logsig','purelin','trainlm'})
net.trainparam.epochs = 500;
net.trainparam.goal = 1e-25;
net.trainparam.lr = .01;
net = train(net,Input,Temp)
TempPr = net(Input)
error = TempPr - Temp
TempPrSmpl = net(Smpl)
The row vector, TempPr, generated by the neural network exactly matches with the target vector, Temp. However, it seems that I am unable to predict values properly. For example I try to predict temperature values for inputs 5 and 6 which I expect them to be less than .07.
But instead the matlab code is returning:
TempPrSmpl =
0.3560 0.3560
Two questions:
Why is the value being returned from MATLAB greater than .07?
Why are there not two different values being returned from MATLAB (one for 5 and one for 6)?
I am trying to build a neural network and have the following code:
for i = 1:num_samples-num_reserved
% Getting the sample and transposing it so it can be multiplied
sample = new_data_a(:,i)';
% Normalizing the input vector
sample = sample/norm(sample);
% Calculating output
outputs = sample*connections;
% Neuron that fired the hardest's index (I) and its output (output)
[output, I] = max(outputs);
% Conections leading to this neuron
neuron_connections = connections(:,I);
% Looping through input components
for j = 1:num_features
% Value of this input component
component_input = sample(j);
% Updating connection weights
delta = 0.7*(component_input - neuron_connections(j));
neuron_connections(j) = neuron_connections(j) + delta;
end
% Copying new connection weights into original matrix
connections(:,I) = neuron_connections/norm(neuron_connections);
if(rem(i,100) == 0)
if(I == 1)
delta_track = [delta_track connections(2,I)];
end
end
% Storing current connections
end
I think I'm doing everything right. This loop is repeated about 600 times so as to progressively update connections. The function to update weights I am using is the standard one I found in textbooks.
However, when I look at the values stored in delta_track These keep oscillating forming a regular pattern.
Any advice?
You can reduce the feedback factor. Then the network may require more time to learn but is less likely to oscillate.
Another common technique is to add a decay, i.e. reducing the factor each iteration.
In general neural networks have the same stability rules as control systems have (because as long as a NN is learning it is a control system) Therefore similar approaches work as for e.g. PID controllers.
I'm new in machine learning (and to stackoverflow as well) and i want to make some classification tasks. I performed two group classifications on my data set (field of speech acoustics) with LIBSVM and Matlab's Pattern Recignition Tool from the Neural network toolbox to create a simple network with one hidden layer. In the hope of higher classification results i want to try Deep Neural Networks, and i found this code: http://www.mathworks.com/matlabcentral/fileexchange/42853-deep-neural-network
I have some difficulty understanding it.
My data is constructed of 127 samples of 19 parameters, so my input number is 19. I want to classify them in two groups: 0 and 1, so my output number is 1. The values in my data set are normalized between 0 and 1.
My code is the following:
clear all
clc
addpath('..');
load('data.mat')
inputdata = inputs;
outputdata = outputs;
datanum = 127;
outputnum = 1;
hiddennum = 3;
inputnum = 19;
% rbm = randRBM(inputnum, outputnum);
% rbm = pretrainRBM( rbm, inputdata );
dbn = randDBN([inputnum, hiddennum, outputnum]);
dbn = pretrainDBN( dbn, inputdata );
dbn = SetLinearMapping( dbn, inputdata, outputdata );
dbn = trainDBN( dbn, inputdata, outputdata );
estimate = v2h( dbn, inputdata )
[rmse AveErrNum] = CalcRmse(dbn, inputdata, outputdata)
The code runs. The rmse is 0.4183, the AveErrNum is 0.1969. What i need is the classification accuracy between my targets (stored in outputdata) and the networks predictions (Accuracy = data classified correctly / all data).
Where do i find the networks predictions after binarization?
Do I use the right type of network for my classification?
Don't I need to divide my data into Training, Validation and Testing samples (like in the case of a simple neural network with one hidden layer)?
Thanks in advance for any help!
I've created a neural network to model a certain (simple) input-output relationship. When I look at the time-series responses plot using the nntrain gui the predictions seem quite adequate, however, when I try to do out of sample prediction the results are nowhere close to the function being modelled.
I've googled this problem extensively and messed around with my code to no avail, I'd really appreciate a little insight into what I've been doing wrong.
I've included a minimal working example below.
A = 1:1000; B = 10000*sin(A); C = A.^2 +B;
Set = [A' B' C'];
input = Set(:,1:end-1);
target = Set(:,end);
inputSeries = tonndata(input(1:700,:),false,false);
targetSeries = tonndata(target(1:700,:),false,false);
inputSeriesVal = tonndata(input(701:end,:),false,false);
targetSeriesVal = tonndata(target(701:end,:),false,false);
inputDelays = 1:2;
feedbackDelays = 1:2;
hiddenLayerSize = 5;
net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
[inputs,inputStates,layerStates,targets] = preparets(net,inputSeries,{},targetSeries);
net.divideFcn = 'divideblock'; % Divide data in blocks
net.divideMode = 'time'; % Divide up every value
% Train the Network
[net,tr] = train(net,inputs,targets,inputStates,layerStates);
Y = net(inputs,inputStates,layerStates);
% Prediction Attempt
delay=length(inputDelays); N=300;
inputSeriesPred = [inputSeries(end-delay+1:end),inputSeriesVal];
targetSeriesPred = [targetSeries(end-delay+1:end), con2seq(nan(1,N))];
netc = closeloop(net);
[Xs,Xi,Ai,Ts] = preparets(netc,inputSeriesPred,{},targetSeriesPred);
yPred = netc(Xs,Xi,Ai);
perf = perform(net,yPred,targetSeriesVal);
figure;
plot([cell2mat(targetSeries),nan(1,N);
nan(1,length(targetSeries)),cell2mat(yPred);
nan(1,length(targetSeries)),cell2mat(targetSeriesVal)]')
legend('Original Targets','Network Predictions','Expected Outputs')
end
I realise narx net with a time delay is probably overkill for this type of problem but I intend on using this example as a base for a more complicated time-series problem in the future.
Kind regards, James
The most likely causes of poor generalization from the training data to new data is that either (1) there was not enough training data to characterize the problem, or (2) the neural network has more neurons and delays than are needed for the problem so it is overfitting the data (i.e. it is having an easy time memorizing the examples instead of having to figure out how they are related.
The fix for (1) is typically more data. The fix for (2) is to reduce the number of tap delays and/or neurons.
Hope this helps!
I'm not sure if you solved the problem yet. But there is at least one more solution to your problem.
Since you are dealing with a time series it is better (at least in this case) to set net.divideFcn = 'dividerand'. The 'divideblock' will only use the first part of the time series for training which may result in lost information about the long-term trends.
Increase the inputdelay, feedbackdelay and hiddenlayersize as following:
inputDelays = 1:30;
feedbackDelays = 1:3;
hiddenLayerSize = 30;
Also change function as
net.divideFcn = 'dividerand';
this changes work for me even though network take time
I am trying to implement Naive Bayes Classifier using a dataset published by UCI machine learning team. I am new to machine learning and trying to understand techniques to use for my work related problems, so I thought it's better to get the theory understood first.
I am using pima dataset (Link to Data - UCI-ML), and my goal is to build Naive Bayes Univariate Gaussian Classifier for K class problem (Data is only there for K=2). I have done splitting data, and calculate the mean for each class, standard deviation, priors for each class, but after this I am kind of stuck because I am not sure what and how I should be doing after this. I have a feeling that I should be calculating posterior probability,
Here is my code, I am using percent as a vector, because I want to see the behavior as I increase the training data size from 80:20 split. Basically if you pass [10 20 30 40] it will take that percentage from 80:20 split, and use 10% of 80% as training.
function[classMean] = naivebayes(file, iter, percent)
dm = load(file);
for i=1:iter
idx = randperm(size(dm.data,1))
%Using same idx for data and labels
shuffledMatrix_data = dm.data(idx,:);
shuffledMatrix_label = dm.labels(idx,:);
percent_data_80 = round((0.8) * length(shuffledMatrix_data));
%Doing 80-20 split
train = shuffledMatrix_data(1:percent_data_80,:);
test = shuffledMatrix_data(percent_data_80+1:length(shuffledMatrix_data),:);
train_labels = shuffledMatrix_label(1:percent_data_80,:)
test_labels = shuffledMatrix_data(percent_data_80+1:length(shuffledMatrix_data),:);
%Getting the array of percents
for pRows = 1:length(percent)
percentOfRows = round((percent(pRows)/100) * length(train));
new_train = train(1:percentOfRows,:)
new_trin_label = shuffledMatrix_label(1:percentOfRows)
%get unique labels in training
numClasses = size(unique(new_trin_label),1)
classMean = zeros(numClasses,size(new_train,2));
for kclass=1:numClasses
classMean(kclass,:) = mean(new_train(new_trin_label == kclass,:))
std(new_train(new_trin_label == kclass,:))
priorClassforK = length(new_train(new_trin_label == kclass))/length(new_train)
priorClassforK_1 = 1 - priorClassforK
end
end
end
end
First, compute the probability of evey class label based on frequency counts. For a given sample of data and a given class in your data set, you compute the probability of evey feature. After that, multiply the conditional probability for all features in the sample by each other and by the probability of the considered class label. Finally, compare values of all class labels and you choose the label of the class with the maximum probability (Bayes classification rule).
For computing conditonal probability, you can simply use the Normal distribution function.