I'm trying to us matlab ANN for predict T as a function of 5 variables,
T = f(x1,x2,x3,x4,x5)
and i assume that there is a linear function such as :
T = ax1 + bx2 + cx3 + dx4 + ex5
i want to find weight vector [a,b,c,d,e].
each training set is timeseries like:
t1,x11,x21,x31,x41,x51
t2,x12,x22,x32,x42,x52
......
......
......
......
......
tn,x1n,x2n,x3n,x4n,x5n
This training set is obtained from suing weight vector w1 i want to reform this vector so that my variables get close to :
X = [X1,X2,X3,X4,X5] (desire matrix)
exactly this is a control problem, buy using Wi vector i run my system(and control my T with : T = W*x) and grab new x vector(that is time sequence) and add this one to the training data and again run the algorithm.
Related
I refer to the link https://stats.stackexchange.com/questions/105516/how-to-implement-a-2-d-gaussian-processes-regression-through-gpml-matlab and create a 2-d Gaussian Process regression. I want to create a 4-d Gaussian Process regression, however the 'meshgrid' only allows 3 inputs([X,Y,Z] = meshgrid(x,y,z)); how do I add another input into meshgrid?
The 3-d code is like:
X1train = linspace(-4.5,4.5,10);
X2train = linspace(-4.5,4.5,10);
X3train = linspace(-4.5,4.5,10);
X = [X1train' X2train' X3train'];
Y = [X1train + X2train + X3train]';
%Testdata
[Xtest1, Xtest2, Xtest3] = meshgrid(-4.5:0.1:4.5, -4.5:0.1:4.5, -4.5:0.1:4.5);
Xtest = [Xtest1(:) Xtest2(:) Xtest3(:)];
% implement regression
[ymu ys2 fmu fs2] = gp(hyp, #infExact, [], covfunc, likfunc, X, Y, Xtest);
If I create an X4train, that means I need an Xtest4, how do I add Xtest4 into meshgrid?
The GPML code is from http://www.gaussianprocess.org/gpml/code/matlab/doc/
You may create n- dimensional grids using ndgrid, but please keep in mind that it does not directly create the same output as meshgrid, you have to convert it first. (How to do that is also explained in the documentation)
Out of curiosity I am trying to fit neural network with rectified linear units to polynomial functions.
For example, I would like to see how easy (or difficult) it is for a neural network to come up with an approximation for the function f(x) = x^2 + x. The following code should be able to do it, but seems to not learn anything. When I run
using Base.Iterators: repeated
ENV["JULIA_CUDA_SILENT"] = true
using Flux
using Flux: throttle
using Random
f(x) = x^2 + x
x_train = shuffle(1:1000)
y_train = f.(x_train)
x_train = hcat(x_train...)
m = Chain(
Dense(1, 45, relu),
Dense(45, 45, relu),
Dense(45, 1),
softmax
)
function loss(x, y)
Flux.mse(m(x), y)
end
evalcb = () -> #show(loss(x_train, y_train))
opt = ADAM()
#show loss(x_train, y_train)
dataset = repeated((x_train, y_train), 50)
Flux.train!(loss, params(m), dataset, opt, cb = throttle(evalcb, 10))
println("Training finished")
#show m([20])
it returns
loss(x_train, y_train) = 2.0100101f14
loss(x_train, y_train) = 2.0100101f14
loss(x_train, y_train) = 2.0100101f14
Training finished
m([20]) = Float32[1.0]
Anyone here sees how I could make the network fit f(x) = x^2 + x?
There seem to be couple of things wrong with your trial that have mostly to do with how you use your optimizer and treat your input -- nothing wrong with Julia or Flux. Provided solution does learn, but is by no means optimal.
It makes no sense to have softmax output activation on a regression problem. Softmax is used in classification problems where the output(s) of your model represent probabilities and therefore should be on the interval (0,1). It is clear your polynomial has values outside this interval. It is usual to have linear output activation in regression problems like these. This means in Flux no output activation should be defined on the output layer.
The shape of your data matters. train! computes gradients for loss(d...) where d is a batch in your data. In your case a minibatch consists of 1000 samples, and this same batch is repeated 50 times. Neural nets are often trained with smaller batches sizes, but a larger sample set. In the code I provided all batches consist of different data.
For training neural nets, in general, it is advised to normalize your input. Your input takes values from 1 to 1000. My example applies a simple linear transformation to get the input data in the right range.
Normalization can also apply to the output. If the outputs are large, this can result in (too) large gradients and weight updates. Another approach is to lower the learning rate a lot.
using Flux
using Flux: #epochs
using Random
normalize(x) = x/1000
function generate_data(n)
f(x) = x^2 + x
xs = reduce(hcat, rand(n)*1000)
ys = f.(xs)
(normalize(xs), normalize(ys))
end
batch_size = 32
num_batches = 10000
data_train = Iterators.repeated(generate_data(batch_size), num_batches)
data_test = generate_data(100)
model = Chain(Dense(1,40, relu), Dense(40,40, relu), Dense(40, 1))
loss(x,y) = Flux.mse(model(x), y)
opt = ADAM()
ps = Flux.params(model)
Flux.train!(loss, ps, data_train, opt , cb = () -> #show loss(data_test...))
I have trained my Neural network model using MATLAB NN Toolbox. My network has multiple inputs and multiple outputs, 6 and 7 respectively, to be precise. I would like to clarify few questions based on it:-
The final regression plot showed at the end of the training shows a very good accuracy, R~0.99. However, since I have multiple outputs, I am confused as to which scatter plot does it represent? Shouldn't we have 7 target vs predicted plots for each of the output variable?
According to my knowledge, R^2 is a better method of commenting upon the accuracy of the model, whereas MATLAB reports R in its plot. Do I treat that R as R^2 or should I square the reported R value to obtain R^2.
I have generated the Matlab Script containing weight, bias and activation functions, as a final Result of the training. So shouldn't I be able to simply give my raw data as input and obtain the corresponding predicted output. I gave the exact same training set using the indices Matlab chose for training (to cross check), and plotted the predicted output vs actual output, but the result is not at all good. Definitely, not along the lines of R~0.99. Am I doing anything wrong?
code:
function [y1] = myNeuralNetworkFunction_2(x1)
%MYNEURALNETWORKFUNCTION neural network simulation function.
% X = [torque T_exh lambda t_Spark N EGR];
% Y = [O2R CO2R HC NOX CO lambda_out T_exh2];
% Generated by Neural Network Toolbox function genFunction, 17-Dec-2018 07:13:04.
%
% [y1] = myNeuralNetworkFunction(x1) takes these arguments:
% x = Qx6 matrix, input #1
% and returns:
% y = Qx7 matrix, output #1
% where Q is the number of samples.
%#ok<*RPMT0>
% ===== NEURAL NETWORK CONSTANTS =====
% Input 1
x1_step1_xoffset = [-24;235.248;0.75;-20.678;550;0.799];
x1_step1_gain = [0.00353982300884956;0.00284355877067267;6.26959247648903;0.0275865874012055;0.000366568914956012;0.0533831576137729];
x1_step1_ymin = -1;
% Layer 1
b1 = [1.3808996210168685;-2.0990163849711894;0.9651733083552595;0.27000953282929346;-1.6781835509820286;-1.5110463684800366;-3.6257438832309905;2.1569498669085361;1.9204156230460485;-0.17704342477904209];
IW1_1 = [-0.032892214008082517 -0.55848270745152429 -0.0063993424771670616 -0.56161004933654057 2.7161844536020197 0.46415317073346513;-0.21395624254052176 -3.1570133640176681 0.71972178875396853 -1.9132557838515238 1.3365248285282931 -3.022721627052706;-1.1026780445896862 0.2324603066452392 0.14552308208231421 0.79194435276493658 -0.66254679969168417 0.070353201192052434;-0.017994515838487352 -0.097682677816992206 0.68844109281256027 -0.001684535122025588 0.013605622123872989 0.05810686279306107;0.5853667840629273 -2.9560683084876329 0.56713425120259764 -2.1854386350040116 1.2930115031659106 -2.7133159265497957;0.64316656469750333 -0.63667017646313084 0.50060179040086761 -0.86827897068177973 2.695456517458648 0.16822164719859456;-0.44666821007466739 4.0993786464616679 -0.89370838440321498 3.0445073606237933 -3.3015566360833453 -4.492874075961689;1.8337574137485424 2.6946232855369989 1.1140472073136622 1.6167763205944321 1.8573696127039145 -0.81922672766933646;-0.12561950922781362 3.0711045035224349 -0.6535751823440773 2.0590707752473199 -1.3267693770634292 2.8782780742777794;-0.013438026967107483 -0.025741311825949621 0.45460734966889638 0.045052447491038108 -0.21794568374100454 0.10667240367191703];
% Layer 2
b2 = [-0.96846557414356171;-0.2454718918618051;-0.7331628718025488;-1.0225195290982099;0.50307202195645395;-0.49497234988401961;-0.21817117469133171];
LW2_1 = [-0.97716474643411022 -0.23883775971686808 0.99238069915206006 0.4147649511973347 0.48504023209224734 -0.071372217431684551 0.054177719330469304 -0.25963474838320832 0.27368380212104881 0.063159321947246799;-0.15570858147605909 -0.18816739764334323 -0.3793600124951475 2.3851961990944681 0.38355142531334563 -0.75308427071748985 -0.1280128732536128 -1.361052031781103 0.6021878865831336 -0.24725687748503239;0.076251356114485525 -0.10178293627600112 0.10151304376762409 -0.46453434441403058 0.12114876632815359 0.062856969143306296 -0.0019628163322658364 -0.067809039768745916 0.071731544062023825 0.65700427778446913;0.17887084584125315 0.29122649575978238 0.37255802759192702 1.3684190468992126 0.60936238465090853 0.21955911453674043 0.28477957899364675 -0.051456306721251184 0.6519451272106177 -0.64479205028051967;0.25743349663436799 2.0668075180209979 0.59610776847961111 -3.2609682919282603 1.8824214917530881 0.33542869933904396 0.03604272669356564 -0.013842766338427388 3.8534510207741826 2.2266745660915586;-0.16136175574939746 0.10407287099228898 -0.13902245286490234 0.87616472446622717 -0.027079111747601223 0.024812287505204988 -0.030101536834009103 0.043168268669541855 0.12172932035587079 -0.27074383434206573;0.18714562505165402 0.35267726325386606 -0.029241400610813449 0.53053853235049087 0.58880054832728757 0.047959541165126809 0.16152268183097709 0.23419456403348898 0.83166785128608967 -0.66765237856750781];
% Output 1
y1_step1_ymin = -1;
y1_step1_gain = [0.114200879346771;0.145581598485951;0.000139011547272197;0.000456244862967996;2.05816254143146e-05;5.27704485488127;0.00284355877067267];
y1_step1_xoffset = [-0.045;1.122;2.706;17.108;493.726;0.75;235.248];
% ===== SIMULATION ========
% Dimensions
Q = size(x1,1); % samples
% Input 1
x1 = x1';
xp1 = mapminmax_apply(x1,x1_step1_gain,x1_step1_xoffset,x1_step1_ymin);
% Layer 1
a1 = tansig_apply(repmat(b1,1,Q) + IW1_1*xp1);
% Layer 2
a2 = repmat(b2,1,Q) + LW2_1*a1;
% Output 1
y1 = mapminmax_reverse(a2,y1_step1_gain,y1_step1_xoffset,y1_step1_ymin);
y1 = y1';
end
% ===== MODULE FUNCTIONS ========
% Map Minimum and Maximum Input Processing Function
function y = mapminmax_apply(x,settings_gain,settings_xoffset,settings_ymin)
y = bsxfun(#minus,x,settings_xoffset);
y = bsxfun(#times,y,settings_gain);
y = bsxfun(#plus,y,settings_ymin);
end
% Sigmoid Symmetric Transfer Function
function a = tansig_apply(n)
a = 2 ./ (1 + exp(-2*n)) - 1;
end
% Map Minimum and Maximum Output Reverse-Processing Function
function x = mapminmax_reverse(y,settings_gain,settings_xoffset,settings_ymin)
x = bsxfun(#minus,y,settings_ymin);
x = bsxfun(#rdivide,x,settings_gain);
x = bsxfun(#plus,x,settings_xoffset);
end
The above one is the automatically generated code. The plot which I generated to cross-check the first variable is below:-
% X and Y are input and output - same as above
X_train = X(results.info1.train.indices,:);
y_train = Y(results.info1.train.indices,:);
out_train = myNeuralNetworkFunction_2(X_train);
scatter(y_train(:,1),out_train(:,1))
To answer your question about R: Yes, you should square R to get the R^2 value. In this case, they will be very close since R is very close to 1.
The graphs give the correlation between the estimated and real (target) values. So R is the strenght of the correlation. You can square it to find the R-square.
The graph you draw and matlab gave are not the graph of the same variables. The ranges or scales of the axes are very different.
First of all, is the problem you are trying to solve a regression problem? Or is it a classification problem with 7 classes converted to numeric? I assume this is a classification problem, as you are trying to get the success rate for each class.
As for your first question: According to the literature it is recommended to use the value "All: R". If you want to get the success rate of each of your classes, Precision, Recall, F-measure, FP rate, TP Rate, etc., which are valid in classification problems. values you need to reach. There are many matlab documents for this (help ROC) and you can look at the details. All the values I mentioned and which I think you actually want are obtained from the confusion matrix.
There is a good example of this.
[x,t] = simpleclass_dataset;
net = patternnet(10);
net = train(net,x,t);
y = net(x);
[c,cm,ind,per] = confusion(t,y)
I hope you will see what you want from the "nntraintool" window that appears when you run the code.
Your other questions have already been answered. Alternatively, you can consider using a machine learning algorithm with open source software such as Weka.
I am trying to learn how to use neural networks in MATLAB and I am starting with a simple example that uses four data points that I split into two row vectors. One of them is Input and the other is Temp. The input vector is a vector from 1 to 4.
Next I run some neural network coding I found from examples. Now I would like for the neural network to predict the outcome of a sample input vector which is a row vector [5 6].
clear all
clc
Input = [1,2,3,4];
Temp = [.25,.15,.1,.07];
Smpl = [5,6]
net = newff(minmax(Input),[20,1],{'logsig','purelin','trainlm'})
net.trainparam.epochs = 500;
net.trainparam.goal = 1e-25;
net.trainparam.lr = .01;
net = train(net,Input,Temp)
TempPr = net(Input)
error = TempPr - Temp
TempPrSmpl = net(Smpl)
The row vector, TempPr, generated by the neural network exactly matches with the target vector, Temp. However, it seems that I am unable to predict values properly. For example I try to predict temperature values for inputs 5 and 6 which I expect them to be less than .07.
But instead the matlab code is returning:
TempPrSmpl =
0.3560 0.3560
Two questions:
Why is the value being returned from MATLAB greater than .07?
Why are there not two different values being returned from MATLAB (one for 5 and one for 6)?
i want to apply the perceptron algorithm for fisheriris data and i was tried this code
function [ ] = Per( )
%PERCEPTON_NN Summary of this function goes here
% Detailed explanation goes here
%%%%%%%%%%%STEP ONE INPUT DATA PREPERATION
%N=3000;
load fisheriris
tr=50; %traning
te=50; %test
epochs =150;
data=meas;
%N = size(meas,1);
%species=nonomil(species)
%figure,plot(data_shuffeled(1,:),data_shuffeled(2,:),'rx');
%%%%%%%%%%%%%%%%%%%STEP TWO INTIALIZE WEIGHT
baise=1;
w=[baise; 1 ; 1;1 ; 1];
eta=0.9; %%learning rate
%%%%%%%%%%%%%%%%%%%%%%%% Rosen Blatt learning Algo
for epoch=1 : epochs
for i=1 : tr
x=[1;data(i,1);data(i,2);data(i,3);data(i,4)]; % input vector
N = size(species,i); %desiard output
y=w'*x; % y=actual output ,w'= transpose w , mmoken y=w.*x
%%%%%%%%%%%%%%% Activation function (hardlimit)(step fn)
y=1*(y>=0)+(-1)*(y<0); % da badl el if
%%%%%%%%%%% Error calcualtion %%%%%%%
err(i)=N-y;
%%%%%%%%%%%%%% update weight %%%%%%%%%5
wnew=w+ eta*err(i)*x;
w=wnew;
end
mse(epoch)=mean(err.^2);
end
%%%%%%%%%%%%%%%%%%%%%% step four classification (testing) %%%%%%%%%%%%%%%%%%5
hold on
for i=1 : te
%x=[1;data(i,1);data(i,2),data(i,3);data(i,4)];
x=[1;data(i,1);data(i,2);data(i,3);data(i,4)];
% d=data_shuffeled(3,i+tr);
N = size(species,i);
y=w'*x;
y=1*(y>=0)+(-1)*(y<0);
if (y==1)
plot(x(2),x(3),x(4),x(5),'rx');
elseif y==-1
plot(x(2),x(3),x(4),x(5),'r&');
end
end
hold off
if abs(N-y)>1E-6
testerro=testerro+1;
end
i wrote this code to make the perceptron algorithm with fisheriris data "meas" as input and species as "output"
any help in the code or any modify on this code .
Thanks .
First, did you know that MATLAB has something for neural network training called the Neural network toolbox?
Second, think data_shuffeled is your own function. There is something called randperm in MATLAB that you should use to shuffle your data.
Third, you want to avoid using for-loops when you can use vectors/matrices in MATLAB.
Instead of doing (for testing)
for i = 1:te,
....
end
You might want to do
X = [ones(te,1), data]; %X is [50x5] so each row of X is x'
y = X*w; %y is [50x1], X is [50x5], w is [5x1]
idx_p1 = y==1; %all the indices of y where y is +1
idx_m1 = y==-1; %all the indicies of y where y is -1
plot(X(idx_p1,1),y(idx_p1),'rx');
plot(X(idx_m1,1),y(idx_m1),'r&');
I don't know how you were using plot with 4-dimensional X so the above just plots with the first feature (column) of X.
Additionally, the training looks strange to me. For one, I don't think you should use N for both the size of data matrix meas and for the desired output. 'yhat' or 'ybar' is a better name. Also, if N is the desired output, then why is it size(species,i) where i loops through 1:50? species is a [150x1] vector. size(species,1) = 150. And size(species,x) where x is 2 to 50 will be 1. Are you sure you want this? Shouldn't it be something like:
yhat = -ones(50,1); %Everything is -1
yhat(strmatch('virginica,species)) = 1; %except virginicas which are +1