I am trying to use ODE45 to find the solution to 2 rotating bars, rotating on vertical plane, that have a torsional spring that creates a moment on the bars only when the angle between them differs from 90 degrees. I am just using a1-b4 as the constants in the diffEQ and are just imputing their values into a matrix before sending it into the function. I keep betting back an error saying that I am sending 6 initial conditions, but only get 5 back from the ODE45 function. Any ideas on how to fix this?
%system1.m
function [dx] = system1(t,x,parameters)
dx = zeros(4,1);
a1 = parameters(1);
a2 = parameters(2);
a3 = parameters(3);
a4 = parameters(4);
b1 = parameters(5);
b2 = parameters(6);
b3 = parameters(7);
b4 = parameters(8);
dx(1) = x(2); %dtheta1 = angular velocity1
dx(2) = x(3); %d(angular velocity1) = angular acceleration1
dx(4) = x(5); %dtheta2 = angular velocity2
dx(5) = x(6); %d(angular velocity2) = angular acceleration2
dx(2) = a1*x(1)+a2*x(4)+a3*x(2)+a4*x(5); %motion equation 1
dx(5) = b1*x(1)+b2*x(4)+b3*x(2)+b4*x(5); %motion equation 2
%CA2Lou.m
%set parameters
clear;
a1 = -12;
a2 = 12;
a3 = 0;
a4 = 0;
b1 = 4;
b2 = -4;
b3 = 0;
b4 = 0;
parameters = [a1 a2 a3 a4 b1 b2 b3 b4];
%set final time
tf = .5;
options = odeset('MaxStep',.05);
%setting initial conditions
InitialConditions = [90 0 0 0 0 0];
[t_sol,x_sol] = ode45(#system1,[0 tf],InitialConditions,[],parameters);
Your size and indexing for dx don't match x. You initialize dx to 4 elements, even though x has 6. Then you assign values to 4 indices of dx (specifically, [1 2 4 5]) which results in a new size for dx of 5 elements, still one less than the 6 it expects.
You probably need to initialize dx like so:
dx = zeros(6, 1);
Then, your first and second motion equations should probably (I'm guessing) be placed in indices 3 and 6:
dx(3) = a1*x(1)+a2*x(4)+a3*x(2)+a4*x(5); %motion equation 1
dx(6) = b1*x(1)+b2*x(4)+b3*x(2)+b4*x(5); %motion equation 2
Related
I'm having trouble on an easy exercise about an artificial neural network with 2 features, a hidden layer of 5 neurons and two possible outputs (0 or 1).
My X matrix is a 51x2 matrix, and y is a 51x1 vector.
I know I'm not supposed to do the while E>1 but I wanted to see if eventually my error would be lower than 1
I'd like to know what I am doing wrong. My error doesn't seem to lower (around 1.5 no matter how much iterations I'm doing). Do you see in the code where I am doing a mistake? I'm supposed to use gradient descent.
function [E, v,w] = costFunction(X, y,alpha1,alpha2)
[m n] = size(X);
E = 1;
v = 2*rand(5,3)-1;
w = 2*rand(2,6)-1;
grad_v=zeros(size(v));
grad_w=zeros(size(w));
K = 2;
E = 2;
while E> 1
a1 = [ones(m,1) X];
z2 = a1 * v';
a2 = sigmoid(z2);
a2 = [ones(size(a2,1),1),a2];
z3 = a2 * w';
h = sigmoid(z3);
cost = sum((-y.*log(h)) - ((1-y).*log(1-h)),2);
E = (1/m)*sum(cost);
Delta1=0;
Delta2=0;
for t = 1:m
a1 = [1;X(t,:)'];
z2 = v * a1;
a2 = sigmoid(z2);
a2 = [1;a2];
z3 = w * a2;
a3 = sigmoid(z3);
d3 = a3 - y(t,:)';
d2 = (w(:,2:end)'*d3).*sigmoidGradient(z2);
Delta2 += (d3*a2');
Delta1 += (d2*a1');
end
grad_v = (1/m) * Delta1;
grad_w = (1/m) * Delta2;
v -= alpha1 * grad_v;
w -= alpha2 * grad_w;
end
end
I was trying to build a 5-layer neural network to classify a 3 classes, 178 instances and 13 features dataset. Basically I was following the guideline given here. I have written down my own code in Matlab and it can successfully run. However, the training result turns out to be very bad. The model keep predict the same class as the output. I could not found where is wrong with my code, or the model doesn't fit for the data? Could someone help me find where the problem is? Thank you very much.
My Matlab training code is shown below:
%% Initialization
numclass = 3; % num of class
c = 13; % num of feature
% for each layer, initialize each parameter w
and each b to a small random value near zero
w1 = normrnd(0,0.01,[c,10]); % Input layer -> layer 2 (10 nodes)
b1 = normrnd(0,0.01,[1,10]);
w2 = normrnd(0,0.01,[10,6]); % layer 2 -> layer 3 (6 nodes)
b2 = normrnd(0,0.01,[1,6]);
w3 = normrnd(0,0.01,[6,4]); % layer 3 -> layer 4 (4 nodes)
b3 = normrnd(0,0.01,[1,4]);
w4 = normrnd(0,0.01,[4,numclass]); % layer 4 -> Output layer (3 nodes/class label)
b4 = normrnd(0,0.01,[1,numclass]);
Iter = 0;
lambda = 0.5; % regularization coefficient
%% Batch Training
while Iter < 200
Iter = Iter+1
d_w1 = 0; d_w2 = 0; d_w3 = 0; d_b1 = 0; d_b2 = 0; d_b3 = 0;
d_w4 = 0; d_b4 = 0;
for i = 1:r
% Forward propagation
a1 = X0(i,:); % X0 is training data, each row represents a instance with 13 features
% Input layer -> Layer 2
z2 = a1*w1+b1;
a2 = sigmoid(z2);
% Layer 2 -> Layer 3
z3 = a2*w2+b2;
a3 = sigmoid(z3);
% Layer 3 -> Layer 4
z4 = a3*w3+b3;
a4 = sigmoid(z4);
% Layer 4 -> Output Layer
z5 = a4*w4+b4;
a5 = sigmoid(z5);
% Backward propagation
y = zeros(1,numclass);
y(Y0(i)) = 1; % Y0 is the training label ({1,2,3} in this case), each element indicates which class the instance belongs to
% Output Layer -> Layer 4
delta5 = (-(y-a5).*d_sigmoid(z5))';
% Output Layer -> Layer 3
delta4 = (w4*delta5).*d_sigmoid(z4');
% Layer 3 -> Layer 2
delta3 = (w3*delta4).*d_sigmoid(z3');
% Layer 2 -> Layer I
delta2 = (w2*delta3).*d_sigmoid(z2');
% Compute the desired partial derivatives
d_w1 = d_w1 + (delta2*a1)';
d_b1 = d_b1 + delta2';
d_w2 = d_w2 + (delta3*a2)';
d_b2 = d_b2 + delta3';
d_w3 = d_w3 + (delta4*a3)';
d_b3 = d_b3 + delta4';
d_w4 = d_w4 + (delta5*a4)';
d_b4 = d_b4 + delta5';
end
eta = 0.8; % leraning rate
% weights and bias updating
w1 = w1 - eta*((1/r*d_w1)+ lambda*w1);
b1 = b1 - eta*1/r*d_b1;
w2 = w2 - eta*((1/r*d_w2)+ lambda*w2);
b2 = b2 - eta*1/r*d_b2;
w3 = w3 - eta*((1/r*d_w3)+ lambda*w3);
b3 = b3 - eta*1/r*d_b3;
w4 = w4 - eta*((1/r*d_w4)+ lambda*w4);
b4 = b4 - eta*1/r*d_b4;
end
sigmoid and d_sigmoid function are shown below:
function y = sigmoid(x);
L=1;
k=10;
x0=0;
y = L./(1+exp(-k*(x-x0)));
end
function y = d_sigmoid(x)
tmp = sigmoid(x);
y = tmp.*(1-tmp);
end
The prediction code is shown below:
%% Prediction: X1 is testing data, and Y1 is a vector of testing label
[r,c] = size(X1);
for i = 1:r
A1 = X1(i,:);
% Input layer -> Layer 2
Z2 = A1*w1+b1;
A2 = sigmoid(Z2);
% Layer 2 -> Layer 3
Z3 = A2*w2+b2;
A3 = sigmoid(Z3);
% Layer 3 -> Layer 4
Z4 = A3*w3+b3;
A4 = sigmoid(Z4);
% Layer 4 -> Output Layer
Z5 = A4*w4+b4;
A5 = sigmoid(Z5);
pred(i) = find(A5==max(A5))
end
error = length(find((pred'-Y1)~=0))
can anybody help me?I cant show the legend lines different colors.How can I do it?
a1 = 0; b1 = 4;
a2 = 4; b2 = 10;
a3 = 6; b3 = 20;
x1=a1:.01:b1;
x2=a2:.01:b2;
x3=a3:.01:b3;
f1 = 1 ./ (b1 - a1);
f2 = 1 ./ (b2 - a2);
f3 = 1 ./ (b3 - a3);
plot(x1,f1,'r',x2,f2,'b',x3,f3,'y');
grid
xlabel('0 < x < 7 , 0.01 örnek aralığında') % x ekseni başlığı
ylabel('Üstel dağılımın olasılık yoğunluk fonksiyonu') % y ekseni başlığı
legend('s1','s2','s3')
You are plotting a vector x1, x2, x3 against a scalar f1, f2, f3. From the documentation for plot():
If one of X or Y is a scalar and the other is a vector, then the plot
function plots the vector as discrete points at the scalar value.
Each data point in your vector is plotted against the corresponding f value as a separate lineseries, giving you 2403 separate line series. In your legend call you add legend strings for the first 3 line series, which are all going to be red since the first 401 lineseries are red. If your desire is to plot a horizontal line you can create vectors from your f variables using repmat()
Using your example:
a1 = 0; b1 = 4;
a2 = 4; b2 = 10;
a3 = 6; b3 = 20;
x1=a1:.01:b1;
x2=a2:.01:b2;
x3=a3:.01:b3;
f1 = repmat((1 ./ (b1 - a1)), size(x1));
f2 = repmat((1 ./ (b2 - a2)), size(x2));
f3 = repmat((1 ./ (b3 - a3)), size(x3));
plot(x1,f1,'r',x2,f2,'b',x3,f3,'y');
grid
xlabel('0 < x < 7 , 0.01 örnek aralığında') % x ekseni başlığı
ylabel('Üstel dağılımın olasılık yoğunluk fonksiyonu') % y ekseni başlığı
legend('s1','s2','s3')
I need to plot parameterized solutions for the following systems of equations using t values from 0 to 0.3 incremented by 0.001 each time:
x′ = 12.3 x − 2.7 y
y′ = 5.7 x − 3.7 y
This is what I have so far, but I'm pretty sure my parametric curves are wrong. I'd be expecting some exponential looking thing, not a lot of straight lines. What am I doing wrong?
A = [ 12.3, -2.7; 5.7, -3.7 ]; %initial matrix
[P D] = eig(A); %finding eigenvalues and eigenvectors
i = [1;4.3]; %initial conditions
H = inv(P)*i; %solving for c1 and c2
t = 0:0.001:0.3;
c1 = 0.2580; %constant
c2 = 4.2761; %constant
B1 = [0.9346;0.3558]; %eigenvector
B2 = [0.1775;0.9841]; %eigenvector
a = 11.2721; %eigenvalue
b = -2.6721; %eigenvalue
x1 = c1*B1*exp(a*t) + c2*B1*exp(b.*t);
x2 = c1*B2*exp(a*t) + c2*B2*exp(b.*t);
plot(x1,x2);
Your problem was calculating x1 and x2. Since B1 and B2 are vectors, doing this:
x1 = c1*B1*exp(a*t) + c2*B1*exp(b.*t);
x2 = c1*B2*exp(a*t) + c2*B2*exp(b.*t);
made x1 and x2 2 by 301 matrices.
The correct result is simpler:
x = c1*B1*exp(a*t) + c2*B2*exp(b*t);
and plotting it gives:
plot(x(1,:),x(2,:));
I asked a question a few days before but I guess it was a little too complicated and I don't expect to get any answer.
My problem is that I need to use ANN for classification. I've read that much better cost function (or loss function as some books specify) is the cross-entropy, that is J(w) = -1/m * sum_i( yi*ln(hw(xi)) + (1-yi)*ln(1 - hw(xi)) ); i indicates the no. data from training matrix X. I tried to apply it in MATLAB but I find it really difficult. There are couple things I don't know:
should I sum each outputs given all training data (i = 1, ... N, where N is number of inputs for training)
is the gradient calculated correctly
is the numerical gradient (gradAapprox) calculated correctly.
I have following MATLAB codes. I realise I may ask for trivial thing but anyway I hope someone can give me some clues how to find the problem. I suspect the problem is to calculate gradients.
Many thanks.
Main script:
close all
clear all
L = #(x) (1 + exp(-x)).^(-1);
NN = #(x,theta) theta{2}*[ones(1,size(x,1));L(theta{1}*[ones(size(x,1),1) x]')];
% theta = [10 -30 -30];
x = [0 0; 0 1; 1 0; 1 1];
y = [0.9 0.1 0.1 0.1]';
theta0 = 2*rand(9,1)-1;
options = optimset('gradObj','on','Display','iter');
thetaVec = fminunc(#costFunction,theta0,options,x,y);
theta = cell(2,1);
theta{1} = reshape(thetaVec(1:6),[2 3]);
theta{2} = reshape(thetaVec(7:9),[1 3]);
NN(x,theta)'
Cost function:
function [jVal,gradVal,gradApprox] = costFunction(thetaVec,x,y)
persistent index;
% 1 x x
% 1 x x
% 1 x x
% x = 1 x x
% 1 x x
% 1 x x
% 1 x x
m = size(x,1);
if isempty(index) || index > size(x,1)
index = 1;
end
L = #(x) (1 + exp(-x)).^(-1);
NN = #(x,theta) theta{2}*[ones(1,size(x,1));L(theta{1}*[ones(size(x,1),1) x]')];
theta = cell(2,1);
theta{1} = reshape(thetaVec(1:6),[2 3]);
theta{2} = reshape(thetaVec(7:9),[1 3]);
Dew = cell(2,1);
DewApprox = cell(2,1);
% Forward propagation
a0 = x(index,:)';
z1 = theta{1}*[1;a0];
a1 = L(z1);
z2 = theta{2}*[1;a1];
a2 = L(z2);
% Back propagation
d2 = 1/m*(a2 - y(index))*L(z2)*(1-L(z2));
Dew{2} = [1;a1]*d2;
d1 = [1;a1].*(1 - [1;a1]).*theta{2}'*d2;
Dew{1} = [1;a0]*d1(2:end)';
% NNRes = NN(x,theta)';
% jVal = -1/m*sum(NNRes-y)*NNRes*(1-NNRes);
jVal = -1/m*(a2 - y(index))*a2*(1-a2);
gradVal = [Dew{1}(:);Dew{2}(:)];
gradApprox = CalcGradApprox(0.0001);
index = index + 1;
function output = CalcGradApprox(epsilon)
output = zeros(size(gradVal));
for n=1:length(thetaVec)
thetaVecMin = thetaVec;
thetaVecMax = thetaVec;
thetaVecMin(n) = thetaVec(n) - epsilon;
thetaVecMax(n) = thetaVec(n) + epsilon;
thetaMin = cell(2,1);
thetaMax = cell(2,1);
thetaMin{1} = reshape(thetaVecMin(1:6),[2 3]);
thetaMin{2} = reshape(thetaVecMin(7:9),[1 3]);
thetaMax{1} = reshape(thetaVecMax(1:6),[2 3]);
thetaMax{2} = reshape(thetaVecMax(7:9),[1 3]);
a2min = NN(x(index,:),thetaMin)';
a2max = NN(x(index,:),thetaMax)';
jValMin = -1/m*(a2min-y(index))*a2min*(1-a2min);
jValMax = -1/m*(a2max-y(index))*a2max*(1-a2max);
output(n) = (jValMax - jValMin)/2/epsilon;
end
end
end
EDIT:
Below I present the correct version of my costFunction for those who may be interested.
function [jVal,gradVal,gradApprox] = costFunction(thetaVec,x,y)
m = size(x,1);
L = #(x) (1 + exp(-x)).^(-1);
NN = #(x,theta) L(theta{2}*[ones(1,size(x,1));L(theta{1}*[ones(size(x,1),1) x]')]);
theta = cell(2,1);
theta{1} = reshape(thetaVec(1:6),[2 3]);
theta{2} = reshape(thetaVec(7:9),[1 3]);
Delta = cell(2,1);
Delta{1} = zeros(size(theta{1}));
Delta{2} = zeros(size(theta{2}));
D = cell(2,1);
D{1} = zeros(size(theta{1}));
D{2} = zeros(size(theta{2}));
jVal = 0;
for in = 1:size(x,1)
% Forward propagation
a1 = [1;x(in,:)']; % added bias to a0
z2 = theta{1}*a1;
a2 = [1;L(z2)]; % added bias to a1
z3 = theta{2}*a2;
a3 = L(z3);
% Back propagation
d3 = a3 - y(in);
d2 = theta{2}'*d3.*a2.*(1 - a2);
Delta{2} = Delta{2} + d3*a2';
Delta{1} = Delta{1} + d2(2:end)*a1';
jVal = jVal + sum( y(in)*log(a3) + (1-y(in))*log(1-a3) );
end
D{1} = 1/m*Delta{1};
D{2} = 1/m*Delta{2};
jVal = -1/m*jVal;
gradVal = [D{1}(:);D{2}(:)];
gradApprox = CalcGradApprox(x(in,:),0.0001);
% Nested function to calculate gradApprox
function output = CalcGradApprox(x,epsilon)
output = zeros(size(thetaVec));
for n=1:length(thetaVec)
thetaVecMin = thetaVec;
thetaVecMax = thetaVec;
thetaVecMin(n) = thetaVec(n) - epsilon;
thetaVecMax(n) = thetaVec(n) + epsilon;
thetaMin = cell(2,1);
thetaMax = cell(2,1);
thetaMin{1} = reshape(thetaVecMin(1:6),[2 3]);
thetaMin{2} = reshape(thetaVecMin(7:9),[1 3]);
thetaMax{1} = reshape(thetaVecMax(1:6),[2 3]);
thetaMax{2} = reshape(thetaVecMax(7:9),[1 3]);
a3min = NN(x,thetaMin)';
a3max = NN(x,thetaMax)';
jValMin = 0;
jValMax = 0;
for inn=1:size(x,1)
jValMin = jValMin + sum( y(inn)*log(a3min) + (1-y(inn))*log(1-a3min) );
jValMax = jValMax + sum( y(inn)*log(a3max) + (1-y(inn))*log(1-a3max) );
end
jValMin = 1/m*jValMin;
jValMax = 1/m*jValMax;
output(n) = (jValMax - jValMin)/2/epsilon;
end
end
end
I've only had a quick eyeball over your code. Here are some pointers.
Q1
should I sum each outputs given all training data (i = 1, ... N, where
N is number of inputs for training)
If you are talking in relation to the cost function, it is normal to sum and normalise by the number of training examples in order to provide comparison between.
I can't tell from the code whether you have a vectorised implementation which will change the answer. Note that the sum function will only sum up a single dimension at a time - meaning if you have a (M by N) array, sum will result in a 1 by N array.
The cost function should have a scalar output.
Q2
is the gradient calculated correctly
The gradient is not calculated correctly - specifically the deltas look wrong. Try following Andrew Ng's notes [PDF] they are very good.
Q3
is the numerical gradient (gradAapprox) calculated correctly.
This line looks a bit suspect. Does this make more sense?
output(n) = (jValMax - jValMin)/(2*epsilon);
EDIT: I actually can't make heads or tails of your gradient approximation. You should only use forward propagation and small tweaks in the parameters to compute the gradient. Good luck!