This question already has answers here:
Why is 24.0000 not equal to 24.0000 in MATLAB?
(6 answers)
Closed 8 years ago.
Here is my whole function:
function val = deceptive_3_nd_function( arr )
alpha = [0.3 0.7];
beta = 0.2;
val = zeros(size(arr,1),1);
for part=1:size(arr,1)
for i=1:size(arr,2)
if 0 <= arr(part, i) && arr(part, i) <= 4/5*alpha(i);
val(part) = val(part) - arr(part, i)/alpha(i) + 4/5;
elseif 4/5*alpha(i) < arr(part, i) && arr(part, i) <= alpha(i)
val(part) = val(part) + 5*arr(part, i)/alpha(i) - 4;
elseif alpha(i) < arr(part, i) && arr(part, i) <= (1+4*alpha(i))/5
* a_ = 5.0 * ( arr(part, i) - alpha(i) );
* b_ = alpha(i)-1.0;
* c_ = a_/b_;
* val(part) = val(part) + c_ + 1.0;
elseif (1+4*alpha(i))/5 < arr(part, i) && arr(part, i) <= 1
val(part) = val(part) + (arr(part, i)-1)/(1-alpha(i)) +4/5;
end
end
val(part) = -(1/size(arr, 2)*val(part))^beta;
end
end
In lines marked with asterisks in got unexpected results. As you can see I tried to isolate a problem and that's where it led me:
K>> arr(part, i)
ans =
0.7600
K>> arr(part, i)==0.76
ans =
1
K>> alpha(i)
ans =
0.7000
K>> alpha(i)==0.7
ans =
1
K>> arr(part, i) - alpha(i)
ans =
0.0600
K>> arr(part, i) - alpha(i) == 0.06
ans =
0
Why is this happening..?
Looks like you've run into floating point error there.
(0.76 - 0.7) == 0.06
ans =
0
num2str(0.76 - 0.7, '%0.20f')
ans =
0.06000000000000005300
Related
Suppose there is a value n input from a user and it goes in to the following for loop code. Is there a way to vectorize the following code?
A = 1:n
B = [1 1;1 1]
for i = 1:n
B = B + A(i)*B;
end
Let's have a look at a specific example:
n = 5;
A = 1:n;
B = [1 1; 1 1];
for i = 1:n
B = B + A(i) * B;
end
B
The result is:
B =
720 720
720 720
First of all, I would re-write the loop:
n = 5;
A = 1:n;
B = [1 1; 1 1];
for i = 1:length(A)
B = B * (A(i) + 1);
end
B
That way, it's more obvious, that your loop variable i simply iterates all elements in A.
Also: B + A(i) * B is the same as B * (A(i) + 1).
Now, we see, that inside the loop, you're basically calculating:
B = B * (A(1) + 1) * (A(2) + 1) * (A(3) + 1) ...
The product over all elements in A (or here: A + 1) can be simplified by using MATLAB's prod function:
n = 5;
A = 1:n;
B = [1 1; 1 1];
B = B * prod(A + 1)
Let's check the result:
B =
720 720
720 720
In that very special case for A = 1:n, the product prod(A + 1) is simply the factorial of n + 1, such that we could also use MATLAB's factorial function:
n = 5;
B = [1 1; 1 1];
B = B * factorial(n + 1)
I am trying to find out the result of the sigmoid function. When I tried to compute it without loop. I did not get the expected result. On the other hand, using for loop getting an actual result.
The question is, Why I am getting different results?
%This is a hypothesis of AND operation
x1 = randi([0,1],[1,4]);
x2 = randi([0,1],[1,4]);
features = [x1;x2]';
theta = [-30 20 20];
x = [ones(length(features),1) features]
z = x * theta';
pred = zeros(length(z),1);
pred = 1 / (1 + exp(-z))
y1 = (pred >= 0.5)
fprintf('Using loop\n')
for i= 1:length(z)
pred(i) = 1 / (1 + exp(-z(i)));
end
pred
y1 = (pred >= 0.5)
Output:
x =
1 1 1
1 0 1
1 0 0
1 0 1
pred =
1.0e-13 *
0 0 0.9358 0
y1 =
0 0 0 0
Using loop
pred =
1.0000 0.0000 0.0000 0.0000
y1 =
1 0 0 0
I am trying to solve following optimization problem:
I am using calculating θ1, θ2 and θ3 for every value of phi discretized between 30 deg. to 150 deg.
function thetas = inverse_kinematics_1(l1,l2,l3,phi)
x = 100;
y = 0;
x1 = x - (l3*cos(phi));
y1 = y - (l3*sin(phi));
a = sqrt(x1^2 + y1^2);
y2 = -y1/a;
x2 = -x1/a;
gamma = atan2(y2,x2);
c = (- x1^2 - y1^2 - l1^2 + l2^2)/(2*l1*a);
d = acos(c);
theta1 = gamma + d;
if theta1 < 0
theta1 = theta1 + 2*pi;
end
e = (y1 - l1*sin(theta1))/l2;
f = (x1 - l1*cos(theta1))/l2;
theta2 = atan2(e,f) - theta1;
if theta2 < 0
theta2 = theta2 + 2*pi;
end
theta3 = (phi)- (theta1 + theta2);
if theta3 < 0
theta3 = theta3 + 2*pi;
end
thetas = [theta1,theta2,theta3].*180/pi;
end
How can I write the constraints in this situation ?
This is my matlab code to predict [1;1;1] given [1;0;1] :
m = 1;
alpha = .00001;
x = [1;0;1;0;0;0;0;0;0];
y = [1;1;1;0;0;0;0;0;0];
theta1 = [4.7300;3.2800;1.4600;0;0;0;4.7300;3.2800;1.4600];
theta1 = theta1 - (alpha/m .* (x .* theta1-y)' * x)'
theta1 = reshape(theta1(1:9) , 3 , 3)
sigmoid(theta1 * [1; 0; 1])
x = [1;0;1;0;0;0;0;0;0];
y = [1; 1; 1;0;0;0;0;0;0];
theta2 = [8.892;6.167;2.745;8.892;6.167;2.745;8.892;6.167;2.745];
theta2 = theta2 - (alpha/m .* (x .* theta2-y)' * x)'
theta2 = reshape(theta2(1:9) , 3 , 3)
sigmoid(theta2 * [1; 0; 1])
x = [1;0;1;0;0;0;0;0;0];
y = [1; 1; 1;0;0;0;0;0;0];
theta3 = [9.446;6.55;2.916;9.351;6.485;2.886;8.836;6.127;2.727];
theta3 = theta3 - (alpha/m .* (x .* theta3-y)' * x)'
theta3 = reshape(theta3(1:9) , 3 , 3)
sigmoid(theta3 * [1; 0; 1])
I'm computing theta1, theta2, theta3 individually but I think they
should be linked between each computation ?
Though gradient descent appears to be working as :
sigmoid(theta1 * [1; 0; 1]) =
0.9999
0.9986
0.9488
sigmoid(theta2 * [1; 0; 1]) =
1.0000
1.0000
0.9959
sigmoid(theta3 * [1; 0; 1]) =
1.0000
1.0000
0.9965
This shows for each theta value (layer in the network) the prediction is moving closer to [1;1;1]
Update : sigmoid function :
function g = sigmoid(z)
g = 1.0 ./ (1.0 + exp(-z));
end
Update2 :
After extended discussion with user davidhigh who provided key insights have made following changes :
x = [1;0;1];
y = [1;1;1];
theta1 =
4.7300 3.2800 1.4600
0 0 0
4.7300 3.2800 1.4600
theta2 =
8.8920 8.8920 8.8920
6.1670 6.1670 6.1670
2.7450 2.7450 2.7450
theta3 =
9.4460 6.5500 2.9160
9.3510 6.4850 2.8860
8.8360 6.1270 2.7270
The crux of my issue is that I don't feed output of each layer into the next layer, once I made this change I get better result :
z1 = sigmoid(theta1 * x)
z1 =
0.9980
0.5000
0.9980
z2 = sigmoid(theta2 * z1)
z2 =
1.0000
1.0000
0.9989
z3 = sigmoid(theta3 * z2)
z3 =
1.0000
1.0000
1.0000
z3 is the predicted value which correctly is [1;1;1;] whereas previously it is approx [1;1;1;]
So, I'm hoping this is a real dumb thing I'm doing, and there's an easy answer. I'm trying to train a 2x3x1 neural network to do the XOR problem. It wasn't working, so I decided to dig in to see what was happening. Finally, I decided to assign the weights my self. This was the weight vector I came up with:
theta1 = [11 0 -5; 0 12 -7;18 17 -20];
theta2 = [14 13 -28 -6];
(In Matlab notation). I deliberately tried to make no two weights be the same (barring the zeros)
And, my code, really simple in matlab is
function layer2 = xornn(iters)
if nargin < 1
iters = 50
end
function s = sigmoid(X)
s = 1.0 ./ (1.0 + exp(-X));
end
T = [0 1 1 0];
X = [0 0 1 1; 0 1 0 1; 1 1 1 1];
theta1 = [11 0 -5; 0 12 -7;18 17 -20];
theta2 = [14 13 -28 -6];
for i = [1:iters]
layer1 = [sigmoid(theta1 * X); 1 1 1 1];
layer2 = sigmoid(theta2 * layer1)
delta2 = T - layer2;
delta1 = layer1 .* (1-layer1) .* (theta2' * delta2);
% remove the bias from delta 1. There's no real point in a delta on the bias.
delta1 = delta1(1:3,:);
theta2d = delta2 * layer1';
theta1d = delta1 * X';
theta1 = theta1 - 0.1 * theta1d;
theta2 = theta2 - 0.1 * theta2d;
end
end
I believe that's right. I tested various parameters (of the thetas) with the finite differences method to see if they were right, and they seemed to be.
But, when I run it, it eventually just all boils down to returning all zeros. If I do xornn(1) (for 1 iteration) I get
0.0027 0.9966 0.9904 0.0008
But, if I do xornn(35)
0.0026 0.9949 0.9572 0.0007
(It's started a descent in the wrong direction) and by the time I get to xornn(45) I get
0.0018 0.0975 0.0000 0.0003
If I run it for 10,000 iterations, it just returns all 0's.
What is going on? Must I add regularization? I would have thought such a simple network wouldn't need it. But, regardless, why does it move away from an obvious good solution that I have hand fed it?
Thanks!
AAARRGGHHH! The solution was simply a matter of changing
theta1 = theta1 - 0.1 * theta1d;
theta2 = theta2 - 0.1 * theta2d;
to
theta1 = theta1 + 0.1 * theta1d;
theta2 = theta2 + 0.1 * theta2d;
sigh
Now tho, I need to figure out how I'm computing the negative derivative somehow when what I thought I was computing was the ... Never mind. I'll post here anyway, just in case it helps someone else.
So, z = is the sum of inputs to the sigmoid, and y is the output of the sigmoid.
C = -(T * Log[y] + (1-T) * Log[(1-y))
dC/dy = -((T/y) - (1-T)/(1-y))
= -((T(1-y)-y(1-T))/(y(1-y)))
= -((T-Ty-y+Ty)/(y(1-y)))
= -((T-y)/(y(1-y)))
= ((y-T)/(y(1-y))) # This is the source of all my woes.
dy/dz = y(1-y)
dC/dz = ((y-T)/(y(1-y))) * y(1-y)
= (y-T)
So, the problem, is that I accidentally was computing T-y, because I forgot about the negative sign in front of the cost function. Then, I was subtracting what I thought was the gradient, but was in fact the negative gradient. And, there. That's the problem.
Once I did that:
function layer2 = xornn(iters)
if nargin < 1
iters = 50
end
function s = sigmoid(X)
s = 1.0 ./ (1.0 + exp(-X));
end
T = [0 1 1 0];
X = [0 0 1 1; 0 1 0 1; 1 1 1 1];
theta1 = [11 0 -5; 0 12 -7;18 17 -20];
theta2 = [14 13 -28 -6];
for i = [1:iters]
layer1 = [sigmoid(theta1 * X); 1 1 1 1];
layer2 = sigmoid(theta2 * layer1)
delta2 = T - layer2;
delta1 = layer1 .* (1-layer1) .* (theta2' * delta2);
% remove the bias from delta 1. There's no real point in a delta on the bias.
delta1 = delta1(1:3,:);
theta2d = delta2 * layer1';
theta1d = delta1 * X';
theta1 = theta1 + 0.1 * theta1d;
theta2 = theta2 + 0.1 * theta2d;
end
end
xornn(50) returns 0.0028 0.9972 0.9948 0.0009 and
xornn(10000) returns 0.0016 0.9989 0.9993 0.0005
Phew! Maybe this will help someone else in debugging their version..