In Matlab I want to create the partial derivative of a cost function called J(theta_0, theta_1) (in order to do the calculations necessary to do gradient descent).
The function J(theta_0, theta_1) is defined as:
Lets say h_theta(x) = theta_1 + theta_2*x. Also: alpha is fixed, the starting values of theta_1 and theta_2 are given. Let's say in this example: alpha = 0.1 theta_1 = 0, theta_2 = 1. Also I have all the values for x and y in two different vectors.
VectorOfX =
5
5
6
VectorOfX =
6
6
10
Steps I took to try to solve this in Matlab: I have no clue how to solve this problem in matlab. So I started off with trying to define a function in Matlab and tried this:
theta_1 = 0
theta_2 = 1
syms x;
h_theta(x) = theta_1 + t2*x;
This worked, but is not what I really wanted. I wanted to get x^(i), which is in a vector. The next thing I tried was:
theta_1 = 0
theta_2 = 1
syms x;
h_theta(x) = theta_1 + t2*vectorOfX(1);
This gives the following error:
Error using sym/subsindex (line 672)
Invalid indexing or function definition. When defining a
function, ensure that the body of the function is a SYM
object. When indexing, the input must be numeric, logical or
':'.
Error in prog1>gradientDescent (line 46)
h_theta(x) = theta_1 + theta_2*vectorOfX(x);
I looked up this error and don't know how to solve it for this particular example. I have the feeling that I make matlab work against me instead of using it in my favor.
When I have to perform symbolic computations I prefer to use Mathematica. In that environment this is the code to get the partial derivatives you are looking for.
J[th1_, th2_, m_] := Sum[(th1 + th2*Subscript[x, i] - Subscript[y, i])^2, {i, 1, m}]/(2*m)
D[J[th1, th2, m], th1]
D[J[th1, th2, m], th2]
and gives
Coming back to MATLAB we can solve this problem with the following code
%// Constants.
alpha = 0.1;
theta_1 = 0;
theta_2 = 1;
X = [5 ; 5 ; 6];
Y = [6 ; 6 ; 10];
%// Number of points.
m = length(X);
%// Partial derivatives.
Dtheta1 = #(theta_1, theta_2) sum(2*(theta_1+theta_2*X-Y))/2/m;
Dtheta2 = #(theta_1, theta_2) sum(2*X.*(theta_1+theta_2*X-Y))/2/m;
%// Loop initialization.
toll = 1e-5;
maxIter = 100;
it = 0;
err = 1;
theta_1_Last = theta_1;
theta_2_Last = theta_2;
%// Iterations.
while err>toll && it<maxIter
theta_1 = theta_1 - alpha*Dtheta1(theta_1, theta_2);
theta_2 = theta_2 - alpha*Dtheta2(theta_1, theta_2);
it = it + 1;
err = norm([theta_1-theta_1_Last ; theta_2-theta_2_Last]);
theta_1_Last = theta_1;
theta_2_Last = theta_2;
end
Unfortunately for this case the iterations does not converge.
MATLAB is not very flexible for symbolic computations, however a way to get those partial derivatives is the following
m = 10;
syms th1 th2
x = sym('x', [m 1]);
y = sym('y', [m 1]);
J = #(th1, th2) sum((th1+th2.*x-y).^2)/2/m;
diff(J, th1)
diff(J, th2)
Related
I'm having some issues getting my RK2 algorithm to work for a certain second-order linear differential equation. I have posted my current code (with the provided parameters) below. For some reason, the value of y1 deviates from the true value by a wider margin each iteration. Any input would be greatly appreciated. Thanks!
Code:
f = #(x,y1,y2) [y2; (1+y2)/x];
a = 1;
b = 2;
alpha = 0;
beta = 1;
n = 21;
h = (b-a)/(n-1);
yexact = #(x) 2*log(x)/log(2) - x +1;
ye = yexact((a:h:b)');
s = (beta - alpha)/(b - a);
y0 = [alpha;s];
[y1, y2] = RungeKuttaTwo2D(f, a, b, h, y0);
error = abs(ye - y1);
function [y1, y2] = RungeKuttaTwo2D(f, a, b, h, y0)
n = floor((b-a)/h);
y1 = zeros(n+1,1); y2 = y1;
y1(1) = y0(1); y2(1) = y0(2);
for i=1:n-1
ti = a+(i-1)*h;
fvalue1 = f(ti,y1(i),y2(i));
k1 = h*fvalue1;
fvalue2 = f(ti+h/2,y1(i)+k1(1)/2,y2(i)+k1(2)/2);
k2 = h*fvalue2;
y1(i+1) = y1(i) + k2(1);
y2(i+1) = y2(i) + k2(2);
end
end
Your exact solution is wrong. It is possible that your differential equation is missing a minus sign.
y2'=(1+y2)/x has as its solution y2(x)=C*x-1 and as y1'=y2 then y1(x)=0.5*C*x^2-x+D.
If the sign in the y2 equation were flipped, y2'=-(1+y2)/x, one would get y2(x)=C/x-1 with integral y1(x)=C*log(x)-x+D, which contains the given exact solution.
0=y1(1) = -1+D ==> D=1
1=y1(2) = C*log(2)-1 == C=1/log(2)
Additionally, the arrays in the integration loop have length n+1, so that the loop has to be from i=1 to n. Else the last element remains zero, which gives wrong residuals for the second boundary condition.
Correcting that and enlarging the computation to one secant step finds the correct solution for the discretization, as the ODE is linear. The error to the exact solution is bounded by 0.000285, which is reasonable for a second order method with step size 0.05.
I am trying to use the fixed point iteration method with initial approximation x(1)=0 to obtain an approximation to the root of the equation f(x)=3x+sin(x)e^x=0.
The stopping criterion is
|x(k+1)-x(k)|<0.0001
x(1) = 0;
n = 100;
for k = 1:n
f(k) = 3*x(k) +sin(x(k))-exp(x(k));
if (abs(f(k))<0.0001)
break;
end
syms x
diff(f(k));
x(k+1) = x(1)- (f(k))/(diff(f(k)));
end
[x' f']
This is the error I am getting: Error using / Matrix dimensions must
agree. Error in prac2Q2 (line 15)
x(k+1) = x(1)- (f(k))/(diff(f(k)));
I would suggest to calculate the derivative by hand and use that term as denominator or to save the derivative in another variable and use this as the denominator.
Derivative as Variable
f(k) = ...;
df(k) = diff(f(k));
x(k+1) = x(k) - f(k) / df(k);
PS: I cannot test this, because I do not have access to the Symbolic Toolbox right now.
If you're looking for the root of 3*x +sin(x)-exp(x) you want to resolve this equation:
3*x + sin(x) - exp(x) = 0
The easiest way will be to isolate x in one side of the equation:
x = (exp(x) - sin(x))/3 % now iterate until x = (exp(x) - sin(x))/3
Now I would recommand to use an easier fixed point method: x(k+1) = (x(k)+f(x(k)))/2
x = 1 % x0
while 1
y = (exp(x)-sin(x))/3; % we are looking for the root not for a fixed point !!! y = f(x)
x = (x+y)/2 % after a few iterations x == y, so x = (x+y)/2 or x = 2x/2
if abs(x-y) < 1e-10
break
end
end
And you obtain the correct result:
x = 0.36042
No need of symbolic math.
As you probably guessed from the title, I'm attempting to do tridiagonal GaussJordan elimination. I'm trying to do it without the default solver. My answers aren't coming out correct and I need some assistance as to where the error is in my code.
I'm getting different values for A/b and x, using the code I have.
n = 4;
#Range for diagonals
ranged = [15 20];
rangesd = [1 5];
#Vectors for tridiagonal matrix
supd = randi(rangesd,[1,n-1]);
d = randi(ranged,[1,n]);
subd = randi(rangesd,[1,n-1]);
#Creates system Ax+b
A = diag(supd,1) + diag(d,0) + diag(subd,-1)
b = randi(10,[1,n])
#Uses default solver
y = A/b
function x = naive_gauss(A,b);
#Forward elimination
for k=1:n-1
for i=k+1:n
xmult = A(i,k)/A(k,k);
for j=k+1:n
A(i,j) = A(i,j)-xmult*A(k,j);
end
b(i) = b(i)-xmult*b(k);
end
end
#Backwards elimination
x(n) = b(n)/A(n,n);
for i=n-1:-1:1
sum = b(i);
for j=i+1:n
sum = sum-A(i,j)*x(j);
end
x(i) = sum/A(i,i)
end
end
x
Your algorithm is correct. The value of y that you compare against is wrong.
you have y=A/b, but the correct syntax to get the solution of the system should be y=A\b.
Consider a difference equation with its initial conditions.
5y(n) + y(n-1) - 3y(n-2) = (1/5^n) u(n), n>=0
y(n-1) = 2, y(n-2) = 0
How can I determine y(n) in Matlab?
Use an approach similar to this (using filter), but specifying initial conditions as done here (using filtic).
I'm assuming your initial conditions are: y(-1)=2, y(-2)=0.
num = 1; %// numerator of transfer function (from difference equation)
den = [5 1 -3]; %// denominator of transfer function (from difference equation)
n = 0:100; %// choose as desired
x = (1/5).^n; %// n is >= 0, so u(n) is 1
y = filter(num, den, x, filtic(num, den, [2 0], [0 0]));
%// [2 0] reflects initial conditions on y, and [0 0] those on x.
Here's a plot of the result, obtained with stem(n,y).
The second line of your code does not give initial conditions, because it refers to the index variable n. Since Matlab only allows positive integer indices, I'll assume that you mean y(1) = 0 and y(2) = 2.
You can get an iteration rule out of your first equation by simple algebra:
y(n) = ( (1/5^n) u(n) - y(n-1) + 3y(n-2) ) / 5
Code to apply this rule in Matlab:
n_max = 100;
y = nan(n_max, 1);
y(1) = 0;
y(2) = 2;
for n = 3 : n_max
y(n) = ( (1/5^n) * u(n) - y(n-1) + 3 * y(n-2) ) / 5;
end
This code assumes that the array u is already defined. n_max specifies how many elements of y to compute.
If I'm to integrate a function
y = -((F+h)M^3(cosh(h*M)+M*beta*sinh(h*M)))/(h*M*cosh(h*M)+(-1+h*M^2*beta)*sinh(h*M))- (alpha*(M^2*(F+h)*(-1+2*h^2*M^2+ cosh(2*h*M)-2*h*M*sinh(2*h*M)))/(8*(h*M*cosh(h*M)+(-1+h*M^2*beta)*sinh(h*M))^2));
with respect to x, where
phi = 0.6;
x = 0.5;
M = 2;
theta = -1:0.5:1.5;
F = theta - 1;
h = 1 + phi*cos(2*pi*x);
alpha = 0.2;beta = 0.0;
I have written an Mfile
function r = parameterIntegrate(F,h,M,beta,alpha,theta,phi)
% defining a nested function that uses one variable
phi = 0.6;
x = 0.5;
r = quad(#testf,0,1 + phi*cos(2*pi*x));
% simpson's rule from 0 to h
function y = testf(x)
h = 1 + phi*cos(2*pi*x);
theta = -1:0.5:1.5;
F = theta - 1;
M = 2;
beta = 0;
alpha = 0;
y = -((F+h)*M^3*(cosh(h*M)+M*beta*sinh(h*M)))/(h*M*cosh(h*M)+(-1+h*M^2*beta)*sinh(h*M))- (alpha*(M^2*(F+h)*(-1+2*h^2*M^2+ cosh(2*h*M)-2*h*M*sinh(2*h*M)))/(8*(h*M*cosh(h*M)+(-1+h*M^2*beta)*sinh(h*M))^2));
end
end
and called the function by
tol = [1e-5 1e-3];
q = quad(#parameterIntegrate, 0, h,tol)
or
q = quad(#parameterIntegrate, 0,1 + phi*cos(2*pi*0.5),tol)
its not working its giving me
Error using ==> plus
Matrix dimensions must agree.
What your error message means is that for some line of code, there are 2 matrices, but the dimensions don't match, so it can't add them. What I suggest you do to solve this is as follows:
Figure out exactly which line of code is causing the problem.
If the line has large numbers of variables, simplify them some.
Remember that if there are any matrixs at all, and you don't want to do matrix multiplication/division, use the .*, ./, and .^.
I suspect that if you change your multiplies/divides with step 3, your problem will go away.