how to implement newton-raphson to calculate the k(i) coefficients of a implicit runge kutta? - matlab

I'm trying to implement a RK implicit 2-order to convection-diffusion equation (1D) with fdm_2nd and gauss butcher coefficients: 'u_t = -uu_x + nu .u_xx' .
My goal is to compare the explit versus implcit scheme. The explicit rk which works well with a little number of viscosity. The curve of explicit schem show us a very nice shock wave.
I need your help to implement correctly the solver of the k(i) coefficient. I don't see how implement the newton method for all k(i).
do I need to implement it for all time-space steps ? or just in time ? The jacobian is maybe wrong but i don't see where. Or maybe i use the jacobian in wrong direction...
Actualy, my code works, but i think it's was wrong somewhere ... also the implicit curve does not move from the initial values.
here my function :
function [t,u] = burgers(t0,U,N,dx)
nu=0.01; %coefficient de viscosité
A=(diag(zeros(1,N))-diag(ones(1,N-1),1)+diag(ones(1,N-1),-1)) / (2*dx);
B=(-2*diag(ones(1,N))+diag(ones(1,N-1),1)+diag(ones(1,N-1),-1)) / (dx).^2;
t=t0;
u = - A * U.^2 + nu .* B * U;
the jacobian :
function Jb = burJK(U,dx,i)
%Opérateurs
a(1,1) = 1/4;
a(1,2) = 1/4 - (3).^(1/2) / 6;
a(2,1) = 1/4 + (3).^(1/2) / 6;
a(2,2) = 1/4;
Jb(1,1) = a(1,1) .* (U(i+1,1) - U(i-1,1))/ (2*dx) - 1;
Jb(1,2) = a(1,2) .* (U(i+1,1) - U(i-1,1))/ (2*dx);
Jb(2,1) = a(2,1) .* (U(i+1,2) - U(i-1,2))/ (2*dx);
Jb(2,2) = a(2,2) .* (U(i+1,2) - U(i-1,2))/ (2*dx) - 1;
Here my newton-code:
iter = 1;
iter_max = 100;
k=zeros(2,N);
k(:,1)=[0.4;0.6];
[w_1,f1] =burgers(n + c(1) * dt,uu + dt * (a(1,:) * k(:,iter)),iter,dx);
[w_2,f2] =burgers(n + c(2) * dt,uu + dt * (a(2,:) * k(:,iter)),iter,dx);
f1 = -k(1,iter) + f1;
f2 = -k(1,iter) + f2;
f(:,1)=f1;
f(:,2)=f2;
df = burJK(f,dx,iter+1);
while iter<iter_max-1 % K_newton
delta = df\f(iter,:)';
k(:,iter+1) = k(:,iter) - delta;
iter = iter+1;
[w_1,f1] =burgers(n + c(1) * dt,uu + dt * (a(1,:) * k(:,iter+1)),N,dx);
[w_2,f2] =burgers(n + c(2) * dt,uu + dt * (a(2,:) * k(:,iter+1)),N,dx);
f1 = -k(1,iter+1) + f1;
f2 = -k(1,iter+1) + f2;
f(:,1)=f1;
f(:,2)=f2;
df = burJK(f,dx,iter);
if iter>iter_max
disp('#');
else
disp('ok');
end
end

I'm a little rusty on exactly how to implement this in matlab, but I can walk your through the general steps and hopefully that will help. First we can consider the equation you are solving to fit the general class of problems that can be posed as
du/dt = F(u), where F is a linear or nonlinear function
For a Runge Kutta scheme you typically recast the problem something like this
k(i) = F(u+dt*a(i,i)*k(i)+ a(i,j)*k(j))
for a given stage. Now comes the tricky part, you you need to make 1-D vector constructed by stacking k(1) onto k(2). So the first half of the elements of the vector are k(1) and the second half are k(2). With this new combined vector you can then change F So that it operates on the two k's separately. This results in
K = FF(u+dt*a*K) where FF is F for the new double k vector, K
Ok, now we can implement the Newton's method. You will do this for each time step and until you have converged on the right answer and use it across all spatial points at the same time. What you do is you guess a K and compute the jacobian of G(K,U) = K-FF(FF(u+dt*a*K). G(K,U) should be only valued at zero when K is at the right solution. So in other words, do you Newton's method on K and when looking for convergence you need to see that it is converging at all spots. I would run the newton's method until max(abs(G(K,U)))< SolverTolerance.
Sorry I can't be more help on the matlab implementation, but hopefully I helped with explaining how to implement the newton's method.

Related

Regularized logistic regresion with vectorization

I'm trying to implement a vectorized version of the regularised logistic regression. I have found a post that explains the regularised version but I don't understand it.
To make it easy I will copy the code below:
hx = sigmoid(X * theta);
m = length(X);
J = (sum(-y' * log(hx) - (1 - y') * log(1 - hx)) / m) + lambda * sum(theta(2:end).^2) / (2*m);
grad =((hx - y)' * X / m)' + lambda .* theta .* [0; ones(length(theta)-1, 1)] ./ m ;
I understand the first part of the Cost equation, If I'm correct it could be represented as:
J = ((-y' * log(hx)) - ((1-y)' * log(1-hx)))/m;
The problem it's the regularization term. Let's take more detail:
Dimensions:
X = (m x (n+1))
theta = ((n+1) x 1)
I don't understand why he let the first term of theta (theta_0) outside of the equation, when in theory the regularized term it's:
and it has to take into account all the thetas
For the gradient descent, I think that this equation it's equivalent:
L = eye(length(theta));
L(1,1) = 0;
grad = (1/m * X'* (hx - y)+ (lambda*(L*theta)/m).
In Matlab indexes begin from 1, and in mathematic indexes begin from 0 (the indexes on the formula which you mentioned are also beginning from 0).
So, in theory, the first term of theta also needs to be let outside of the equation.
And as for your second question, you right! It is an equivalent clean equation!

How to equate coefficients of two quadratic equations to find the unknowns

I have these equations, e1=S^2-k2*s-k1 and e2=s^2+0.7*s+0.12 and e1=e2
now, by the naked eye you can see, k1=-0.12 and k2=-0.7. But I need a matlab code to evaluate this. Please help.
Thank You
How about this?
% Define your functions
f1 =#(s, k) s.^2 - k(2).*s - k(1)
f2 =#(s) s.^2 + 0.7*s + 0.12
% Define your initial guess
k0 = rand(2,1);
% Make sure you have enough equations
SMin = -5;
SMax = 5;
NEqn = 10 * length(k0)
S = (SMax - SMin) * rand(NEqn , 1) + SMin;
% find the coefficients
options = optimoptions('fsolve','Display','none','MaxFunctionEvaluations', 10000, 'TolPCG', 1E-10, 'TolFun', 1E-10);
x = fsolve(#(k) f1(S, k) - f2(S), k0 )

Solving Set of Second Order ODEs with Matlab ODE45 function

Introduction
NOTE IN CODE AND DISUSSION:
A single d is first derivative A double d is second derivative
I am using Matlab to simulate some dynamic systems through numerically solving the governing LaGrange Equations. Basically a set of Second Order Ordinary Differential Equations. I am using ODE45. I found a great tutorial from Mathworks (link for tutorial below) on how to solve a basic set of second order ordinary differential equations.
https://www.mathworks.com/academia/student_center/tutorials/source/computational-math/solving-ordinary-diff-equations/player.html
Based on the tutorial I simulated the motion for an elastic spring pendulum by obtaining two second order ordinary differential equations (one for angle theta and the other for spring elongation) shown below:
theta double prime equation:
M*thetadd*(L + del)^2 + M*g*sin(theta)*(L + del) + M*deld*thetad*(2*L + 2*del) = 0
del (spring elongation) double prime equation:
K*del + M*deldd - (M*thetad^2*(2*L + 2*del))/2 - M*g*cos(theta) = 0
Both equations above have form ydd = f(x, xd, y, yd)
I solved the set of equations by a common reduction of order method; setting column vector z to [theta, thetad, del, deld] and therefore zd = [thetad, thetadd, deld, deldd]. Next I used two matlab files; a simulation file and a function handle file for ode45. See code below of simulation file and function handle file:
Simulation File
%ElasticPdlmSymMainSim
clc
clear all;
%Define parameters
global M K L g;
M = 1;
K = 25.6;
L = 1;
g = 9.8;
% define initial values for theta, thetad, del, deld
theta_0 = 0;
thetad_0 = .5;
del_0 = 1;
deld_0 = 0;
initialValues = [theta_0, thetad_0, del_0, deld_0];
% Set a timespan
t_initial = 0;
t_final = 36;
dt = .01;
N = (t_final - t_initial)/dt;
timeSpan = linspace(t_final, t_initial, N);
% Run ode45 to get z (theta, thetad, del, deld)
[t, z] = ode45(#OdeFunHndlSpngPdlmSym, timeSpan, initialValues);
Here is the function handle file:
function dz = OdeFunHndlSpngPdlmSym(~, z)
% Define Global Parameters
global M K L g
% Take output from SymDevFElSpringPdlm.m file for fy1 and fy2 and
% substitute into z2 and z4 respectively
% z1 and z3 are simply z2 and z4
% fy1=thetadd=z(2)= -(M*g*sin(z1)*(L + z3) + M*z2*z4*(2*L + 2*z3))/(M*(L + z3)^2)
% fy2=deldd=z(4)=((M*(2*L + 2*z3)*z2^2)/2 - K*z3 + M*g*cos(z1))/M
% return column vector [thetad; thetadd; deld; deldd]
dz = [z(2);
-(M*g*sin(z(1))*(L + z(3)) + M*z(2)*z(4)*(2*L + 2*z(3)))/(M*(L + z(3))^2);
z(4);
((M*(2*L + 2*z(3))*z(2)^2)/2 - K*z(3) + M*g*cos(z(1)))/M];
Question
However, I am coming across systems of equations where the variables can not be solved for explicitly as is the case with spring pendulum example. For one case I have the following set of ordinary differential equations:
y double prime equation
ydd - .5*L*(xdd*sin(x) + xd^2*cos(x) + (k/m)*y - g = 0
x double prime equation
.33*L^2*xdd - .5*L*ydd*sin(x) - .33*L^2*C*cos(x) + .5*g*L*sin(x) = 0
L, g, m, k, and C are given parameters.
Note that x'' term appears in y'' equation and y'' term appears in x'' equation so I am not able to use reduction of order method. Can I use Matlab ODE45 to solve the set of ordinary differential equations in the second example in a manner similar to first example?
Thanks!
This problem can be solved by working out some of the math by hand. The equations are linear in xdd and ydd so it should be straightforward to solve.
ydd - .5*L*(xdd*sin(x) + xd^2*cos(x)) + (k/m)*y - g = 0
.33*L^2*xdd - .5*L*ydd*sin(x) - .33*L^2*C*cos(x) + .5*g*L*sin(x) = 0
can be rewritten as
-.5*L*sin(x)*xdd + ydd = -.5*L*xd^2*cos(x) - (k/m)*y + g
.33*L^2*xdd - .5*L*sin(x)*ydd = .33*L^2*C*cos(x) - .5*g*L*sin(x)
which is the form A*x=b.
For more complex systems, you can look into the fsolve function.

Matlab - Unexpected Results from Differential Equation Solver Ode45

I am trying to solve a differential equation with the ode solver ode45 with MATLAB. I have tried using it with other simpler functions and let it plot the function. They all look correct, but when I plug in the function that I need to solve, it fails. The plot starts off at y(0) = 1 but starts decreasing at some point when it should have been an increasing function all the way up to its critical point.
function [xpts,soln] = diffsolver(p1x,p2x,p3x,p1rr,y0)
syms x y
yp = matlabFunction((p3x/p1x) - (p2x/p1x) * y);
[xpts,soln] = ode45(yp,[0 p1rr],y0);
p1x, p2x, and p3x are polynomials and they are passed into this diffsolver function as parameters.
p1rr here is the critical point. The function should diverge after the critical point, so i want to integrate it up to that point.
EDIT: Here is the code that I have before using diffsolver, the above function. I do pade approximation to find the polynomials p1, p2, and p3. Then i find the critical point, which is the root of p1 that is closest to the target (target is specified by user).
I check if the critical point is empty (sometimes there might not be a critical point in some functions). If its not empty, then it uses the above function to solve the differential equation. Then it plots the x- and y- points returned from the above function basically.
function error = padeapprox(m,n,j)
global f df p1 p2 p3 N target
error = 0;
size = m + n + j + 2;
A = zeros(size,size);
for i = 1:m
A((i + 1):size,i) = df(1:(size - i));
end
for i = (m + 1):(m + n + 1)
A((i - m):size,i) = f(1:(size + 1 - i + m));
end
for i = (m + n + 2):size
A(i - (m + n + 1),i) = -1;
end
if det(A) == 0
error = 1;
fprintf('Warning: Matrix is singular.\n');
end
V = -A\df(1:size);
p1 = [1];
for i = 1:m
p1 = [p1; V(i)];
end
p2 = [];
for i = (m + 1):(m + n + 1)
p2 = [p2; V(i)];
end
p3 = [];
for i = (m + n + 2):size
p3 = [p3; V(i)];
end
fx = poly2sym(f(end:-1:1));
dfx = poly2sym(df(end:-1:1));
p1x = poly2sym(p1(end:-1:1));
p2x = poly2sym(p2(end:-1:1));
p3x = poly2sym(p3(end:-1:1));
p3fullx = p1x * dfx + p2x * fx;
p3full = sym2poly(p3fullx); p3full = p3full(end:-1:1);
p1r = roots(p1(end:-1:1));
p1rr = findroots(p1r,target); % findroots eliminates unreal roots and chooses the one closest to the target
if ~isempty(p1rr)
[xpts,soln] = diffsolver(p1x,p2x,p3fullx,p1rr,f(1));
if rcond(A) >= 1e-10
plot(xpts,soln); axis([0 p1rr 0 5]); hold all
end
end
I saw some examples using another function to generate the differential equation but i've tried using the matlabFunction() method with other simpler functions and it seems like it works. Its just that when I try to solve this function, it fails. The solved values start becoming negative when they should all be positive.
I also tried using another solver, dsolve(). But it gives me an implicit solution all the time...
Does anyone have an idea why this is happening? Any advice is appreciated. Thank you!
Since your code seems to work for simpler functions, you could try to increase the accuracy options of the ode45 solver.
This can be achieved by using odeset:
options = odeset('RelTol',1e-10,'AbsTol',1e-10);
[T,Y] = ode45(#function,[tspan],[y0],options);

Shorten this in Matlab

Let x = [1,...,t] be a vector with t components and A and P arrays. I asked myself whether there is any chance to shorten this, as it looks very cumbersome:
for n = 1:t
for m = 1:n
H(n,m) = A(n,m) + x(n) * P(n,m)
end
end
My suggestion: bsxfun(#times,x,P) + A;
e.g.
A = rand(3);
P = rand(3);
x = rand(3,1);
for n = 1:3
for m = 1:3
H(n,m) = A(n,m) + x(n) * P(n,m);
end
end
H2 = bsxfun(#times,x,P) + A;
%//Check that they're the same
all(H(:) == H2(:))
returns
ans = 1
EDIT:
Amro is right! To make the second loop is dependent on the first use tril:
H2 = tril(bsxfun(#times,x,P) + A);
Are the matrices square btw because that also creates other problems
tril(A + P.*repmat(x',1,t))
EDIT. This is for when x is row vector.
If x is a column vector, then use tril(A + P.*repmat(x,t,1))
If your example code is correct, then H(i,j) = 0 for any j > i, e.g. X(1,2).
For t = 3 for example, you would have.
H =
'A(1,1) + x(1) * P(1,1)' [] []
'A(2,1) + x(2) * P(2,1)' 'A(2,2) + x(2) * P(2,2)' []
'A(3,1) + x(3) * P(3,1)' 'A(3,2) + x(3) * P(3,2)' 'A(3,3) + x(3) * P(3,3)'
Like I pointed out in the comments, unless it was a typo mistake, the second for-loop counter depends on that of the first for-loop...
In case it was intentional, I came up with the following solution:
% some random data
t = 10;
x = (1:t)';
A = rand(t,t);
P = rand(t,t);
% double for-loop
H = zeros(t,t);
for n = 1:t
for m = 1:n
H(n,m) = A(n,m) + x(n) * P(n,m);
end
end
% vectorized using linear-indexing
[a,b] = ndgrid(1:t,1:t);
idx = sub2ind([t t], nonzeros(tril(a)), nonzeros(tril(b)));
xidx = nonzeros(tril(a));
HH = zeros(t);
HH(tril(true(t))) = A(idx) + x(xidx).*P(idx);
% check the results are the same
assert(isequal(H,HH))
I like #Dan's solution better. The only advantage here is that I do not compute unnecessary values (since the upper half of the matrix is zeros), while the other solution computes the full matrix and then cut back the extra stuff.
A good start would be
H = A + x*P
This may not be a working solution, you'll have to check conformability of arrays and vectors, and make sure that you're using the correct multiplication, but this should be enough to point you in the right direction. If you're new to Matlab be aware that vectors can be either 1xn or nx1, ie row and column vectors are different species unlike in so many programming languages. If x isn't what you want on the rhs, you may want its transpose, x' in Matlab.
Matlab is, from one point of view, an array language, explicit loops are often unnecessary and frequently not even a good way to go.
Since the range for second loop is 1:n, you can take the lower triangle parts of matrices A and P for calculation
H = bsxfun(#times,x(:),tril(P)) + tril(A);