I want to plot the Lorentz system (s=10, r=28, b=8/3) for a start condition which should give 0 since it's a start condition in the plane spanned by v1 and v2 and trough the critical point (0,0,0) (equation of the plane : -x + (-9- \sqrt(1201))/56 * y = 0) When I use the Runge-Kutta method to plot the solution with as start conditions ((-9-sqrt(1201))/56,1,10) my graph doesn't converge to 0 and I don't know why.
[X,Y,Z,T] = Runge(T0,(-9-sqrt(1201))/56,1,10,h,1000);
plot(T,X);
plot(T,Y);
plot(T,Z);
I expect a solution in which the lines in the plots go to 0. But I get some random function which goes up and down all the time.
Could this be caused by approximations?
Thanks in advance
This is the function Runge
function [X,Y,Z,T] = Runge(t0,x0,y0,z0,h,n)
X=[x0];
Y=[y0];
Z=[z0];
T=[t0];
k1x = 0;
k1y = 0;
k1z = 0;
k2x=0;
k2y=0;
k2z=0;
k3x=0;
k3y=0;
k3z=0;
k4x=0;
k4y=0;
k4z=0;
for k = 1:n
T(k+1)= T(k) +h;
k1x = F(X(k),Y(k),Z(k));
k1y = G(X(k),Y(k),Z(k));
k1z = H(X(k),Y(k),Z(k));
k2x = F(X(k)+h/2*k1x,Y(k) + h/2*k1y, Z(k) + h/2*k1z);
k2y= G(X(k)+h/2*k1x,Y(k) + h/2*k1y, Z(k) + h/2*k1z);
k2z= H(X(k)+h/2*k1x,Y(k) + h/2*k1y, Z(k) + h/2*k1z);
k3x= F(X(k)+h/2*k2x,Y(k)+h/2*k2y,Z(k) + h/2*k2z);
k3y=G(X(k)+h/2*k2x,Y(k)+h/2*k2y,Z(k) + h/2*k2z);
k3z=H(X(k)+h/2*k2x,Y(k)+h/2*k2y,Z(k) + h/2*k2z);
k4x= F(X(k)+h*k3x,Y(k)+h*k3y,Z(k)+h*k3z);
k4y=G(X(k)+h*k3x,Y(k)+h*k3y,Z(k)+h*k3z);
k4z=H(X(k)+h*k3x,Y(k)+h*k3y,Z(k)+h*k3z);
X(k+1) = X(k) + h/6 * (k1x + 2*k2x + 2*k3x + k4x);
Y(k+1) = Y(k) + h/6 * (k1y + 2*k2y + 2*k3y + k4y);
Z(k+1) = Z(k) + h/6 * (k1z + 2*k2z + 2*k3z + k4z);
end
end
There is no such plane for the non-linear system
Why do you think this plane is invariant?
In the system
F=#(x,y,z) sigma*(y-x);
G=#(x,y,z) x*(rho-z)-y;
H=#(x,y,z) x*y-beta*z;
no non-trivial linear combination of the first two equations is independent of z, which would be required to get an invariant expression in only x and y.
The invariant sub-spaces of the linearization are close to the stable and unstable manifolds, but they are not identical to them. So if you are on some invariant subspace of the linearization, you are in fact some distance away from any curved sub-manifolds related to the non-linear system, and numerical as well as exact solutions will move towards the chaotic attractor.
The Lorentz system is chaotic
Apart from that your observation is correct that floating point errors accumulate to drive the numerical solution away from the exact solution and any properties of it.
Related
I'm given the a(k) matrix and e(n) and I need to compute y(n) from the following eq:
y(n) = sum(k = 1 to 10 )( a(k)*y(n-k) ) + e(n).
I have the a matrix(filter coefficients) and the e matrix (the residual), therefore there is only 1 unknown: y, which is built by the previous 10 samples of y.
An example to this equation:
say e(0) (my first residual sample) = 3
and y(-10) to y(-1) = 0
then y(0), my first sample in the signal y, would just be e(0) = 3:
y(0) = a(1)*y(-1) + a(2)*y(-2) + .... + a(10)*y(-10) + e(0) = e(0) = 3
and if e(1) = 4, and a(1) = 5, then
y(1) = a(1)*y(0) + a(2)*y(-1) + a(3)&y(-2) + ... + a(10)*y(-9) + e(1) = 19
The problem is
I don't know how to do this without loops because, say, y(n) needs y(n-1), so I need to immediately append y(n-1) into my matrix in order to get y(n).
If n (the number of samples) = say, 10,000,000, then using a loop is not ideal.
What I've done so far
I have not implemented anything. The only thing I've done for this particular problem is research on what kind of Matlab functions I could use.
What I need
A Matlab function that, given an equation and or input matrix, computes the next y(n) and automatically appends that to the input matrix, and then computes the next y(n+1), and automatically append that to the input matrix and so on.
If there is anything regarding my approach
That seems like it's on the wrong track, or my question isn't clear enough, of if there is no such matlab function that exists, then I apologize in advance. Thank you for your time.
I would like to solve the following equation: tan(x) = 1/x
What I did:
syms x
eq = tan(x) == 1/x;
sol = solve(eq,x)
But this gives me only one numerical approximation of the solution. After that I read about the following:
[sol, params, conds] = solve(eq, x, 'ReturnConditions', true)
But this tells me that it can't find an explicit solution.
How can I find numerical solutions to this equation within some given range?
I've never liked using solvers "blindly", that is, without some sort of decent initial value selection scheme. In my experience, the values you will find when doing things blindly, will be without context as well. Meaning, you'll often miss solutions, think something is a solution while in reality the solver exploded, etc.
For this particular case, it is important to realize that fzero uses numerical derivatives to find increasingly better approximations. But, derivatives for f(x) = x · tan(x) - 1 get increasingly difficult to accurately compute for increasing x:
As you can see, the larger x becomes, the better f(x) approximates a vertical line; fzero will simply explode! Therefore it is imperative to get an estimate as closely to the solution as possible before even entering fzero.
So, here's a way to get good initial values.
Consider the function
f(x) = x · tan(x) - 1
Knowing that tan(x) has Taylor expansion:
tan(x) ≈ x + (1/3)·x³ + (2/15)·x⁵ + (7/315)·x⁷ + ...
we can use that to approximate the function f(x). Truncating after the second term, we can write:
f(x) ≈ x · (x + (1/3)·x³) - 1
Now, key to realize is that tan(x) repeats with period π. Therefore, it is most useful to consider the family of functions:
fₙ(x) ≈ x · ( (x - n·π) + (1/3)·(x - n·π)³) - 1
Evaluating this for a couple of multiples and collecting terms gives the following generalization:
f₀(x) = x⁴/3 - 0π·x³ + ( 0π² + 1)x² - (0π + (0π³)/3)·x - 1
f₁(x) = x⁴/3 - 1π·x³ + ( 1π² + 1)x² - (1π + (1π³)/3)·x - 1
f₂(x) = x⁴/3 - 2π·x³ + ( 4π² + 1)x² - (2π + (8π³)/3)·x - 1
f₃(x) = x⁴/3 - 3π·x³ + ( 9π² + 1)x² - (3π + (27π³)/3)·x - 1
f₄(x) = x⁴/3 - 4π·x³ + (16π² + 1)x² - (4π + (64π³)/3)·x - 1
⋮
fₙ(x) = x⁴/3 - nπ·x³ + (n²π² + 1)x² - (nπ + (n³π³)/3)·x - 1
Implementing all this in a simple MATLAB test:
% Replace this with the whole number of pi's you want to
% use as offset
n = 5;
% The coefficients of the approximating polynomial for this offset
C = #(npi) [1/3
-npi
npi^2 + 1
-npi - npi^3/3
-1];
% Find the real, positive polynomial roots
R = roots(C(n*pi));
R = R(imag(R)==0);
R = R(R > 0);
% And use these as initial values for fzero()
x_npi = fzero(#(x) x.*tan(x) - 1, R)
In a loop, this can produce the following table:
% Estimate (polynomial) Solution (fzero)
0.889543617524132 0.860333589019380 0·π
3.425836967935954 3.425618459481728 1·π
6.437309348195653 6.437298179171947 2·π
9.529336042900365 9.529334405361963 3·π
12.645287627956868 12.645287223856643
15.771285009691695 15.771284874815882
18.902410011613000 18.902409956860023
22.036496753426441 22.036496727938566 ⋮
25.172446339768143 25.172446326646664
28.309642861751708 28.309642854452012
31.447714641852869 31.447714637546234
34.586424217960058 34.586424215288922 11·π
As you can see, the approximant is basically equal to the solution. Corresponding plot:
To find a numerical solution to a function within some range, you can use fzero like this:
fun = #(x)x*tan(x)-1; % Multiplied by x so fzero has no issue evaluating it at x=0.
range = [0 pi/2];
sol = fzero(fun,range);
The above would return just one solution (0.8603). If you want additional solutions, you will have to call fzero more times. This can be done, for example, in a loop:
fun = #(x)tan(x)-1/x;
RANGE_START = 0;
RANGE_END = 3*pi;
RANGE_STEP = pi/2;
intervals = repelem(RANGE_START:RANGE_STEP:RANGE_END,2);
intervals = reshape(intervals(2:end-1),2,[]).';
sol = NaN(size(intervals,1),1);
for ind1 = 1:numel(sol)
sol(ind1) = fzero(fun, mean(intervals(ind1,:)));
end
sol = sol(~isnan(sol)); % In case you specified more intervals than solutions.
Which gives:
[0.86033358901938;
1.57079632679490; % Wrong
3.42561845948173;
4.71238898038469; % Wrong
6.43729817917195;
7.85398163397449] % Wrong
Note that:
The function is symmetric, and so are its roots. This means you can solve on just the positive interval (for example) and get the negative roots "for free".
Every other entry in sol is wrong because this is where we have asymptotic discontinuities (tan transitions from +Inf to -Inf), which is mistakenly recognized by MATLAB as a solution. So you can just ignore them (i.e. sol = sol(1:2:end);.
Multiply the equation by x and cos(x) to avoid any denominators that can have the value 0,
f(x)=x*sin(x)-cos(x)==0
Consider the normalized function
h(x)=(x*sin(x)-cos(x)) / (abs(x)+1)
For large x this will be increasingly close to sin(x) (or -sin(x) for large negative x). Indeed, plotting this this is already visually true, up to an amplitude factor, for x>pi.
For the first root in [0,pi/2] use the Taylor approximation at x=0 of second degree x^2-(1-0.5x^2)==0 to get x[0]=sqrt(2.0/3) as root approximation, for the higher ones take the sine roots x[n]=n*pi, n=1,2,3,... as initial approximations in the Newton iteration xnext = x - f(x)/f'(x) to get
n initial 1. Newton limit of Newton
0 0.816496580927726 0.863034004302817 0.860333589019380
1 3.141592653589793 3.336084918413964 3.425618459480901
2 6.283185307179586 6.403911810682199 6.437298179171945
3 9.424777960769379 9.512307014150883 9.529334405361963
4 12.566370614359172 12.635021895208379 12.645287223856643
5 15.707963267948966 15.764435036320542 15.771284874815882
6 18.849555921538759 18.897518573777646 18.902409956860023
7 21.991148575128552 22.032830614521892 22.036496727938566
8 25.132741228718345 25.169597069842926 25.172446326646664
9 28.274333882308138 28.307365162331923 28.309642854452012
10 31.415926535897931 31.445852385744583 31.447714637546234
11 34.557519189487721 34.584873343220551 34.586424215288922
I am trying to calculate some integrals, for example:
a = 1/sqrt(2);
b = -5;
c = 62;
d = 1;
f = exp(-x^2-y^2)*(erfc((sym(a) + 1/(x^2+y^2)*(sym(b)*x+sym(d)*y))*sqrt((x^2+y^2)*sym(10.^(c/10))))...
+ erfc((sym(a) - 1/(x^2+y^2)*(sym(b)*x+sym(d)*y))*sqrt((x^2+y^2)*sym(10.^(c/10)))));
h = int(int(f,x,-Inf,Inf),y,-Inf,Inf);
It will occur error like this:
Warning: Explicit integral could not be found.
Then, I try to use vpato calculate that integral,and get the result like this
vpa(int(int(f,x,-Inf,Inf),y,-Inf,Inf),5)
numeric::int(numeric::int(exp(- x^2 - y^2)*(erfc(((6807064429273519*x^2)/4294967296 + (6807064429273519*y^2)/4294967296)^(1/2)*(2^(1/2)/2 + (5*x - y)/(x^2 + y^2))) + erfc(((6807064429273519*x^2)/4294967296 + (6807064429273519*y^2)/4294967296)^(1/2)*(2^(1/2)/2 - (5*x - y)/(x^2 + y^2)))), x == -Inf..Inf), y == -Inf..Inf)
I already tried to change the interval [-Inf,Inf] to [-100,100], and get the same above result:
numeric::int(numeric::int(exp(- x^2 - y^2)*(erfc(((6807064429273519*x^2)/4294967296 + (6807064429273519*y^2)/4294967296)^(1/2)*(2^(1/2)/2 + (5*x - y)/(x^2 + y^2))) + erfc(((6807064429273519*x^2)/4294967296 + (6807064429273519*y^2)/4294967296)^(1/2)*(2^(1/2)/2 - (5*x - y)/(x^2 + y^2)))), x == -100..100), y == -100..100)
My question is why vpa in this case could not return to a real value?
There are something wrong in above Matlab code? (I, myself, could not find the bug so far)
Thank you in advance for your help.
It is unlikely that there is an analytic solution to this integral so using int may not be a good choice. In some cases vpa can be used to for a numeric solution. When this fails (by returning a call to itself) it may be for several reason: the integral may not exist, the integral may converge too slowly, singularities may cause issues, the integrand may be highly oscillatory or non-smooth, etc. Mathematica also struggles with this integral.
You can try calculating the integral numerically using integral2:
a = 1/sqrt(2);
b = -5;
c = 62;
d = 1;
f = #(x,y)exp(-x.^2-y.^2).*(erfc((a + 1./(x.^2+y.^2).*(b*x+d*y)).*sqrt((x.^2+y.^2)*10^(c/10)))...
+erfc((a - 1./(x.^2+y.^2).*(b*x+d*y)).*sqrt((x.^2+y.^2)*10^(c/10))));
h = integral2(f,-Inf,Inf,-Inf,Inf)
which returns 5.790631184403967. This compares well with Mathematica's numerical integration using NIntegrate. You can try specifying smaller absolute and relative tolerances for integral2 to get more accurate values, but this will result in much slower compute times.
I am currently trying to implement a machine learning algorithm that involves the logistic loss function in MATLAB. Unfortunately, I am having some trouble due to numerical overflow.
In general, for a given an input s, the value of the logistic function is:
log(1 + exp(s))
and the slope of the logistic loss function is:
exp(s)./(1 + exp(s)) = 1./(1 + exp(-s))
In my algorithm, the value of s = X*beta. Here X is a matrix with N data points and P features per data point (i.e. size(X)=[N,P]) and beta is a vector of P coefficients for each feature such that size(beta)=[P 1].
I am specifically interested in calculating the average value and gradient of the Logistic function for given value of beta.
The average value of the Logistic function w.r.t to a value of beta is:
L = 1/N * sum(log(1+exp(X*beta)),1)
The average value of the slope of the Logistic function w.r.t. to a value of b is:
dL = 1/N * sum((exp(X*beta)./(1+exp(X*beta))' X, 1)'
Note that size(dL) = [P 1].
My issue is that these expressions keep producing numerical overflows. The problem effectively comes from the fact that exp(s)=Inf when s>1000 and exp(s)=0 when s<-1000.
I am looking for a solution such that s can take on any value in floating point arithmetic. Ideally, I would also really appreciate a solution that allows me to evaluate the value and gradient in a vectorized / efficient way.
How about the following approximations:
– For computing L, if s is large, then exp(s) will be much larger than 1:
1 + exp(s) ≅ exp(s)
and consequently
log(1 + exp(s)) ≅ log(exp(s)) = s.
If s is small, then using the Taylor series of exp()
exp(s) ≅ 1 + s
and using the Taylor series of log()
log(1 + exp(s)) ≅ log(2 + s) ≅ log(2) + s / 2.
– For computing dL, for large s
exp(s) ./ (1 + exp(s)) ≅ 1
and for small s
exp(s) ./ (1 + exp(s)) ≅ 1/2 + s / 4.
– The code to compute L could look for example like this:
s = X*beta;
l = log(1+exp(s));
ind = isinf(l);
l(ind) = s(ind);
ind = (l == 0);
l(ind) = log(2) + s(ind) / 2;
L = 1/N * sum(l,1)
I found a good article about this problem.
Cutting through a lot of words, we can simplify the argument to stating that the original expression
log(1 + exp(s))
can be rewritten as
log(exp(s)*(exp(-s) + 1))
= log(exp(s)) + log(exp(-s) + 1)
= s + log(exp(-s) + 1)
This stops overflow from occurring - it doesn't prevent underflow, but by the time that occurs, you have your answer (namely, s). You can't just use this instead of the original, since it will still give you problems. However, we now have the basis for a function that can be written that will be accurate and won't produce over/underflow:
function LL = logistic(s)
if s<0
LL = log(1 + exp(s));
else
LL = s + logistic(-s);
I think this maintains reasonably good accuracy.
EDIT now to the meat of your question - making this vectorized, and allowing the calculation of the slope as well. Let's take these one at a time:
function LL = logisticVec(s)
LL = zeros(size(s));
LL(s<0) = log(1 + exp(s(s<0)));
LL(s>=0) = s(s>=0) + log(1 + exp(-s(s>=0)));
To obtain the average you wanted:
L = logisticVec(X*beta) / N;
The slope is a little bit trickier; note I believe you may have a typo in your expression (missing a multiplication sign).
dL/dbeta = sum(X * exp(X*beta) ./ (1 + exp(X*beta))) / N;
If we divide top and bottom by exp(X*beta) we get
dL = sum(X ./ (exp(-X*beta) + 1)) / N;
Once again, the overflow has gone away and we are left with underflow - but since the underflowed value has 1 added to it, the error this creates is insignificant.
I have a function
function y = testf(x,F,phi,M,beta,alpha)
y = -((F+(1 + phi.*cos(2.*pi.*x))).*M.^3.*(cosh((1 + phi.*cos(2.*pi.*x)).*M)+M.*beta.*sinh((1 + phi.*cos(2.*pi.*x)).*M)))./((1 + phi.*cos(2.*pi.*x)).*M.*cosh((1 + phi.*cos(2.*pi.*x)).*M)+(-1+(1 + phi.*cos(2.*pi.*x)).*M.^2.*beta).*sinh((1 + phi.*cos(2.*pi.*x)).*M))- (alpha.*(M.^2.*(F+(1 + phi.*cos(2.*pi.*x))).*(-1+2.*(1 + phi.*cos(2.*pi.*x)).^2.*M.^2+ cosh(2.*(1 + phi.*cos(2.*pi.*x)).*M)-2.*(1 + phi.*cos(2.*pi.*x)).*M.*sinh(2.*(1 + phi.*cos(2.*pi.*x)).*M)))./(8.*((1 + phi.*cos(2.*pi.*x)).*M.*cosh((1 + phi.*cos(2.*pi.*x)).*M)+(-1+(1 + phi.*cos(2.*pi.*x)).*M.^2.*beta).*sinh((1 + phi.*cos(2.*pi.*x)).*M)).^2));
integrating with
q = quad(#(x) testf (x, F, phi,M, beta, alpha), 0, h);
when q = 0 and x,F,phi,M,beta, how do I find alpha and draw the streamline?
It would be great if you gave some numbers, but here is how you would start this. It assumes you are using a version of Matlab that has MuPad as the Symbolic Engine.
First of all, I wouldn't use quad because symbolic expressions will be involved, use int instead.
If I understood you correctly, have the values of x, F, phi, M, beta and you would like to solve for alpha when q = 0
%define the known variables first
syms alpha %defining symbolic object
Now, the following may not work because it's a huge function:
q = int(y,x,0,h)
If it did, all you have to do is solve and evaluate the results for alpha (this may not work as well):
alpha = eval( solve( 'q' , alpha ) )
If the above didn't achieve anything, you might what to look at the 'IgnoreAnalyticConstraints' option.