I tried to find out how MATLAB computes the step size (not the initial one) to solve ODEs with, for example, the ode45 solver. The source code is really complex, so does anyone know hot it works?
You should be aware that the step size is dynamically adapted, there no "The" step size.
To get a general simplified idea: The total error E is composed of the atomic errors of every time step. In first order it is summation, more exactly there is some kind of cumulative magnification of the atomic errors involved.
A sensible approach would be that every time step of length h should have an atomic error of about E·h/T, where T is the length of the integration interval. The order 4 method has an local error of C·h^5 where C is in zeroth order a polynomial in the first 4 derivatives of the ODE function. Since the method computes an order 4 and an order 5 step, call them y4 and y5, one can take y5 as the more precise one so that approximately C·h^5 = |y4-y5|. This allows to compute C and to adapt the step size a·h to get the desired atomic error, since one can solve C·(a·h)^5=E/T·(a·h) to get
a = pow( E/T·h/norm(y4-y5), 1/4)
This does not need to be terribly exact, so that one can just use the adapted step size for the next step if the atomic error is not largely out of range.
Another approach is to compare if the local error |y4-y5|/h falls inside a bracket around the desired local error E/T and increase/decrease the step size by a constant factor, with a repetition of the step if the step size needed to be reduced.
There is more to the advanced/actual implementations, taking into account relative and absolute error goals, detecting stiffness, i.e., where the local error formula breaks down, …
Related
Howdy just had a numerical analysis question about RKF45. The documentation is a little mysterious in terms if is 4th or 5th order, and even on the wikipedia page "is a method of order O(h^4) with an error estimator of order O(h^5)"
https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.RK45.html
https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta%E2%80%93Fehlberg_method
Is the idea you first do the 4th order method, and then the interpolate of the numerical solution is 5th order to the true solution? So overall the output of this numerical method is 5th order?
Thanks a lot!
classical embedded methods like Fehlberg 4(5) aka RKF45
The method can proceed with the state updates of its 4th or 5th order method, designed for the 4th, nowadays used with 5th. The step size is variable, the optimal step size is always determined for the 4th order method, with the difference to the 5th order method serving as estimate of the local error, which in most cases is very accurate.
The relation between error tolerance and number of ODE function evaluations will mostly correspond to the 4th order, for example dividing the error tolerance by 16 should result in a step size sequence that is locally about half of the original sequence, doubling the number of ODE function evaluations.
If an exact solution is computable, the error of the numerical integration should be proportional to the tolerance if the 4th order steps are taken, and proportional to the power 5/4 of the tolerance if the 5th order steps are taken.
Extrapolation methods like DoPri (4)5 aka ode45 or RK45
The method proceeds with the values of the 5th order method. It uses a variable step size that is controlled via the difference between the 4th order method and the 5th order method, augmented to behave loosely like the actual local (unit step) error of the 5th order method. Despite being more a guidance to the local error size, it is good enough for the actual error to remain in the region of or below the error tolerance and for the step size to not leave the stability region of the 5th order method. The method was explicitly designed to show this behavior.
For test equations that are smooth, one can thus expect that in a log-log diagram of number of evaluations of the ODE function vs. given error tolerance or actual error against an exact solution you get a curve that is mostly linear with slope about 5. (One would of course compute it the other way, get number of function evaluations and actual error for some spread of error tolerances.)
See https://personal.math.ubc.ca/%7Efeldman/math/vble.pdf for some experiments with low-order embedded methods, starting with using the Euler method as embedded method in the explicit midpoint and Heun method.
I have two data set, let us name them "actual speed" and "desired speed". My main objective is to match actual speed with the desired speed.
But for doing that in my case, I need to tune FF(1x10), Integral(10x8) and Proportional gain table(10x8).
My approach till now was as follows:-
First, start the iteration with having 0.1 as the initial value in the first cells(FF[0]) of the FF table
Then find the R-square or Co-relation between two dataset( i.e. Actual Speed and Desired Speed)
Increment the value of first cell(FF[0]) by 0.25 and then again compute R-square or Co-relation of two data set.
Once the cell(FF[0]) value reaches 2(Gains Maximum value. Already defined by the lab). Evaluate R-square and re-write the gain value in FF[0] which gives min. error between the two curve.
Then tune the Integral and Proportional table in the same way for the same RPM Range
Once It is tune then go for higher RPM range and repeat step 2-5 (RPM Range: 800-1000; 1000-1200;....;3000-3200)
Now the problem is that this process is taking way too long time to complete. For example it takes around 1 Hr. time to tune one cell of FF. Which is actually very slow.
If possible, Please suggest any other approach which I can try to tune the tables. I am using MATLAB R2010a and I can't shift to any other version of MATLAB because my controller can communicate with this version only and I can't use any app for tuning since my GUI is already communicating with the controller and those two datasets are being made in real-time
In the given figure, lets us take (X1,Y1) curve as Desired speed and (X2,Y2) curve as Actual speed
UPDATE
I am writing a matlab code where i calculate the max-min.
I am using matlab's "fminimax" to solve the following problem:
ki=G(i,:);
ki(i)=0;
fs(i)=-((G(i,i)*pt(i)+sum(ki.*pt)+C1)-(C2*(sum(ki.*pt)+C1)));
G: is a system matrix. pt: is the optimization variable.
When the actual system matrix is used, the "fminimax" stops after one iteration and returns the initial value of "pt", no matter what the initial value for "pt", i.e. no solution is found. (the initial value is defined as X0 in the documentation). The system has the following parameters: G is in the order of e-11, pt is in the order of e-1, and c1 is in the order of e-14.
when i try a randomly generated test matrix and different parameters, the "fminimax" finds a solution for the problem, and everything works fine. G in order of e-2, pt in order of e-2, c1 is in the order of e-7.
I tried to scale the actual system: "fminimax" lasted more than one iteration, however, it still returned the initial value of pt, i.e. it couldn't find a solution.
I tried to change the tolerance of the "fminmax", using "options" [StepTolerance, OptimalityTolerance, ConstraintTolerance, and functiontolerance]. There were no impact at all. still no solution.
I thought that the problem might be that the precision of "fminimax" is not that high, or it is not suitable to solve the problem. i think it is also slow.
i downloaded CPLX, and i wanted to transform the max-min problem into linear programing, using a method i found in a book. However, when i tried my code on a simple minimax it didn't give the same solution.
I thought of using CVX for example, but the problem is not convex.
What might be the problem?
P.S. the system matrix, G, has different realizations, i tried some of them. However, the "fminimax" responds in the same way for all of them, i.e. it wasn't able to find an adequate solution.
I am not convinced that the optimization solvers are broken. If the problem is nonconvex, then there can be multiple local minimizers. Given the information you have provided, we have no way of knowing whether you started at an initial condition.
The first place you need to start is by getting more information from the optimization exit condition... Did it finish because it hit the iteration limit? (I hope not since it isn't doing many iterations)... Did it finish because a tolerance was hit (e.g. the function did not change by more than xxxx)? Or perhaps it could not find a feasible solution? (I don't know if you have any constraints that need to be met).
More than likely, I wold guess that you are starting at a local minimizer without realizing it. So you need to determine whether you are indeed at a local minimizer by looking at the jacobian of the function evaluated at your initial guess. Either calculate it analytically or use a finite step approximation....
I have the following problem.
for i=1:3000
[~,dist(i,1)]=knnsearch(C(selectedIndices,:),C);
end
Let me explain the code above. Matrix C is a huge matrix (300000 x 1984). C(selectedIndices,:) is a subset of 100 elements of C depending on the value of i. It means: For i=1, first 100 points of C are selected, for i==2, C(101:200,:) is selected. As you can see, the second argument remains constant.
Is there any way to make this work faster. I have tried the following:
- [~,dist(i,1)]=knnsearch(C,C); %obviously goes out of memory
send a bigger chunk of selectedIndices instead of sending just 100. This adds a little bit post-processing which I am not worried about. But this doesn't work since it takes equivalent amount of time. For example, if I send 100 points of C at a time, it takes 60 seconds. If I send 500, it takes 380 seconds with the post-processing.
Tried using parfor as: different sets of selectedIndices will be executed parallely. It doesn't work as two copies of big matrix C may have got created (not sure how parfor works), but I am sure that computer becomes very slow in turn negating the advantage of parfor.
Haven't tried yet: break both arguments into smaller chunks and send it in parfor. Do you think this will make any difference?
I am open to any suggestion i.e. if you feel braking a matrix in some different way may speed up the computation, do suggest it. Since, at the end I only care about finding closest point from a set of points (here each set has 100 points) for each point in C.
I want to speed up the convergence of a convex optimization problem in MATLAB.
My objective function is convex having three parameters and I am using gradient ascent for the maximization.
Right now I am manually writing the iteration with the termination condition being the difference between the new parameter value and old parameter value is very small (around 0.0000001). I cannot terminate based upon the number of iterations because it doesn't guarantee that it has converged to the optimum solution.
So, it takes a lot of time to converge - almost 2 days! Is there any way to speed this up?
Actually my objective function has only three parameters. I know that my first parameter's value should be greater than that of the second.
So starting with the initial condition, the second parameter's value starts increasing rapidly. After it has reached a certain point, the first parameter's value starts increasing rapidly. While the first parameter's value starts increasing, the second parameter's value starts decreasing slowly. Eventually, I have the first parameter's value greater than that of second.
Is there any way to speed up the process? 2 days is a very long time. Furthermore, calculating the gradient is also time consuming. It needs a lot of matrix computations.
I don't want to start with the defined parameter values like parameter1's value greater than that of second. Also it's not necessary that the first parameter always has to be greater than the the second. I just know which parameter value should be greater. Any suggestions?
If the calculation of gradients is very slow and you still want to do a manual implementation you could try this, it will take more steps but could be a lot quicker as the steps are so simple:
Define a stepsize
Try all the points where your variable moves -1, 0 or 1 times in the direction of the stepsize (3^3 = 27 possibilities)
Pick the best one
If the best one is your previous one, multiply the stepsize with a factor 0.5
Of course the success of this process depends on the properties of your function. Furthermore it should be noted that a much simpler solution could be to set the desired difference to something like 0.0001