I have recieved some old code which uses the function fmincon and the algorithm LevenbergMarquardt to optimize my parameters. However this algorithm is no longer available in this function.
Since I am new to Matlab I am not sure about what the best alternative is.
I have tried to simply change the function to the ones that are compatible with LevenbergMarquardt but this doesn't seem to work.
Below are the vector of options and the fmincon function. "S", "A" and "b" are starting values for the parameters, "lb" and "ub" are upper and lower bounds.
Please write if anything is unclear or you you need additional information.
options_ = optimset('LevenbergMarquardt', 'on','TolFun',1e-6,'TolX',1e-6, 'HessUpdate', 'steepdesc', 'Display','iter', 'LargeScale', 'off', 'MaxFunEvals', 100000, 'MaxIter', 100000);
[ out_p, fval, exitfflag ] = fmincon(#MyLikelihoodFunction, S, A, b, [], [], lb, ub, [], options_);
What did you try and what went wrong?
I'd start with fmincon with all default options. That will give you the interior-point algorithm. Set 'Display' to 'iter' to see how the algorithm progresses. If the problem is large (although the old code has 'LargeScale' off), you might try 'HessianApproximation' set to 'lbfgs'.
A and b are not part of the starting point. Those define the linear inequality constraints. There's more info on that in the documentation link I gave above.
Related
I am writing MATLAB code with the intention to do some fittings. I simulated a plot using a second-order Gaussian (see my code below) and tried fitting using the lsqcurvefit function. Unfortunately, MATLAB returns the same guess values as 'optimized' parameters and apparently gets stuck in a local minimum.
Can someone please provide some advice as to what might be wrong here? I know that if the guess is far from the 'true' values then this can happen, but I expected MATLAB to return an answer which was closer to true value. Improving the initial guess to [29,0] (which is much closer to the actual value) results in the same output: that the initial value is a local minimum.
%%%%%%%%%%
function z= testfunction(x, xdata);
sigma=x(1)/(2*sqrt(2*log(2)));
z=((xdata.^2-2*x(2)*xdata-sigma.^2+x(2)^2)./(sigma^5*sqrt(2*pi))).*exp(-(xdata-x(2)).^2/(2.*sigma.^2));
end
%%%%%%%%
% Simulate Data
xdata= -50:1:50;
ydata = testfunction([30,0],xdata);
% Fit Data
xfit = lsqcurvefit(#testfunction,[19,-4],xdata, ydata );
xfit(1)
xfit(2)
yfit=testfunction([xfit(1),xfit(2)],xdata);
% Plot Data;
plot(xdata,yfit);
hold on;
plot(xdata,ydata,'o')
Output:
Initial point is a local minimum.
Optimization completed because the size of the gradient at the initial point
is less than the default value of the optimality tolerance.
<stopping criteria details>
ans =
19
ans =
-4
Short answer: check the stopping criteria details and change the stopping criteria accordingly:
options = optimoptions('lsqcurvefit','OptimalityTolerance', 1e-16, 'FunctionTolerance', 1e-16);
xfit = lsqcurvefit(#testfunction,[19,-4],xdata, ydata, [], [], options);
What is the problem?
lsqcurvefit is a numerical solver and therefore uses a stopping criteria to determine if the local minimum is sufficiently reached. In general, you never reach the exact solution. Therefore, the solution to your problem is to change the stopping criteria to request a more accurate solution.
How to solve it?
By clicking on stopping criteria details, you receive the following explanation:
Optimization completed: The final point is the initial point. The
first-order optimality measure, 7.254593e-07, is less than
options.OptimalityTolerance = 1.000000e-06.
Optimization Metric Options
relative first-order optimality = 7.25e-07 OptimalityTolerance = 1e-06 (default)
So, you should decrease the OptimalityTolerance (f.e. to 1e-16):
options = optimoptions('lsqcurvefit','OptimalityTolerance', 1e-16);
xfit = lsqcurvefit(#testfunction,[19,-4],xdata, ydata, [], [], options);
The above image visualizes the new result, which is better than the previous one, but not yet very good. By inspecting the stopping criteria again, you will see that you also need to change FunctionTolerance:
options = optimoptions('lsqcurvefit','OptimalityTolerance', 1e-16, 'FunctionTolerance', 1e-16);
Why are the default options that bad?
Note that you need to tweak the stopping criteria, because your function returns relative small values. Multiplying z with a factor of 1000, without any option specification would also result in a nice fit:
My question regards using the scipy.interpolate.RectBivariateSpline function to interpolate a 2D mesh. I am essentially trying to emulate functionality of Matlab's interp2 function.
For a particular (light) use case, I call RectBivariateSpline on x and y vectors, which are regularly spaced, monotonically increasing vectors, e.g.:
x = np.arange(0., 20., 1.) # shape (20,)
y = np.arange(0., 20., 1.) # shape (20,)
and some 2D field, e.g.:
fld = np.reshape(np.arange(0., 400., 1.), (20, 20)) # shape (20, 20)
i.e.:
fn = sp.interpolate.RectBivariateSpline(x, y, fld)
To evaluate the interpolation, then, at particular xi, yi coordinates (or arrays, obeying numpy broadcasting), the docs advise to call the RectBivariateSpline.ev function, i.e.:
val1 = fn.ev(1, 1) # Ans: 21
val2 = fn.ev([1, 2], [1, 2]) # Ans: [21, 22]
This allows the user to find one interpolated value, for say (xi, yi) = (1.5, 1.5), or many interpolated values, say on a particular domain (regular or irregular grid).
My question is this: for large xi, yi arrays, how can I make the fn.ev call faster? Compared to the Matlab interp2 call, it is pretty slow (by an order of magnitude or worse).
Following the function call in the debugger, I have found that ev is actually a wrapper for RectBivariateSpline.__call__. This again is a wrapper for a call to the fitpack functions (in C / Fortran) on which Scipy's interpolation functionality is based. The __call__ function has an optional keyword grid, defaulted to False and absent from ev, which allows you to pass two vectors defining an orthogonal grid. Calling with grid set to True leads to the call of the fitpack routine bispev which is MUCH faster than the documented ev function which calls fitpack bispeu. (I assume the performance boost is because bispev exploits the regular grid, while bispeu may simply loop through all the index pairs... though I'm not sure).
In any event, I want to call an .ev like function in such a way that I can interpolate a grid that may not be completely regular (but is close) and that will be faster than the current call to bispeu. "Regularizing" the grid and going with bispev IS an option, and the end results are PRETTY close, and the routine is MUCH faster that way (by a factor of 20!)... however, Matlab interp2 allows the slightly irregular grid and computes with the same speed. I have considered trying to write my own C function, but I'm highly doubtful that such a lowly one as I can do any better than what's already in Scipy and written by geniuses. :)
So.. is there a way I can have my cake and eat it too? Is there some awesome little tricky way I can call the spline evaluator? I have thought about making my own bespoke wrapper for the fitpack calls... but the documentation for fitpack is not readily available (certainly not in the Scipy release I have), and I'd like to avoid that extra work if possible. Also note that this problem is especially irksome because I'm having to call it twice, once for the real and imaginary components of my original field (Matlab takes complex meshes). At the end of the day, I want to give Matlab the BOOT... but this speed issue could be a killer.
Thank you for your time.
I have a question about solving a system of equations and initial guesses of the solution. I want to solve a system of equations where "x", a Tx1 vector, are my unknowns, "a" a Tx1 vector and "B" a TxT matrix. "f" is the function I want to solve for. I want to solve for "x" such that "f==0":
x = sym('x', [T,1]);
f = -x+1-(1+erf((a - B*x)./sqrt(2)))/2; % -x+1-normcdf(a-B*x)
Spp = solve(f==0, x);
I use Matlab's solve (or vpasolve) functions to obtain values. If the entries of "B" are above a certain value I should observe a jump for changing values of "a" (which I do). However, depending on the initial guess of the solution, i.e. for example either the initial guess is 1 or 0, the position of the jump occurs at different values for "a", a hysteresis cycle occurs.
I solved the equation using fzero for T=1. I specified the initial guess and indeed was able to observe the hysteresis cycle. For T>1, fzero does not work anymore and I tried solve as well as vpasolve. solve does not allow initial guesses and for vpasolve I even get with examples from Matlab's help site an error whenever I include more than the system of equations and the unknown variables, i.e. vpasolve(eqn,var) works fine but for vpasolve(eqn,var,init_guess) I get the following error:
Error using getEqnsVars (line 50) Expecting two arguments: a vector of
equations and a vector of variables.
Error in sym/vpasolve (line 91) [eqns,vars] =
getEqnsVars(varargin{1:end-1});
What am I doing wrong? Is there another function I could try?
Edit: I didn't use 'fsolve' but 'fzero' to find the roots.
You can use slightly different definition of function f and try fsolve. Here you don't have to explicitly define x as symbolic variables.
f = #(x) -x+1-(1+erf((a - B*x)./sqrt(2)))/2; % -x+1-normcdf(a-B*x)
initial_guess = zeros(T,1);
Spp = fsolve(f,initial_guess);
I'm having trouble understanding and applying the use of nlinfit function in Matlab. So, let's say I'm given vectors
x = [1, 2, 3, 4, 5]
y = [2.3, 2.1, 1.7, .95, .70]
and I'm asked to fit this data to an exponential form (I don't know if the numbers will work, I made them up) where y = A*e^(Bx) + C (A/B/C are constants).
My understanding is that nlinfit takes 4 arguments, the two vectors, a modelfunction which in this case should be the equation I have above, and then beta0, which I don't understand at all. My question is how do you implement the modelfunction in nlinft, and how do you find beta0 (when only working with 2 vectors you want to plot/fit) and how should it be implemented? Can someone show me an example so that I can apply this function for any fit? I suspect I'll be using this a lot in the future and really want to learn it.
Check out the second example in the docs: http://www.mathworks.com/help/stats/nlinfit.html
Basically you pass a function handle as your modelfunction parameter. Either make a function in a file and then just pass it the function name with an # in front or else make an anonymous function like this:
nlinfit(x, y, #(b,x)(b(1).*exp(b(2).*x) + b(3)), beta0)
You'll notice that in the above I have stuck all your parameters into a single vector. The first parameter of your function must be a vector of all the points you are trying to solve for (i.e. A, B and C in your case) and the second must be x.
As woodchips has said beta0 is your starting point so your best guess (doesn't have to be great) of your A, B and C parameters. so something like [1 1 1] or rand(3,1), it is very problem specific though. You should play around with a few. Just remember that this is a local search function and thus can get stuck on local optima so your starting points can actually be quite important.
beta0 is your initial guess at the parameters. The better your guess, the more likely you will see convergence to a viable solution. nlinfit is no more than an optimization. It has to start somewhere.
I set my options to
options=optimset('LevenbergMarquardt', 'on')
and then employ lsqcurvefit like below,
[x,resnorm,residual,exitflag,output] = lsqcurvefit(#myfun, [0.01 0.3], xdata, ydata, [-inf -inf], [inf inf], options)
but the problem is that I don't now why I will get for output :
output =
firstorderopt: 3.4390e-07
iterations: 4
funcCount: 15
cgiterations: 0
algorithm: 'large-scale: trust-region reflective Newton'
message: [1x425 char]
Does this mean Matlab did not use the algorithm Levenberg Marquardt?
But I did set my options to levenberg Marquardt algorithm!!!
I'd appreciate any help.
Sometimes a specific algorithm is not suitable for a specific configuration of an optimization problem. In these cases Matlab "falls back" to its default optimization algorithm.
It might be the case that for your specific problem/configuration Matlab is unable to use Levenberg-Marquardt algorithm.
Read the docs carefully to see if this is the case.
I can't say for certain, but the constaints ([-inf -inf], [inf inf]) could be your problem. The documentation for lsqcurvefit strictly says the LMA cannot be used for constrained problems. If constraints are included, it will fall back to trust-region.
Yes, your constraints are mathematically equivalent to 'no constaints', but I have no idea how the MATLAB function itself will interpret those. I tried to recreate the problem on my end, but optimset('LevenbergMarquardt', 'on') has been deprecated and generates an error (implying you have a relatively old version). Even when using the new syntax (optimset('Algorithm', 'levenberg-marquardt')), it behaves correctly on my end (using 2011b). To not have constraints, the correct approach is to use empty matrices (i.e. []).
Yes, the question is a month old, but someone else may find the answer useful.