I set my options to
options=optimset('LevenbergMarquardt', 'on')
and then employ lsqcurvefit like below,
[x,resnorm,residual,exitflag,output] = lsqcurvefit(#myfun, [0.01 0.3], xdata, ydata, [-inf -inf], [inf inf], options)
but the problem is that I don't now why I will get for output :
output =
firstorderopt: 3.4390e-07
iterations: 4
funcCount: 15
cgiterations: 0
algorithm: 'large-scale: trust-region reflective Newton'
message: [1x425 char]
Does this mean Matlab did not use the algorithm Levenberg Marquardt?
But I did set my options to levenberg Marquardt algorithm!!!
I'd appreciate any help.
Sometimes a specific algorithm is not suitable for a specific configuration of an optimization problem. In these cases Matlab "falls back" to its default optimization algorithm.
It might be the case that for your specific problem/configuration Matlab is unable to use Levenberg-Marquardt algorithm.
Read the docs carefully to see if this is the case.
I can't say for certain, but the constaints ([-inf -inf], [inf inf]) could be your problem. The documentation for lsqcurvefit strictly says the LMA cannot be used for constrained problems. If constraints are included, it will fall back to trust-region.
Yes, your constraints are mathematically equivalent to 'no constaints', but I have no idea how the MATLAB function itself will interpret those. I tried to recreate the problem on my end, but optimset('LevenbergMarquardt', 'on') has been deprecated and generates an error (implying you have a relatively old version). Even when using the new syntax (optimset('Algorithm', 'levenberg-marquardt')), it behaves correctly on my end (using 2011b). To not have constraints, the correct approach is to use empty matrices (i.e. []).
Yes, the question is a month old, but someone else may find the answer useful.
Related
I have recieved some old code which uses the function fmincon and the algorithm LevenbergMarquardt to optimize my parameters. However this algorithm is no longer available in this function.
Since I am new to Matlab I am not sure about what the best alternative is.
I have tried to simply change the function to the ones that are compatible with LevenbergMarquardt but this doesn't seem to work.
Below are the vector of options and the fmincon function. "S", "A" and "b" are starting values for the parameters, "lb" and "ub" are upper and lower bounds.
Please write if anything is unclear or you you need additional information.
options_ = optimset('LevenbergMarquardt', 'on','TolFun',1e-6,'TolX',1e-6, 'HessUpdate', 'steepdesc', 'Display','iter', 'LargeScale', 'off', 'MaxFunEvals', 100000, 'MaxIter', 100000);
[ out_p, fval, exitfflag ] = fmincon(#MyLikelihoodFunction, S, A, b, [], [], lb, ub, [], options_);
What did you try and what went wrong?
I'd start with fmincon with all default options. That will give you the interior-point algorithm. Set 'Display' to 'iter' to see how the algorithm progresses. If the problem is large (although the old code has 'LargeScale' off), you might try 'HessianApproximation' set to 'lbfgs'.
A and b are not part of the starting point. Those define the linear inequality constraints. There's more info on that in the documentation link I gave above.
Actually I have to calculate values of 3 variables from probably 8 or 9 non-linear equations(may be more for accuracy).
I was using lsqnonlin and fsolve.
Using lsqnonlin, it says solver stopped prematurely (mainly due to value of iteration, funEvals and tolerance) and the output is far away from exact solution. I tried but I don't know on what basis I should set those parameters.
Using fsolve, it says no solution found.
I also used LMFnlsq and LMFsolve but it gives the output nowhere near the exact solution? I tried to change other parameters too but I could not bring those solutions to my desired values.
Is there any other way to solve these overdetermined non-linear equations?
My code till now:
x0 = [20 40 275];
eqn = #(x)[((((x(1)-Sat(1,1))^2+(x(2)-Sat(1,2))^2+(x(3)-Sat(1,3))^2))-dis(1)^2);
((((x(1)-Sat(2,1))^2+(x(2)-Sat(2,2))^2+(x(3)-Sat(2,3))^2))-dis(2)^2);
((((x(1)-Sat(3,1))^2+(x(2)-Sat(3,2))^2+(x(3)-Sat(3,3))^2))- dis(3)^2);
((((x(1)-Sat(4,1))^2+(x(2)-Sat(4,2))^2+(x(3)-Sat(4,3))^2))- dis(4))^2;
((((x(1)-Sat(5,1))^2+(x(2)-Sat(5,2))^2+(x(3)-Sat(5,3))^2))- dis(5))^2;
((((x(1)-Sat(6,1))^2+(x(2)-Sat(6,2))^2+(x(3)-Sat(6,3))^2))- dis(6))^2;
((((x(1)-Sat(7,1))^2+(x(2)-Sat(7,2))^2+(x(3)-Sat(7,3))^2))- dis(7))^2;
((((x(1)-Sat(8,1))^2+(x(2)-Sat(8,2))^2+(x(3)-Sat(8,3))^2))- dis(8))^2;
((((x(1)-Sat(9,1))^2+(x(2)-Sat(9,2))^2+(x(3)-Sat(9,3))^2))- dis(9))^2;
((((x(1)-Sat(10,1))^2+(x(2)-Sat(10,2))^2+(x(3)-Sat(10,3))^2))- dis(10))^2];
lb = [0 0 0];
ub = [100 100 10000];
options = optimoptions('lsqnonlin','MaxFunEvals',3000,'MaxIter',700,'TolFun',1e-18);%,'TolX',1);
x= lsqnonlin(eqn,x0,lb,ub,options)
**Error:**
**Solver stopped prematurely.**
lsqnonlin stopped because it exceeded the iteration limit,
options.MaxIter = 700 (the selected value).
x = 20.349 46.633 9561.5
Hoping for some suggestions!
Thanks in advance!
I usually model this explicitly:
min w'w
f_i(x) = w_i
w is a free variable
L<=x<=U
It should be easy to calculate a feasible (but non-optimal) solution in advance. If you can find a "good" initial solution that would be even better. Then use a general purpose NLP solver (e.g. fmincon) and pass on your initial feasible solution (both x and w). The best thing is to use a modeling system that allows automatic differentiation. Otherwise you should provide correct and precise gradients (and if needed second derivatives). See also the advice here.
The allfitdist function in MATLAB for normally distributed data return 'rayleigh' as best fit distribution!
Here is the link of function: http://www.mathworks.in/matlabcentral/fileexchange/34943-fit-all-valid-parametric-probability-distributions-to-data
data = normrnd(5,3,1e4,1);
[D PD] = allfitdist(data,'PDF');
D(1)
DistName: 'rayleigh'
NLogL: 2.4515e+04 - 1.5959e+03i
BIC: 4.9038e+04 - 3.1919e+03i
AIC: 4.9031e+04 - 3.1919e+03i
AICc: 4.9031e+04 - 3.1919e+03i
ParamNames: {'B'}
ParamDescription: {'scale'}
Params: 4.1166
Paramci: [2x1 double]
ParamCov: 4.2366e-04
Support: [1x1 struct]
So weird as it is an example included in file.
The second best fit is normal. The results are sorted by BIC, just wondering if I should change the sorting criteria or something is wrong with what I am doing.
Following user3278640's answer, I tried out several versions of matlab. 2012b showed the normal distribution, while 2013b and 2014b showed the rayleigh as the best approximation.
While inspecting the values, it seems that the negative log-likelihood of the rayleigh in 2013b and 2014b is a complex number. I don't think that makes much sense, so I changed the code around line 209 from:
%If NLL is non-finite number, produce error to ignore distribution
if ~isfinite(NLL)
error('non-finite NLL');
end
Into:
%If NLL is non-finite or complex number, produce error to ignore
%distribution
if ~isfinite(NLL) || ~isreal(NLL)
error('non-finite or non-real NLL');
end
That filters out the rayleigh distribution. Not really sure what changed between versions, but this might help.
For saving time of the others, I have to highlight that this problem arises when you are using function ALLFITDIST in MATLAB 2014a. In version 2010b, everything is fine.
I won't accept this answer as a correct one, since it does not contain any information relating to the part of function that cause this dependency and problem.
The following command
syms x real;
f = #(x) log(x^2)*exp(-1/(x^2));
fp(x) = diff(f(x),x);
fpp(x) = diff(fp(x),x);
and
solve(fpp(x)>0,x,'Real',true)
return the result
solve([0.0 < (8.0*exp(-1.0/x^2))/x^4 - (2.0*exp(-1.0/x^2))/x^2 -
(6.0*log(x^2)*exp(-1.0/x^2))/x^4 + (4.0*log(x^2)*exp(-1.0/x^2))/x^6],
[x == RD_NINF..RD_INF])
which is not what I expect.
The first question: Is it possible to force Matlab's solve to return the set of all solutions?
(This is related to this question.) Moreover, when I try to solve the equation
solve(fpp(x)==0,x,'Real',true)
which returns
ans =
-1.5056100417680902125994180096313
I am not satisfied since all solutions are not returned (they are approximately -1.5056, 1.5056, -0.5663 and 0.5663 obtained from WolframAlpha).
I know that vpasolve with some initial guess can handle this. But, I have no idea how I can generally find initial guessed values to obtain all solutions, which is my second question.
Other solutions or suggestions for solving these problems are welcomed.
As I indicated in my comment above, sym/solve is primarily meant to solve for analytic solutions of equations. When this fails, it tries to find a numeric solution. Some equations can have an infinite number of numeric solutions (e.g., periodic equations), and thus, as per the documentation: "The numeric solver does not try to find all numeric solutions for [the] equation. Instead, it returns only the first solution that it finds."
However, one can access the features of MuPAD from within Matlab. MuPAD's numeric::solve function has several additional capabilities. In particular is the 'AllRealRoots' option. In your case:
syms x real;
f = #(x)log(x^2)*exp(-1/(x^2));
fp(x) = diff(f(x),x);
fpp(x) = diff(fp(x),x);
s = feval(symengine,'numeric::solve',fpp(x)==0,x,'AllRealRoots')
which returns
s =
[ -1.5056102995536617698689500437312, -0.56633904710786569620564475006904, 0.56633904710786569620564475006904, 1.5056102995536617698689500437312]
as well as a warning message.
My answer to this question provides other way that various MuPAD solvers can be used, particularly if you can isolate and bracket your roots.
The above is not going to directly help with your inequalities other than telling you where the function changes sign. For those you could try:
s = feval(symengine,'solve',fpp(x)>0,x,'Real')
which returns
s =
(Dom::Interval(0, Inf) union Dom::Interval(-Inf, 0)) intersect solve(0 < 2*log(x^2) - 3*x^2*log(x^2) + 4*x^2 - x^4, x, Real)
Try plotting this function along with fpp.
While this is not a bug per se, The MathWorks still might be interested in this difference in behavior and poor performance of sym/solve (and the underlying symobj::solvefull) relative to MuPAD's solve. File a bug report if you like. For the life of me I don't understand why they can't better unify these parts of Matlab. The separation makes not sense from the perspective of a user.
Recently I'm doing some training of HMM, I used the HMM toolbox. But I have some problems and couldn't resolve them.
I train my hmm as shown below. There's no problems here.
[LL, prior1, transmatrix1, observematrix1] = dhmm_em(data, prior0, transmatrix0, observematrix0);
I use the Viterbi algorithm to find the most-probable path through the HMM state trellis.
function path = viterbi_path(prior, transmat, obslik);
Now there's a problem. I don't know what the "obslik" means. Is it the observematrix1?
I want to get the probability of a sequence, but I don't know whether I should use the "fwdback" function or not. If I should, what the "obslik" means then?
function [alpha, beta, gamma, loglik, xi_summed, gamma2] = fwdback(init_state_distrib, transmat, obslik, varargin);
Thanks!!!
I didn't understand the comments. Now I understand it.
The "obslik" here isn't equal to the observematrix1. Before using Viterbi_path function, we should compute the obslik:
obslik = multinomial_prob(data(m,:), observematrix1);
the data matrix is the observematrix0, observe-matrix before training.
Am I right?