I want to minimize some function J using gradient information. I found two functions in scipy that may do the job, scipy.optimize.fmin_tnc (here) and scipy.optimize.minimize (here), and I implemented them. However, now I need the stepwise output of the function evaluations at each step of the (e.g. Newton) algorithm to plot its convergence. Is it possible to get this vector somehow out of these functions? It is not part of default return values as it seems.
Related
I tried to use ode45 to solve an equation, and get output like the following. I get the idea it is trying to estimate using nearby points (as explained here https://www.mathworks.com/videos/solving-odes-in-matlab-6-ode45-117537.html). By my understanding, it should solve the equation in one round of computation? but the output looks like ode45 is an iterative algorithm (so that it generates output that repeat the '... steps ... failed attempt ... function evaluations' over and over again)? If it is iterative, could you help give some detail or references? Thanks!
ode45 is an iterative adaptive ODE solver. That is, it uses a 5th order (FSAL) method to propose the an update using some stepsize h. Then it does the same again, but now with a 4th order method, then it compared those two updates to one another, if the difference is less than some local tolerance, it accepts the proposed update. If the difference is larger than some local tolerance, the update is rejected and the stepsize is lowered (in some smart way).
To reduce the cost of using both a 4th and 5th order method, those two methods uses (roughly) the same function evaluations.
As for your output, it is, as also noted by #LutzL, not the standard output, which might point to an error in your code.
I'm trying to perform logistic regression to do classification using MATLAB. There seem to be two different methods in MATLAB's statistics toolbox to build a generalized linear model 'glmfit' and 'fitglm'. I can't figure out what the difference is between the two. Is one preferable over the other?
Here are the links for the function descriptions.
http://uk.mathworks.com/help/stats/glmfit.html
http://uk.mathworks.com/help/stats/fitglm.html
The difference is what the functions output. glmfit just outputs a vector of the regression coefficients (and some other stuff if you ask for it). fitglm outputs a regression object that packs all sorts of information and functionality inside (See the docs on GeneralizedLinearModel class). I would assume the fitglm is intended to replace glmfit.
In addition to Dan's answer, I would like to add the following.
The function fitglm, like newer functions from the statistics toolbox, accepts more flexible inputs than glmfit. For example, you can use a table as the data source, specifyy a formula of the form Y ~ X1 + X2 + ..., and use categorical variables.
As a side note, the function lassoglm uses (depends on) glmfit.
How to minimize multivariate function in Matlab by using derivatives?
So far, for minimizing single variable functions I used fminunc,
but now I need to work with multivariate functions.
Thank you
Use fminunc. If you want to use the gradient, just return it as the 2nd output of your objective function. You'll also need to indicate in an options object that you are passing the gradient.
options = optimoptions('fminunc','GradObj','on');
I believe the documentation has info on passing the Hessian (if one exists).
I need to fit data in quite an indirect way. The original data to be recovered in the fit is some linear function with small oscillations and drifts on it, that I would like to identify. Let's call this f(t). We can not record this parameter in the experiment directly, but only indirectly, let's say as g(f) = sin(a f(t)). (The real transfer funcion is more complex, but it should not play a role in here)
So if f(t) changes direction towards the turning points of the sin function, it is difficult to identify and I tried an alternative approach to recover f(t) than just the inverse function of g and some data continuing guesses:
I create a model function fm(t) which undergoes the same and known transfer function g() and fit g(fm(t)) to the data. As the dataset is huge, I do this piecewise for successive chunks of data guaranteeing the continuity of fm across the whole set.
A first try was to use linear functions using the optimize.leastsq, where the error estimate is derived from g(fm). It is not completely satisfactory, and I think it would be far better to fit a spline to the data to get fspline(t) as a model for f(t), guaranteeing the continuity of the data and of its derivative.
The problem with it is, that spline fitting from the interpolate package works on the data directly, so I can not wrap the spline using g(fspline) and do the spline interpolation on this. Is there a way this can be done in scipy?
Any other ideas?
I tried quadratic functions and fixing the offset and slope such to match the ones of the preceeding fitted chunk of data, so there is only one fitting parameter, the curvature, which very quickly starts to deviate
Thanks
What you would need is a matrix of spline basis functions, b(t), so you can approximate f(t) as a linear combination of spline basis function
f(t) = np.dot(b(t), coefs)
and then estimate the coefficients, coefs, by optimize.leastsq.
However, spline basis functions are not readily available in python, as far as I know (unless you borrow experimental scripts or search through the code of some packages).
Instead you could also use polynomials, for example
b(t) = np.polynomial.chebvander(t, order)
and use a polynomial approximation instead of the splines.
The structure of this problem is very similar to generalized linear models where g is your known link function and similar to index problems in econometrics.
It would be possible to use the scipy splines in an indirect way if you create artificial data
y_i = f(t_i)
where f(t_i) are scipy.interpolate splines, and the y_i are the parameters to be estimated in the least squares optimization. (Loosely based on a script that I saw some time ago that used this for creating a different kind of smoothing splines than the scipy version. I don't remember where I saw this.)
Thank you for these comments. I tried out the polynomial basis suggested above, but polynomials are no option for my needs, ads they tend to create ringing, which is difficult to condition.
The solution on using splines I now found is quite simple and straightforward, and I think it is what you meant by "using the splines in an indirect way".
The fitting function f(t) is obtained by the interpolate.splev(x, (t,c,k)) function, but providing the spline coefficients c by the omptimize.leastsq function. In this way, f(t) is no direct spline fit (as one would usually obtain with the splrep(x, y) function) but indirectly optimized in the fit, and therefore it is possible to use the link function g on it. The initial guess for c might be obtained by one evaluation of splrep(xinit, yinit, t=knots) on model data.
One trick is to restrict the number of knots for the spline to below the number of datapoints by explicitly specifying them during the function call of splrep() and giving this reduced set during the evaluation using splev().
I am looking for numerical integration with matlab. I know that there is a trapz function in matlab but the precision is not good enough. By searching it online, I found there is a quad function there it seems only accept symbolic expression as input. My data is all discrete and one-dimensional. Is that any way to use quad on my data? Thanks.
An answer to your question would be no. The only way to perform numerical integration for data with no expression in Matlab is by using the trapz function. If it's not accurate enough for you, try writing your own quad function as Li-aung said, it's very simple, this may help.
Another method you may try is to use the powerful Curve Fitting Tool cftool to make a fit then use the integrate function which can operate on cfit objects (it has a weird convention, the upper limit is the first argument!). I don't think you will get much accurate answers than trapz, it depends on the fit.
Use the spline function in MATLAB to interpolate your data, then integrate this data. This is the standard method for integrating data in discrete form.
You can use quadl() to integrate your data if you first create a function in which you interpolate them.
function f = int_fun(x,xdata,ydata)
f = interp1(xdata,ydata,x);
And then feed it to the quadl() function:
integral = quadl(#int_fun,A,B,[],[],x,y) % syntax to pass extra arguments
% to the function
Integration of a function of one variable is the computation of the area under the curve of the graph of the function. For this answer I'll leave aside the nasty functions and the corner cases and all the twists and turns that trip up writers of numerical integration routines, most of which are probably not relevant here.
Simpson's rule is an approach to the numerical integration of a function for which you have a code to evaluate the function at points within its domain. That's irrelevant here.
Let's suppose that your data represents a time series of values collected at regular intervals. Then you can plot your data as a histogram with bars of equal width. The integrand you seek is the sum of the areas of the bars in the histogram between the limits you are interested in.
You should be able to apply this approach to data sets where the x-axis (ie the width of the bars in the histogram) does not show time, to the situation where the bars are not of equal width, to the situation where the data crosses the x-axis, and most reasonable data sets, quite easily.
The discretisation of your data establishes a limit to the accuracy of the result you can get. If, for example, your time series is sampled at 1sec intervals you can't integrate over an interval which is not a whole number of seconds by this approach. But then, you don't really have the data on which to compute a figure with any more accuracy by any approach. Sure, you can use Matlab (or anything else) to generate extra digits of precision but they don't carry any meaning.