Both fmincon and fminunc report a hessian matrix. I could not find in the documentation about how is the hessian calculated. Is it the average hessian of all sample data point?
The returned Hessian will typically correspond to either the last, or next-to-last, iterate ("point") prior to termination. Several details come into play though (BFGS approximation? User supplied Hessian evalutation?, etc.), but all that is nicely summarized in the documentation.
Related
Can Fisher Information matrix be calculated for any matrix? I am doing my work on image processing field for face identification. How can I calculate Fisher information matrix on my input image(which is a matrix of pixels indeed)?
You can use empirical Fisher information, however you would need to specify a parametric likelihood for your data. Given that you know the form of your likelihood, you can evaluate the Hessian at your parameter values. Intuitively, if the Hessian or curvature of log likelihood is high you are more certain about the parameter estimates. To compute the Fisher information matrix you would then take the empirical average of the observed information matrix.
I have two covariances of size 6*6, one is supposed be the true covariance and the other is the Maximum likelihood estimate for my covariance. Is there any way I could validate my estimated covariance?
I don't know how exactly you determined your covariance matrix, but generally it is a good first step to check the confidence intervals of your estimators.
Heuristically speaking a wide confidence interval suggests that your estimator has a lot of uncertainty.
Take a look at the Matlab function corrcoef, which also gives lower and upper bounds for the estimated correlation coefficients,
cf. https://uk.mathworks.com/help/matlab/ref/corrcoef.html#bunkanr .
Maybe using this function on your data gives you a good starting point. If you use your own function to estimate the ML estimators, you will have to add the confidence intervals yourself.
When solving the log likelihood expression for autoregressive models, I cam across the variance covariance matrix Tau given under slide 9 Parameter estimation of time series tutorial. Now, in order to use
fminsearch
to maximize the likelihood function expression, I need to express the likelihood function where the variance covariance matrix arises. Can somebody please show with an example how I can implement (determinant of Gamma)^-1/2 ? Any other example apart from autoregressive model will also do.
How about sqrt(det(Gamma)) for the sqrt-determinant and inv(Gamma) for inverse?
But if you do not want to implement it yourself you can look at yulewalkerarestimator
UPD: For estimation of autocovariance matrix use xcov
also, this topic is a bit more explained here
In minimizing a convex objective function, does it mean that the Hessian matrix at minimizer should be PSD? If fminunc in Matlab returns a hessian which is not psd what does it mean? am I using a wrong objective?
I do that in environments other than matlab.
Non-PSD means you can't take the Cholesky transform of it (i.e. the matrix square-root), so you can't use it to get standard errors, for example.
To get a good hessian, your objective function has to be really smooth, because you're taking a second derivative, which doubly amplifies any noise. If possible, it is best to use analytic derivatives rather than finite-difference. That is, if you really need the hessian.
I would like to know if anybody knows how I can plot an integral calculated using quad/quadl, or if this is possible.
I read that I can set the trace parameter to be non-zero, and this results in the information of each iteration being provided, but I'm not sure how and if I can use the information to plot an integral.
Thanks.
quad and quadl do not compute an integral function anyway, i.e., an integral as a function of the parameter. And since tools like this work iteratively, refining their estimate until it satisfies a tolerance on the global value, they are not easily made to produce the plot you desire.
You can do what you desire by using a differential equation solver to generate the solution, ode45 for example.