Does anyone know what criteria is used to decide the value of "epsilon" for each parameter in the calculation of the Jacobian matrix in scipy? I am using scipy.curve_fit where I get the covariance matrix, but I need to do this fit in matlab as well where I have to manually calculate the Jacobian and convert to covariance matrix. I have not been able to find the criteria by which scipy decides on the default shift in parameter (i.e. epsilon) used to determine the function gradient
Related
I need to compute efficiently and in a numerically stable way the inverse CDF F^-1(y) (cumulative distribution function) of a probability function, assuming that both the PDF f(x) and the CDF F(x) are known analytically but the inverse CDF is not. I am doing this in MATLAB.
This is a root-finding problem for F(x)-y and I could use fzero:
invcdf = #(y, x0) fzero(#(x) cdf(x) - y, x0);
However, fzero is for a generic nonlinear function.
I wonder if there is some function, or I can write some algorithm that uses the explicit information that F(x) is a cdf (for example, we know that it is monotonically non-decreasing and we have its derivative, f(x)).
FYI, the shape of the PDFs I am working with is generic mixtures of Gaussian distributions multiplied by a polynomial of arbitrary degree (the CDF can be computed analytically in this case, although it's not pretty and it becomes expensive for polynomials with many terms). Note that I need to compute the inverse CDF for millions of CDFs within this class; a lookup table is not feasible.
For more mathematical details see also this related question on Math Exchange (here I am asking specifically for a MATLAB solution).
I am solving a hug optimization problem that takes a lot of time to converge to a solution. This is for the reason that Matlab uses finite difference method for calculating the Gradient of objective functions and nonlinear constraint and also constructing Hessian matrix. But there is an option in fmincon solver that allow you to supply the analytic derivative of functions and constraints.
For this reason I wanted to know how can I calculate the Grad of the namely function which is given here both in mathematical aspect and symbolic math tool. I should note that still I want the gradient of the objective in the vector format. (not by extracting Eq1 in 5 equation.)
Lets assume we have these optimization variables
Pd=[x1 x2 x3 x4]
Now we define these 2 variables based on optimization vector i.e.,Pd
Pdn=[pd(1);mo;Pd(2);0;Pd(4)]
Pgn=[pd(2);Pd(1);m1;Pd(4),Pd(1)]
Now this is the equation that I want to take the gradient from:
Eq1=Sin(Pdn)+Pdn+Pgn.^2
I wanted to fit an arbitrary function to my data set. Therefore, I used lsqcurvefit in MATLAB. Now I want to give weight to the fit procedure, meaning when curve fitting function (lsqcurvefit) is calculating the residue of the fit, some data point are more important than the others. To be more specific I want to use statistical weighting method.
w=1/y(x),
where w is a matrix contains the weight of each data point and y is the data set.
I cannot find anyway to make weighted curve fitting with lsqcurvefit. Is there any trick I should follow or is there any other function rather than lsqcurvefit which do it for me?
For doing weighting, I find it much easier to use lsqnonlin which is the function that lsqcurvefit calls to do the actual fitting.
You first have to define a function that you are trying to minimize, ie. a cost function. You need to pass in your weighting function as an extra parameter to your function as a vector:
x = yourIndependentVariable;
y = yourData;
weightVector = sqrt(abs(1./y));
costFunction = #(A) weightVector.*(yourModelFunction(A) - y);
aFit = lsqnonlin(costFunction,aGuess);
The reason for the square root in the weighting function definition is that lsqnonlin requires the residuals, not the squared residuals or their sum, so you need to pre-unsquare the weights.
Alternatively, if you have the Statistics Toolbox, you can use nlinfit which will accept a weighting vector/matrix as one of the optional inputs.
I have to calculate:
gamma=(I-K*A^-1)*OLS;
where I is the identity matrix, K and A are diagonal matrices of the same size, and OLS is the ordinary least squares estimate of the parameters.
I do this in Matlab using:
gamma=(I-A\K)*OLS;
However I then have to calculate:
gamma2=(I-K^2*A-2)*OLS;
I calculate this in Matlab using:
gamma2=(I+A\K)*(I-A\K)*OLS;
Is this correct?
Also I just want to calculate the variance of the OLS parameters:
The formula is simple enough:
Var(B)=sigma^2*(Delta)^-1;
Where sigma is a constant and Delta is a diagonal matrix containing the eigenvalues.
I tried doing this by:
Var_B=Delta\sigma^2;
But it comes back saying matrix dimensions must agree?
Please can you tell me how to calculate Var(B) in Matlab, as well as confirming whether or not my other calculations are correct.
In general, matrix multiplication does not commute, which makes A^2 - B^2 not equal to (A+B)*(A-B). However your case is special, because you have an identity matrix in the equation. So your method for finding gamma2 is valid.
'Var_B=Delta\sigma^2' is not a valid mldivide expression. See the documentation. Try Var_B=sigma^2*inv(Delta). The function inv returns a matrix inverse. Although this function can also be applied in your expression to find gamma or gamma2, the use of the operator \ is more recommended for better accuracy and faster computation.
When using MATLAB's lsqnonlin function, I am trying to give a user-defined Jacobian matrix, as described in the documentation.
The output of the objective function used in lsqnonlin should be a vector of unsquared values, which, when squared and summed, give the energy. However, should the Jacobian be the partial derivates of the squared or unsquared values?
The unsquared values is correct.