Incompatibility between functools.partial and scipy - scipy

I need to pass the inverse of a cdf as an argument to function.
I cannot understand why this does not work:
def InvNormal(q,a,b):
return norm.ppf(q,a,b)
F_invs = functools.partial(InvNormal,0,1)
Doing
print(InvNormal(0.3,0,1))
print(F_invs(0.3))
I get
-0.5244005127080409
-inf
Any help is appreciated!
Bernardo
I tried it with different random variables.

Related

Using MLE function to estimate the parameters of a custom distribution

I am trying to use mle() function in MATLAB to estimate the parameters of a 6-parameter custom distribution.
The PDF of the custom distribution is
and the CDF is
where Γ(x,y) and Γ(x) are the upper incomplete gamma function and the gamma function, respectively. α, θ, β, a, b, and c are the parameters of the custom distribution. K is given by
Given a data vector 'data', I want to estimate the parameters α, θ, β, a, b, and c.
So, far I have come up with this code:
data = rand(20000,1); % Since I cannot upload the acutal data, we may use this
t = 0:0.0001:0.5;
fun = #(w,a,b,c) w^(a-1)*(1-w)^(b-1)*exp^(-c*w);
% to estimate the parameters
custpdf = #(data,myalpha,mybeta,mytheta,a,b,c)...
((integral(#(t)fun(t,a,b,c),0,1)^-1)*...
mybeta*...
igamma(myalpha,((mytheta/t)^mybeta)^(a-1))*...
(mytheta/t)^(myalpha*mybeta+1)*...
exp(-(mytheta/t)^mybeta-(c*(igamma(myalpha,(mytheta/t)^mybeta)/gamma(myalpha)))))...
/...
(mytheta*...
gamma(myalpha)^(a+b-1)*...
(gamma(myalpha)-igamma(myalpha,(mytheta/t)^mybeta))^(1-b));
custcdf = #(data,myalpha,mybeta,mytheta,a,b,c)...
(integral(#(t)fun(t,a,b,c),0,1)^-1)*...
integral(#(t)fun(t,a,b,c),0,igamma(myalpha,(mytheta/t)^mybeta)^mybeta/gamma(myalpha));
phat = mle(data,'pdf',custpdf,'cdf',custcdf,'start',0.0);
But I get the following error:
Error using mlecustom (line 166)
Error evaluating the user-supplied pdf function
'#(data,myalpha,mybeta,mytheta,a,b,c)((integral(#(t)fun(t,a,b,c),0,1)^-1)*mybeta*igamma(myalpha,((mytheta/t)^mybeta)^(a-1))*(mytheta/t)^(myalpha*mybeta+1)*exp(-(mytheta/t)^mybeta-(c*(igamma(myalpha,(mytheta/t)^mybeta)/gamma(myalpha)))))/(mytheta*gamma(myalpha)^(a+b-1)*(gamma(myalpha)-igamma(myalpha,(mytheta/t)^mybeta))^(1-b))'.
Error in mle (line 245)
phat = mlecustom(data,varargin{:});
Caused by:
Not enough input arguments.
I tried to look into the error lines but I can't figure out where the error actually is.
Which function lacks fewer inputs? Is it referring to fun? Why would mle lack fewer inputs when it is trying to estimate the parameters?
Could someone kindly help me debug the error?
Thanks in advance.
exp() is a function, not a variable, precise the argument
exp^(-c*w) ---> exp(-c*w)
The starting point concerns the 6 parameters, not only one
0.1*ones(1,6)
In custcdf mle requires the upper bound of the integral to be a
scalar, I did some trial and error and the range is [2~9]. for the
trial some values lead to negative cdf or less than 1 discard them.
Then use the right one to compute the upper bound see if it's the
same as the one you predefined.
I re-write all the functions, check them out
The code is as follow
Censored = ones(5,1);% All data could be trusted
data = rand(5,1); % Since I cannot upload the acutal data, we may use this
f = #(w,a,b,c) (w.^(a-1)).*((1-w).^(b-1)).*exp(-c.*w);
% to estimate the parameters
custpdf = #(t,alpha,theta,beta, a,b,c)...
(((integral(#(w)f(w,a,b,c), 0,1)).^-1).*...
beta.*...
((igamma(alpha, (theta./t).^beta)).^(a-1)).*...
((theta./t).^(alpha.*beta + 1 )).*...
exp(-(((theta./t).^beta)+...
c.*igamma(alpha, (theta./t).^beta)./gamma(alpha))))./...
(theta.*...
((gamma(alpha)).^(a+b-1)).*...
((gamma(alpha)-...
igamma(alpha, (theta./t).^beta)).^(1-b)));
custcdf = #(t,alpha,theta,beta, a,b,c)...
((integral(#(w)f(w,a,b,c), 0,1)).^-1).*...
(integral(#(w)f(w,a,b,c), 0,2));
phat = mle(data,'pdf',custpdf,'cdf',custcdf,'start', 0.1.*ones(1,6),'Censoring',Censored);
Result
phat = 0.1017 0.1223 0.1153 0.1493 -0.0377 0.0902

Solve trigonometrics equations in Matlab

I'm trying to solve for t trigonometric equations in Matlab, as i.e. 7*cos(t) + 5*sin(t) = 0. I would solve it as: sin(t)/cos(t) = -7/5 and I would find it as arctan(-7/5) = -0.9505.
I have tried to do it on matlab using solve function:
syms t
theta = solve(7*cos(t) + 5*sin(t)==0, t);
disp(theta);
But I get -(log(- 12/37 - (35*i)/37)*i)/2 instead of -0.9505. Could someone explain me why I get this answer from solve and how to get -0.9505?
The expression is the exact result, expressed symbolically (due to the use of syms).
To make Matlab display the result in the format your looking for use double(theta)
which should give you:
double(theta)
ans =
-0.9505

Using Interpolation in Matlab to return a function

I've got a silly problem, I'm looking to take a few data points, fit a polynomial function through it and then differentiate that function to get that particular functions optimal point. As such I have done some reading online and I've used the Matlab 'spline' function. Here is some code:
a = [50; 100; 150;200;250;300;350]
b = [56;23;22;18;14;15;21]
y = spline(a,b)
But when I used diff(y) I get the following error:
??? Error using ==> diff
Function 'diff' is not supported for class 'struct'.
I'm not too familiar with Matlab, so any help would really be appreciated
As per comments:
y = polyfit(a,b,2)
syms x
g = y(1)*x^2 + y(2)*x + y(3)
diff(g)
and you get the derivative of the function g. Much thanks to the guys in the comment section!

trying to understand scipy's optimize.minimize function, getting indexerror

I'm trying to write code that will optimize a multivariate function using sklearn's optimize function, but it keeps returning an IndexError, and I'm not sure where to go from here.
The code is this:
revcoeff = coefficients[::-1]
xdot = np.zeros(0)
normfeat1 = normfeat1.reshape(-1,1)
xdot = np.append(normfeat1, normfeat2.reshape(-1,1), axis=1)
a = revcoeff[1:3]
b = xdot[0, :]
seed = np.zeros(5) #does seed need to be the coefficients? not sure
fun = lambda x: np.multiply((1/666), np.power(np.sum(np.dot(a, xdot[x, :])-medianv[x]),2)) #costfunction
optsol = optimize.minimize(fun, seed)
where there are two features I'm using in my nearest neighbors algorithm. Coefficients for the fitted regression model are given into the array "coefficients".
What I'm having trouble understanding is 1) why my code is throwing a "IndexError: arrays used as indicies must be of integer or boolean type"....and also partially I'm confused by the optimize.minimize function itself. It takes in two input values, the function and x0 (an ndarray with initial guesses). What should x0 be, the coefficients values? Or do I pick random values, and how many are necessary?
np.zeros() does not return integers by default. Try, for example, np.zeros(5, dtype=int) instead. It won't solve all of the problems with your code, though. You'll see some other error message.
Also, notice, that 1/666 returns 0 instead of 0.00150150. You probably want 1/666.0.
It would be helpful if you could clean up your code as half of it is of no use.

fitting curve with nonlinear function in matlab

I've a problem using matlab. I need to fit a dataset with a nonlinear function like:
f=alfa*(1+beta*(zeta))^(1/3)
where alfa and beta are the coefficients to be found. I want to use the least squares method. How can I do this with the command lsqcurvefit? Otherwise, there are other ways to solve my problem?
Thank so much.
Here there is the dataset:
zeta val
0.001141174 1.914017718
0.010606563 1.36090774
0.021610291 1.906194276
0.070026172 1.87606762
0.071438139 1.877264055
0.081679327 1.859341737
0.101181292 2.518896436
0.107877774 2.772125094
0.205038829 3.032759627
0.211802706 1.483644094
0.561521724 2.424261001
0.61500615 2.559041397
0.647249191 2.949944577
0.943396226 2.84068921
1.091107474 3.453699422
1.175260761 2.604008404
1.837813003 4.00262983
2.057613169 4.565849247
2.083333333 3.779001445
3.188521323 4.430824069
4.085801839 7.766971568
4.22832981 5.711800741
4.872107186 4.949950059
9.756097561 10.78574156
you have to use the fit-function with fitType=Power2
fitobject = fit(zeta2,val,'Power2')
you can also use the cftool to manually determine your coefficients, especially if you want to keep the (1/3). Maybe Least-Squares is not the best solution for your data, as woodchips said.
be aware that you have to substitute your zeta:
zeta2 = 1+beta*(zeta)
you can determine the coefficients as follows:
coeffvalues(fitobject)