I am trying to create an array inside numba jit function. I tried these two options and both return various errors
#njit
def monte_carlo_ou(T: int, nsamples: int = 100, dt:float=0.1, sigma_g:float =1.0, ):
particles = np.zeros(T, nsamples, dtype=float)
velocity_particles = np.zeros(nsamples , dtype=float)
return particles
monte_carlo_ou(100, 50, dt=0.1, sigma_g=1.0,)
TypingError: Cannot determine Numba type of <class 'numba.core.ir.UndefinedType'>
#njit
def monte_carlo_ou(T: int, nsamples: int = 100, dt:float=0.1, sigma_g:float =1.0, ):
particles = numba.float32[:, :]
velocity_particles = numba.float32[:]
return particles
monte_carlo_ou(100, 50, dt=0.1, sigma_g=1.0,)
Error :
TypingError: No implementation of function Function(<built-in function getitem>) found for signature:
getitem(class(float32), UniTuple(slice<a:b> x 2))
There are 22 candidate implementations:
- Of which 22 did not match due to:
Overload of function 'getitem': File: <numerous>: Line N/A.
With argument(s): '(class(float32), UniTuple(slice<a:b> x 2))':
No match.
During: typing of intrinsic-call at /var/folders/43/m_k444pn2z7c6ygxj0y3_c4r0000gn/T/ipykernel_58943/4011707724.py (6)
During: typing of static-get-item at /var/folders/43/m_k444pn2z7c6ygxj0y3_c4r0000gn/T/ipykernel_58943/4011707724.py (6)
Any advise on how to get started? All tutorials seems to be very limited
The problem is your first in-function line: the shape in np.zeros should be one tuple argument, not multiple scalar arguments.
particles = np.zeros(shape=(T, nsamples), dtype=float) worked for me.
shape= is not necessary but is explicit about how you should provide the shape of the array.
Related
I have my function:
function [result] = my_func(x,y)
result = y^2*(1-3*x)-3*y;
endfunction
Also, my vector with Ts, my function address and my initial variable x_0
load file_with_ts
# Add my limits as I also want to calculate those
# (all values in file_with_ts are within those limits.)
t_points = [-1, file_with_ts, 2]
myfunc = str2func("my_func")
x_0 = 0.9142
I am trying to execute the following line:
lsode_d1 = lsode(myfunc, x_0, t_points)
And expecting a result, but getting the following error:
INTDY-- T (=R1) ILLEGAL
In above message, R1 = 0.7987082301475D+00
T NOT IN INTERVAL TCUR - HU (= R1) TO TCUR (=R2)
In above, R1 = 0.8091168896311D+00 R2 = 0.8280400838323D+00
LSODE-- TROUBLE FROM INTDY. ITASK = I1, TOUT = R1
In above message, I1 = 1
In above message, R1 = 0.7987082301475D+00
error: lsode: invalid input detected (see printed message)
error: called from
main at line 20 column 10
Also, the variable sizes are:
x_0 -> 1x1
t_points -> 1x153
myfunc -> 1x1
I tried transposing the t_points vector
using #my_func instead of the str2func function
I tried adding multiple variables as the starting point (instead of x_0 I entered [x_0; x_1])
Tried changing my function header from my_func(x, y) to my_func(y, x)
Read the documentation and confirmed that my_func allows x to be a vector and returns a vector (whenever x is a vector).
EDIT: T points is the following 1x153 matrix (with -1 and 2 added to the beggining and the end respectively):
-4.9451e-01
-4.9139e-01
-4.7649e-01
-4.8026e-01
-4.6177e-01
-4.5412e-01
-4.4789e-01
-4.2746e-01
-4.1859e-01
-4.0983e-01
-4.0667e-01
-3.8436e-01
-3.7825e-01
-3.7150e-01
-3.5989e-01
-3.5131e-01
-3.4875e-01
-3.3143e-01
-3.2416e-01
-3.1490e-01
-3.0578e-01
-2.9267e-01
-2.9001e-01
-2.6518e-01
-2.5740e-01
-2.5010e-01
-2.4017e-01
-2.3399e-01
-2.1491e-01
-2.1067e-01
-2.0357e-01
-1.8324e-01
-1.8112e-01
-1.7295e-01
-1.6147e-01
-1.5424e-01
-1.4560e-01
-1.1737e-01
-1.1172e-01
-1.0846e-01
-1.0629e-01
-9.4327e-02
-8.0883e-02
-6.6043e-02
-6.6660e-02
-6.1649e-02
-4.7245e-02
-2.8332e-02
-1.8043e-02
-7.7416e-03
-6.5142e-04
1.0918e-02
1.7619e-02
3.4310e-02
3.3192e-02
5.2275e-02
5.5756e-02
6.8326e-02
8.2764e-02
9.5195e-02
9.4412e-02
1.1630e-01
1.2330e-01
1.2966e-01
1.3902e-01
1.4891e-01
1.5848e-01
1.7012e-01
1.8026e-01
1.9413e-01
2.0763e-01
2.1233e-01
2.1895e-01
2.3313e-01
2.4092e-01
2.4485e-01
2.6475e-01
2.7154e-01
2.8068e-01
2.9258e-01
3.0131e-01
3.0529e-01
3.1919e-01
3.2927e-01
3.3734e-01
3.5841e-01
3.5562e-01
3.6758e-01
3.7644e-01
3.8413e-01
3.9904e-01
4.0863e-01
4.2765e-01
4.2875e-01
4.3468e-01
4.5802e-01
4.6617e-01
4.6885e-01
4.7247e-01
4.8778e-01
4.9922e-01
5.1138e-01
5.1869e-01
5.3222e-01
5.4196e-01
5.4375e-01
5.5526e-01
5.6629e-01
5.7746e-01
5.8840e-01
6.0006e-01
5.9485e-01
6.1771e-01
6.3621e-01
6.3467e-01
6.5467e-01
6.6175e-01
6.6985e-01
6.8091e-01
6.8217e-01
6.9958e-01
7.1802e-01
7.2049e-01
7.3021e-01
7.3633e-01
7.4985e-01
7.6116e-01
7.7213e-01
7.7814e-01
7.8882e-01
8.1012e-01
7.9871e-01
8.3115e-01
8.3169e-01
8.4500e-01
8.4168e-01
8.5705e-01
8.6861e-01
8.8211e-01
8.8165e-01
9.0236e-01
9.0394e-01
9.2033e-01
9.3326e-01
9.4164e-01
9.5541e-01
9.6503e-01
9.6675e-01
9.8129e-01
9.8528e-01
9.9339e-01
Credits to Lutz Lehmann and PierU.
The problem lied in the array t_points not being a monotonous array. Adding a sort(t_points) before doing any calculations fixed the error.
I'm still not very familiar with numba and my problem is that I have the piece of code bellow that I use for randomize the edges of graphs.
This code is simply used to swap some edges in a connectivity matrix given the number of desired swaps and a seed for the random number generator.
My problem is that when I try to use it with numba to speed it up I did not menage to run it. The error it returns is also pasted bellow.
#nb.jit(nopython=True)
def _randomize_adjacency_wei(A, n_swaps, seed):
np.random.seed(seed)
# Number of nodes
n_nodes = A.shape[0]
# Copy the adj. matrix
Arnd = A.copy()
# Choose edges that will be swaped
edges = np.random.choice(n_nodes, size=(4, n_swaps), replace=True).T
#itr = range(n_swaps)
#for it in tqdm(itr) if verbose else itr:
it = 0
for it in range(n_swaps):
i,j,k,l = edges[it,:]
if len(np.unique([i,j,k,l]))<4:
continue
else:
# Old values of weigths
w_ij,w_il,w_kj,w_kl=Arnd[i,j],Arnd[i,l],Arnd[k,j],Arnd[k,l]
# Swaping edges
Arnd[i,j]=Arnd[j,i]=w_il
Arnd[k,l]=Arnd[l,k]=w_kj
Arnd[i,l]=Arnd[l,i]=w_ij
Arnd[k,j]=Arnd[j,k]=w_kl
return Arnd
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function unique at 0x7f1a1c03b0d0>) found for signature:
>>> unique(list(int64)<iv=None>)
There are 2 candidate implementations:
- Of which 2 did not match due to:
Overload in function 'np_unique': File: numba/np/arrayobj.py: Line 1915.
With argument(s): '(list(int64)<iv=None>)':
Rejected as the implementation raised a specific error:
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Unknown attribute 'ravel' of type list(int64)<iv=None>
File "../../../home/vinicius/anaconda3/lib/python3.8/site-packages/numba/np/arrayobj.py", line 1918:
def np_unique_impl(a):
b = np.sort(a.ravel())
^
During: typing of get attribute at /home/vinicius/anaconda3/lib/python3.8/site-packages/numba/np/arrayobj.py (1918)
File "../../../home/vinicius/anaconda3/lib/python3.8/site-packages/numba/np/arrayobj.py", line 1918:
def np_unique_impl(a):
b = np.sort(a.ravel())
^
raised from /home/vinicius/anaconda3/lib/python3.8/site-packages/numba/core/typeinfer.py:1071
During: resolving callee type: Function(<function unique at 0x7f1a1c03b0d0>)
During: typing of call at <ipython-input-165-90ffd30fe0e8> (19)
File "<ipython-input-165-90ffd30fe0e8>", line 19:
def _randomize_adjacency_wei(A, n_swaps, seed):
<source elided>
i,j,k,l = edges[it,:]
if len(np.unique([i,j,k,l]))<4:
^
Thanks in advance,
Vinicius
According to the comments, you are passing a list to np.unique() but this is not supported by Numba.
Modifying the code this way:
i, j, k, l = e = edges[it, :]
if len(np.unique(e)) < 4:
...
The following example doesn't produce any errors:
>>> A = np.random.randint(0, 5, (8,8))
>>> r = _randomize_adjacency_wei(A, 4, 33)
I am trying to run a hyperparameters optimization with Talos. As I have a lot of parameters to test, I want to use a 'grid_downsample' argument that will select 30% of all possible hyperparameters combinations. However when I run my code I get: TypeError: __init__() got an unexpected keyword argument 'grid_downsample'
I tested the code below without the 'grid_downsample' option and with less hyperparameters.
#load data
data = pd.read_csv('data.txt', sep="\t", encoding = "latin1")
# split into input (X) and output (y) variables
Y = np.array(data['Y'])
data_bis = data.drop(['Y'], axis = 1)
X = np.array(data_bis)
p = {'activation':['relu'],
'optimizer': ['Nadam'],
'first_hidden_layer': [12],
'second_hidden_layer': [12],
'batch_size': [20],
'epochs': [10,20],
'dropout_rate':[0.0, 0.2]}
def dnn_model(x_train, y_train, x_val, y_val, params):
model = Sequential()
#input layer
model.add(Dense(params['first_hidden_layer'], input_shape=(1024,)))
model.add(Dropout(params['dropout_rate']))
model.add(Activation(params['activation']))
#hidden layer 2
model.add(Dense(params['second_hidden_layer']))
model.add(Dropout(params['dropout_rate']))
model.add(Activation(params['activation']))
# output layer with one node
model.add(Dense(1))
model.add(Activation(params['activation']))
# Compile model
model.compile(loss='binary_crossentropy', optimizer=params['optimizer'], metrics=['accuracy'])
out = model.fit(x_train, y_train,
batch_size=params['batch_size'],
epochs=params['epochs'],
validation_data=[x_val, y_val],
verbose=0)
return out, model
scan_object = ta.Scan(X, Y, model=dnn_model, params=p, experiment_name="test")
reporting = ta.Reporting(scan_object)
report = reporting.data
report.to_csv('./Random_search/dnn/report_talos.txt', sep = '\t')
This code works well. If I change the scan_object as the end to: scan_object = ta.Scan(X, Y, model=dnn_model, grid_downsample=0.3, params=p, experiment_name="test"), it gives me the error: TypeError: __init__() got an unexpected keyword argument 'grid_downsample' while I was expecting to have the same results format as a normal grid search but with less combinations. What am I missing? Did the name of the argument change? I'm using Talos 0.6.3 in a conda environment.
Thank you!
might be too late for you now but they've switched it to fraction_limit. It would give this for you
scan_object = ta.Scan(X, Y, model=dnn_model, params=p, experiment_name="test", fraction_limit = 0.1)
Sadly, the doc isn't well updated
Check out their examples on GitHub:
https://github.com/autonomio/talos/blob/master/examples/Hyperparameter%20Optimization%20with%20Keras%20for%20the%20Iris%20Prediction.ipynb
I am trying to fit data to a function f(x) that is an integral over T. x is the upper boarder of the Integral. I am trying to do it with scipy.curve_fit() but I don't know how to write my integral as a function that can be passed to curve_fit.
I had a look at similar problems but I didn't see anything that fits to my problem.
I cannot provide any guess values for A and Ea since I have no clue at all what they could be as of now.
from scipy import optimize
import matplotlib.pyplot as plt
import numpy as np
from scipy import interpolate
from scipy import integrate
class get_Ton:
def __init__(self):
self.data=np.genfromtxt('test3.csv', delimiter=',', skip_header=8)
def loop(self):
def fit_tangent():
tck = interpolate.splrep(self.x, self.y, k=2, s=0)
dev_1 = interpolate.splev(self.x, tck, der=1)
def integrand(T, A, Ea):
return A*np.exp(-Ea/(8.314*T))
def polyn(x, A, Ea):
return integrate.quad(integrand, 25, x, args=(A, Ea))[0]
vcurve = np.vectorize(polyn)
p, e = optimize.curve_fit(vcurve, self.x, self.y, [2000, 150])
xd = np.linspace(50, 70, 100)
plt.plot(self.x, self.y, "o")
plt.plot(xd, vcurve(xd, *p))
for i in range((list(np.shape(self.data)[1:2]))[0]):
if i % 2 == 0:
self.temp=self.data[:,i]
self.scat=self.data[:,i+1]
self.x=[26.192, 26.861000000000001, 27.510000000000002, 28.178000000000001, 28.856000000000002, 29.515000000000001, 30.183, 30.823, 31.5, 32.158999999999999, 32.856000000000002, 33.515000000000001, 34.145000000000003, 34.823, 35.491, 36.168999999999997, 36.837000000000003, 37.533999999999999, 38.164000000000001, 38.832000000000001, 39.481000000000002, 40.158999999999999, 40.826999999999998, 41.496000000000002, 42.164000000000001, 42.832000000000001, 43.500999999999998, 44.188000000000002, 44.837000000000003, 45.505000000000003, 46.173000000000002, 46.832000000000001, 47.500999999999998, 48.188000000000002, 48.828000000000003, 49.496000000000002, 50.173999999999999, 50.813000000000002, 51.481000000000002, 52.112000000000002, 52.808999999999997, 53.439, 54.116, 54.765999999999998, 55.453000000000003, 56.101999999999997, 56.761000000000003, 57.429000000000002, 58.078000000000003, 58.737000000000002, 59.442999999999998, 60.082999999999998, 60.770000000000003, 61.448, 62.125999999999998, 62.756, 63.414999999999999, 64.082999999999998, 64.742000000000004, 65.420000000000002, 66.087999999999994, 66.747, 67.415000000000006]
self.y=[1553.5, 1595.0, 1497.8, 1695.5999999999999, 1328.7, 1279.0, 1547.8, 1412.8, 1037.0, 1473.5, 1447.4000000000001, 1532.5999999999999, 1484.2, 1169.5, 1395.2, 1183.5999999999999, 949.01999999999998, 1238.0999999999999, 1225.5999999999999, 924.80999999999995, 1650.5999999999999, 803.96000000000004, 1245.7, 1190.0, 1207.0, 1294.0, 1174.9000000000001, 1229.8, 1260.0, 1129.2, 1142.9000000000001, 987.63999999999999, 1389.5999999999999, 1366.0, 1102.0999999999999, 1325.5, 1258.9000000000001, 1285.7, 1217.5, 871.47000000000003, 820.24000000000001, 1388.7, 1391.0, 1400.3, 2482.5999999999999, 3360.5999999999999, 7013.5, 11560.0, 16525.0, 22538.0, 32556.0, 43878.0, 59093.0, 67977.0, 75949.0, 82316.0, 90213.0, 90294.0, 99928.0, 128240.0, 181280.0, 226380.0, 223260.0]
fit_tangent()
plt.ylim((-100,1000000))
plt.show()
def main():
this=get_Ton()
this.loop()
if __name__ == "__main__":
main()
Three issues here. First, the function polyn does not depend on the variable of integration T, since this variable is integrated out. Remove T from its list of parameters. Accordingly, drop one of numerical values in trueydata computation:
trueydata = vcurve(truexdata, 3, 4)
Second, quad returns a tuple (integral_value, integral_error). Use [0] to return only the integral value.
def polyn(x, A, Ea):
return integrate.quad(integrand, 25, x, args=(A, Ea))[0]
Third, provide an initial guess for parameter values to curve_fit. If you don't, it will likely report unable to determine the number of parameters to fit. Even if successful, it will blindly use all-ones for the initial guess. An initial guess supplied by a human with an understanding of the optimization problem is often the difference between success and failure of multivariable optimization.
popt, pcov = optimize.curve_fit(vcurve, xdata, ydata, [2, 3])
I've written a python script to evaluate a physical quantity and, for some reason, python decided to raise an OverflowError for no justified reason. Here's the script
import numpy as np
from math import sqrt,cos, log, pi
from scipy import integrate as sciint
from scipy import optimize as sciopt
rt=np.inf
ymin=cos(np.radians(0.5))
def func(u,y, D, rt):
return (1.+u)**(-4)/u/sqrt(u*u-D**2*(1-y*y))
def lim_u(y, D, rt):
return [D*sqrt(1-y*y), rt]
def lim_y(D, rt):
return [ymin,1]
def Jfactor(D,rt,r0,rho0,tmax):
ymin=cos(np.radians(tmax))
Dprime=D/r0
rtprime=rt/r0
Msun2kpc5_GeVcm5 = 4463954.894661358
cst = 4*pi*rho0**2*r0*Msun2kpc5_GeVcm5
res = sciint.nquad(func, ranges=[lim_u, lim_y], args=(Dprime,rtprime),
opts=[{'epsabs':1.e-10,'epsrel':1.e-10,'limit':1000},
{'epsabs':1.e-10,'epsrel':1.e-10,'limit':1000}])
return cst*res[0]
def deltaJ(rho0,J,D,rt,r0,tmax):
#return J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.)
return long(J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.))
D=104.
rt=np.inf
tmax=0.5
J = 10.**18
for j,r0 in enumerate(np.logspace(-1.,np.log10(5),10)):
results = sciopt.minimize_scalar(deltaJ,bracket=(7.,9.),args=(J,D,rt,r0,tmax))
print r0,results.x,results.fun
Very quickly: I want scipy.optimize.minimize_scalar to minimize deltaJ but here's (part of) the error message I get
File "test2.py", line 26, in deltaJ
return long(J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.))
OverflowError: (34, 'Numerical result out of range')
Now, if J = 10**18, Jfactor(D,rt,r0,1.,tmax) is ~ 5, I expect minimize_scalar to easily scan over rho0 to find ~ 8.5. Instead I get this error message.
As you can see, I've even bracketed the range and tried using long(), but nothing helped. Using instead another minimizing method, eg. bounded, gives again a weird result (function value of ~ -7e17)...
Does anybody have an idea to have this working? Thank you