I am trying to fit data to a function f(x) that is an integral over T. x is the upper boarder of the Integral. I am trying to do it with scipy.curve_fit() but I don't know how to write my integral as a function that can be passed to curve_fit.
I had a look at similar problems but I didn't see anything that fits to my problem.
I cannot provide any guess values for A and Ea since I have no clue at all what they could be as of now.
from scipy import optimize
import matplotlib.pyplot as plt
import numpy as np
from scipy import interpolate
from scipy import integrate
class get_Ton:
def __init__(self):
self.data=np.genfromtxt('test3.csv', delimiter=',', skip_header=8)
def loop(self):
def fit_tangent():
tck = interpolate.splrep(self.x, self.y, k=2, s=0)
dev_1 = interpolate.splev(self.x, tck, der=1)
def integrand(T, A, Ea):
return A*np.exp(-Ea/(8.314*T))
def polyn(x, A, Ea):
return integrate.quad(integrand, 25, x, args=(A, Ea))[0]
vcurve = np.vectorize(polyn)
p, e = optimize.curve_fit(vcurve, self.x, self.y, [2000, 150])
xd = np.linspace(50, 70, 100)
plt.plot(self.x, self.y, "o")
plt.plot(xd, vcurve(xd, *p))
for i in range((list(np.shape(self.data)[1:2]))[0]):
if i % 2 == 0:
self.temp=self.data[:,i]
self.scat=self.data[:,i+1]
self.x=[26.192, 26.861000000000001, 27.510000000000002, 28.178000000000001, 28.856000000000002, 29.515000000000001, 30.183, 30.823, 31.5, 32.158999999999999, 32.856000000000002, 33.515000000000001, 34.145000000000003, 34.823, 35.491, 36.168999999999997, 36.837000000000003, 37.533999999999999, 38.164000000000001, 38.832000000000001, 39.481000000000002, 40.158999999999999, 40.826999999999998, 41.496000000000002, 42.164000000000001, 42.832000000000001, 43.500999999999998, 44.188000000000002, 44.837000000000003, 45.505000000000003, 46.173000000000002, 46.832000000000001, 47.500999999999998, 48.188000000000002, 48.828000000000003, 49.496000000000002, 50.173999999999999, 50.813000000000002, 51.481000000000002, 52.112000000000002, 52.808999999999997, 53.439, 54.116, 54.765999999999998, 55.453000000000003, 56.101999999999997, 56.761000000000003, 57.429000000000002, 58.078000000000003, 58.737000000000002, 59.442999999999998, 60.082999999999998, 60.770000000000003, 61.448, 62.125999999999998, 62.756, 63.414999999999999, 64.082999999999998, 64.742000000000004, 65.420000000000002, 66.087999999999994, 66.747, 67.415000000000006]
self.y=[1553.5, 1595.0, 1497.8, 1695.5999999999999, 1328.7, 1279.0, 1547.8, 1412.8, 1037.0, 1473.5, 1447.4000000000001, 1532.5999999999999, 1484.2, 1169.5, 1395.2, 1183.5999999999999, 949.01999999999998, 1238.0999999999999, 1225.5999999999999, 924.80999999999995, 1650.5999999999999, 803.96000000000004, 1245.7, 1190.0, 1207.0, 1294.0, 1174.9000000000001, 1229.8, 1260.0, 1129.2, 1142.9000000000001, 987.63999999999999, 1389.5999999999999, 1366.0, 1102.0999999999999, 1325.5, 1258.9000000000001, 1285.7, 1217.5, 871.47000000000003, 820.24000000000001, 1388.7, 1391.0, 1400.3, 2482.5999999999999, 3360.5999999999999, 7013.5, 11560.0, 16525.0, 22538.0, 32556.0, 43878.0, 59093.0, 67977.0, 75949.0, 82316.0, 90213.0, 90294.0, 99928.0, 128240.0, 181280.0, 226380.0, 223260.0]
fit_tangent()
plt.ylim((-100,1000000))
plt.show()
def main():
this=get_Ton()
this.loop()
if __name__ == "__main__":
main()
Three issues here. First, the function polyn does not depend on the variable of integration T, since this variable is integrated out. Remove T from its list of parameters. Accordingly, drop one of numerical values in trueydata computation:
trueydata = vcurve(truexdata, 3, 4)
Second, quad returns a tuple (integral_value, integral_error). Use [0] to return only the integral value.
def polyn(x, A, Ea):
return integrate.quad(integrand, 25, x, args=(A, Ea))[0]
Third, provide an initial guess for parameter values to curve_fit. If you don't, it will likely report unable to determine the number of parameters to fit. Even if successful, it will blindly use all-ones for the initial guess. An initial guess supplied by a human with an understanding of the optimization problem is often the difference between success and failure of multivariable optimization.
popt, pcov = optimize.curve_fit(vcurve, xdata, ydata, [2, 3])
Related
I try to reproduce a simple linear regression x = A†b using pytorch. But I get completely different numbers.
So first I use plain numpy and do
A_pinv = np.linalg.pinv(A)
betas = A_pinv.dot(b)
print(((b - A.dot(betas))**2).mean())
print(betas)
which results in:
364.12875
[0.43196774 0.14436531 0.42414093]
Now I try to get similar enough numbers using pytorch:
# re-implement via pytoch model using built-ins
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import TensorDataset, DataLoader
# We'll create a TensorDataset, which allows access to rows from inputs and targets as tuples.
# We'll also create a DataLoader, to split the data into batches while training.
# It also provides other utilities like shuffling and sampling.
inputs = to.from_numpy(A)
targets = to.from_numpy(b)
train_ds = TensorDataset(inputs, targets)
batch_size = 5
train_dl = DataLoader(train_ds, batch_size, shuffle=True)
# define model, loss and optimizer
new_model = nn.Linear(source_variables, predict_variables, bias=False)
loss_fn = F.mse_loss
opt = to.optim.SGD(new_model.parameters(), lr=1e-10)
def fit(num_epochs, new_model, loss_fn, opt):
for epoch in tnrange(num_epochs, desc="epoch"):
for xb,yb in train_dl:
# Generate predictions
pred = new_model(xb)
loss = loss_fn(pred, yb)
# Perform gradient descent
loss.backward()
opt.step()
opt.zero_grad()
if epoch % 1000 == 0:
print((new_model.weight, loss))
print('Training loss: ', loss_fn(model(inputs), targets))
# fit the model
fit(10000, new_model, loss_fn, opt)
It prints as the last result:
tensor([[0.0231, 0.5185, 0.4589]], requires_grad=True), tensor(271.8525, grad_fn=<MseLossBackward>))
Training loss: tensor(378.2871, grad_fn=<MseLossBackward>)
As you can see these numbers are completely different so I must have made a mistake somewhere ...
Here are the numbers for A and b to reproduce the result:
A = np.array([[2822.48, 2808.48, 2810.92],
[2832.94, 2822.48, 2808.48],
[2832.57, 2832.94, 2822.48],
[2824.23, 2832.57, 2832.94],
[2854.88, 2824.23, 2832.57],
[2800.71, 2854.88, 2824.23],
[2798.36, 2800.71, 2854.88],
[2818.46, 2798.36, 2800.71],
[2805.37, 2818.46, 2798.36],
[2815.44, 2805.37, 2818.46]], dtype=float32)
b = np.array([2832.94, 2832.57, 2824.23, 2854.88, 2800.71, 2798.36, 2818.46, 2805.37, 2815.44, 2834.4 ], dtype=float32)
I have a code (a code we saw in a class) of a recurrent neural network that reads a given text and tries to produce its own text similar to the example. The code is written in python and uses the pytorch library. I wanted to modify to see whether I could increase its speed by using GPU instead of CPU and I made some tests on google collaboratory. The GPU version of the code runs fine but is about three times slower than the CPU version. I do not know the details of GPU architecture so I can not really understand why it is slower. I know that GPUs can do more arithmetic operations per cycle but have more limited memory so I am curious if I am having a memory issue. I also tried using CUDA with a generative adversarial network and in this case it was almost ten times faster. Any tips on this would be welcome.
The code (CUDA version) is below. I am new at this stuff so sorry if some of the terminology is not correct.
The architecture is input->encoder->recursive network->decoder->output.
import torch
import time
import numpy as np
from torch.autograd import Variable
import matplotlib.pyplot as plt
from google.colab import files
#uploding text on google collab
uploaded = files.upload()
for fn in uploaded.keys():
print('User uploaded file "{name}" with length {length} bytes'.format(
name=fn, length=len(uploaded[fn])))
#data preprocessing
with open('text.txt','r') as file:
#with open closes the file after we are done with it
rawtxt=file.read()
rawtxt = rawtxt.lower()
#a function that assigns a number to each unique character in the text
def create_map(rawtxt):
letters = list(set(rawtxt))
lettermap = dict(enumerate(letters)) #gives each letter in the list a number
return lettermap
num_to_let = create_map(rawtxt)
#inverse to num_to_let
let_to_num =dict(zip(num_to_let.values(), num_to_let.keys()))
print(num_to_let)
#turns a text of characters into text of numbers using the mapping
#given by the input mapdict
def maparray(txt, mapdict):
txt = list(txt)
for k, letter in enumerate(txt):
txt[k]=mapdict[letter]
txt=np.array(txt)
return txt
X=maparray(rawtxt, let_to_num) #the data text in numeric format
Y= np.roll(X, -1, axis=0) #shifted data text in numeric format
X=torch.LongTensor(X)
Y=torch.LongTensor(Y)
#up to here we are done with data preprocessing
#return a random batch for training
#this reads a random piece inside data text
#with the size chunk_size
def random_chunk(chunk_size):
k=np.random.randint(0,len(X)-chunk_size)
return X[k:k+chunk_size], Y[k:k+chunk_size]
nchars = len(num_to_let)
#define the recursive neural network class
class rnn(torch.nn.Module):
def __init__(self,input_size,hidden_size,output_size, n_layers=1):
super().__init__()
self.input_size = input_size
self.hidden_size = hidden_size
self.output_size = output_size
self.n_layers= n_layers
self.encoder = torch.nn.Embedding (input_size, hidden_size)
self.rnn = torch.nn.RNN(hidden_size, hidden_size, n_layers, batch_first=True)
self.decoder = torch.nn.Linear (hidden_size, output_size)
def forward (self,x,hidden):
x=self.encoder(x.view(1,-1))
output, hidden = self.rnn(x.view(1,1,-1), hidden)
output = self.decoder(output.view(1,-1))
return output, hidden
def init_hidden(self):
return Variable(torch.zeros(self.n_layers , 1 , self.hidden_size)).cuda()
#hyper-params
lr = 0.009
no_epochs = 50
chunk_size = 150
myrnn = rnn(nchars, 150, nchars,1)
myrnn.cuda()
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(myrnn.parameters(), lr=lr)
t0 = time.time()
for epoch in range(no_epochs):
totcost=0
generated = ''
for _ in range(len(X)//chunk_size):
h=myrnn.init_hidden()
cost = 0
x, y=random_chunk(chunk_size)
x, y= Variable(x).cuda(), Variable(y).cuda()
for i in range(chunk_size):
out, h = myrnn.forward(x[i],h)
_, outl = out.data.max(1)
letter = num_to_let[outl[0]]
generated+=letter
cost += criterion(out, y[i])
optimizer.zero_grad()
cost.backward()
optimizer.step()
totcost+=cost
totcost/=len(X)//chunk_size
print('Epoch', epoch, 'Avg cost/chunk: ', totcost)
print(generated[0:750],'\n\n\n')
t1 = time.time()
total = t1-t0
print('total',total)
#we encode each word into a vector of fixed size
I am trying to build a neural network model with one hidden layer (1024 nodes). The hidden layer is nothing but a relu unit. I am also processing the input data in batches of 128.
The inputs are images of size 28 * 28. In the following code I get the error in line
_, c = sess.run([optimizer, loss], feed_dict={x: batch_x, y: batch_y})
Error: TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder_64:0", shape=(128, 784), dtype=float32) is not an element of this graph.
Here is the code I have written
#Initialize
batch_size = 128
layer1_input = 28 * 28
hidden_layer1 = 1024
num_labels = 10
num_steps = 3001
#Create neural network model
def create_model(inp, w, b):
layer1 = tf.add(tf.matmul(inp, w['w1']), b['b1'])
layer1 = tf.nn.relu(layer1)
layer2 = tf.matmul(layer1, w['w2']) + b['b2']
return layer2
#Initialize variables
x = tf.placeholder(tf.float32, shape=(batch_size, layer1_input))
y = tf.placeholder(tf.float32, shape=(batch_size, num_labels))
w = {
'w1': tf.Variable(tf.random_normal([layer1_input, hidden_layer1])),
'w2': tf.Variable(tf.random_normal([hidden_layer1, num_labels]))
}
b = {
'b1': tf.Variable(tf.zeros([hidden_layer1])),
'b2': tf.Variable(tf.zeros([num_labels]))
}
init = tf.initialize_all_variables()
train_prediction = tf.nn.softmax(model)
tf_valid_dataset = tf.constant(valid_dataset)
tf_test_dataset = tf.constant(test_dataset)
model = create_model(x, w, b)
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(model, y))
optimizer = tf.train.GradientDescentOptimizer(0.5).minimize(loss)
#Process
with tf.Session(graph=graph1) as sess:
tf.initialize_all_variables().run()
total_batch = int(train_dataset.shape[0] / batch_size)
for epoch in range(num_steps):
loss = 0
for i in range(total_batch):
batch_x, batch_y = train_dataset[epoch * batch_size:(epoch+1) * batch_size, :], train_labels[epoch * batch_size:(epoch+1) * batch_size,:]
_, c = sess.run([optimizer, loss], feed_dict={x: batch_x, y: batch_y})
loss = loss + c
loss = loss / total_batch
if epoch % 500 == 0:
print ("Epoch :", epoch, ". cost = {:.9f}".format(avg_cost))
print("Minibatch accuracy: %.1f%%" % accuracy(predictions, batch_labels))
valid_prediction = tf.run(tf_valid_dataset, {x: tf_valid_dataset})
print("Validation accuracy: %.1f%%" % accuracy(valid_prediction.eval(), valid_labels))
test_prediction = tf.run(tf_test_dataset, {x: tf_test_dataset})
print("TEST accuracy: %.1f%%" % accuracy(test_prediction.eval(), test_labels))
This worked for me
from keras import backend as K
and after predicting my data i inserted this part of code
then i had again loaded the model.
K.clear_session()
i faced this problem in production server,
but in my pc it was running fine
...........
from keras import backend as K
#Before prediction
K.clear_session()
#After prediction
K.clear_session()
Variable x is not in the same graph as model, try to define all of these in the same graph scope. For example,
# define a graph
graph1 = tf.Graph()
with graph1.as_default():
# placeholder
x = tf.placeholder(...)
y = tf.placeholder(...)
# create model
model = create(x, w, b)
with tf.Session(graph=graph1) as sess:
# initialize all the variables
sess.run(init)
# then feed_dict
# ......
If you use django server, just runserver with --nothreading
for example:
python manage.py runserver --nothreading
I had the same issue with flask. adding --without-threads flag to flask run or threaded=False to app.run() fixed it
In my case, I was using loop while calling in CNN multiple times, I fixed my problem by doing the following:
# Declare this as global:
global graph
graph = tf.get_default_graph()
# Then just before you call in your model, use this
with graph.as_default():
# call you models here
Note: In my case too, the app ran fine for the first time and then gave the error above. Using the above fix solved the problem.
Hope that helps.
The error message TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("...", dtype=dtype) is not an element of this graph can also arise in case you run a session outside of the scope of its with statement. Consider:
with tf.Session() as sess:
sess.run(logits, feed_dict=feed_dict)
sess.run(logits, feed_dict=feed_dict)
If logits and feed_dict are defined properly, the first sess.run command will execute normally, but the second will raise the mentioned error.
You can also experience this while working on notebooks hosted on online learning platforms like Coursera. So, implementing following code could help get over with the issue.
Implement this at the topmost block of Notebook file:
from keras import backend as K
K.clear_session()
Similar to #javan-peymanfard and #hmadali-shafiee, I ran into this issue when loading the model in an API. I was using FastAPI with uvicorn. To fix the issue I just set the API function definitions to async similar to this:
#app.post('/endpoint_name')
async def endpoint_function():
# Do stuff here, including possibly (re)loading the model
I've written a python script to evaluate a physical quantity and, for some reason, python decided to raise an OverflowError for no justified reason. Here's the script
import numpy as np
from math import sqrt,cos, log, pi
from scipy import integrate as sciint
from scipy import optimize as sciopt
rt=np.inf
ymin=cos(np.radians(0.5))
def func(u,y, D, rt):
return (1.+u)**(-4)/u/sqrt(u*u-D**2*(1-y*y))
def lim_u(y, D, rt):
return [D*sqrt(1-y*y), rt]
def lim_y(D, rt):
return [ymin,1]
def Jfactor(D,rt,r0,rho0,tmax):
ymin=cos(np.radians(tmax))
Dprime=D/r0
rtprime=rt/r0
Msun2kpc5_GeVcm5 = 4463954.894661358
cst = 4*pi*rho0**2*r0*Msun2kpc5_GeVcm5
res = sciint.nquad(func, ranges=[lim_u, lim_y], args=(Dprime,rtprime),
opts=[{'epsabs':1.e-10,'epsrel':1.e-10,'limit':1000},
{'epsabs':1.e-10,'epsrel':1.e-10,'limit':1000}])
return cst*res[0]
def deltaJ(rho0,J,D,rt,r0,tmax):
#return J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.)
return long(J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.))
D=104.
rt=np.inf
tmax=0.5
J = 10.**18
for j,r0 in enumerate(np.logspace(-1.,np.log10(5),10)):
results = sciopt.minimize_scalar(deltaJ,bracket=(7.,9.),args=(J,D,rt,r0,tmax))
print r0,results.x,results.fun
Very quickly: I want scipy.optimize.minimize_scalar to minimize deltaJ but here's (part of) the error message I get
File "test2.py", line 26, in deltaJ
return long(J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.))
OverflowError: (34, 'Numerical result out of range')
Now, if J = 10**18, Jfactor(D,rt,r0,1.,tmax) is ~ 5, I expect minimize_scalar to easily scan over rho0 to find ~ 8.5. Instead I get this error message.
As you can see, I've even bracketed the range and tried using long(), but nothing helped. Using instead another minimizing method, eg. bounded, gives again a weird result (function value of ~ -7e17)...
Does anybody have an idea to have this working? Thank you
After importing this data file from Matlab with scipy.io.loadmat, things appeared to work fine until we tried to calculate the conditioning number of one of the matrixes within.
Here's the minimum amount of code that reproduces for us:
import scipy
import numpy
stuff = scipy.io.loadmat("dati-esercizio1.mat")
numpy.linalg.cond(stuff["A"])
Here's the extended stacktrace courtesy of iPython:
In [3]: numpy.linalg.cond(A)
---------------------------------------------------------------------------
LapackError Traceback (most recent call last)
/snip/<ipython-input-3-15d9ef00a605> in <module>()
----> 1 numpy.linalg.cond(A)
/snip/python2.7/site-packages/numpy/linalg/linalg.py in cond(x, p)
1409 x = asarray(x) # in case we have a matrix
1410 if p is None:
-> 1411 s = svd(x,compute_uv=False)
1412 return s[0]/s[-1]
1413 else:
/snip/python2.7/site-packages/numpy/linalg/linalg.py in svd(a, full_matrices, compute_uv)
1313 work = zeros((lwork,), t)
1314 results = lapack_routine(option, m, n, a, m, s, u, m, vt, nvt,
-> 1315 work, -1, iwork, 0)
1316 lwork = int(work[0])
1317 work = zeros((lwork,), t)
LapackError: Parameter a has non-native byte order in lapack_lite.dgesdd
All obvious ideas (like flattening and reshaping the matrix or recreating the matrix from scratch reassigning it element by element) failed. How can I want to massage the data, then, in order to make it more agreeable with numpy?
It's a bug, fixed some time ago: https://github.com/numpy/numpy/pull/235
Workaround:
np.linalg.cond(stuff['A'].newbyteorder('='))
This works for me:
In [33]: stuff = loadmat('dati-esercizio1.mat')
In [34]: a = stuff['A']
In [35]: try: np.linalg.cond(a)
....: except: print "Fail!"
Fail!
In [36]: b = np.array(a, dtype='>d')
In [37]: np.linalg.cond(b)
Out[37]: 62493201976.673141
In [38]: np.all(a == b) # Verify they hold the same data.
Out[38]: True
Apparently it's something wrong with the byte order (endianness?) of each number in the resulting ndarray and not just with the ndarray object itself.
Something like this but more elegant should do the trick:
n, m = A.shape()
B = numpy.empty_like(A)
for i in xrange(n):
for j in xrange(m):
B[i,j] = float(A[i,j])
del A
B = A
print numpy.linalg.cond(A) # 62493210091.354507
(For some reason an in-place replacement still gives that error - so there's something wrong with the byte order of the whole object, too.)