I am having some difficulty to replciate Matlab's regress function output using scipy linregress function.
Here is the data
pupVals = [3.40621246270059, 3.39361245626222, 3.37960434985898, 3.36422005129091, 3.34750704680340, 3.32952815194744,
3.31035997313977, 3.29009299139087, 3.26883227275573, 3.24668690892347, 3.22376158035874, 3.20015774115851,
3.17597533905954, 3.15130446393993, 3.12622016870185, 3.10078008972008, 3.07503065688319, 3.04901239477419,
3.02276360223545, 2.99632186567962, 2.96972062051014]
defVals = [ -0.676069340690958, -0.681299901070580, -0.687103823182625, -0.693490568594932, -0.700466450335011,
-0.708027898057120, -0.716167334731881, -0.724878961238070, -0.734138903033690, -0.743919748266449,
-0.754183596434113, -0.764899153546520, -0.776036371746185, -0.787551515041980, -0.799395767551789,
-0.811524533752894, -0.823893965641479, -0.836469554360099, -0.849231693452297, -0.862141390106238,
-0.875133488454364]
res = linregress(pupVals, defVals)
the matlab output looks like below
in matlab I make defvals as a 21x2 matrix with ones in first column as suggested in matlab regress help documentation
[SlopeFit,CIS] = regress(pupvals, defvals)
SlopeFit =
4.8830
2.1939
CIS =
4.8604 4.9056
2.1644 2.2234
and I use SlopeFit(2) and CIS(2,1), CIS(2,2)
Any suggestions on how to replicate the results?
thanks
Try the following:
res = linregress(pupVals, defVals)
SlopeFit = np.array([res.intercept, res.slope])
errs = np.array([res.stderr,intercept_stderr])
CIS = SlopeFit[:,None] + 1.96*errs[:,None]*np.array([[1,-1]])
Related
I am trying to run a hyperparameters optimization with Talos. As I have a lot of parameters to test, I want to use a 'grid_downsample' argument that will select 30% of all possible hyperparameters combinations. However when I run my code I get: TypeError: __init__() got an unexpected keyword argument 'grid_downsample'
I tested the code below without the 'grid_downsample' option and with less hyperparameters.
#load data
data = pd.read_csv('data.txt', sep="\t", encoding = "latin1")
# split into input (X) and output (y) variables
Y = np.array(data['Y'])
data_bis = data.drop(['Y'], axis = 1)
X = np.array(data_bis)
p = {'activation':['relu'],
'optimizer': ['Nadam'],
'first_hidden_layer': [12],
'second_hidden_layer': [12],
'batch_size': [20],
'epochs': [10,20],
'dropout_rate':[0.0, 0.2]}
def dnn_model(x_train, y_train, x_val, y_val, params):
model = Sequential()
#input layer
model.add(Dense(params['first_hidden_layer'], input_shape=(1024,)))
model.add(Dropout(params['dropout_rate']))
model.add(Activation(params['activation']))
#hidden layer 2
model.add(Dense(params['second_hidden_layer']))
model.add(Dropout(params['dropout_rate']))
model.add(Activation(params['activation']))
# output layer with one node
model.add(Dense(1))
model.add(Activation(params['activation']))
# Compile model
model.compile(loss='binary_crossentropy', optimizer=params['optimizer'], metrics=['accuracy'])
out = model.fit(x_train, y_train,
batch_size=params['batch_size'],
epochs=params['epochs'],
validation_data=[x_val, y_val],
verbose=0)
return out, model
scan_object = ta.Scan(X, Y, model=dnn_model, params=p, experiment_name="test")
reporting = ta.Reporting(scan_object)
report = reporting.data
report.to_csv('./Random_search/dnn/report_talos.txt', sep = '\t')
This code works well. If I change the scan_object as the end to: scan_object = ta.Scan(X, Y, model=dnn_model, grid_downsample=0.3, params=p, experiment_name="test"), it gives me the error: TypeError: __init__() got an unexpected keyword argument 'grid_downsample' while I was expecting to have the same results format as a normal grid search but with less combinations. What am I missing? Did the name of the argument change? I'm using Talos 0.6.3 in a conda environment.
Thank you!
might be too late for you now but they've switched it to fraction_limit. It would give this for you
scan_object = ta.Scan(X, Y, model=dnn_model, params=p, experiment_name="test", fraction_limit = 0.1)
Sadly, the doc isn't well updated
Check out their examples on GitHub:
https://github.com/autonomio/talos/blob/master/examples/Hyperparameter%20Optimization%20with%20Keras%20for%20the%20Iris%20Prediction.ipynb
I have the following code that I wish to estimate the parameters of a custom distribution using MATLAB's function mle(). For more details on the distribution.
The main code is:
x = [0 0 0 0 0.000649967501624919 0.00569971501424929 0.0251487425628719 0.0693465326733663 0.155342232888356 0.284835758212089 0.458277086145693 0.658567071646418 0.908404579771011 1.17284135793210 1.43977801109945 1.71951402429879 1.98925053747313 2.27553622318884 2.57147142642868 2.80390980450977 3.03829808509575 3.26583670816459 3.45642717864107 3.65106744662767 3.81950902454877 3.98275086245688 4.11259437028149 4.24683765811709 4.35043247837608 4.43832808359582 4.58427078646068 4.62286885655717 4.68361581920904 4.75686215689216 4.80245987700615 4.84005799710015 4.86280685965702 4.91675416229189 4.92725363731813 4.90890455477226 4.96570171491425 4.92315384230789 4.95355232238388 4.92790360481976 4.93135343232838 4.90310484475776 4.90885455727214 4.86765661716914 4.85490725463727 4.81940902954852 4.81450927453627 4.78621068946553 4.74206289685516 4.71791410429479 4.69961501924904 4.65706714664267 4.63611819409030 4.60176991150443 4.57512124393780 4.53507324633768 4.48252587370631 4.47062646867657 4.43127843607820 4.39963001849908 4.37598120093995 4.29548522573871 4.31033448327584 4.21708914554272 4.21913904304785 4.18669066546673 4.16719164041798 4.09774511274436 4.07989600519974 4.02869856507175 3.98485075746213 3.95785210739463 3.93945302734863 3.90240487975601 3.87025648717564 3.81185940702965 3.78461076946153 3.74091295435228 3.71666416679166 3.67276636168192 3.65846707664617 3.61361931903405 3.58712064396780 3.55452227388631 3.53082345882706 3.49197540122994 3.48582570871456 3.46512674366282 3.41227938603070 3.36278186090695 3.35528223588821 3.31238438078096 3.27213639318034 3.23863806809660 3.24173791310434 3.19339033048348 3.20118994050298 3.16489175541223 3.10739463026849 3.09484525773711 3.08094595270237 3.02129893505325 3.02309884505775 2.99375031248438 2.95765211739413 2.93230338483076 2.89560521973901 2.87805609719514 2.85440727963602 2.82285885705715 2.80175991200440 2.79091045447728 2.73901304934753 2.72701364931753 2.73441327933603 2.71646417679116 2.68236588170592 2.65551722413879 2.63356832158392 2.60361981900905 2.58147092645368 2.57697115144243 2.54287285635718 2.53502324883756 2.47702614869257 2.50387480625969 2.46487675616219 2.45722713864307 2.42707864606770 2.41762911854407 2.39823008849558 2.38708064596770 2.34058297085146 2.35613219339033 2.32123393830309 2.30503474826259 2.27613619319034 2.27248637568122 2.25113744312784 2.24908754562272 2.22703864806760 2.20583970801460 2.17244137793110 2.15709214539273 2.16469176541173 2.12139393030348 2.12809359532023 2.11389430528474 2.09774511274436 2.07629618519074 2.07459627018649 2.05394730263487 2.04724763761812 2.01684915754212 2.01684915754212 2.00409979501025 1.98955052247388 1.96540172991350 1.95890205489726 1.93035348232588 1.92295385230738 1.90605469726514 1.89785510724464 1.87070646467677 1.88000599970002 1.86295685215739 1.84420778961052 1.82510874456277 1.80480975951202 1.80785960701965 1.80870956452177 1.77581120943953 1.76771161441928 1.77131143442828 1.76636168191590 1.75081245937703 1.73156342182891 1.69876506174691 1.70836458177091 1.70376481175941 1.67196640167992 1.68101594920254 1.66586670666467 1.66061696915154 1.64296785160742 1.63291835408230 1.62506874656267 1.62516874156292 1.60556972151392 1.59007049647518 1.59187040647968 1.57947102644868 1.57577121143943 1.54527273636318 1.57237138143093 1.54637268136593 1.54802259887006 1.50492475376231 1.52077396130193 1.50417479126044 1.50162491875406 1.50062496875156 1.48957552122394 1.47997600119994 1.47027648617569 1.44452777361132 1.45407729613519 1.44272786360682 1.43247837608120 1.41657917104145 1.40787960601970 1.39323033848308 1.40282985850707 1.39403029848508 1.38233088345583 1.37888105594720 1.37943102844858 1.36183190840458 1.34808259587021 1.34503274836258 1.33703314834258 1.33308334583271 1.32253387330633 1.32698365081746 1.29963501824909 1.30758462076896 1.29103544822759 1.29473526323684 1.27413629318534 1.26858657067147 1.27888605569722 1.26063696815159 1.27863606819659 1.25168741562922 1.23913804309785 1.24788760561972 1.22308884555772 1.24198790060497 1.22133893305335 1.20678966051697 1.20098995050247 1.20343982800860 1.18779061046948 1.19024048797560 1.17194140292985 1.17369131543423 1.16869156542173 1.15814209289536 1.15429228538573 1.15904204789761 1.12774361281936 1.15344232788361 1.13744312784361 1.12909354532273 1.12479376031198 1.11099445027749 1.11469426528674 1.11064446777661 1.10464476776161 1.10309484525774 1.10689465526724 1.07654617269137 1.07884605769712 1.07359632018399 1.06864656767162 1.07544622768862 1.06689665516724 1.04884755762212 1.06164691765412 1.04979751012449 1.04529773511324 1.02839858007100 1.03634818259087 1.01709914504275 1.02089895505225 1.01024948752562 1.01549922503875 1.01319934003300 1.01404929753512 1.00839958002100 0.995400229988501 0.989850507474626 0.978801059947003 0.977551122443878 0.980450977451127 0.975451227438628 0.969201539923004 0.964151792410380 0.964601769911504 0.958802059897005 0.955702214889256 0.948602569871506 0.960751962401880 0.941352932353382 0.928653567321634 0.949002549872506 0.937053147342633 0.913854307284636 0.916204189790510 0.915454227288636 0.902604869756512 0.909454527273636 0.895505224738763 0.898355082245888 0.894455277236138 0.902454877256137 0.883705814709265 0.888405579721014 0.876356182190891 0.881555922203890 0.878156092195390 0.868456577171141 0.870406479676016 0.863906804659767 0.862456877156142 0.858757062146893 0.851307434628269 0.851107444627769 0.833908304584771 0.843507824608770 0.831708414579271 0.836858157092145 0.829058547072646 0.828508574571272 0.822908854557272 0.820508974551273 0.815559222038898 0.819709014549273 0.809609519524024 0.813409329533523 0.800759962001900 0.806609669516524 0.806959652017399 0.792260386980651 0.787660616969152 0.783810809459527 0.794960251987401 0.771061446927654 0.788910554472276 0.789510524473776 0.763061846907655 0.776761161941903 0.767561621918904 0.773611319434028 0.750262486875656 0.765811709414529 0.765911704414779 0.748012599370032 0.741612919354032 0.757312134393280 0.752612369381531 0.741362931853407 0.742212889355532 0.741912904354782 0.743162841857907 0.732963351832408 0.732813359332033 0.733363331833408 0.721913904304785 0.716664166791661 0.726713664316784 0.709764511774411 0.700064996750163 0.710764461776911 0.717664116794160 0.707314634268287 0.707114644267787 0.705614719264037 0.709164541772911 0.696665166741663 0.680765961701915 0.686715664216789 0.694465276736163 0.683015849207540 0.681715914204290 0.694465276736163 0.688615569221539 0.680665966701665 0.672316384180791 0.672866356682166 0.656517174141293 0.665316734163292 0.671566421678916 0.666266686665667 0.652917354132293 0.663366831658417 0.651917404129794 0.663816809159542 0.661366931653417 0.647017649117544 0.655167241637918 0.647867606619669 0.636918154092295 0.645467726613669 0.633118344082796 0.640217989100545 0.634668266586671 0.618669066546673 0.635068246587671 0.632568371581421 0.623118844057797 0.623868806559672 0.623718814059297 0.621368931553422 0.623768811559422 0.608419579021049 0.616019199040048 0.609869506524674 0.606569671516424 0.614019299035048 0.610269486525674 0.596520173991300 0.595570221488926 0.593270336483176 0.596670166491675 0.598470076496175 0.597770111494425 0.593720313984301 0.592770361481926 0.585420728963552 0.580870956452177 0.584120793960302 0.580270986450677 0.577971101444928 0.579021048947553 0.572821358932053 0.585970701464927 0.572921353932303 0.567071646417679 0.569971501424929 0.571271436428179 0.568421578921054 0.567421628918554 0.569521523923804 0.563721813909305 0.558772061396930 0.562171891405430 0.557872106394680 0.549072546372681 0.558722063896805 0.536973151342433 0.561021948902555 0.544172791360432 0.552122393880306 0.553072346382681 0.546222688865557 0.551472426378681 0.540772961351932 0.541122943852807 0.542772861356932 0.530323483825809 0.526023698815059 0.529273536323184 0.524573771311435 0.525923703814809 0.524923753812309 0.516474176291185 0.527273636318184 0.527723613819309 0.518424078796060 0.517874106294685 0.516074196290186 0.517924103794810 0.523173841307935 0.514474276286186 0.513174341282936 0.498875056247188 0.518024098795060 0.507924603769812 0.505524723763812 0.507174641267937 0.502874856257187 0.502624868756562 0.500624968751562 0.510824458777061 0.490925453727314 0.492675366231688 0.489925503724814 0.478126093695315 0.485775711214439 0.491775411229439 0.489925503724814 0.491325433728314 0.487225638718064 0.485725713714314 0.485675716214189 0.477676116194190 0.483875806209690 0.478026098695065 0.470176491175441 0.471926403679816 0.483625818709065 0.469376531173441 0.474026298685066 0.467826608669567 0.462426878656067];
Censored = ones(1,size(x,2));%
custpdf = #eval_custpdf;
custcdf = #eval_custcdf;
phat = mle(x,'pdf', custpdf,'cdf', custcdf,'start',[1 0.1 0.3 0.1 0.01 -0.3],...
'lowerbound',[0 0 0 0 0 -inf],'upperbound',[inf inf inf inf inf inf],'Censoring',Censored);
% Cheking how close the estimated PDF and CDF match with those from the data x
t = 0.001:0.001:0.5;
figure();
plot(t,x);hold on
plot(t,custpdf(t, phat(1), phat(2), phat(3), phat(4), phat(5), phat(6)))
figure();
plot(t,cumsum(x)./sum(x));hold on
plot(t,custcdf(t, phat(1), phat(2), phat(3), phat(4), phat(5), phat(6)))
The functions are:
function out = eval_custpdf(x,myalpha,mybeta,mytheta,a,b,c)
first_integral = integral(#(x) eval_K(x,a,b,c),0,1).^-1;
theta_t_ratio = (mytheta./x);
incomplete_gamma = igamma(myalpha,theta_t_ratio.^mybeta);
n_gamma = gamma(myalpha);
exponent_term = exp(-theta_t_ratio.^mybeta-(c.*(incomplete_gamma./n_gamma)));
numerator = first_integral.* mybeta.*incomplete_gamma.^(a-1).*...
theta_t_ratio.^(myalpha*mybeta+1).*exponent_term;
denominator = mytheta.* n_gamma.^(a+b-1).* (n_gamma-incomplete_gamma.^mybeta).^(1-b);
out = numerator./denominator;
end
function out = eval_custcdf(x,myalpha,mybeta,mytheta,a,b,c)
first_integral = integral(#(x) eval_K(x,a,b,c),0,1).^-1;
theta_t_ratio = (mytheta./x);
incomplete_gamma = igamma(myalpha,theta_t_ratio.^mybeta);
n_gamma = gamma(myalpha);
second_integral = integral(#(x) eval_K(x,a,b,c),0, incomplete_gamma.^mybeta./n_gamma);
% |<----- PROBLEMATIC LINE ----->|
out = first_integral*second_integral;
end
function out = eval_K(x,a,b,c)
out = x.^(a-1).*(1-x).^(b-1).*exp(-c.*x);
end
The integral that is causing the problem is the second intergral in the function eval_custcdf() as its upper limit is an array (denoted by PROBLEMATIC LINE).
Is there a way to take a single value from the array x such that the upper limit remains a scalar? And then calculate the cdf such that the output of the cdf is an array? Using a forloop, maybe? But I cannot seem to figure how to implement that?
How can I work around this problem?
Any help would be appreciated.
Thanks in advance.
eval_custcdf is a function expected to return 1D array of length n for
a given n data input.
I use a for loop to compute the output for a given input, then
return the whole array as output of eval_custcdf
I passed the input array elements one at a time
This is how eval_custcdf may look like
function out = eval_custcdf(x,myalpha,mybeta,mytheta,a,b,c)
out = zeros(size(x));
for i = 1: length(x)
first_integral = integral(#(w) eval_K(w,a,b,c),0,1).^-1;
theta_t_ratio = (mytheta./x(i));
incomplete_gamma = igamma(myalpha,theta_t_ratio.^mybeta);
n_gamma = gamma(myalpha);
second_integral = integral(#(w) eval_K(w,a,b,c),0, incomplete_gamma.^mybeta./n_gamma);
out(i) = first_integral*second_integral;
end
end
I try to reproduce a simple linear regression x = A†b using pytorch. But I get completely different numbers.
So first I use plain numpy and do
A_pinv = np.linalg.pinv(A)
betas = A_pinv.dot(b)
print(((b - A.dot(betas))**2).mean())
print(betas)
which results in:
364.12875
[0.43196774 0.14436531 0.42414093]
Now I try to get similar enough numbers using pytorch:
# re-implement via pytoch model using built-ins
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import TensorDataset, DataLoader
# We'll create a TensorDataset, which allows access to rows from inputs and targets as tuples.
# We'll also create a DataLoader, to split the data into batches while training.
# It also provides other utilities like shuffling and sampling.
inputs = to.from_numpy(A)
targets = to.from_numpy(b)
train_ds = TensorDataset(inputs, targets)
batch_size = 5
train_dl = DataLoader(train_ds, batch_size, shuffle=True)
# define model, loss and optimizer
new_model = nn.Linear(source_variables, predict_variables, bias=False)
loss_fn = F.mse_loss
opt = to.optim.SGD(new_model.parameters(), lr=1e-10)
def fit(num_epochs, new_model, loss_fn, opt):
for epoch in tnrange(num_epochs, desc="epoch"):
for xb,yb in train_dl:
# Generate predictions
pred = new_model(xb)
loss = loss_fn(pred, yb)
# Perform gradient descent
loss.backward()
opt.step()
opt.zero_grad()
if epoch % 1000 == 0:
print((new_model.weight, loss))
print('Training loss: ', loss_fn(model(inputs), targets))
# fit the model
fit(10000, new_model, loss_fn, opt)
It prints as the last result:
tensor([[0.0231, 0.5185, 0.4589]], requires_grad=True), tensor(271.8525, grad_fn=<MseLossBackward>))
Training loss: tensor(378.2871, grad_fn=<MseLossBackward>)
As you can see these numbers are completely different so I must have made a mistake somewhere ...
Here are the numbers for A and b to reproduce the result:
A = np.array([[2822.48, 2808.48, 2810.92],
[2832.94, 2822.48, 2808.48],
[2832.57, 2832.94, 2822.48],
[2824.23, 2832.57, 2832.94],
[2854.88, 2824.23, 2832.57],
[2800.71, 2854.88, 2824.23],
[2798.36, 2800.71, 2854.88],
[2818.46, 2798.36, 2800.71],
[2805.37, 2818.46, 2798.36],
[2815.44, 2805.37, 2818.46]], dtype=float32)
b = np.array([2832.94, 2832.57, 2824.23, 2854.88, 2800.71, 2798.36, 2818.46, 2805.37, 2815.44, 2834.4 ], dtype=float32)
I have a code (a code we saw in a class) of a recurrent neural network that reads a given text and tries to produce its own text similar to the example. The code is written in python and uses the pytorch library. I wanted to modify to see whether I could increase its speed by using GPU instead of CPU and I made some tests on google collaboratory. The GPU version of the code runs fine but is about three times slower than the CPU version. I do not know the details of GPU architecture so I can not really understand why it is slower. I know that GPUs can do more arithmetic operations per cycle but have more limited memory so I am curious if I am having a memory issue. I also tried using CUDA with a generative adversarial network and in this case it was almost ten times faster. Any tips on this would be welcome.
The code (CUDA version) is below. I am new at this stuff so sorry if some of the terminology is not correct.
The architecture is input->encoder->recursive network->decoder->output.
import torch
import time
import numpy as np
from torch.autograd import Variable
import matplotlib.pyplot as plt
from google.colab import files
#uploding text on google collab
uploaded = files.upload()
for fn in uploaded.keys():
print('User uploaded file "{name}" with length {length} bytes'.format(
name=fn, length=len(uploaded[fn])))
#data preprocessing
with open('text.txt','r') as file:
#with open closes the file after we are done with it
rawtxt=file.read()
rawtxt = rawtxt.lower()
#a function that assigns a number to each unique character in the text
def create_map(rawtxt):
letters = list(set(rawtxt))
lettermap = dict(enumerate(letters)) #gives each letter in the list a number
return lettermap
num_to_let = create_map(rawtxt)
#inverse to num_to_let
let_to_num =dict(zip(num_to_let.values(), num_to_let.keys()))
print(num_to_let)
#turns a text of characters into text of numbers using the mapping
#given by the input mapdict
def maparray(txt, mapdict):
txt = list(txt)
for k, letter in enumerate(txt):
txt[k]=mapdict[letter]
txt=np.array(txt)
return txt
X=maparray(rawtxt, let_to_num) #the data text in numeric format
Y= np.roll(X, -1, axis=0) #shifted data text in numeric format
X=torch.LongTensor(X)
Y=torch.LongTensor(Y)
#up to here we are done with data preprocessing
#return a random batch for training
#this reads a random piece inside data text
#with the size chunk_size
def random_chunk(chunk_size):
k=np.random.randint(0,len(X)-chunk_size)
return X[k:k+chunk_size], Y[k:k+chunk_size]
nchars = len(num_to_let)
#define the recursive neural network class
class rnn(torch.nn.Module):
def __init__(self,input_size,hidden_size,output_size, n_layers=1):
super().__init__()
self.input_size = input_size
self.hidden_size = hidden_size
self.output_size = output_size
self.n_layers= n_layers
self.encoder = torch.nn.Embedding (input_size, hidden_size)
self.rnn = torch.nn.RNN(hidden_size, hidden_size, n_layers, batch_first=True)
self.decoder = torch.nn.Linear (hidden_size, output_size)
def forward (self,x,hidden):
x=self.encoder(x.view(1,-1))
output, hidden = self.rnn(x.view(1,1,-1), hidden)
output = self.decoder(output.view(1,-1))
return output, hidden
def init_hidden(self):
return Variable(torch.zeros(self.n_layers , 1 , self.hidden_size)).cuda()
#hyper-params
lr = 0.009
no_epochs = 50
chunk_size = 150
myrnn = rnn(nchars, 150, nchars,1)
myrnn.cuda()
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(myrnn.parameters(), lr=lr)
t0 = time.time()
for epoch in range(no_epochs):
totcost=0
generated = ''
for _ in range(len(X)//chunk_size):
h=myrnn.init_hidden()
cost = 0
x, y=random_chunk(chunk_size)
x, y= Variable(x).cuda(), Variable(y).cuda()
for i in range(chunk_size):
out, h = myrnn.forward(x[i],h)
_, outl = out.data.max(1)
letter = num_to_let[outl[0]]
generated+=letter
cost += criterion(out, y[i])
optimizer.zero_grad()
cost.backward()
optimizer.step()
totcost+=cost
totcost/=len(X)//chunk_size
print('Epoch', epoch, 'Avg cost/chunk: ', totcost)
print(generated[0:750],'\n\n\n')
t1 = time.time()
total = t1-t0
print('total',total)
#we encode each word into a vector of fixed size
I am trying to build a neural network model with one hidden layer (1024 nodes). The hidden layer is nothing but a relu unit. I am also processing the input data in batches of 128.
The inputs are images of size 28 * 28. In the following code I get the error in line
_, c = sess.run([optimizer, loss], feed_dict={x: batch_x, y: batch_y})
Error: TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder_64:0", shape=(128, 784), dtype=float32) is not an element of this graph.
Here is the code I have written
#Initialize
batch_size = 128
layer1_input = 28 * 28
hidden_layer1 = 1024
num_labels = 10
num_steps = 3001
#Create neural network model
def create_model(inp, w, b):
layer1 = tf.add(tf.matmul(inp, w['w1']), b['b1'])
layer1 = tf.nn.relu(layer1)
layer2 = tf.matmul(layer1, w['w2']) + b['b2']
return layer2
#Initialize variables
x = tf.placeholder(tf.float32, shape=(batch_size, layer1_input))
y = tf.placeholder(tf.float32, shape=(batch_size, num_labels))
w = {
'w1': tf.Variable(tf.random_normal([layer1_input, hidden_layer1])),
'w2': tf.Variable(tf.random_normal([hidden_layer1, num_labels]))
}
b = {
'b1': tf.Variable(tf.zeros([hidden_layer1])),
'b2': tf.Variable(tf.zeros([num_labels]))
}
init = tf.initialize_all_variables()
train_prediction = tf.nn.softmax(model)
tf_valid_dataset = tf.constant(valid_dataset)
tf_test_dataset = tf.constant(test_dataset)
model = create_model(x, w, b)
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(model, y))
optimizer = tf.train.GradientDescentOptimizer(0.5).minimize(loss)
#Process
with tf.Session(graph=graph1) as sess:
tf.initialize_all_variables().run()
total_batch = int(train_dataset.shape[0] / batch_size)
for epoch in range(num_steps):
loss = 0
for i in range(total_batch):
batch_x, batch_y = train_dataset[epoch * batch_size:(epoch+1) * batch_size, :], train_labels[epoch * batch_size:(epoch+1) * batch_size,:]
_, c = sess.run([optimizer, loss], feed_dict={x: batch_x, y: batch_y})
loss = loss + c
loss = loss / total_batch
if epoch % 500 == 0:
print ("Epoch :", epoch, ". cost = {:.9f}".format(avg_cost))
print("Minibatch accuracy: %.1f%%" % accuracy(predictions, batch_labels))
valid_prediction = tf.run(tf_valid_dataset, {x: tf_valid_dataset})
print("Validation accuracy: %.1f%%" % accuracy(valid_prediction.eval(), valid_labels))
test_prediction = tf.run(tf_test_dataset, {x: tf_test_dataset})
print("TEST accuracy: %.1f%%" % accuracy(test_prediction.eval(), test_labels))
This worked for me
from keras import backend as K
and after predicting my data i inserted this part of code
then i had again loaded the model.
K.clear_session()
i faced this problem in production server,
but in my pc it was running fine
...........
from keras import backend as K
#Before prediction
K.clear_session()
#After prediction
K.clear_session()
Variable x is not in the same graph as model, try to define all of these in the same graph scope. For example,
# define a graph
graph1 = tf.Graph()
with graph1.as_default():
# placeholder
x = tf.placeholder(...)
y = tf.placeholder(...)
# create model
model = create(x, w, b)
with tf.Session(graph=graph1) as sess:
# initialize all the variables
sess.run(init)
# then feed_dict
# ......
If you use django server, just runserver with --nothreading
for example:
python manage.py runserver --nothreading
I had the same issue with flask. adding --without-threads flag to flask run or threaded=False to app.run() fixed it
In my case, I was using loop while calling in CNN multiple times, I fixed my problem by doing the following:
# Declare this as global:
global graph
graph = tf.get_default_graph()
# Then just before you call in your model, use this
with graph.as_default():
# call you models here
Note: In my case too, the app ran fine for the first time and then gave the error above. Using the above fix solved the problem.
Hope that helps.
The error message TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("...", dtype=dtype) is not an element of this graph can also arise in case you run a session outside of the scope of its with statement. Consider:
with tf.Session() as sess:
sess.run(logits, feed_dict=feed_dict)
sess.run(logits, feed_dict=feed_dict)
If logits and feed_dict are defined properly, the first sess.run command will execute normally, but the second will raise the mentioned error.
You can also experience this while working on notebooks hosted on online learning platforms like Coursera. So, implementing following code could help get over with the issue.
Implement this at the topmost block of Notebook file:
from keras import backend as K
K.clear_session()
Similar to #javan-peymanfard and #hmadali-shafiee, I ran into this issue when loading the model in an API. I was using FastAPI with uvicorn. To fix the issue I just set the API function definitions to async similar to this:
#app.post('/endpoint_name')
async def endpoint_function():
# Do stuff here, including possibly (re)loading the model