I am trying to model a reactor, which means I have a set of differential equations I need to solve. One of the primary inputs to these differential equations, is the rate equation given by:
[
Highlighted is what I have named "term 4". I have wrote the equation in MATLAB within the function I am solving as follows:
T = var(1);
P = var(2);
FA = var(3);
FB = var(4);
FC = var(5);
FD = var(6);
F_tot = FA + FB + FC + FD + F_E0 + F_F0 + F_G0; %mol/hr
molA = FA/F_tot;
molB = FB/F_tot;
molC = FC/F_tot;
molD = FD/F_tot;
K = exp((4577.8/T)-4.33);
P_atm = P*0.00986923; %atm
FPress = P_atm^(0.5-(P_atm/250));
r_A = -FPress * (1.69*10^9) * exp(-88000/(R * T)) * (molA^0.9) * (molB ^0.31) * (molC^-0.156) * (molD ^ -0.05) * (1-((molC * molD)/(molA * molB * K)));
The system of ODEs is as below
diffeqs = [
(H_rxn*r_A)/(F_A0 * (CO_eng + H20_eng + CO2_eng + H2_eng + Ar_eng +N2_eng + CH4_eng))
- ((alpha/2*p)*(T/T_0))
r_A
(H20/CO)*r_A
-(H2/CO)*r_A
-(CO2/CO)*r_A]
At the first integration, the error between my hand calculated term 4 is small and probably just due to rounding errors.
After tspan = 20k, the error is huge. I don't understand how this happens as the terms within the equation seem to be as they should be.
Any help would be appreciated. If more info needed just comment and I will add.
Related
I want to plot the Lorentz system (s=10, r=28, b=8/3) for a start condition which should give 0 since it's a start condition in the plane spanned by v1 and v2 and trough the critical point (0,0,0) (equation of the plane : -x + (-9- \sqrt(1201))/56 * y = 0) When I use the Runge-Kutta method to plot the solution with as start conditions ((-9-sqrt(1201))/56,1,10) my graph doesn't converge to 0 and I don't know why.
[X,Y,Z,T] = Runge(T0,(-9-sqrt(1201))/56,1,10,h,1000);
plot(T,X);
plot(T,Y);
plot(T,Z);
I expect a solution in which the lines in the plots go to 0. But I get some random function which goes up and down all the time.
Could this be caused by approximations?
Thanks in advance
This is the function Runge
function [X,Y,Z,T] = Runge(t0,x0,y0,z0,h,n)
X=[x0];
Y=[y0];
Z=[z0];
T=[t0];
k1x = 0;
k1y = 0;
k1z = 0;
k2x=0;
k2y=0;
k2z=0;
k3x=0;
k3y=0;
k3z=0;
k4x=0;
k4y=0;
k4z=0;
for k = 1:n
T(k+1)= T(k) +h;
k1x = F(X(k),Y(k),Z(k));
k1y = G(X(k),Y(k),Z(k));
k1z = H(X(k),Y(k),Z(k));
k2x = F(X(k)+h/2*k1x,Y(k) + h/2*k1y, Z(k) + h/2*k1z);
k2y= G(X(k)+h/2*k1x,Y(k) + h/2*k1y, Z(k) + h/2*k1z);
k2z= H(X(k)+h/2*k1x,Y(k) + h/2*k1y, Z(k) + h/2*k1z);
k3x= F(X(k)+h/2*k2x,Y(k)+h/2*k2y,Z(k) + h/2*k2z);
k3y=G(X(k)+h/2*k2x,Y(k)+h/2*k2y,Z(k) + h/2*k2z);
k3z=H(X(k)+h/2*k2x,Y(k)+h/2*k2y,Z(k) + h/2*k2z);
k4x= F(X(k)+h*k3x,Y(k)+h*k3y,Z(k)+h*k3z);
k4y=G(X(k)+h*k3x,Y(k)+h*k3y,Z(k)+h*k3z);
k4z=H(X(k)+h*k3x,Y(k)+h*k3y,Z(k)+h*k3z);
X(k+1) = X(k) + h/6 * (k1x + 2*k2x + 2*k3x + k4x);
Y(k+1) = Y(k) + h/6 * (k1y + 2*k2y + 2*k3y + k4y);
Z(k+1) = Z(k) + h/6 * (k1z + 2*k2z + 2*k3z + k4z);
end
end
There is no such plane for the non-linear system
Why do you think this plane is invariant?
In the system
F=#(x,y,z) sigma*(y-x);
G=#(x,y,z) x*(rho-z)-y;
H=#(x,y,z) x*y-beta*z;
no non-trivial linear combination of the first two equations is independent of z, which would be required to get an invariant expression in only x and y.
The invariant sub-spaces of the linearization are close to the stable and unstable manifolds, but they are not identical to them. So if you are on some invariant subspace of the linearization, you are in fact some distance away from any curved sub-manifolds related to the non-linear system, and numerical as well as exact solutions will move towards the chaotic attractor.
The Lorentz system is chaotic
Apart from that your observation is correct that floating point errors accumulate to drive the numerical solution away from the exact solution and any properties of it.
I am doing first steps in machine learning. Firstly, I try to create a simple algorithm, for example, linear regression of two variables. So, this manual (https://towardsdatascience.com/linear-regression-using-gradient-descent-in-10-lines-of-code-642f995339c0) is best example of coding this one. When I transfer this code, but it does not work. More correct, it prints unreal parameters of regression. Please, help me to overcome the problem. The script below.
x_1 = range(1,100)
y_1 = range(1,100)
N = float(len(y_1))
epochs=1000
m_current = b_current = 0
learning_rate=0.01
for i in range(epochs):
for X,y in zip(x_1, y_1):
y_current = (m_current * X) + b_current
cost = (y-y_current)/N
m_gradient = -(2/N)*(X * (y - y_current))
b_gradient = -(2/N)*(y - y_current)
m_current = m_current - (learning_rate * m_gradient)
b_current = b_current - (learning_rate * b_gradient)
print(m_current)
print(b_current)
print(cost)
/*print
1.9999 i excpect 0.9999999 or 1
9.2333 i excpect 0.00000001 or 0
101.11 i excpect 0.1
*/
I know this question has been asked in various forms, but I can't really find any answer I can understand and use. So forgive me if this is a basic question, 'cause I'm a newbie to these tools(theano/keras)
Problem to Solve
Monitor variables in Neural Networks
(e.g. input/forget/output gate values in LSTM)
What I'm currently getting
no matter in which stage I'm getting those values, I'm getting something like :
Elemwise{mul,no_inplace}.0
Elemwise{mul,no_inplace}.0
[for{cpu,scan_fn}.2, Subtensor{int64::}.0, Subtensor{int64::}.0]
[for{cpu,scan_fn}.2, Subtensor{int64::}.0, Subtensor{int64::}.0]
Subtensor{int64}.0
Subtensor{int64}.0
Is there any way I can't monitor(e.g. print to stdout, write to a file, etc) them?
Possible Solution
Seems like callbacks in Keras can do the job, but it doesn't work either for me. I'm getting same thing as above
My Guess
Seems like I'm making very simple mistakes.
Thank you very much in advance, everyone.
ADDED
Specifically, I'm trying to monitor input/forget/output gating values in LSTM.
I found that LSTM.step() is for computing those values:
def step(self, x, states):
h_tm1 = states[0] # hidden state of the previous time step
c_tm1 = states[1] # cell state from the previous time step
B_U = states[2] # dropout matrices for recurrent units?
B_W = states[3] # dropout matrices for input units?
if self.consume_less == 'cpu': # just cut x into 4 pieces in columns
x_i = x[:, :self.output_dim]
x_f = x[:, self.output_dim: 2 * self.output_dim]
x_c = x[:, 2 * self.output_dim: 3 * self.output_dim]
x_o = x[:, 3 * self.output_dim:]
else:
x_i = K.dot(x * B_W[0], self.W_i) + self.b_i
x_f = K.dot(x * B_W[1], self.W_f) + self.b_f
x_c = K.dot(x * B_W[2], self.W_c) + self.b_c
x_o = K.dot(x * B_W[3], self.W_o) + self.b_o
i = self.inner_activation(x_i + K.dot(h_tm1 * B_U[0], self.U_i))
f = self.inner_activation(x_f + K.dot(h_tm1 * B_U[1], self.U_f))
c = f * c_tm1 + i * self.activation(x_c + K.dot(h_tm1 * B_U[2], self.U_c))
o = self.inner_activation(x_o + K.dot(h_tm1 * B_U[3], self.U_o))
with open("test_visualization.txt", "a") as myfile:
myfile.write(str(i)+"\n")
h = o * self.activation(c)
return h, [h, c]
And as it's in the code above, I tried to write the value of i into a file, but it only gave me values like :
Elemwise{mul,no_inplace}.0
[for{cpu,scan_fn}.2, Subtensor{int64::}.0, Subtensor{int64::}.0]
Subtensor{int64}.0
So I tried i.eval() or i.get_value(), but both failed to give me values.
.eval() gave me this:
theano.gof.fg.MissingInputError: An input of the graph, used to compute Subtensor{::, :int64:}(<TensorType(float32, matrix)>, Constant{10}), was not provided and not given a value.Use the Theano flag exception_verbosity='high',for more information on this error.
and .get_value() gave me this:
AttributeError: 'TensorVariable' object has no attribute 'get_value'
So I backtracked those chains(which line calls which functions..) and tried to get values at every steps I found but in vain.
Feels like I'm in some basic pitfalls.
I use the solution described in the Keras FAQ:
http://keras.io/getting-started/faq/#how-can-i-visualize-the-output-of-an-intermediate-layer
In detail:
from keras import backend as K
intermediate_tensor_function = K.function([model.layers[0].input],[model.layers[layer_of_interest].output])
intermediate_tensor = intermediate_tensor_function([thisInput])[0]
yields:
array([[ 3., 17.]], dtype=float32)
However I'd like to use the functional API but I can't seem to get the actual tensor, only the symbolic representation. For example:
model.layers[1].output
yields:
<tf.Tensor 'add:0' shape=(?, 2) dtype=float32>
I'm missing something about the interaction of Keras and Tensorflow here but I'm not sure what. Any insight much appreciated.
One solution is to create a version of your network that is truncated at the LSTM layer of which you want to monitor the gate values, and then replace the original layer with a custom layer in which the stepfunction is modified to return not only the hidden layer values, but also the gate values.
For instance, say you want to access the access the gate values of a GRU. Create a custom layer GRU2 that inherits everything from the GRU class, but adapt the step function such that it returns a concatenation of the states you want to monitor, and then takes only the part containing the previous hidden layer activations when computing the next activations. I.e:
def step(self, x, states):
# get prev hidden layer from input that is concatenation of
# prev hidden layer + reset gate + update gate
x = x[:self.output_dim, :]
###############################################
# This is the original code from the GRU layer
#
h_tm1 = states[0] # previous memory
B_U = states[1] # dropout matrices for recurrent units
B_W = states[2]
if self.consume_less == 'gpu':
matrix_x = K.dot(x * B_W[0], self.W) + self.b
matrix_inner = K.dot(h_tm1 * B_U[0], self.U[:, :2 * self.output_dim])
x_z = matrix_x[:, :self.output_dim]
x_r = matrix_x[:, self.output_dim: 2 * self.output_dim]
inner_z = matrix_inner[:, :self.output_dim]
inner_r = matrix_inner[:, self.output_dim: 2 * self.output_dim]
z = self.inner_activation(x_z + inner_z)
r = self.inner_activation(x_r + inner_r)
x_h = matrix_x[:, 2 * self.output_dim:]
inner_h = K.dot(r * h_tm1 * B_U[0], self.U[:, 2 * self.output_dim:])
hh = self.activation(x_h + inner_h)
else:
if self.consume_less == 'cpu':
x_z = x[:, :self.output_dim]
x_r = x[:, self.output_dim: 2 * self.output_dim]
x_h = x[:, 2 * self.output_dim:]
elif self.consume_less == 'mem':
x_z = K.dot(x * B_W[0], self.W_z) + self.b_z
x_r = K.dot(x * B_W[1], self.W_r) + self.b_r
x_h = K.dot(x * B_W[2], self.W_h) + self.b_h
else:
raise Exception('Unknown `consume_less` mode.')
z = self.inner_activation(x_z + K.dot(h_tm1 * B_U[0], self.U_z))
r = self.inner_activation(x_r + K.dot(h_tm1 * B_U[1], self.U_r))
hh = self.activation(x_h + K.dot(r * h_tm1 * B_U[2], self.U_h))
h = z * h_tm1 + (1 - z) * hh
#
# End of original code
###########################################################
# concatenate states you want to monitor, in this case the
# hidden layer activations and gates z and r
all = K.concatenate([h, z, r])
# return everything
return all, [h]
(Note that the only lines I added are at the beginning and end of the function).
If you then run your network with GRU2 as last layer instead of GRU (with return_sequences = True for the GRU2 layer), you can just call predict on your network, this will give you all hidden layer and gate values.
The same thing should work for LSTM, although you might have to puzzle a bit to figure out how to store all the outputs you want in one vector and retrieve them again afterwards.
Hope that helps!
You can use theano's printing module for printing during execution (and not during definition, which is what you're doing and the reason why you're not getting values, but their abstract definition).
Print
Just use the Print function. Don't forget to use the output of Print to continue your graph, otherwise the output will be disconnected and Print will most likely be removed during optimisation. And you will see nothing.
from keras import backend as K
from theano.printing import Print
def someLossFunction(x, ref):
loss = K.square(x - ref)
loss = Print('Loss tensor (before sum)')(loss)
loss = K.sum(loss)
loss = Print('Loss scalar (after sum)')(loss)
return loss
Plot
A little bonus you might enjoy.
The Print class has a global_fn parameter, to override the default callback to print. You can provide your own function and directly access to the data, to build a plot for instance.
from keras import backend as K
from theano.printing import Print
import matplotlib.pyplot as plt
curve = []
# the callback function
def myPlottingFn(printObj, data):
global curve
# Store scalar data
curve.append(data)
# Plot it
fig, ax = plt.subplots()
ax.plot(curve, label=printObj.message)
ax.legend(loc='best')
plt.show()
def someLossFunction(x, ref):
loss = K.sum(K.square(x - ref))
# Callback is defined line below
loss = Print('Loss scalar (after sum)', global_fn=myplottingFn)(loss)
return loss
BTW the string you passed to Print('...') is stored in the print object under property name message (see function myPlottingFn). This is useful for building multi-curves plot automatically
I performed a surface fit of some data in Matlab using the fit function.
To be more specific:
func_trim= fit( [hopper_volume, water_ballast_volume], trim, 'poly55');
It returned a multi-variable polynomial:
Linear model Poly55:
func_trim(x,y) = p00 + p10*x + p01*y + p20*x^2 + p11*x*y + p02*y^2 + p30*x^3
+ p21*x^2*y + p12*x*y^2 + p03*y^3 + p40*x^4 + p31*x^3*y
+ p22*x^2*y^2 + p13*x*y^3 + p04*y^4 + p50*x^5 + p41*x^4*y
+ p32*x^3*y^2 + p23*x^2*y^3 + p14*x*y^4 + p05*y^5
Coefficients (with 95% confidence bounds):
p00 = -4.742 (-4.745, -4.74)
p10 = 5.068e-05 (4.969e-05, 5.167e-05)
p01 = 0.001447 (0.001436, 0.001459)
p20 = -3.565e-09 (-3.731e-09, -3.399e-09)
p11 = -6.534e-08 (-6.7e-08, -6.369e-08)
p02 = -1.6e-07 (-1.815e-07, -1.385e-07)
p30 = 5.919e-13 (5.795e-13, 6.043e-13)
p21 = 4.683e-12 (4.552e-12, 4.815e-12)
p12 = 1.129e-11 (9.908e-12, 1.267e-11)
p03 = 1.471e-10 (1.282e-10, 1.659e-10)
p40 = -2.017e-17 (-2.059e-17, -1.975e-17)
p31 = -1.562e-16 (-1.611e-16, -1.513e-16)
p22 = -6.93e-16 (-7.444e-16, -6.417e-16)
p13 = -9.909e-16 (-1.551e-15, -4.308e-16)
p04 = -6.751e-14 (-7.516e-14, -5.986e-14)
p50 = 2.446e-22 (2.392e-22, 2.5e-22)
p41 = 2.186e-21 (2.118e-21, 2.254e-21)
p32 = 1.321e-20 (1.243e-20, 1.4e-20)
p23 = 3.805e-20 (2.969e-20, 4.642e-20)
p14 = 2.262e-20 (-6.978e-20, 1.15e-19)
p05 = 1.059e-17 (9.424e-18, 1.175e-17)
I now want to use this object in the simulink environment.
One option would be to explicitly define this in simulink by just hard-coding it in. However I don't prefer this because the coefficients might change (quite frequently) over time and I would have to repeat this every time.
So in short. Are there any more elegant ways to use this object straight in simulink?
The easiest thing to do is to wrap the fit object into a function handle and then call that from an Interpreted MATLAB function block.
To create the function handle:
>> myFunc = #(u)feval(func_trim,u);
Then with the Interpreted MATLAB function block you'll need to
Mux the x and y signals together, then feed the combined signal into the block.
Use "myFunc(u)" - without the double quotes - as the name of the function to call.
The above is not the fastest approach from an execution pespective, nor will it allow for code generation, but it is by far the easiest/quickest to get something up an running.
I have a series of experimental kinetic data in a form of Concentration = f(time), namely:
L(time) = f0(time)
A(time) = f1(time)
B(time) = f2(time)
C(time) = f3(time)
D(time) = f4(time)
P(time) = f5(time)
A(time) + ... + P(Time) = known value
I have a model represented as a system of differential equations:
dL/dt = -k_tot*MR3*(L)^2, where k_tot = k_LA + k_LB + k_LC + k_LD
dMR3/dt = -k_tot*MR3*L
dP/dt = k_AP*A + k_BP*B + k_CP*C + k_DP*D
dA/dt = 2*k_LA*MR3*(L)^2 - k_AP*A
dB/dt = 2*k_LB*MR3*(L)^2 - k_BP*B
dC/dt = 2*k_LC*MR3*(L)^2 - k_CP*C
dD/dt = 2*k_LA*MR3*(L)^2 - k_DP*D
I would like to use ODE model (ode45) in order to:
find/estimate the k_XY (k_LA, k_LB, ..., k_AP etc)
curve-fit the calculated data with the experimental ones.
The idea is to set up the ODE-based function, which would allow to fit a specified data for a specified diff.equation within the given system of equations.
Could someone help with the example of Matlab code, please?
Thanks much in advance!