I am following this tutorial on a Lokta Volterra (Predator-prey) problem. I managed to product a plot that looks like theirs. However, I want a definitive proof that my implementation is correct.
My first idea is to calculate rabbits[t]-rabbits[t-1] and compare with the analytical equation of the number of rabbits to see if they are the same. However I cannot make the test code work.
The test code
def test(rabbits, foxes, ts, t):
'''
rabbits, foxes, ts: array of numbers of rabbits, foxes and time points
t: a specific timestep (t < len(ts))
'''
calculated_drabbit_dt=(rabbits[t+1]-rabbits[t])
analytic_drabbit_dt=a*rabbits[t]-b*rabbits[t]*foxes[t] # equation is also in link above, rabbit is called u
print([calculated_drabbit_dt], [analytic_drabbit_dt])
assert np.isclose([calculated_drabbit_dt], [analytic_drabbit_dt])
test(rabbits_2, foxes_2, t, t=166)
Why did I do wrong in my test code?
Related
What is the idea behind double QN?
The Bellman equation used to calculate the Q values to update the online network follows the equation:
value = reward + discount_factor * target_network.predict(next_state)[argmax(online_network.predict(next_state))]
The Bellman equation used to calculate the Q value updates in the original DQN is:
value = reward + discount_factor * max(target_network.predict(next_state))
but the target network for evaluating the action is updated using weights of the
online_network and the value and fed to the target value is basically old q value of the action.
any ideas how adding another networks based on weights from the first network helps?
I really liked the explanation from here:
https://becominghuman.ai/beat-atari-with-deep-reinforcement-learning-part-2-dqn-improvements-d3563f665a2c
"This is actually quite simple: you probably remember from the previous post that we were trying to optimize the Q function defined as follows:
Q(s, a) = r + γ maxₐ’(Q(s’, a’))
Because this definition is recursive (the Q value depends on other Q values), in Q-learning we end up training a network to predict its own output, as we pointed out last time.
The problem of course is that at each minibatch of training, we are changing both Q(s, a) and Q(s’, a’), in other words, we are getting closer to our target but also moving our target! This can make it a lot harder for our network to converge.
It thus seems like we should instead use a fixed target so as to avoid this problem of the network “chasing its own tail”, but of course that isn’t possible since the target Q function should get better and better as we train."
Is there any function in matlab to create stationary series? Or somewhere specific that I can get this kind of series? I'm implementing a method and I think the lack of a stationary series is a problem.
Not much to go on here, please try to be more specific w.r.t. the "method" and the "series"... A common error is to use OLS regression on a non-stationary process in which case results could be spurious. To avoid this, when a variable is suspected of being non-stationary, a transformation of that variable can sometimes break that non-stationarity. For example, an autocorrelated AR(1) process with a unit root (non-stationary) will yield the (stationary) innovation if you subtract two successive values from each other (e.g. using the diff function).
To answer your question more directly (but perhaps less usefully), any Matlab function generating a random number without changing the distribution through time will generate a "stationary process"... e.g. plot(randn(1000,1)) shows a "process" which represents pure innovations. If these innovations were in an AR(1) with a unit root (i.e. X(t+1)=X(t)+u) process we could get something like:
AR1 = cumsum(u);
innov=diff(AR1);
plot([AR1(2:end),innov])
As you can see, the AR1 process is non-stationary since it has a unit root. trivially, x is in fact the innovation, but "reverse engineering" it using diff just shows that in this case you can recover from the non-stationary series something stationary.
Finally, a common test to look for non-stationarity is the augmented Dickey Fuller test (a unit root test), which is implemented in Matlab: https://www.mathworks.com/help/econ/adftest.html
I'm trying to solve an ODE in Matlab and I have the following problem:
my code is as bellow:
xinit=[0.19;25;0;7];
t=0:768:76800; %% 101 cells
[t1,y]=ode45(#Model_Bio,t,xinit);
In the function #Model_Bio I have a parameter that I need to read its corresponding values for each time step (101 values)!
So, My #Model_Bio is somehow as bellow:
load 'mydata'
a=mydata;
.....
the problem is that, when I execute the ode45(#Model_Bio,t,xinit), it calls my function with an automated time step (for example 50 times!) and my problem is that I cannot assign the values for each of my time step (101)!
Also, I think its not a good idea to change the time step of the ode the same as my 101 steps!
Anyone that help me on this is really appreciated!
It seems like you need to provide a wrapper of your data that interpolates it for arbitrary t, for example
my_interp = #(t) interp1(my_data_t, my_data_x, t)
http://se.mathworks.com/help/matlab/ref/interp1.html
and then implement your RHS (#Model_Bio) in terms of my_interp
Recently I'm working on a genetic circuits modeling which include a delay differential equation:
∂A/∂t=CA*[1−(d/d0)^4]*P(α,τ)− γa*A/(1+f*(A+ I))
∂I/∂t=CI*[1−(d/d0)4]*P(α,τ)− γi*I/1+f*(A+ I)
∂Hi/∂t=bI/(1+kI)−γh*A*Hi/(1+g*A)+D(He−Hi)
∂He/∂t=(−d/1−d)*D*(He−Hi)−µ*He
P(α,τ)= (δ+α*H(t)^2)/(1+k1*H(t)^2)
H(t)=Hi(t−τ)
the polt of the Hi is supposed to oscillate, however the MATLAB code I wrote doesn't seem to work that way. Can anyone please take a look what's wrong with my code?
history=[0;0;0;0];
tau=10;
sol=dde23(#ddex2de,[tau], zeros(4,1),[0,200]);
figure
plot(sol.x,sol.y(3,:),'--r',sol.x,sol.y(2,:),'-b')
function dydt=ddex2de(t,y,Z)
ylag1=Z(:,1);
H=ylag1(3);
x1=y(1);%A
x2=y(2);%I
x3=y(3);%Hi
x4=y(4);%He
dx1dt=CA*(1-(d/d0)^4)*(delta+alpha*(H.^n1))/(1+k1*(H.^n1)) - gammaA*x1/(1+f*(x1+x2));
dx2dt=CI*(1-(d/d0)^4)*(delta+alpha*power(H, n1))/(1+k1*power(H, n1)) - gammaI*x2/(1+f*(x1+x2));
dx3dt=(b*x2/(1+k*x2))-(gammaH*x1*x3/(1+g*x1))+D*(x4-x3);
dx4dt=(-d/(1-d))*D*(x4-x3)-mu*x4;
dydt=[dx1dt,dx2dt,dx3dt,dx4dt]';
end
In your sample code most of the variables are undefined, what are the values of: d d0 CA CI delta alpha ... ? It might be that some of these parameters has a wrong value.
Alternatively try to change time interval tspan, as your input value is [0,200] to something longer, say [0,10000] or more, as your oscillation might take more time than 200 units for one cycle.
Just few questions, i hope someone will find time to answer :).
What if we have COUPLED model example: system of n indepedent variables X and n nonlinear partial differential equations PDEf(X,PDEf(X)) with respect to TIME that depends of X,PDEf(X)(partial differential equation depending of variables X ). Can you give some advice? Here is one example:
Let’s say that c is output, or desired variable. Let’s say that r is independent variable.Partial differential equation looks like:
∂c/∂t=D*1/r+∂c/∂r+2(D* (∂^2 c)/(∂r^2 ))
D=constant
r=0:0.1:Rp- Matlab syntaxis, how to represent same in Modelica (I use integrator,but didn't work)?
Here is a code (does not work):
model PDEtest
/* Boundary conditions
1. delta(c)/delta(r)=0 for r=0
2. delta(c)/delta(r)=-j*d for r=Rp*/
parameter Real Rp=88*1e-3; // length
parameter Real initialConc=1000;
parameter Real Dp=1e-14;
parameter Integer np=10; // num. of points
Real cp[np](start=fill(initialConc,np));
Modelica.Blocks.Continuous.Integrator r(k=1); // independent x1
Real j;
protected
parameter Real dr=Rp/np;
parameter Real ts= 0.01; // for using when loop (sample(0,ts) )
algorithm
j:=sin(time); // this should be indepedent variable like x2
r.u:=dr;
while r.y<=Rp loop
for i in 2:np-1 loop
der(cp[i]):=2*Dp/r.y+(cp[i]-cp[i-1])/dr+2*(Dp*(cp[i+1]-2*cp[i]+cp[i-1])/dr^2);
end for;
if r.y==Rp then
cp[np]:=-j*Dp;
end if;
cp[1]:=if time >=0 then initialConc else initialConc;
end while;
annotation (uses(Modelica(version="3.2")));
end PDEtest;
Here are more questions:
This code don’t work in OpenModelica 1.8.1, also don’t work in Dymola 2013demo. How can we have continuos function of variable c, not array of functions ?
Can we place values of array cp in combiTable? And how?
If instead “algorithm” stay “equation” code can’t be succesfull checked.Why? In OpenModelica, error is :could not flattening model :S.
Is there any simplified way to use a set of equation (PDE’s) that are coupled? I know for PDEs library in Modelica, but I think they are complicated. I want to write a function for solving PDE and call these function in “main model”, so that output of function be continuos function of “c”.I don’t know what for doing with array of functions.
Can you give me advice how to understand Modelica language, if we “speak” like in Matlab? For example: Values of independent variable r,we can specife in Matlab, like r=0:TimeStep:Rp…How to do same in Modelica? And please explain me how section “equation” works, is there similarity with Matlab, and is there necessary sequancial approach?
Cheers :)
It's hard to answer your question, since you assuming that Modelica ~ Matlab, but that's not the case. So I won't comment your code, since it's really wrong. Let me give you an example model to the burger equation. Maybe you could use it as starting point.
model burgereqn
Real u[N+2](start=u0);
parameter Real h = 1/(N+1);
parameter Integer N = 10;
parameter Real v = 234;
parameter Real Pi = 3.14159265358979;
parameter Real u0[N+2]={((sin(2*Pi*x[i]))+0.5*sin(Pi*x[i])) for i in 1:N+2};
parameter Real x[N+2] = { h*i for i in 1:N+2};
equation
der(u[1]) = 0;
for i in 2:N+1 loop
der(u[i]) = - ((u[i+1]^2-u[i-1]^2)/(4*(x[i+1]-x[i-1])))
+ (v/(x[i+1]-x[i-1])^2)*(u[i+1]-2*u[i]+u[i+1]);
end for;
der(u[N+2]) = 0;
end burgereqn;
Your further questions:
cp is an continuous variable and the array is representing
every discretization point.
Why you should want to do that, as far as I understand cp is
your desired solution variable.
You should try to use almost always equation section
algorithm sections are usually used in functions. I'm pretty
sure you can represent your desire behaviour with equations.
I don't know that library, but the hard thing on a pde is the
discretization and the solving it self. You may run into issues
while solving the pde with a modelica tool, since usually
a Modelica tool has no specialized solving algorithm for pdes.
Please consider for that question further references. You could
start with Modelica.org.