Can anyone point me to suitable techniques for working with timescales other than seconds?
An example is the Lotka-Volterra 'classic model' on the following site:
https://mbe.modelica.university/behavior/equations/population/
The resulting graph is shown with an x-axis from 1-120 seconds, but obviously that is not realistic for the rabbit/fox example. I've adjusted it in the following code snippet to give an idea of what I am looking for (with the assumption that alpha, beta, gamma and delta are actually rates/day).
My adjustments are a bit clunky and I'm sure there must be a nicer way, I just can't work it out.
I do want something compatible with the standard library and am using OpenModelica. Thanks!
model ClassicModel "This is the typical equation-oriented model"
parameter Real alpha=0.1 "Reproduction rate of prey per day";
parameter Real beta=0.02 "Mortality rate of prey per predator per day";
parameter Real gamma=0.4 "Mortality rate of predator per day";
parameter Real delta=0.02 "Reproduction rate of predator per day";
parameter Real x0=10 "Start value of prey population";
parameter Real y0=10 "Start value of predator population";
Real x "Prey population";
Real y "Predator population";
Real alpha_S=alpha/(60*60*24) "Reproduction rate of prey per second";
Real beta_S=beta/(60*60*24) "Mortality rate of prey per predator per second";
Real gamma_S=gamma/(60*60*24) "Mortality rate of predator per second";
Real delta_S=delta/(60*60*24) "Reproduction rate of predator per second";
initial equation
x=x0;
y=y0;
equation
der(x) = x*(alpha_S-beta_S*y);
der(y) = y*(delta_S*x-gamma_S);
end ClassicModel;
I think what you did is correct, and the issue is in the example. As you say, the rates are probably per day, but as it's just a demonstration it's easier to learn from it if the time numbers are not huge.
An improvements I would make is to omit the second set of parameters, and define the original ones as parameter Real alpha=0.1/(60*60*24) etc.; this way the structure of the code stays simpler, it's clear and directly known what the actual rate comes out to (in case you compare to analytical results or similar), but you can still clearly adjust in the source code.
Another thing is that you can add parameter in front of your _S quantities, as they will not change during execution.
Related
I refrained from asking for help until now, but as my thesis' deadline creeps ever closer and I do not know anybody with experience in RL, I'm trying my luck here.
TLDR;
I have not found an academic/online resource which helps me understand the correct representation of the environment as an observation space. I would be very thankful for any links or for giving me a starting point of how to model the specifics of my environment in an observation space.
Short thematic introduction
The goal of my research is to determine the viability of RL for strategy development in motorsports. This is currently achieved by simulating (lots of!) races and calculating the resulting race time (thus end-position) of different strategic decisions (which are the timing of pit stops + amount of laps to refuel for). This demands a manual input of expected inlaps (the lap a pit stop occurs) for all participants, which implicitly limits the possible strategies by human imagination as well as the amount of possible simulations.
Use of RL
A trained RL agent could decide on its own when to perform a pit stop and how much fuel should be added, in order to minizime the race time and react to probabilistic events in the simulation.
The action space is discrete(4) and represents the options to continue, pit and refuel for 2,4,6 laps respectively.
Problem
The observation space is of POMDP nature and needs to model the agent's current race position (which I hope is enough?). How would I implement the observation space accordingly?
The training is performed using OpenAI's Gym framework, but a general explanation/link to article/publication would also be appreciated very much!
Your observation could be just an integer which represents round or position the agent is in. This is obviously not a sufficient representation so you need to add more information.
A better observation could be the agents race position x1, the round the agent is in x2 and the current fuel in the tank x3. All three of these can be represented by a real number. Then you can create your observation by concating these to a vector obs = [x1, x2, x3].
For a simple model in Dymola, the Start attribute works to provide initial conditions for the DOE equations, like the following examples.
model QuiescentModelUsingStart "Find steady-state solutions to LotkaVolterra equations"
parameter Real alpha=0.1 "Reproduction rate of prey";
parameter Real beta=0.02 "Mortality rate of predator per prey";
parameter Real gamma=0.4 "Mortality rate of predator";
parameter Real delta=0.02 "Reproduction rate of predator per prey";
Real x(start=10) "Prey population";
Real y(start=10) "Predator population";
initial equation
der(x) = 0;
der(y) = 0;
equation
der(x) = x*(alpha-beta*y);
der(y) = y*(delta*x-gamma);
end QuiescentModelUsingStart;
But for the complicated model like a power plant model, which is a strong nonlinear model, it is a lot more complicated.
Based on the Modelica by example(https://mbe.modelica.university/behavior/equations/variables/), the start attribute may also be used as an initial guess if the variable has been chosen as an iteration variable.
So, what is the process of initializing a model in Dymola? Would Dymola take the "equation" part into consideration during initialization, and set the derivate as zero, so it could Find the Steady-State as Initial Conditions?
Or Dymola just uses the "start attributes" and "initial equation" part to get a group of initial values?
How should I ensure that the initialization values I use could make up a steady-state?
Probably an excerpt from the Modelica Language Specification describes what you are looking for:
Before any operation is carried out with a Modelica model [e.g., simulation or linearization], initialization takes place to assign consistent values for all variables present in the model. During this phase, also the derivatives, der(..), and the pre-variables, pre(..), are interpreted as unknown algebraic variables. The initialization uses all equations and algorithms that are utilized in the intended operation [such as simulation or linearization].
This is the first part of Section 8.6, which is about three pages and should give you a pretty good insight on what happens during initialization. It also discusses the start attribute with fixed=true/false.
I have a battery model in Modelica. PNet is the value of power flowing through battery (PNet is positive for charging and negative for discharging). This oscillates based on a load. I want to calculate the number of cycles that the battery is put through and also the depth of discharge comin in from each of these cycles.
This is a pretty generic question so my answer will be rather generic as well. Also it is not clear to me, what you are referring to as a cycle. Wikipedia mentions deep and shallow discharge and there are some others as well.
Some general note: In Modelica the when statement is useful for counting. You can read through Section 8.3.5 of the Modelica Language Specification to get full information on this.
The below examples computes how often the variable PNet turns positive, which should respond to the number of shallow cycles above. Some description for the model:
The model noiseSource computes a random number which is then filtered by a first order (PT1) element to compute PNet. The filter should likely be skipped in the original example, it is only there to smooth the trajectory a bit.
The code in the when statement is executed once at the time when the condition turns true, which enables the counting.
The pre statement accesses the value of cycles right before the when statement got active, which enables counting how often the condition occurred.
The start=0 in cycles(start=0) sets the starting value for the variable cycles, which is necessary as you cannot use cycles = 0 as this would generate an equation for cycles, which is not what you want.
The inner model globalSeed is necessary for the noiseSource to work.
Here is the actual code:
model CycleCounter
inner Modelica.Blocks.Noise.GlobalSeed globalSeed;
Modelica.Blocks.Noise.NormalNoise noiseSource;
parameter Modelica.SIunits.Time T = 1e-3 "Time constant of PT1 element to filter random signal to compute PNet";
Integer cycles(start=0) "Counts the number of ";
Real PNet "Random value";
equation
der(PNet) = (noiseSource.y - PNet)/T;
when PNet > 0 then
cycles = pre(cycles)+1;
end when;
annotation (uses(Modelica(version="3.2.3")));
end CycleCounter;
And the result from simulating in Dymola:
I have two data set, let us name them "actual speed" and "desired speed". My main objective is to match actual speed with the desired speed.
But for doing that in my case, I need to tune FF(1x10), Integral(10x8) and Proportional gain table(10x8).
My approach till now was as follows:-
First, start the iteration with having 0.1 as the initial value in the first cells(FF[0]) of the FF table
Then find the R-square or Co-relation between two dataset( i.e. Actual Speed and Desired Speed)
Increment the value of first cell(FF[0]) by 0.25 and then again compute R-square or Co-relation of two data set.
Once the cell(FF[0]) value reaches 2(Gains Maximum value. Already defined by the lab). Evaluate R-square and re-write the gain value in FF[0] which gives min. error between the two curve.
Then tune the Integral and Proportional table in the same way for the same RPM Range
Once It is tune then go for higher RPM range and repeat step 2-5 (RPM Range: 800-1000; 1000-1200;....;3000-3200)
Now the problem is that this process is taking way too long time to complete. For example it takes around 1 Hr. time to tune one cell of FF. Which is actually very slow.
If possible, Please suggest any other approach which I can try to tune the tables. I am using MATLAB R2010a and I can't shift to any other version of MATLAB because my controller can communicate with this version only and I can't use any app for tuning since my GUI is already communicating with the controller and those two datasets are being made in real-time
In the given figure, lets us take (X1,Y1) curve as Desired speed and (X2,Y2) curve as Actual speed
UPDATE
Is it possible to use the previous value of the time varying variable
for eg:
Suppose I have pipe whose inlet temperature is 298K with a specified uniform mass flow(m_flow), now suppose i am heating the pipe using a heater of 100 watts.
The outlet temperature will be attain a higher temperature of suppose 302K, now if i have to use this outlet temperature as my inlet temperature (in the sense i am recircuilating the water), how would i be doing it?
is it possible to update the value of the inlet temperature based on the outlet temperature at the previous timestep? so that for the next iteration the inlet temperature will be the same as the oulet temperature in the previous iteration (in other words the fluid would be recirculating).
Thanks
You cannot access the value in the previous time step. The closest you can get in Modelica is using delay(exp,T) to get the value T units of time ago.
The timestep does not enter into it at all. A model that uses information about timestep is just wrong. Nature doesn't know or care about integration time steps, the model should reflect that.
It seems to me what you want to capture is transport delay. Transport delay is the delay introduced by the time it takes for molecules, electrons, etc. through the system. So presumably what you wish to model is the time it takes the inlet fluid to reach the exit. Again, this has nothing to do with the integration timestep but rather the velocity of the fluid and the distance it must travel. Once you know how long that takes (by either a priori knowledge of the system of by looking at the simulation results themselves), you can follow Marco's suggestion of using the delay operator.
In order to setup a proper model for the system you described I suggest you to look at the example :
Modelica.Thermal.FluidHeatFlow.Examples.IndirectCooling
of the modelica standard library ver. 3.2. Instead of one pipe you can put an ambient or control volume component to better suit you needs. Moreover using continous and differentiable equations (the delay function is not) you will benefit from some of the advantages of the Modelica code, e.g. you will be able to reuse your models in a much wider range of cases, solve inverse problems, solve initial value problems, ...
I hope this helps,
Marco