I have a battery model in Modelica. PNet is the value of power flowing through battery (PNet is positive for charging and negative for discharging). This oscillates based on a load. I want to calculate the number of cycles that the battery is put through and also the depth of discharge comin in from each of these cycles.
This is a pretty generic question so my answer will be rather generic as well. Also it is not clear to me, what you are referring to as a cycle. Wikipedia mentions deep and shallow discharge and there are some others as well.
Some general note: In Modelica the when statement is useful for counting. You can read through Section 8.3.5 of the Modelica Language Specification to get full information on this.
The below examples computes how often the variable PNet turns positive, which should respond to the number of shallow cycles above. Some description for the model:
The model noiseSource computes a random number which is then filtered by a first order (PT1) element to compute PNet. The filter should likely be skipped in the original example, it is only there to smooth the trajectory a bit.
The code in the when statement is executed once at the time when the condition turns true, which enables the counting.
The pre statement accesses the value of cycles right before the when statement got active, which enables counting how often the condition occurred.
The start=0 in cycles(start=0) sets the starting value for the variable cycles, which is necessary as you cannot use cycles = 0 as this would generate an equation for cycles, which is not what you want.
The inner model globalSeed is necessary for the noiseSource to work.
Here is the actual code:
model CycleCounter
inner Modelica.Blocks.Noise.GlobalSeed globalSeed;
Modelica.Blocks.Noise.NormalNoise noiseSource;
parameter Modelica.SIunits.Time T = 1e-3 "Time constant of PT1 element to filter random signal to compute PNet";
Integer cycles(start=0) "Counts the number of ";
Real PNet "Random value";
equation
der(PNet) = (noiseSource.y - PNet)/T;
when PNet > 0 then
cycles = pre(cycles)+1;
end when;
annotation (uses(Modelica(version="3.2.3")));
end CycleCounter;
And the result from simulating in Dymola:
Related
I want to adjust the mass flow rate of a pump by using similarities laws so if I want to head and flow rate equation. I need to use old value and new value of head so that I can calculate new value of mass flow rate.
Can anybody tell me how can I write this in program in OpenModellica.
Generally Modelica is not designed to access "old values" of a continuous variable, as this is usually not necessary to model physical behavior. For discrete events there is the pre() operator, for clocks previous() but none of them is useful in continuous modeling.
What is common in continuous modeling, is to have relations based on the derivative der() of a variable, but not "old values". Personally I would double-check if you really need an old value or if that is just a form of abstraction chosen due to limitations of other software...
Still you can use the delay() operator as shown below to delay a signal by a fixed or a variable amount of time.
model DelayExample
Real x, y_fix, y_var;
equation
x = time;
y_fix = delay(x, 100e-3);
y_var = delay(x, min(time/2,0.2), 0.2);
end DelayExample;
The result is the following:
If you want to do use graphical modeling, you can use blocks from Modelica.Blocks.Nonlinear, namely
Modelica.Blocks.Nonlinear.FixedDelay
Modelica.Blocks.Nonlinear.PadeDelay
Modelica.Blocks.Nonlinear.VariableDelay
I refrained from asking for help until now, but as my thesis' deadline creeps ever closer and I do not know anybody with experience in RL, I'm trying my luck here.
TLDR;
I have not found an academic/online resource which helps me understand the correct representation of the environment as an observation space. I would be very thankful for any links or for giving me a starting point of how to model the specifics of my environment in an observation space.
Short thematic introduction
The goal of my research is to determine the viability of RL for strategy development in motorsports. This is currently achieved by simulating (lots of!) races and calculating the resulting race time (thus end-position) of different strategic decisions (which are the timing of pit stops + amount of laps to refuel for). This demands a manual input of expected inlaps (the lap a pit stop occurs) for all participants, which implicitly limits the possible strategies by human imagination as well as the amount of possible simulations.
Use of RL
A trained RL agent could decide on its own when to perform a pit stop and how much fuel should be added, in order to minizime the race time and react to probabilistic events in the simulation.
The action space is discrete(4) and represents the options to continue, pit and refuel for 2,4,6 laps respectively.
Problem
The observation space is of POMDP nature and needs to model the agent's current race position (which I hope is enough?). How would I implement the observation space accordingly?
The training is performed using OpenAI's Gym framework, but a general explanation/link to article/publication would also be appreciated very much!
Your observation could be just an integer which represents round or position the agent is in. This is obviously not a sufficient representation so you need to add more information.
A better observation could be the agents race position x1, the round the agent is in x2 and the current fuel in the tank x3. All three of these can be represented by a real number. Then you can create your observation by concating these to a vector obs = [x1, x2, x3].
Modelica modeling is the first principle modeling, so how to test the model and set an effective benchmark is important, for example, I could design a fluid network as my wish, but when building a dynamic simulation model, I need to know the detailed geometry structure and parameters to set up every piece of my model. Usually, I would build a steady-state model with simple energy and mass conservation laws, then design every piece of equipment based on the corresponding design manual, but when I put every dynamic component together, when simulation till steady-state, the result is different from the steady-state model more or less. So I was wondering if I should modify my workflow to make the dynamic model agree with the steady-state model. Any suggestions are welcome.
#dymola #modelica
To my understanding of the question, your parameter values are fixed and physically known. I would attempt the following approach as a heuristic to identify the (few) component(s) that one needs to carefully investigate in order to understand how they influence or violates the assumed first principles.
This is just as a first trial and it could be subject to further improvement and fine-tuning.
Consider the set of significant set of variables xd(p,t) \in R^n and parameters p. Note that p also includes significant start values. p in R^m includes only the set of additional parameters not available in the steady state model.
Denote the corresponding variables of the steady state model by x_s
Denote a time point where the dynamic model is "numerically" in "semi-" steady-state by t*
Consider the function C(xd(p,t*),xs) = ||D||^2 with D = xd(p,t*) - xs
It could be beneficial to describe C as a vector rather than a single valued function.
Compute the partial derivatives of C w.t. p expressed in terms of dxd/dp, i.e.
dC/dp = d[D^T D]/dp
= d[(x_d-x_s)^T (x_d - x_s)]/dp
= (dx_d/dp)^T D + ...
Consider scaling the above function, i.e. dC/dp * p/C (avoid expected numerical issues via some epsilon-tricks)
Here you get a ranking of most significant parameters which are causing the apparent differences. The hopefully few number of components including these parameters could be the ones causing such violation.
If this still does not help, may be due to expected high correlation among parameters, I would go further and consider a dummy parameter identification problem, out of which a more rigorous ranking of significant model parameters can be obtained.
If the Modelica language had capabilities for expressing dynamic parameter sensitivities, all the above computation can be easily carried out as a single Modelica model (with a slightly modified formulation).
For instance, if we had something like der(x,p) corresponding to dx/dp, one could simply state
dcdp = der(C,p)
An alternative approach is proposed via the DerXP library
For a simple model in Dymola, the Start attribute works to provide initial conditions for the DOE equations, like the following examples.
model QuiescentModelUsingStart "Find steady-state solutions to LotkaVolterra equations"
parameter Real alpha=0.1 "Reproduction rate of prey";
parameter Real beta=0.02 "Mortality rate of predator per prey";
parameter Real gamma=0.4 "Mortality rate of predator";
parameter Real delta=0.02 "Reproduction rate of predator per prey";
Real x(start=10) "Prey population";
Real y(start=10) "Predator population";
initial equation
der(x) = 0;
der(y) = 0;
equation
der(x) = x*(alpha-beta*y);
der(y) = y*(delta*x-gamma);
end QuiescentModelUsingStart;
But for the complicated model like a power plant model, which is a strong nonlinear model, it is a lot more complicated.
Based on the Modelica by example(https://mbe.modelica.university/behavior/equations/variables/), the start attribute may also be used as an initial guess if the variable has been chosen as an iteration variable.
So, what is the process of initializing a model in Dymola? Would Dymola take the "equation" part into consideration during initialization, and set the derivate as zero, so it could Find the Steady-State as Initial Conditions?
Or Dymola just uses the "start attributes" and "initial equation" part to get a group of initial values?
How should I ensure that the initialization values I use could make up a steady-state?
Probably an excerpt from the Modelica Language Specification describes what you are looking for:
Before any operation is carried out with a Modelica model [e.g., simulation or linearization], initialization takes place to assign consistent values for all variables present in the model. During this phase, also the derivatives, der(..), and the pre-variables, pre(..), are interpreted as unknown algebraic variables. The initialization uses all equations and algorithms that are utilized in the intended operation [such as simulation or linearization].
This is the first part of Section 8.6, which is about three pages and should give you a pretty good insight on what happens during initialization. It also discusses the start attribute with fixed=true/false.
I'm working on a feed forward artificial neural network (ffann) that will take input in form of a simple calculation and return the result (acting as a pocket calculator). The outcome wont be exact.
The artificial network is trained using genetic algorithm on the weights.
Currently my program gets stuck at a local maximum at:
5-6% correct answers, with 1% error margin
30 % correct answers, with 10% error margin
40 % correct answers, with 20% error margin
45 % correct answers, with 30% error margin
60 % correct answers, with 40% error margin
I currently use two different genetic algorithms:
The first is a basic selection, picking two random from my population, naming the one with best fitness the winner, and the other the loser. The loser receives one of the weights from the winner.
The second is mutation, where the loser from the selection receives a slight modification based on the amount of resulting errors. (the fitness is decided by correct answers and incorrect answers).
So if the network outputs a lot of errors, it will receive a big modification, where as if it has many correct answers, we are close to a acceptable goal and the modification will be smaller.
So to the question: What are ways I can prevent my ffann from getting stuck at local maxima?
Should I modify my current genetic algorithm to something more advanced with more variables?
Should I create additional mutation or crossover?
Or Should I maybe try and modify my mutation variables to something bigger/smaller?
This is a big topic so if I missed any information that could be needed, please leave a comment
Edit:
Tweaking the numbers of the mutation to a more suited value has gotten be a better answer rate but far from approved:
10% correct answers, with 1% error margin
33 % correct answers, with 10% error margin
43 % correct answers, with 20% error margin
65 % correct answers, with 30% error margin
73 % correct answers, with 40% error margin
The network is currently a very simple 3 layered structure with 3 inputs, 2 neurons in the only hidden layer, and a single neuron in the output layer.
The activation function used is Tanh, placing values in between -1 and 1.
The selection type crossover is very simple working like the following:
[a1, b1, c1, d1] // Selected as winner due to most correct answers
[a2, b2, c2, d2] // Loser
The loser will end up receiving one of the values from the winner, moving the value straight down since I believe the position in the array (of weights) matters to how it performs.
The mutation is very simple, adding a very small value (currently somewhere between about 0.01 and 0.001) to a random weight in the losers array of weights, with a 50/50 chance of being a negative value.
Here are a few examples of training data:
1, 8, -7 // the -7 represents + (1+8)
3, 7, -3 // -3 represents - (3-7)
7, 7, 3 // 3 represents * (7*7)
3, 8, 7 // 7 represents / (3/8)
Use a niching techniche in the GA. A useful alternative is niching. The score of every solution (some form of quadratic error, I think) is changed in taking account similarity of the entire population. This maintains diversity inside the population and avoid premature convergence an traps into local optimum.
Take a look here:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.100.7342
A common problem when using GAs to train ANNs is that the population becomes highly correlated
as training progresses.
You could try increasing mutation chance and/or effect as the error-change decreases.
In English. The population becomes genetically similar due to crossover and fitness selection as a local minim is approached. You can introduce variation by increasing the chance of mutation.
You can do a simple modification to the selection scheme: the population can be viewed as having a 1-dimensional spatial structure - a circle (consider the first and last locations to be adjacent).
The production of an individual for location i is permitted to involve only parents from i's local neighborhood, where the neighborhood is defined as all individuals within distance R of i. Aside from this restriction no changes are made to the genetic system.
It's only one or a few lines of code and it can help to avoid premature convergence.
References:
TRIVIAL GEOGRAPHY IN GENETIC PROGRAMMING (2005) - Lee Spector, Jon Klein