I'm trying to set up a new medium model on the basis of Modelica.Media. My model extends from PartialMixtureMedium. When simulating an open vessel (taken from Modelica.Fluid), which has neither feed nor drain but is heated from the outside, the mass balance is not fulfilled. While der(m) remains zero, m increases heavily.
Does anybody have a hint for me?
Thanks in advance
TIMO.
I am not sure that your problem is with the medium model. If partial derivatives with respect to composition or density are wrong (and used in the model), it is possible that the medium model causes the problem, but without more information I cannot determine this. There can be other reasons, the open vessel is only valid for liquids, and if your fluid model would be boiling or in the two phase region, it may be outside the validity region of the vessel model.
Hubertus
Related
I am fairly new to constraint programming, so apologies for any misconceptions.
Assume the following example problem:
A pump is moving water from a bottomless lake into a tightly constrained reservoir at a set rate. There is demand on the reservoir over the day with each hour having different demand. The goal is to ensure the reservoir stays within limits by turning the pump on and off.
Example data for a 6hr problem:
res_bounds = [0, 20]
demand = [0,8,1,13,11,12]
A feasible solution would be:
pump_schedule = [1,1,0,1,1,1]
Giving a volume profile of
[10,12,11,8,7,5]
When modelling the above example with a MIP, the "standard" formulation is discretized 1-hour steps with an underlying mass-balance model. These models, however, tend to scale poorly when the time steps become too fine or the size of the problem gets out of hand.
At first glance, CP offers a solution with features such as interval variables for efficiently scheduling the pump at a very fine resolution. However, tracking the reservoir levels proves a challenge as the model has no visibility of the levels in the middle of the pump running interval.
Google's or-tools offers AddReservoirConstraintWithActive to model the problem exactly as I described it, but seems to require discrete boolean time variables, running into the same issue as the MIP solvers.
In short, is there a way in constraint programming to model such reservoir constraints without boolean time variables? I tried looking around online but found nothing that would help so far.
Thanks in advance.
LBM focuses on fluid clusters, and uses the macro fluid density and velocity to calculate the equilibrium distribution function, and then uses the evolution equation to achieve system iteration. But if we add the same fluid to the lattice grid points in the LBM or reduce the existing fluid continuously, how should we recalculate the macro fluid density and velocity? Or how should the distribution function at the lattice grid point be recalculated? Can LBM simulate a scenario where fluid is continuously added or reduced to the system? For example, water keeps flowing from the tap.
The traditional lattice-Boltzmann method (e.g. the D2Q9 lattice in 2D) can only be applied to incompressible flows. Put in simple terms this means that there can't be more mass entering the domain than exiting it: The mass inside the domain is roughly the same throughout the simulation. This simplification of the generally compressible Navier-Stokes equations can not only be applied to incompressible fluids (such as water) but also to low-Mach number flows like the flow around a car (for more details see here). Yet the traditional lattice-Boltzmann method can't describe multi-phase and free-surface flows as well as flows with sinks and sources (which all result in a change of density of the system).
Any inlet or outlet conditions in the incompressible lattice-Boltzmann method falls in one of the following categories:
Periodic boundaries (the populations that exit the domain on one side enter it again on the other side)
Pressure-drop-periodic boundaries (such as Zhang/Kwok) for periodic flow but with an additional term for compensating for a pressure drop inside the domain due to friction
Velocity and pressure boundaries (generally a velocity inlet and a pressure outlet): There exist various formulations of these to make sure that the moments of the distribution are actually conserved and they have different characteristics regarding numeric stability. Most of them enforce some sort of symmetry and extrapolation of higher moments. The simplest ones are the ones by Zou/He but others like Guo's extrapolation method are significantly more stable for under-resolved and turbulent (high Reynolds number flows). This review discusses different ones in more detail.
You can have a look at this small code I have written in C++ for 2D and 3D simulations if you are interested in more details on how this actually works.
That being said there exist though several variations of lattice-Boltzmann methods in research that allow for multi-component or multi-phase flows (e.g. by introducing additional distributions) or compressible flows (with lattices with more discrete velocities and potentially a second lattice) but they are still exotics and you won't find many implementations around.
I am working on driving industrial robots with neural nets and so far it is working well. I am using the PPO algorithm from the OpenAI baseline and so far I can drive easily from point to point by using the following rewarding strategy:
I calculate the normalized distance between the target and the position. Then I calculate the distance reward with.
rd = 1-(d/dmax)^a
For each time step, I give the agent a penalty calculated by.
yt = 1-(t/tmax)*b
a and b are hyperparameters to tune.
As I said this works really well if I want to drive from point to point. But what if I want to drive around something? For my work, I need to avoid collisions and therefore the agent needs to drive around objects. If the object is not straight in the way of the nearest path it is working ok. Then the robot can adapt and drives around it. But it gets more and more difficult to impossible to drive around objects which are straight in the way.
See this image :
I already read a paper which combines PPO with NES to create some Gaussian noise for the parameters of the neural network but I can't implement it by myself.
Does anyone have some experience with adding more exploration to the PPO algorithm? Or does anyone have some general ideas on how I can improve my rewarding strategy?
What you describe is actually one of the most important research areas of Deep RL: the exploration problem.
The PPO algorithm (like many other "standard" RL algos) tries to maximise a return, which is a (usually discounted) sum of rewards provided by your environment:
In your case, you have a deceptive gradient problem, the gradient of your return points directly at your objective point (because your reward is the distance to your objective), which discourage your agent to explore other areas.
Here is an illustration of the deceptive gradient problem from this paper, the reward is computed like yours and as you can see, the gradient of your return function points directly to your objective (the little square in this example). If your agent starts in the bottom right part of the maze, you are very likely to be stuck in a local optimum.
There are many ways to deal with the exploration problem in RL, in PPO for example you can add some noise to your actions, some other approachs like SAC try to maximize both the reward and the entropy of your policy over the action space, but in the end you have no guarantee that adding exploration noise in your action space will result in efficient of your state space (which is actually what you want to explore, the (x,y) positions of your env).
I recommend you to read the Quality Diversity (QD) literature, which is a very promising field aiming to solve the exploration problem in RL.
Here is are two great resources:
A website gathering all informations about QD
A talk from ICLM 2019
Finally I want to add that the problem is not your reward function, you should not try to engineer a complex reward function such that your agent is able to behave like you want. The goal is to have an agent that is able to solve your environment despite pitfalls like the deceptive gradient problem.
I was wondering how a mechanical stop can be modeled most efficiently.
I do a hydraulic simulation with a controlled hydraulic cylinder in OpenModelica. For the hydraulic cylinder I use the sweptVolume model from the Modelica Standard Library.
What bugs me about this model is that there is no mechanical stop if the piston reaches the bottom of the cylinder.
I tried several ideas with no good result. I tried to reset the displacement of the piston to zero, if it hits the bottom, via an if-expression. But this is not really a good option due to the fact that the volume is calculated using the piston's displacement.
I then tried to introduce a force that equals the force applied to the piston, if the piston hits the stop. This option didn't work either, because in this case the pressure inside the cylinder can not be calculated.
The third try was to use the MSL model of MassWithStopAndFriction linked to the translational flange of the sweptVolume model, but this model seems to be broken for me.
Now I count on you as a competent community to bring in some more ideas for me to test.
Depending on your application, you may deploy the Hydraulics library? The library aims to model (compressible) fluid power systems and contains cylinders with end-stops. Its scope is different than the Fluid package you are using.
Using when and/or if statements for this task, I'd strongly discourage from experience. You may get one cylinder to work, but using that in a larger system will definitely get you into numerical problems. Have a look at the Mechanics package and analyse if the ElastoGap can be of any use to you.
i am reading in soft computing algorithms ,currently in "Particle Swarm Optimization ",i understand the technique in general but ,i stopped at mathematical or physics part which i can't imagine or understand how it works or how it affect the flying,that part is the first part in the equation which update the velocity which is called the "Inertia Factor"
the complete update velocity equation is :
i read in one article in section 2.3 "Ineteria Factor" that:
"This variation of the algorithm aims to balance two possible PSO tendencies (de-
pendent on parameterization) of either exploiting areas around known solutions
or explore new areas of the search space. To do so this variation focuses on the
momentum component of the particles' velocity equation 2. Notice that if you
remove this component the movement of the particle has no memory of the pre-
vious direction of movement and it will always explore close to a found solution.
On the other hand if the velocity component is used, or even multiplied by a w
(inertial weight, balances the importance of the momentum component) factor
the particle will tend to explore new areas of the search space since it cannot
easily change its velocity towards the best solutions. It must rst \counteract"
the momentum previously gained, in doing so it enables the exploration of new
areas with the time \spend counteracting" the previous momentum. This vari-
ation is achieved by multiplying the previous velocity component with a weight
value, w."
the full pdf at: https://www.google.com.eg/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CDIQFjAA&url=http%3A%2F%2Fweb.ist.utl.pt%2F~gdgp%2FVA%2Fdata%2Fpso.pdf&ei=0HwrUaHBOYrItQbwwIDoDw&usg=AFQjCNH8vChXHXWz_ydHxJKAY0cUa94n-g
but i can't also imagine how physicaly or numerically this is happend and how this factor affect going from exploration level to exploitative level ,so need a numerical example to see how it's work and imagine how it's work.
also ,in Genetic Algorithm there's a schema theorem which is a proof of GA success of finding optimum solution,is there's such athoerm for PSO.
It's not easy to explain PSO using mathematics (see Wikipedia article for example).
But you can think like this: the equation has 3 parts:
particle speed = inertia + local memory + global memory
So you control the 'importance' of these components by varying the coefficientes in each part.
There's no analytical way to see this, unless you make the stocastic part constant and ignore things like particle-particle interation.
Exploit: take advantage of the best know solutions (local and global).
Explore: search in new directions, but don't ignore the best know solutions.
In a nutshell, you can control how much importance to give for the particle current speed (inertia), the particle memory of the best know solution, and the particle memory of the swarm best know solution.
I hope it can help you!
Br's
Inertia was not the part of the original PSO algorithm introduced by Kennedy and Eberhart in 1995. It's been three years until Shi and Eberhart published this extension and showed (to some extent) that it works better.
One can set that value to a constant (supposedly [0.8 to 1.2] is best).
However, the point of the parameter is to balance exploitation and exploration of space, and
authors got best results when they defined the parameter with a linear function, that decreases over time from [1.4 to 0].
Their rationale was that first one should exploit solutions to find a good seed and later exploit area around the seed.
My feeling about it is that the closer you are to 0, the more chaotic turns particles make.
For a detailed answer refer to Shi, Eberhart 1998 - "A modified Particle Swarm Optimizer".
Inertia controls the influence of the previous velocity.
When high, cognitive and social components are less relevant. (particle keeps going its way, exploring new portions of the space)
When low, particle explores better the space where the best-so-far optimum has been found
Inertia can change over time: Start high, later decrease