I was wondering how a mechanical stop can be modeled most efficiently.
I do a hydraulic simulation with a controlled hydraulic cylinder in OpenModelica. For the hydraulic cylinder I use the sweptVolume model from the Modelica Standard Library.
What bugs me about this model is that there is no mechanical stop if the piston reaches the bottom of the cylinder.
I tried several ideas with no good result. I tried to reset the displacement of the piston to zero, if it hits the bottom, via an if-expression. But this is not really a good option due to the fact that the volume is calculated using the piston's displacement.
I then tried to introduce a force that equals the force applied to the piston, if the piston hits the stop. This option didn't work either, because in this case the pressure inside the cylinder can not be calculated.
The third try was to use the MSL model of MassWithStopAndFriction linked to the translational flange of the sweptVolume model, but this model seems to be broken for me.
Now I count on you as a competent community to bring in some more ideas for me to test.
Depending on your application, you may deploy the Hydraulics library? The library aims to model (compressible) fluid power systems and contains cylinders with end-stops. Its scope is different than the Fluid package you are using.
Using when and/or if statements for this task, I'd strongly discourage from experience. You may get one cylinder to work, but using that in a larger system will definitely get you into numerical problems. Have a look at the Mechanics package and analyse if the ElastoGap can be of any use to you.
Related
I am working on driving industrial robots with neural nets and so far it is working well. I am using the PPO algorithm from the OpenAI baseline and so far I can drive easily from point to point by using the following rewarding strategy:
I calculate the normalized distance between the target and the position. Then I calculate the distance reward with.
rd = 1-(d/dmax)^a
For each time step, I give the agent a penalty calculated by.
yt = 1-(t/tmax)*b
a and b are hyperparameters to tune.
As I said this works really well if I want to drive from point to point. But what if I want to drive around something? For my work, I need to avoid collisions and therefore the agent needs to drive around objects. If the object is not straight in the way of the nearest path it is working ok. Then the robot can adapt and drives around it. But it gets more and more difficult to impossible to drive around objects which are straight in the way.
See this image :
I already read a paper which combines PPO with NES to create some Gaussian noise for the parameters of the neural network but I can't implement it by myself.
Does anyone have some experience with adding more exploration to the PPO algorithm? Or does anyone have some general ideas on how I can improve my rewarding strategy?
What you describe is actually one of the most important research areas of Deep RL: the exploration problem.
The PPO algorithm (like many other "standard" RL algos) tries to maximise a return, which is a (usually discounted) sum of rewards provided by your environment:
In your case, you have a deceptive gradient problem, the gradient of your return points directly at your objective point (because your reward is the distance to your objective), which discourage your agent to explore other areas.
Here is an illustration of the deceptive gradient problem from this paper, the reward is computed like yours and as you can see, the gradient of your return function points directly to your objective (the little square in this example). If your agent starts in the bottom right part of the maze, you are very likely to be stuck in a local optimum.
There are many ways to deal with the exploration problem in RL, in PPO for example you can add some noise to your actions, some other approachs like SAC try to maximize both the reward and the entropy of your policy over the action space, but in the end you have no guarantee that adding exploration noise in your action space will result in efficient of your state space (which is actually what you want to explore, the (x,y) positions of your env).
I recommend you to read the Quality Diversity (QD) literature, which is a very promising field aiming to solve the exploration problem in RL.
Here is are two great resources:
A website gathering all informations about QD
A talk from ICLM 2019
Finally I want to add that the problem is not your reward function, you should not try to engineer a complex reward function such that your agent is able to behave like you want. The goal is to have an agent that is able to solve your environment despite pitfalls like the deceptive gradient problem.
we want to publish an Open-Source for integrating Reinforcement Learning to Smartgrid optimization.
We use OpenModelica as GUI, PyFMI for the import to Python and Gym.
Nearly everything is running, but a possibility to connect or disconnect additional loads during the simulation is missing. Everything we can do for now is a variation of the parameters of existing loads, which gives some flexibility, but way less then the possibility to switch loads on and off.
Using the implemented switches in OpenModelica is not really an option. They just place a resistor at this spot, giving it either a very low or very high resistance. First, its not really decoupled, and second, high resistances make the ODE-system stiff, which makes it really hard (and costly) to solve it. In tests our LSODA solver (in stiff cases basically a BDF) ran often in numerical errors, regardless of how the jacobian was calculated (analytically by directional derivatives or with finite differences).
Has anyone an idea how we can implement a real "switching effect"?
Best regards,
Henrik
Ideal connection and disconnection of components during simulation
requires structure variability, which is not fully supported
by Modelica (yet). See also this answer https://stackoverflow.com/a/30487641/8725275
One solution for this problem is to translate all possible
model structures in advance and switch the simulation model if certain conditions are met. As there is some overhead involved, this approach only makes sense, when the model is not switched very often.
There is a python framework, which was built to support this process: DySMo. The tool was written by Alexandra Mehlhase, who made a lot of interesting publications regarding structure variability, e.g. An example of beneficial use of
variable-structure modeling to enhance an existing rocket model.
The paper Simulating a Variable-structure Model of an Electric Vehicle for Battery Life Estimation Using Modelica/Dymola and Python of Moritz Stueber is also worth a look. It contains a nice introduction about variable structure systems and available solutions.
For my project, I am planning to conduct simulations in Simulink for an investigation on viscoelasticity behaviour that 3D printed plastic coupons seem to exhibit. Viscoelastic behaviour can be represented by different spring-dashpot combinations such as the ones seen in the Maxwell or Kevin-Voigt models.
For this purpose, I will be using Simscape elements to represent the layers of the 3D-printed object using spring, mass and dampers in the system. For my first attempt, I have used just a simple two mass system with spring-dashpot in series and observe their displacement using the scope element. For the the mechanical source, I have tried using both an ideal translational velocity source as well as an ideal force source.
Here is my Simulink diagram:
Initially, I tried using an ideal translational velocity source, I encountered two warnings:
Thus, I attempted to use an ideal force source instead as most of the examples for modelling in spring-mass-damper using Simscape that I found used that as a mechanical source instead, which managed to work. (I only changed the source from an ideal translational velocity to an ideal force source)
However for the purpose of my experiment, it would serve me better to use the ideal translational velocity source instead as the strain testing was conducted at a constant speed of 10mm/s that is to say a constant velocity rather than a constant force which would imply that the speed is changing.
I am not sure whether the warnings obtained when using the ideal translational velocity element would affect my results and how I can deal with the warnings, would really appreciate any help I can get as I am super new to Simulink! :)
I am working on modeling and controlling of a hydraulic system. Modeling of the system is modeled in Matlab simscape in simulink environment which is looks like this
and for basic controlling to control the piston position (Piston Pos in figure) I have established simple feedback to check the position.
While I run the simulation when this comes to control the position Simulation takes too much time. For example if I gave desired piston position 300 mm than while output comes to around 290-294 mm simulation time reaches at around 5.18sec than it is stuck on that for longer time.
I want to know that, is there any way to speed up the simulation ?
I am using Matlab simulink solver ode23t due to simscape modeling.
Speeding up simulations in general is vast subject. It seems the issue here is an event which triggers multiple small time-step in the variable step solver.
This can be perfectly normal, for example a clutch engaging, or a valve opening.
To check whether or nor this is the case you can execute (make sure time-logging is enabled):
semilogy(tout(2:end), diff(tout))
Sharp downward spikes indicate small time-steps were taken. For a more in-depth analysis you can use the Solver Profiler:
https://www.mathworks.com/help/simulink/ug/examine-solver-behavior-using-solver-profiler.html
This will give you detailed information as to which components are causing solver resets.
Such behavior can be difficult to debug if you're not used to the tool. I'd highly recommend getting in touch with MathWorks tech support if the behavior persists. They'll be able to look at your model and diagnose the issue.
I'm simulating a shaft system in Simulink, where I have to find the displacement of a mass. I'm not sure how to model this in Simulink because of the shaft and pulley. I'm looking through the documentation and the closest thing I see to a shaft is the wheel and axle block. But the shafts are connected by a flexible shaft, which is similar to a spring. Any ideas?
This is a fairly trivial task when using SimScape, which is especially made to simulate physical systems. You'll find most of the blocks you need ready from the library.
I've used SimScape to create a model of a complete hybrid truck... In Simulink it can be done, but you'll need to build your own differential equations for the task. In your case, the flexible axle could be translated to another block with a spring/damper system inside.
If you haven't got access to SimScape, you may also consider to use .m (matlab) files to write your differential equations. This can then be used as a block in Simulink, varying (only) a few parameters over time.
Take this step by step:
1. Draw a free body diagram, write out equations for all the forces as a function of displacement, velocity and acceleration of every element (including rotation obviously). For instance, you know that force on the box m will be *c*dy/dt* plus whatever the pulley experiences.
2. Sort out the rotation of the rod first. You know that *T=I*d(omega)/dt* if you get rid of the rest of the system. So, do something analogous to the car engine example of MatLab: Divide the input T by I to get the acceleration, integrate it to get velocity and one more time to get rotational displacement.
3. Keep adding bits one by one. First, you know that there will be a moment proportional to k*(theta_1-theta_2) acting. This will oppose the motion of rod 1 and act to create motion of rod 2. Add a new "branch" to your model to get theta_2 same way you got theta_1.
4. Go on including further elements...