I'm working on a Vehicle Routing Problem. In my cost function I need to find the current search depth in order to calculate a deferred cost which is dependent on the current length of the intermediate solution. Is this information available via some method? This is my distance cost function:
def distance_callback(from_index, to_index):
"""Returns the shortest path distance between the two nodes"""
from_node = self.routing_manager.IndexToNode(from_index)
to_node = self.routing_manager.IndexToNode(to_index)
return self.distance_matrix[from_node][to_node]
See discussion on:
https://groups.google.com/forum/#!topic/or-tools-discuss/lw_zdalvm6k
The current approach is not possible as the distance callback is called in many places, and is usually cached, especially if written in python.
The original request is to have time dependent demands. It can be modeled with duplicate nodes, in disjunctions, with non overlapping time windows, and different demands.
Related
I refrained from asking for help until now, but as my thesis' deadline creeps ever closer and I do not know anybody with experience in RL, I'm trying my luck here.
TLDR;
I have not found an academic/online resource which helps me understand the correct representation of the environment as an observation space. I would be very thankful for any links or for giving me a starting point of how to model the specifics of my environment in an observation space.
Short thematic introduction
The goal of my research is to determine the viability of RL for strategy development in motorsports. This is currently achieved by simulating (lots of!) races and calculating the resulting race time (thus end-position) of different strategic decisions (which are the timing of pit stops + amount of laps to refuel for). This demands a manual input of expected inlaps (the lap a pit stop occurs) for all participants, which implicitly limits the possible strategies by human imagination as well as the amount of possible simulations.
Use of RL
A trained RL agent could decide on its own when to perform a pit stop and how much fuel should be added, in order to minizime the race time and react to probabilistic events in the simulation.
The action space is discrete(4) and represents the options to continue, pit and refuel for 2,4,6 laps respectively.
Problem
The observation space is of POMDP nature and needs to model the agent's current race position (which I hope is enough?). How would I implement the observation space accordingly?
The training is performed using OpenAI's Gym framework, but a general explanation/link to article/publication would also be appreciated very much!
Your observation could be just an integer which represents round or position the agent is in. This is obviously not a sufficient representation so you need to add more information.
A better observation could be the agents race position x1, the round the agent is in x2 and the current fuel in the tank x3. All three of these can be represented by a real number. Then you can create your observation by concating these to a vector obs = [x1, x2, x3].
Modelica modeling is the first principle modeling, so how to test the model and set an effective benchmark is important, for example, I could design a fluid network as my wish, but when building a dynamic simulation model, I need to know the detailed geometry structure and parameters to set up every piece of my model. Usually, I would build a steady-state model with simple energy and mass conservation laws, then design every piece of equipment based on the corresponding design manual, but when I put every dynamic component together, when simulation till steady-state, the result is different from the steady-state model more or less. So I was wondering if I should modify my workflow to make the dynamic model agree with the steady-state model. Any suggestions are welcome.
#dymola #modelica
To my understanding of the question, your parameter values are fixed and physically known. I would attempt the following approach as a heuristic to identify the (few) component(s) that one needs to carefully investigate in order to understand how they influence or violates the assumed first principles.
This is just as a first trial and it could be subject to further improvement and fine-tuning.
Consider the set of significant set of variables xd(p,t) \in R^n and parameters p. Note that p also includes significant start values. p in R^m includes only the set of additional parameters not available in the steady state model.
Denote the corresponding variables of the steady state model by x_s
Denote a time point where the dynamic model is "numerically" in "semi-" steady-state by t*
Consider the function C(xd(p,t*),xs) = ||D||^2 with D = xd(p,t*) - xs
It could be beneficial to describe C as a vector rather than a single valued function.
Compute the partial derivatives of C w.t. p expressed in terms of dxd/dp, i.e.
dC/dp = d[D^T D]/dp
= d[(x_d-x_s)^T (x_d - x_s)]/dp
= (dx_d/dp)^T D + ...
Consider scaling the above function, i.e. dC/dp * p/C (avoid expected numerical issues via some epsilon-tricks)
Here you get a ranking of most significant parameters which are causing the apparent differences. The hopefully few number of components including these parameters could be the ones causing such violation.
If this still does not help, may be due to expected high correlation among parameters, I would go further and consider a dummy parameter identification problem, out of which a more rigorous ranking of significant model parameters can be obtained.
If the Modelica language had capabilities for expressing dynamic parameter sensitivities, all the above computation can be easily carried out as a single Modelica model (with a slightly modified formulation).
For instance, if we had something like der(x,p) corresponding to dx/dp, one could simply state
dcdp = der(C,p)
An alternative approach is proposed via the DerXP library
I have agents in a grid and for any arbitrary agent I want to somehow define a radius r and find all the other agents in that area given r. Is this possible, is there any implemented method for that in AnyLogic?
If by 'in a grid' you mean they are in continuous space but with integer (x,y) coordinates, you can use the standard function agentsInRange (which internally is using the standard function distanceTo).
If the agents are in an actual grid (i.e., a discrete space with (row, column) coordinates), you can't use the standard continuous/GIS space functions like distanceTo or agentsInRange. (You get runtime exceptions.)
In that case, you would have to write your own function to calculate distances (probably via Pythagoras' theorem given the row and column values) to each other agent, and return only those within the given threshold.
Below is such a sample function which assumes your agents are of type Thing and live in a population called things (and uses the straight-line distance in number-of-cells between agents as the 'distance between them'). You supply the function with the 'source' agent (that you want to get other agents in range of) and the desired range. (There are lots of slightly different ways of coding this depending on, for example, whether you want it to work for any population of any agent type.)
There is. Check the agentsInRange(double distance) function, see help article here.
So basically, I have a warehouse represented by a graph and each node in it contains a certain amount of 3 objects (A, B, C). I have to use Dijkstra to find the shortest path the robot should take in order to take an amount of each item provided as the input and minimize the time.
Also, each time the robot picks an object, the robot's speed goes slower so the time it takes for him to travel a vertice isn't equal to its distance anymore. The given equation is Time = Distance * k where k is a constant associated with the robot (k= 1 + mass carried) and type A objects have a mass of 1kg, B objects of 3kg and C objects of 5kg.
My question is how can I modify or use the Dijkstra's algorithm given that I have to take into account the objects that I have to pick and the decrease of speed.
Thanks in advance!
When calculating the cost to go to a node, have another variable to account for time such that more items result in a larger number meaning more costly. So the total cost to go to a node is the sum of the cost between nodes and the variable accounting for time. The rest of Dijkstra should still work.
Problem: It appears to me that a fundamental property of a clustering method c() is whether we can combine the results c(A) and c(B) by some function f() of two clusterings in a way that we do not have to apply the full clustering c(A+B) again but instead do f(c(A),c(B)) and still end up with the same result:
c(A+B) == f(c(A),c(B))
I suppose that a necessary condition for some c() to have this property is that it is determistic, that is the order of its internal processing is irrelevant for the result. However, this might not be sufficient.
It would be really nice to have some reference where to look up which cluster methods support this and what a good f() looks like in the respective case.
Example: At the moment I am thinking about DBSCAN which should be deterministic if I allow border points to belong to multiple clusters at the same time (without connecting them):
One point is reachable from another point if it is in its eps-neighborhood
A core point is a point with at least minPts reachable
An edge goes from every core point to all points reachable from it
Every point with incoming edge from a core point is in the same cluster as the latter
If you miss the noise points then assume that each core node reaches itself (reflexivity) and afterwards we define noise points to be clusters of size one. Border points are non-core points. Afterwards if we want a partitioning, we can assign randomly the border points that are in multiple clusters to one of them. I do not consider this relevant for the method itself.
Supposedly the only clustering where this is efficiently possible is single linkage hierarchical clustering, because edges removed from A x A and B x B are not necessary for finding the MST of the joined set.
For DBSCAN precisely, you have the problem that the core point property can change when you add data. So c(A+B) likely has core points that were not core in either A not B. This can cause clusters to merge. f() supposedly needs to re-check all data points, i.e., rerun DBSCAN. While you can exploit that core points of the subset must be core of the entire set, you'll still need to find neighbors and missing core points.