I would like to solve routing problems with passengers. At most 4 passengers can be present in a vehicle. At each stop either passengers enter the vehicle or leave the vehicle. As I understand this can not be modeled with a demands dimension, as the cumulative variable is only allowed to increase (non-negative transits). What's the approach here?
You can have negative transits.
You cannot have negative cumulative variables.
Related
I've a question, in my simulation model I wanted to create some randomness of quantities of agents arrival. In my model the chance is 25% that the quantity is 1, 50% that quantity is 2 and 25% that the quantity is 3. I want to generate these quantities by using the split function of anylogic and than in the split function I want to add a distribution, which gives the values based on this distribution.
But is there a distribution in Anylogic, which fits to this kind of estimations or should I really do it manually. I was already thinking about using a rounded value of a normal distribution, but I'm not sure if this is correct.
Thanks.
You can use a custom distribution to create any shape of distribution you want (it is under the Agent palette).
In your case, you would need to pick one of type discrete as shown below.
Then, in the split block you would specify the number of copies as
customDistribution()
I was wondering if anyone could please let me know the prototype of Integral function in Anylogic. Thanks a lot.
For example, how to put the lower and upper bound of integral?
There's no such thing as an integral, nevertheless, the concept of stock and flow on a system dynamics model is equivalent to an integral
The lower and upper bound are your simulation initial and end time... And if you want to define other lower and upper bounds, you have to make your flows equal to 0 until you reach the initial time and after you reach the final time.
What you are integrating is the sum of the flows that are going in minus the sum of the flows going out of the stock and the result of that integral, is the value of the stock.
Of course, you can only integrate based on time, and using an approximation method such as Euler.
I have a sequential data (one instance per time) to be clustered into two classes. I want to use the sequential version of K-means (sequential K-means) for this task.
Upon randomly specifying the centers of the two clusters for the algorithm at the beginning, I want for the distance between them to be as max as possible (i.e., very away from each other) so the distribution of the the resulting two clusters will not be affected by the initial centers.
Is my thinking correct? if so, how can I do that?
Rather try to best estimate the true means. That is the optimum steategy.
If you just want to make them far apart, that can lead to badly assigned points inbetween.
I'm new to reinforcement learning and q-learning and I'm trying to understand concepts and try to implement them. Most of material I have found use CNN layers to process image input. I think I would rather start with something simpler than than, so I use grid world.
This is what I have already implemented. I implemented an environment by following MDP and have 5x5 grid, with fixed agent position (A) and target position (T). Start state could look like this.
-----
---T-
-----
-----
A----
Currently I represent my state as a 1-dimensional vector of length 25 (5x5) where 1 is on position where Agent is, otherwise 0, so for example
the state above will be repsented as vector
[1, 0, 0, ..., 0]
I have successfully implemented solutions with Q table and simple NN with no hidden layer.
Now, I want to move little further and make the task more complicated by making Target position random each episode. Because now there is no correlation between my current representation of state and actions, my agents act randomly. In order to solve my problem, first I need to adjust my state representation to contain some information like distance to the target, direction or both. The problem is, that I don't how to represent my state now. I have come up with some ideas:
[x, y, distance_T]
[distance_T]
two 5x5 vectors, one for Agent's position, one for Target's position
[1, 0, 0, ..., 0], [0, 0, ..., 1, 0, ..., 0]
I know that even if I will figure out the state representation, my implemented model will not be able to solve the problem and I will need to move toward hidden layers, experience replay, frozen target network and so on, but I only want to verify the model failure.
In conclusion, I want to ask how to represent such state as an input for neural network. If there are any sources of informations, articles, papers etc. which I have missed, feel free to post them.
Thank you in advance.
In Reinforcement Learning there is no right state representation. But there are wrong state representations. At least, that is to say that Q-learning and other RL techniques make a certain assumption about the state representation.
It is assumed that the states are states of a Markov Decision Process (MDP). An MDP is one where everything you need to know to 'predict' (even in a probabilistic way) is available in the current state. That is to say that the agent must not need memory of past states to make a decision.
It is very rarely the case in real life that you have a Markov decision process. But many times you have something close, which has been empirically shown to be enough for RL algorithms.
As a "state designer" you want to create a state that makes your task as close as possible to an MDP. In your specific case, if you have the distance as your state there is very little information to predict the next state, that is the next distance. Some thing like the current distance, the previous distance and the previous action is a better state, as it gives you a sense of direction. You could also make your state be the distance and the direction to which the target is at.
Your last suggestion of two matrices is the one I like most. Because it describes the whole state of the task without giving away the actual goal of the task. It also maps well to convolutional networks.
The distance approach will probably converge faster, but I consider it a bit like cheating because you practically tell the agent what it needs to look for. In more complicated cases this will rarely be possible.
Your last suggestion is the most general way to represent states as an input for function approximators, especially for Neural Networks. By that representation, you can also add more dimensions that will stand for non-accessible blocks and even other agents. So, you generalize the representation and might apply it to other RL domains. You will also have the chance to try Convolutional NNs for bigger grids.
On each day, I have 10000 position pairs in the form of (x,y); Up to now, I have collected 30 days. I want to select a position pair from each day so that all the positions pairs have similar coordinates (x,y) value. By similar I mean the euclidean distance is minimized between any two pairs. How to do it in matlab with efficiency. Because with brute force, it is almost impossible.
In brute force case, we have 10000^30 possibilities, each operation say needs 10^-9 second,
It will run forever.
One idea would be to use the k-means algorithm or one of its variations. it is relatively easy to implement (it is also part of the Statistics Toolbox) and has a runtime about O(nkl).
Analysing all the possibilities will give you for sure the best result you are looking for.
If you want an approximate result you can consider the first two days and analyse all the possibilities for these 2 days and pick the best result. Then when analyse the next day keep the result obtained previously and find the point of the third column closest to the previous two.
In this way you will obtain an approximate solution but with a less computational complexity.