I'm building a cost effectiveness model (infectious disease related) in NetLogo consisting of an intervention and "status quo" cohort that should behave identically except for the influence of the intervention. I can reproduce base case results (default parameter values) using the random-seed statement. However, when I change parameters influence intervention efficacy (e.g., compliance) it changes the random number generation process and then causes the outcomes in the status quo cohort to change. How can I get around this? How do I isolate one cohort from changes taking place in another cohort with respect to the random number generation process? the with-local-randomness statement doesn't really help me ..
Unfortunately, you face two serious problems: NetLogo really does not provide access to random number generators other than the global one, and much of the access to that PRNG is implicit. So for example, whenever you use ask, you are implicitly drawing from the global PRNG.
You may be able to use with-local-randomness. E.g., use it every time you ask turtles outside the status-quo cohort to do something. Or, to use with-local-randomness for the special cohort, you could start the simulation by generating a seed sequence that you then consume as the simulation runs, resetting the random state each time you ask that cohort to do something (or generate random numbers, etc.).
Do the cohorts need to be run simultaneously (ie do they interact)? If not, could you simply run two completely separate experiments - one for the control and one for the intervention scenario.
Related
We’re generating the data that we might get from a shop floor to run, test, and validate our machine learning models. We first have here a discrete event simulation model for our manufacturing system. Each production order is seen as an agent, which then goes through different processes with a queue (waiting time) and delays (firstly production time, secondly logistics time).
enter image description here
But sometimes we have one process, for example, printing (code 5A, after the second Select5Output), with three different machines, which do not have a particular capacity. It’s time when we divide our order into parts and send them to those machines (very randomly, subjectively).
The data we take is from flowchart_process_states_log in Database.
The data we take is from flowchart_process_states_log in Database.
My questions here are:
How can we define the number of products in each order? Ex. we’re printing card, for one order it may be 10k, for another 8k or 33k. Can we define it as agent’s parameter? Then how can we vary them (stochastically, no exact number needed).
How can we split those 10k cards into three different machines? And then how to get back an complete agent with 10k? The Agent ID should remain the same as we trace and analyse them in ML model. Is it reasonable to see an order as an agent?
How can we multiply the number of our agent after a process? Ex. After cutting 10k pieces we have 20k.
We have the distribution for delay ex. triangle distribution. But we want some disturbances, when it suddenly takes 2 days for that delay instead of 3-4 hours as normal. How to do it?
Thank you in advance for your effort. Every help is highly appreciated, because we're here and learning together. Thank you !!
I have an AnyLogic simulation model using trucks and forklifts as agents (Transporter type), and among other things would like to identify each time one of them becomes within a certain distance of another one (example within 5 meters). I will record this count as a variable within the main space of the model. I am using path-guided navigation.
I have seen a method "agentsInRange" which will probably be able to do the trick, but am not sure where to call this from. I assume I should be able to use the AL functionality of "Min distance to obstacle" (TransporterFleet) and "Collision detection timeout" (TransporterControl)?
Thanks in advance!
Since there don't seem to be pre-built functions for this, afaik, the easiest way is to:
add an int variable to your transporter agent type counter
add an event to your transporter type checkCollision that triggers every second or so
in the event, loop across the entire population of transporters and count the number that are closer than X meters (use distanceTo(otherTransporter) and write your own custom code)
add that number to counter
Note that this might be very inefficient computationally as it is quite brute-force. But might be good enough :)
I have an optimization program where I have a main program and three subprograms (functions) in MATLAB. I declared rng('shuffle','twister') in my main program but I thought that I needed to declare the same rng('shuffle','twister') under my functions since they also use random sampling. My question is if it is necessary to declare rng('shuffle','twister') in my functions since it greatly degrades the computation time. I seem to be getting the same answers anyway. Is there a way around this?
You do not need to repeatedly run rng(...) in your functions, just once when you start MATLAB if you want to get different numbers each time. The random number functions in MATLAB (i.e. rand, randn, randi, etc.) share a global/system-wide generator, so there is no need to reseed it except when you restart MATLAB.
Since all of these functions access the same underlying stream, a call to one affects the values produced by the others at subsequent calls.
Hence, numbers generated in the different functions and in repeated calls to the functions will be different whether or not you reseed the generator.
More about the 'shuffle' option from this page, which indicates that not only is it not useful to re-seed frequently, but it may actually be undesirable from a statistical standpoint:
'shuffle' is a very easy way to reseed the random number generator. You might think that it's a good idea, or even necessary, to use it to get "true" randomness in MATLAB. For most purposes, though, it is not necessary to use 'shuffle' at all. Choosing a seed based on the current time does not improve the statistical properties of the values you'll get from rand, randi, and randn, and does not make them "more random" in any real sense. While it is perfectly fine to reseed the generator each time you start up MATLAB, or before you run some kind of large calculation involving random numbers, it is actually not a good idea to reseed the generator too frequently within a session, because this can affect the statistical properties of your random numbers.
I am thinking to implement a learning strategy for different types of agents in my model. To be honest, I still do not know what kind of questions should I ask first or where to start.
I have two types of agents which I want them to learn by experience, they have a pool of actions which each has different reward based on specific situations that might happen.
I am new to reinforcement Learning methods, therefore any suggestions on what kind of questions should I ask myself is welcomed :)
Here is how I am going forward to formulate my problem:
Agents have a lifetime and they keep track of a few things that matter for them and these indicators are different for different agents, for example, one agent wants to increase A another wants B more than A.
States are points in an agent's lifetime which they
Have more than one option (I do not have a clear definition for
States as they might happen a few times or not happen at all because
Agents move around and they might never face a situation)
The reward is the an increase or decrease in an indicator that agents can get from an action in a specific State, and agent do not know what would be the gain if he chose another action.
The gain is not constant, the states are not well defined and there is no formal transition of one state into another,
For example agent can decide to share with one of the co-located agent (Action 1) or with all of the agents at the same location(Action 2) If certain conditions hold true Action A will be more rewarding for that agent, while in other conditions Action 2 will have higher reward; my problem is I did not see any example with unknown rewards since sharing in this scenario also depends on the other agent's characteristics (which affects the conditions of reward system) and in different states it will be different.
In my model there is no relationship between the action and the following state,and that makes me wonder if its ok to think about RL in this situation at all.
What I am looking to optimize here is the ability for my agents to reason about current situation in a better way and not only respond to their need which is triggered by their internal states. They have a few personalities which can define their long term goal and can affect their decision making in different situations, but I want them to remember what action in a situation helped them to increase their preferred long term goal.
In my model there is no relationship between the action and the following state,and that makes me wonder if its ok to think about RL in this situation at all.
This seems strange. What do actions do if not change state? Note that agents don't have to necessarily know how their actions will change their state. Similarly, actions could change the state imperfectly (a robots treads could skid out so it doesn't actually move when it tries to). In fact, some algorithms are specifically designed for this uncertainty.
In any case, even if the agents are moving around the state space without having any control, it can still learn the rewards for the different states. Indeed, many RL algorithms involve moving around the state space semi-randomly to figure out what the rewards are.
I do not have a clear definition for States as they might happen a few times or not happen at all because Agents move around and they might never face a situation
You might consider expanding what goes into what you consider to be a "state". For instance, the position seems like it should definitely go into the variables identifying a state. Not all states need to have rewards (although good RL algorithms typically infer a measure of goodness of neutral states).
I would recommend clearly defining the variables that determine an agent's state. For instance, the state space could be current-patch X internal-variable-value X other-agents-present. In the simplest case, the agent can observe all of the variables that make up their state. However, there are algorithms that don't require this. An agent should always be in a state, even if the state has no reward value.
Now, concerning unknown reward. That's actually totally okay. Reward can be a random variable. In that case, a simple way to apply standard RL algorithms would be to use the expected value of the variable when making decisions. If the distribution is unknown, then the algorithm could just use the mean of the rewards observed so far.
Alternatively, you could include the variables that determine the reward in the definition of the state. That way, if the reward changes, then it is literally in a different state. For example, suppose a robot is on top of a building. It needs to get to the top of the building in front of it. If it just moves forward, it falls to ground. Thus, that state has a very low reward. However, if it first places a plank that goes from one building to the other, and then moves forward, the reward changes. To represent this, we could include plank-in-place as a variable so that putting the board in place actually changes the robot's current state and the state that would result from moving forward. Thus, the reward itself has not changed; it's just in a different state.
Hopefully this helps!
UPDATE 2/7/2018: A recent upvote reminded me of the existence of this question. In the years since it was asked, I've actually dived into RL in NetLogo to a much greater extent. In particular, I've made a python extension for NetLogo, primarily to make it easier to integrate machine learning algorithms in with model. One of the demos of the extension trains a collection of agents using deep Q-learning as the model runs.
I have a Perl script that uses rand to generate pseudorandom integers in some range. I want it to be random (i.e. not set the seed by myself to some constant), but also want to be able to reproduce the results of a specific run if needed.
What would you do?
McWafflestix says:
Possibly you want to have a default randomly determined seed, that will give you complete randomness when desired, but which can be set prior to a run manually to give reproducibility.
The obvious way to implement this is to follow your normal seeding process (either manually from a strong random source, or letting perl do it automatically on the first call to rand), then use the first generated random value as the seed, and record it. If you want to reproduce later, just use a recorded value for the seed.
# something like this?
if ( defined $input_rand_seed ) {
srand($input_rand_seed);
} else {
my $seed = rand(); # or something fancier
log_random_seed($seed);
srand($seed);
}
If the purpose is to be able to reproduce simulation paths which incorporate random shocks (say, when you are running an economic model to produce projections, I would give up on the idea of storing the seed, but rather store each sequence alongside the model data.
Note that the built in rand is subject to vagaries of the rand implementation provided by the C runtime. On all Windows machines and across all perl versions I have used, this usually means that rand will only ever produce 32768 unique values.
That is severely limited for any serious purpose. In simulations, a crucial criterion is that random sequences used be independent of each other so that each run can be considered an independent realization.
In fact, if you are going to run a simulation 1,000 times, I would pre-produce 1,000 corresponding random sequences using known-good generators that are consistent across platforms and store them with the model inputs.
You can update the simulations using the same sequences or a new set if parameter estimates change when you get new data.
Log the seed for each run and provide a method to call the script and set the seed?
Why don't you want to set the seed, but at the same time set the seed? As I've said to you before, you need to explain why you don't want to do something so we know what you are actually asking.
You might just set it yourself only in certain conditions:
srand( $ENV{SOME_SEED} ) if defined $ENV{SOME_SEED};
If you don't call srand, rand calls it for you automatically but it doesn't report the seed that it used (at least not until Perl 5.14).
It's really just a simple programming problem. Just turn what you outlined into the code that does what you said.
Your goals are at odds with each other. One one hand, you want a self-seeding, completely random sequence of integers; on the other hand, you want reproducibility. Completely random and reproducibility are at odds with each other.
You can set the seed to something you want. Possibly you want to have a default randomly determined seed, that will give you complete randomness when desired, but which can be set prior to a run manually to give reproducibility.