I need to create a very large grid of patches to have GIS information of a very large network (such as a city-wide network). My question is how to get NetLogo to model such a world? When I set the max-pxcor and max-pycor to large numbers, it stop working. I need a world of for example size 50000 * 50000.
Thanks for your help.
See http://ccl.northwestern.edu/netlogo/docs/faq.html#howbig , which says in part: “The NetLogo engine has no fixed limits on size...”
It's highly unlikely that you'll be able to fit a 50,000 x 50,000 world, in your computer though — that's 2.5 billion patches. Memory usage in NetLogo is proportional to the number of agents, and patches are agents too.
You might take Stephin Guerin's advice at http://netlogo-users.18673.x6.nabble.com/Re-Rumors-of-Relogo-tp4869241p4869247.html on how to avoid needing an enormous patch grid when modeling transportation networks.
Related
I am working on driving industrial robots with neural nets and so far it is working well. I am using the PPO algorithm from the OpenAI baseline and so far I can drive easily from point to point by using the following rewarding strategy:
I calculate the normalized distance between the target and the position. Then I calculate the distance reward with.
rd = 1-(d/dmax)^a
For each time step, I give the agent a penalty calculated by.
yt = 1-(t/tmax)*b
a and b are hyperparameters to tune.
As I said this works really well if I want to drive from point to point. But what if I want to drive around something? For my work, I need to avoid collisions and therefore the agent needs to drive around objects. If the object is not straight in the way of the nearest path it is working ok. Then the robot can adapt and drives around it. But it gets more and more difficult to impossible to drive around objects which are straight in the way.
See this image :
I already read a paper which combines PPO with NES to create some Gaussian noise for the parameters of the neural network but I can't implement it by myself.
Does anyone have some experience with adding more exploration to the PPO algorithm? Or does anyone have some general ideas on how I can improve my rewarding strategy?
What you describe is actually one of the most important research areas of Deep RL: the exploration problem.
The PPO algorithm (like many other "standard" RL algos) tries to maximise a return, which is a (usually discounted) sum of rewards provided by your environment:
In your case, you have a deceptive gradient problem, the gradient of your return points directly at your objective point (because your reward is the distance to your objective), which discourage your agent to explore other areas.
Here is an illustration of the deceptive gradient problem from this paper, the reward is computed like yours and as you can see, the gradient of your return function points directly to your objective (the little square in this example). If your agent starts in the bottom right part of the maze, you are very likely to be stuck in a local optimum.
There are many ways to deal with the exploration problem in RL, in PPO for example you can add some noise to your actions, some other approachs like SAC try to maximize both the reward and the entropy of your policy over the action space, but in the end you have no guarantee that adding exploration noise in your action space will result in efficient of your state space (which is actually what you want to explore, the (x,y) positions of your env).
I recommend you to read the Quality Diversity (QD) literature, which is a very promising field aiming to solve the exploration problem in RL.
Here is are two great resources:
A website gathering all informations about QD
A talk from ICLM 2019
Finally I want to add that the problem is not your reward function, you should not try to engineer a complex reward function such that your agent is able to behave like you want. The goal is to have an agent that is able to solve your environment despite pitfalls like the deceptive gradient problem.
After reading a few papers on Neuro Evolution, more specifically NEAT, I realised that there was very little information regarding how you should weight each synapse at the start of the Neural Network. I understand that at the start, using NEAT, all the input neurons are connected to the output neuron, and then evolution takes place from there. However, should you weight each synapse randomly at the start, or simply set each one to 1?
It doesn't really matter a lot - it matters most how you mutate the weights of the connections in a genome.
However, setting the weights of each genome's connections to a random value is best: it acts like a small random search in the 'right' direction. If you'd set all the weights the same in for each genome, then weights in genomes will be extremely similar: keep in mind that a genome has a lot of connections, and with a mutation rate of 0.3 and two mutation options for example, only 15% of the population will have at least óne different weight after just 1 generation.
So make it something random, like random() * .2 - .1 (distribute between [-0.1, 0.1]). Just figure out what values work best for you.
Suppose in a model the NetLogo world is 160 X 101. So, In such a world how many stationary agents and moving agents can possibly be created. Can this world will able to support 100000 moving agents (no die) or possible support by NetLogo to moving agents in a single model.
No formal restrictions, just performance and resources. See How to model a very large world in NetLogo? I have personally modelled with 50,000 agents and it's still reasonable speed. However, get your model working with a MUCH smaller size before expanding as it will slow down.
I am messing about with GIS and what not. So I want to have as many patches as I can alas when I go too large 3600 x 1800 I end up with a crash and a verbose error starting with A InvocationTargetException.
Is it system based or in the network code?
How big can I go?
This is answered in the NetLogo FAQ at http://ccl.northwestern.edu/netlogo/docs/faq.html#howbig :
The NetLogo engine has no fixed limits on size [...]
In practice, the main limit is memory. Are you sure you need that high of patch resolution?
In cases where your model has turtles sampling raster data, you can use high resolution GIS raster datasets while keeping patch resolution low and then have turtles sample the dataset on the fly instead of importing into patch variables. Look closely at how the GIS Gradient Example is written in ModelLibrary/ CodeExamples/ GIS.
In my engineering class we are programming a "non-trivial" predator/prey pursuit problem.
Here's the gist of the situation:
There is a prey that is trying to escape a predator. Each can be modeled as a particle that can be animated in MATLAB (we have to use this coding language).
The prey:
can maneuver (turn) easier than the predator can
The predator:
can move faster than the prey
I have to create code for both the predator and the prey, which will be used in a class competition.
This is basically what the final product will look like:
http://www.brown.edu/Departments/Engineering/Courses/En4/Projects/pred_prey.gif
The goal is to catch the other team's prey in the shortest amount of time, and for my prey to become un-catchable for the other team's predator (or at least escape for a long period of time).
Here are the specific design constraints:
3. Design Constraints:
Predator and prey can only move in the x-y plane
Simulations will run for a time period of 250 seconds.
Both predator and prey will be subjected to three forces: (a) The propulsive force; (b) a viscous drag
force; and (c) a random time-varying force. (all equations given)
1. The propulsive forces will be determined by functions provided by the two competing groups
The predator is assumed to catch the prey if the distance between predator and prey drops below 1m.
You may not use the rand() function in computing your predator/prey forces – the only random forces
should be those generated by the script provided. (EOM with random forces are impossible for the
ODE solver to integrate, and it ends up in an infinite loop).
For the competition, we will provide the MATLAB code that will compute and animate the trajectories of
the competitors, and will determine the winner of each contest. The test code will be working in SI units.
I am looking for any resources that may be able to help me with some strategy. I have looked at basic pursuit curves, but I would love to look at some examples where the prey is not moving in a straight line. Any other coding advice or strategies would be greatly appreciated!
It's a good idea to start with the fundamentals in any field, and you can't go past the work of Issacs (Differential Games: A mathematical theory with applications to warfare and pursuit, control and optimization). This will almost certainly end up being a reference in any academic research project you may end up writing up.
Steven Lavalle's excellent book Motion Planning has a number of aspects that may be of interest including a section on visibility based pursuit evasion.
As for many mathematical topics, Wolfram Mathworld has some good diagrams and links that might get you thinking in the right direction (eg Pursuit Curves).
If you want to have a look at a curious problem in the area that is well understood try the Homicidal chauffeur problem - this will at least give you some grounds for comparing complexity / efficiency of different techniques. In particular, this is probably a good way to get a feel for level set methods (the paper Homicidal Chau eur Game. Computation of Level Sets of the Value Function by Patsko and Turova appears to have a number of images that might be helpful)