What is the fitness function used to solve an inverted pendulum ?
I am evolving neural networks with genetic algorithm. And I don't know how to evaluate each individual.
I tried minimize the angle of pendulum and maximize distance traveled at the end of evaluation time (10 s), but this won't work.
inputs for neural network are: cart velocity, cart position, pendulum angular velocity and pendulum angle at time (t). The output is the force applied at time (t+1)
thanks in advance.
I found this paper which lists their objective function as being:
Defined as:
where "Xmax = 1.0, thetaMax = pi/6, _X'max = 1.0, theta'Max =
3.0, N is the number of iteration steps, T = 0.02 * TS and Wk are selected positive weights." (Using specific values for angles, velocities, and positions from the paper, however, you will want to use your own values depending on the boundary conditions of your pendulum).
The paper also states "The first and second terms determine the accumulated sum of
normalised absolute deviations of X1 and X3 from zero and the third term when minimised, maximises the survival time."
That should be more than enough to get started with, but i HIGHLY recommend you read the whole paper. Its a great read and i found it quite educational.
You can make your own fitness function, but i think the idea of using a position, velocity, angle, and the rate of change of the angle the pendulum is a good idea for the fitness function. You can, however, choose to use those variables in very different ways than the way the author of the paper chose to model their function.
It wouldn't hurt to read up on harmonic oscillators either. They take the general form:
mx" + Bx' -kx = Acos(w*t)
(where B, or A may be 0 depending on whether or not the oscillator is damped/undamped or driven/undriven respectively).
Related
I'm trying to model basic gravitation for an object of negligible mass around a massive body. I've followed the examples provided in the ODE suite documentation, but the results I'm getting are plainly ridiculous.
Here's the function I'm calling with ode45:
function dy = rigid(t,y)
dy = zeros(4,1);
%Position
xx=y(1);
yy=y(2);
%Radius
r=(xx.^2+yy.^2).^0.5;
%Constants
M=10^30;
G=6.67*10^-11;
%dX/dt
dy(1)=y(3); %vx
dy(3)=-M.*G.*xx.*r.^-3; %ax
%dY/dt
dy(2)=y(4); %vy
dy(4)=-M.*G.*yy.*r.^-3; %ay
And here are the solver lines:
%Init
x_0=1;
y_0=1;
vx_0=0;
vy_0=5;
%ODEs
[T,Y] = ode45(#rigid,[0 1000],[x_0 y_0 vx_0 vy_0]);
%Vectors
x=Y(:,1);
y=Y(:,2);
%Plot
plot(x,y)
xlabel('x');
ylabel('y');
title('y=f(x)');
I get a linear plot at the end. Even with initial speed, the position doesn't change over a period of several steps. The only thing I can think of is that I've misunderstood the way to set up my system of ODEs.
I've been pondering this for a while now, and I'm really short on ideas having had done a few searches on the web.
Summary: There are problems in integrating Hamiltonian systems with normal numerical integrators, and your special initial conditions aggravate this to the point where the numerical solution has no resemblance with the correct one.
There's nothing wrong with your implementation per se. However, the initial conditions you use are not the best. The constants G and M you use are in SI units, which means the coordinates are in m, the speeds are in m/s, and time is in s. These lines
x_0=1;
y_0=1;
vx_0=0;
vy_0=5;
[T,Y] = ode45(#rigid,[0 1000],[x_0 y_0 vx_0 vy_0]);
therefore mean that you are asking for an orbit with a radius of about 1.4 meters and an orbital speed of 5 m/s, and you want this orbit over a period of 17 minutes. Imagine there actually was an object just meters away from a mass of 10^30 kilograms!
So let's try to ask for something more realistic, similar to Earths' orbit, and look at it over 1 year:
x_0=149.513e9;
y_0=0;
vx_0=0;
vy_0=29.78e3;
[T,Y] = ode45(#rigid,[0 31.536e6],[x_0 y_0 vx_0 vy_0]);
And the result is
which looks as expected.
But there is a second problem here. Let's look at this orbit over a period of 10 years ([0 315.36e6]):
Now we no longer get a closed orbit, but a spiral! That's because the numerical integration proceeds with limited precision, and for this set of equations this leads (physically speaking) to a loss of energy. The precision can be increased using parameters to ode45, but ultimately the problem will persist.
Now let's go back to your original parameters and have a look at the result:
This "orbit" is a straight line towards the origin (the sun). Which could be ok, since a straight oscillation is a possible special case of an elliptic orbit. But looking at the coordinates over time:
We see that there is no oscillation. The "planet" falls into the sun and stays there. What's happening here is the same effect as with the larger orbit: Imprecise integration leads to a loss of energy. Moreover, the numerical integration gets stuck: We asked for a period of 1000 s, but the integration does not proceed beyond 1.6e-10 seconds.
As far as I can tell, neither ode45 nor any of the other standard Matlab integrators are adequate to deal with this problem. There are special numerical integration algorithms designed to do so, specifically for Hamiltonian systems, called symplectic integrators. There is a file exchange entry that provides different implementations. Also see this question and answers for more pointers.
I have written a simple code for a model in NetLogo. At the same time the model is well studied through ordinary differential equations in literature. Now I would like to compare some plots of model obtained by both NetLogo and Matlab (used to solve differential equations). I used "ticks" command to increment the time in NetLogo, where as Matlab uses time in seconds. What kind of precautions ( changes ) should I keep in mind in order to compare the plots obtained by NetLogo and Matlab.
In general, the ticks-axis of the plots from NetLogo should be some constant scalar of the time-axis of the MatLab plots. That scalar is often referred to in simulations as dt or "step size": the time per tick. If you were just using NetLogo to numerically solve differential equations (not recommended, though possible), you would likely explicitly set this to something (just as you do when numerically solving in MatLab). In most NetLogo models, however, the step size is implicit.
Some common parameters that correspond to step size in models:
speed of agents
rates of growth or decay
rates of diffusion
So, for example, if we're modeling traffic on a street with a speed limit of 100 kph (= (100000 m) / (60 min * 60 sec) = 27.8 m/s), and our patch-size is equal to 1 m and our agents travel at most 0.5 per tick, then we have:
27.8 m/s = (0.5 patches/tick) * (1 m/patches) / (step-size s/tick) = (0.5 m/tick) / (step-size s/tick)
step-size s/tick = (0.5 m/tick) / (27.8 m/s) = 0.018 s/tick
So, in this case, each tick is about 0.018 seconds.
Basically, you should try to find some "per tick" parameter in your model that corresponds to "per second" parameter in the differential equations. Then, you should be able to determine how many seconds per tick there are by comparing these parameters.
Alternatively, you could kind of cheat by just comparing plots, seeing how they line up, and then determine the step size like that. Then, you can work backwards to figure out what parameters in your models are determining the step size.
Please help. The condensation algorithm steps to track an object in a frame are:
Initialize a point and choose N random points around it and set the weight to be 1/N.
Propagate the points as per the dynamic model (constant velocity model).
Now, calculate the weights of each particle using an observation model. Till now, there are N particles.
Now, in the resampling step pick N particles from the above set of particles?
What? How can we pick N particles from N particles?
How can we pick N particles from N particles?
Pick N particles with replacement, i.e one particle can be chosen more than one time according to the weight you assign to it.
If you have 3 particles , with weight [0.1 0.2 0.7 ] , then choose the 3rd particle twice and 2nd particle one,hence you have selected get 3 particles from 3 particles itself.
There are many techniques to do this step i.e resampling N particles.Even I am trying to write a code for this part only.
Some sites where you can learn about resampling is :-
Udacity - Artificial Intelligence for Robotics - Link to the course page
IEEE paper- tutorial on Particle filter for online .... by Arulampalam ,gordon maskell, This is a highly cited paper,and almost every where people have taken references for particle filter from here only.
This paper is lecture tutorial in which they have explained resampling nicely, just follow the algorithm, and i think your code will do the resampling Link
I don't know how much detail is required...forgive me if most of this is known. The particle filter attempts to estimate the posterior distribution p(x_t|y_1,...,y_t) based on the observations y_1,...,y_t. The "correction step" relies on the simplification:
p(x_t|y_1,...,y_t) = p(y_t|x_t)p(x_t|y_1,...,y_t-1)/p(y_t|y_1,...,y_t-1)
N points are sampled from this posterior distribution, then evolved according to the right-hand side to approximate the next posterior. We are not dealing with normals so we need to approximate more than just 2 moments. The N points for the next step are then resampled from the new posterior, rather than just using the old points, wherever they have eveolved. The reason is the well-known degeneracy effect - that all points but 1 will tend to 0.
So we are not picking N particles from N particles, but rather throwing away the old particles, and resampling N from the new estimate of the posterior.
Please can you help me understand how to calculate the Mean-Squared Displacement for a single particle moving randomly within a given period of time. I have read a lot of articles on this (including Saxton,1991,Single-Particle Tracking: The Distribution of Diffusion Coefficients), but still confused (not getting the right answer).
Let me start by showing you how I do it and please correct me if I'm wrong:
The way I'm doing it is as follows:
1.Run the program from t=0 to t=100
2.Calculate the displacement, (s(t)-s(t+tau)), at each timestep (ie. at t=1,2,3,...100) and store it in a vector
3.Square the answer to number 2
4.find the mean to the answer of 3
In essence, this is what I'm doing in Matlab
%Initialise the lattice with a square consisting of 16 nonzero lattice sites then proceed %as follows to calculate the MSD:
for t=1:tend
% Allow the particle to move randomly in the lattice. Then do the following
[row,col]=find(lattice>0);
centroid=mean([row col]);
xvec=[xvec centroid(2)];
yvec=[yvec centroid(1)];
k=length(xvec)-1; % Time
dt=1;
diffx = xvec(1:k) - xvec((1+dt):(k+dt));
diffy = yvec(1:k) - yvec((1+dt):(k+dt));
xsquare = diffx.^2;
ysquare = diffy.^2;
MSD=mean(xsquare+ysquare);
end
I'm trying to find the MSD in order to compute the diffusion co-efficient. Note that I'm modelling a collection of lattice sites (16) to represent a single particle (more biologically realistic), instead of just one. I have been brief with the comment within the for loop as it is quite long, but I'm happy to send it to you.
So far, I'm getting very small MSD values (in the range of 0.001-1), whereas I'm supposed to get values in the range of (10-50). The particle moves very large distances so surely my range of 0.001-1 cannot be right!
This is an extract from the article which I'm trying to reproduce their figure:
" We began by running some simulations in 1D for a single
cell. We allowed the cell to move for a given number of
Monte Carlo time steps (MCS), worked out the mean square
distance traveled in that time, repeated this process 500
times, and evaluate the mean squared distance for this t.
We then repeated this process ten times to get the mean of
. The reason for this choice of repetitions was to
keep the time required to run the simulations within a reasonable
level yet ensuring that the standard deviation of the
mean was relatively small (<7%)".
You can access the article here "From discrete to a continuous model of biological cell movement, 2004, by Turner et al., Physical Review E".
Any hints are greatly appreciated.
How many dimensions does the particle move along ?
I don't have Matlab right now, but here is how I'd do that over one dimension :
% pos is the vector of positions
delta = pos(2:100) - pos(1:99);
meanSquared = mean(delta .* delta);
First of all, why have a particle cover multiple lattice sites? What counts for MSD, in the end, is the displacement of the centroid, which can be represented as a point. If your particle (or cell) is large, or only takes large steps, you can always just make a wider grid. Also, if you're trying to reproduce a figure from somewhere else, you should really use the same algorithm.
For your Monte Carlo simulation, what do you do? If all you really want is get a displacement, you can generate a bunch of random movement vectors in one go (using rand or randi), and use cumsum to calculate the positions. Also, have you plotted your random walks to make sure the data is sensible?
Then, your code looks a bit funny (see comments). Why don't you just use the code provided in this answer to calculate MSD from the positions?
for t=1:tend
% Allow the particle to move randomly in the lattice. Then do the following
[row,col]=find(lattice>0); %# what do you do this for?
centroid=mean([row col]);
xvec=[xvec centroid(2)];
yvec=[yvec centroid(1)]; %# till here, I have no idea what you want to do
k=length(xvec)-1; % Time %# you should subtract dt here
dt=1; %# dt should depend on t!
diffx = xvec(1:k) - xvec((1+dt):(k+dt));
diffy = yvec(1:k) - yvec((1+dt):(k+dt));
xsquare = diffx.^2;
ysquare = diffy.^2;
MSD=mean(xsquare+ysquare);
end
I am rewriting a Monte Carlo simulation model in MATLAB with an emphasis on readability. The model involves many particles, represented as (x,y,z), following a random walk over a small set of states with certain termination probabilities. The information relevant for output is the number of particles that terminate in a given state.
The simulation requires enough particles that running it for each particle individually is cost prohibitive. Vectorization seems to be the way to get performance out of MATLAB, but is there any idiomatic way of creating a vectorized version of this simulation in MATLAB?
I'm beating my head against the wall to accomplish this - I've even tried creating a (nStates x nParticles) matrix representing each particle-state combination, but this approach quickly spirals out of control in terms of readability since particles bounce from state to state independently of one another. Should I just bite the bullet and switch to a language more suitable for this?
Just write the code as you normally would. Almost all matlab functions can accept and return vectorized input. For instance, to simulate a brownian motion of N particles in 1 dimension
position = zeros([N 1]); %start at origin
sigma = sqrt(D * dt); %D is diffusion coefficient, dt is time step
for j = 1:numSteps
position = position + sigma*randn(size(position));
end
if you wanted to have a different sigma for each position, you would make sigma a vector the same size as position and use "dot times" notation to indicate element by element operation
position = position + sigma.*randn(size(position));
if the scattering was an arbitrary function of position and some random element, you would just have to write a vectorized function, e.g.
function newstep = step(position)
%diffusion in a overdamped harmonic potential
newstep = -dt*k*position + D*randn(size(position));
for j = 1:numsteps; position = position + step(position);
and so on