I have implemented Roulette wheel selection in GA.
TotalFitness=sum(Fitness);
ProbSelection=zeros(PopLength,1);
CumProb=zeros(PopLength,1);
for i=1:PopLength
ProbSelection(i)=Fitness(i)/TotalFitness;
if i==1
CumProb(i)=ProbSelection(i);
else
CumProb(i)=CumProb(i-1)+ProbSelection(i);
end
end
SelectInd=rand(PopLength,1);
for i=1:PopLength
flag=0;
for j=1:PopLength
if(CumProb(j)<SelectInd(i) && CumProb(j+1)>=SelectInd(i))
SelectedPop(i,1:IndLength)=CurrentPop(j+1,1:IndLength);
flag=1;
break;
end
end
if(flag==0)
SelectedPop(i,1:IndLength)=CurrentPop(1,1:IndLength);
end
end
Now i was trying to implement rank selection in GA. I learned that:
Rank selection first ranks the population and then every chromosome receives fitness from this ranking.
The worst will have fitness 1, second worst 2 etc. and the best will have fitness N (number of chromosomes in population).
I saw these link1 and link2 and what i understood is that:
First i will sort the Fitness value of the Population.
Then if the Population number is 10 then i will give the probability of selection to the Population like 0.1,0.2,0.3,...,1.0 .
Then i will calculate cumulative Fitness like roulette wheel.
And the next steps is same as roulette wheel.
My Implementation:
NewFitness=sort(Fitness);
NewPop=round(rand(PopLength,IndLength));
for i=1:PopLength
for j=1:PopLength
if(NewFitness(i)==Fitness(j))
NewPop(i,1:IndLength)=CurrentPop(j,1:IndLength);
break;
end
end
end
CurrentPop=NewPop;
ProbSelection=zeros(PopLength,1);
CumProb=zeros(PopLength,1);
for i=1:PopLength
ProbSelection(i)=i/PopLength;
if i==1
CumProb(i)=ProbSelection(i);
else
CumProb(i)=CumProb(i-1)+ProbSelection(i);
end
end
SelectInd=rand(PopLength,1);
for i=1:PopLength
flag=0;
for j=1:PopLength
if(CumProb(j)<SelectInd(i) && CumProb(j+1)>=SelectInd(i))
SelectedPop(i,1:IndLength)=CurrentPop(j+1,1:IndLength);
flag=1;
break;
end
end
if(flag==0)
SelectedPop(i,1:IndLength)=CurrentPop(1,1:IndLength);
end
end
Am i understanding the algo wrong?? If it is then can anyone give me any idea how to modify my roulette wheel to rank selection??
If the population has N individuals, the best individual gets rank N and the worst 1 then
TotalFitness = sum(Fitness);
should be changed with:
TotalFitness = (N + 1) * N / 2;
(probably TotalFitness isn't anymore the right name for the variable, but let it go)
(N + 1) * N / 2 is just the sum of the ranks:
1 + 2 + ... + N = (N + 1) * N / 2
The probability of selection should be changed from:
ProbSelection(i) = Fitness(i) / TotalFitness;
to
ProbSelection(i) = i / TotalFitness;
Here using the rank instead of the fitness and assuming that the first individual of the population is the worst and the last is the best (sorted population).
Hence the complexity of the rank selection algorithm is dominated by the complexity of the sorting (O(N * log(N)).
You can see that the probability of selection for the worst individual is:
1 / ((N + 1) * N / 2) = 2 / ((N + 1) * N)
and the probability for the best individual is:
N / (((N + 1) * N / 2)) = 2 * (N + 1)
This is a linear rank selection: the ranks are in a linear progression. There are other schemes of rank selection (e.g. exponential).
Related
Given the equation to approximate pi
I need to the number of terms (n) that are needed to obtain an approximation that is within 10^(-12) of the actual value of pi. The code I have to find the n looks like this:
The while loop statement I have seems to never end, so I feel like my code must be wrong.
Try something along these lines (transcribed from your image), incrementing the number of approximation terms n inside your infinite while loop:
s = 1
n = 1
while true
s = abs(pi - approximate_pi(n))
if s <= 0.001
break
end
n = n + 1
end
On a related note, this calculation is a little bit pointless if you know the value of pi beforehand. Termination condition should be on the absolute magnitude of the n-th term.
The way you're doing it makes sense only if you're trying to find out minimum n for which your approximation series produces the result within some margin of error.
Edit. So, normally you would do it like this:
n = 1;
sum_running = 0
sum_target = (pi^2 - 8) / 16;
while true
sum_running += 1 / ((2*n-1)^2 * (2*n+1)^2);
if abs(sum_target - sum_running) <= 10e-12
break
end
n += 1;
end
pi_approx = sqrt(16*sum_running + 8)
There's no need to keep recalculating pi approximation up to n terms, for each new n. This is has O(n) complexity, while your initial solution had O(n^2), so it's much faster for large n.
I'm looking to create a vector of autocorrelated data points in MATLAB, with the lag 1 higher than lag 2, and so on.
If I look at the lag 1 data pairs (1, 2), (3, 4), (5, 6), ..., then the correlation is relatively higher, but then at lag 2 it's reduced.
I found a way to do this in R
x <- filter(rnorm(1000), filter=rep(1,3), circular=TRUE)
However, I'm not sure how to do the same thing in MATLAB. Ideally I'd like to be able to fine tune exactly how autocorrelated the data is.
Math:
A group of standard models for autocorrelation in stationary time series are so called "auto regressive model" eg. an autoregressive model with 1 term is known as an AR(1) and is:
y_t = a + b*y_{t-1} + e_t
AR(1) sounds simplistic, but it turns it's a quite powerful tooll. Eg. an AR(p) with p autoregressive terms is actually an AR(1) on a p dimensional vector. (Check Wikipedia page.) Note also b=1, gives a non-stationary random walk.
A more intuitive way to write what's going on (in stationary case with |b| < 1) is define u = a / (1 - b) (turns out u is unconditional mean of AR(1)), then with some algebra:
y_t - u = b * ( y_{t-1} - u) + e_t
That is, the difference from the unconditional mean u gets hit with some decay term b and then a shock term e_t gets added. (you want -1<b<1 for stationarity)
Code:
Since e_t denotes the shock term, this is super easy to simulate. Eg. to simulate an AR(1):
a = 0; b = .4; sigma = 1; T = 1000;
y0 = a / (1 - b); %eg initialize to unconditional mean of stationary time series
y = zeros(T,1);
y(1) = a + b * y0 + randn() * sigma;
for t = 2:T
y(t) = a + b * y(t-1) + randn() * sigma;
end
This code isn't mean to be fast, but illustrative. An AR(1) model implies a certain type of correlation structure, but adding AR or MA terms, you can fit some pretty funky stuff. (MA is moving average model)
Can test sample autocorrelation with autocorr(y). For reference, the bible on time series mathematics is Hamilton's book Time Series Analysis.
I want to calculate the sum of a series in MATLAB
The assignment is on the last page question 3 on this webpage
The series's sum should be pi but when I used MATLAB to prove that it just does not converge to pi
What am I doing wrong in my code?
function f = equation(n)
for i=1:n;
f=4*sum(((-1)^i)/(2*i+1))
if f >= pi
break;
else
continue;
end
end
end
EDIT :
function f = equation(n)
i=0:n;
f=4*sum((-1).^i ./(2*i+1));
plot(f,'x')
end
It does not work to plot(f) , what am I doing wrong ?
There are a few things you are doing wrong. First, you aren't keeping track of the running sum, you are just defining each term inside the loop. You should instead have something like
function total = equation(n)
total = 0;
for i = 1:n
total = total + (-1)^i / (2 * i + 1);
if total > pi
break;
end
end
total = 4 * total;
end
Second, you don't want the break statement there. If you plot the partial sums of this series, you can see that it oscillates either side of pi -
If you break as soon as you have exceeded pi, then you will break too early (after the first term, in fact!)
So your code should look like
function total = equation(n)
total = 0;
for i = 1:n
total = total + (-1)^i / (2*i+1);
end
total = 4 * total;
end
Additionally, this series will only converge to pi if you start it at zero, rather than one -
function total = equation(n)
total = 0;
for i = 0:n
total = total + (-1)^i / (2*i+1);
end
total = 4 * total;
end
Finally, you can simplify dramatically by vectorizing your code -
function total = equation(n)
indices = 0:n;
total = 4 * sum((-1).^indices ./ (2 * indices + 1));
end
To add to the rest of the answers. This code will compute all the approximations of pi from n=1 to n=100.
n=100;
f=[];
for jj=1:n
ii=0:jj;
f=[f 4*sum( ((-1).^ii)./(2.*ii+1) )];
end;
hold on
plot(f)
plot(1:size(f,2),ones(size(f))*pi)
Your code does not make sense: why would you sum a single element, then discard the result the next cycle? Your purpose would be better served by:
function f = no_kitty_is_my_pi_you_cant_have_it(n)
%'Sorry for the South Park reference/pun'
k = 0:n;
f = 4 * sum((-1).^k ./ (2*k+1));
end
And, as side note on MATLAB programming practices: choosing i and j as names for variables is not a good idea, because these two are built-in functions that give you the imaginary unit (the square root of -1).
everyone I have created a neural network with 1600 input, one hidden layer with different number of neurons nodes and 24 output neurons.
My code shown that I can decrease the error each epoch, but the output of hidden layer always is 1. Due to this reason, the weight adjusted always produce same result for my testing data.
I try different number of neuron nodes and learning rate in the ANN and also randomly initialize my initial weight. I use sigmoid function as my activate function since my output is either 1 or 0 in different output.
May I know that what is the main reason that causes the output of hidden layer always is 1 and how should i solve it?
My purpose for this neural network is to recognize 24 hand shape for alphabet, I try intensities data in my first phase of project.
I have try 30 hidden neural nodes also 100 neural nodes even 1000 neural nodes but the output of hidden layer still is 1. Due to this reason, all of the outcome in testing data is always similar.
I added the code for my network
Thanks
g = inline('logsig(x)');
[row, col] = size(input);
numofInputNeurons = col;
weight_input_hidden = rand(numofInputNeurons, numofFirstHiddenNeurons);
weight_hidden_output = rand(numofFirstHiddenNeurons, numofOutputNeurons);
epochs = 0;
errorMatrix = [];
while(true)
if(totalEpochs > 0 && epochs >= totalEpochs)
break;
end
totalError = 0;
epochs = epochs + 1;
for i = 1:row
targetRow = zeros(1, numofOutputNeurons);
targetRow(1, target(i)) = 1;
hidden_output = g(input(1, 1:end)*weight_input_hidden);
final_output = g(hidden_output*weight_hidden_output);
error = abs(targetRow - final_output);
error = sum(error);
totalError = totalError + error;
if(error ~= 0)
delta_final_output = learningRate * (targetRow - final_output) .* final_output .* (1 - final_output);
delta_hidden_output = learningRate * (hidden_output) .* (1-hidden_output) .* (delta_final_output * weight_hidden_output');
for m = 1:numofFirstHiddenNeurons
for n = 1:numofOutputNeurons
current_changes = delta_final_output(1, n) * hidden_output(1, m);
weight_hidden_output(m, n) = weight_hidden_output(m, n) + current_changes;
end
end
for m = 1:numofInputNeurons
for n = 1:numofFirstHiddenNeurons
current_changes = delta_hidden_output(1, n) * input(1, m);
weight_input_hidden(m, n) = weight_input_hidden(m, n) + current_changes;
end
end
end
end
totalError = totalError / (row);
errorMatrix(end + 1) = totalError;
if(errorThreshold > 0 && totalEpochs == 0 && totalError < errorThreshold)
break;
end
end
I see a few obvious errors that need fixing in your code:
1) You have no negative weights when initialising. This is likely to get the network stuck. The weight initialisation should be something like:
weight_input_hidden = 0.2 * rand(numofInputNeurons, numofFirstHiddenNeurons) - 0.1;
2) You have not implemented bias. That will severely limit the ability of the network to learn. You should go back to your notes and figure that out, it is usually implemented as an extra column of 1's inserted into input and activation vectors/matrix before determining the activations of each layer, and there should be a matching additional column of weights.
3) Your delta for output layer is wrong. This line
delta_final_output = learningRate * (targetRow - final_output) .* final_output .* (1 - final_output);
. . . is not the delta for the output layer activations. It has some extra unwanted factors.
The correct delta for logloss objective function and sigmoid activation in output layer would be:
delta_final_output = (final_output - targetRow);
There are other possibilities, depending on your objective function, which is not shown. You original code is close to correct for mean squared error, which would probably still work if you changed the sign and removed the factor of learningRate
4) Your delta for hidden layer is wrong. This line:
delta_hidden_output = learningRate * (hidden_output) .* (1-hidden_output) .* (delta_final_output * weight_hidden_output');
. . . is not the delta for the hidden layer activations. You have multiplied by the learningRate for some reason (combined with the other delta that means you have a factor of learningRate squared).
The correct delta would be:
delta_hidden_output = (hidden_output) .* (1-hidden_output) .* (delta_final_output * weight_hidden_output');
5) Your weight update step needs adjusting to match fixes to (3) and (4). These lines:
current_changes = delta_final_output(1, n) * hidden_output(1, m);
would need to be adjusted to get correct sign and learning rate multiplier
current_changes = -learningRate * delta_final_output(1, n) * hidden_output(1, m);
That's 5 bugs from looking through the code, I may have missed some. But I think that's more than enough for now.
I want to make the code below fast. It takes so long time to run, and I got this error:
Warning: FOR loop index is too large. Truncating to 2147483647.
I need to calculate over 3^100 so... is it impossible?
function sodiv = divisorSum(n)
sodiv = 0;
for i=1:n
if (mod(n,i) == 0)
sodiv = sodiv + i;
end
end
end
function finalSum1 = formular1(N,n)
finalSum1 = 0;
for k = 1:N
finalSum1 = finalSum1 + (divisorSum(k) * divisorSum(3^n*(N-k)));
end
end
Nv=100;
nv=[1:20];
for i=1:length(nv)
tic;
nfunc1(i)=formular1(Nv,nv(i));
nt1(i)=toc;
sprintf('nt1 : %d finished, %f', i,nt1(i))
end
The purpose of this code is to check the algorithm's calculation time.
The algorithm is too general and inefficient for this particular problem.
I understand you want to sum the divisors of 3^100. But these divisors are easily determined.
S = 1 + 3 + 3^2 + 3^3 + ... + 3^100, a geometric series.
3*S = 3 + 3^2 + ... + 3^101
subtract
2*S = 3^101 - 1
S = (3^101 - 1)/2
This code will never finish, because it is so inefficient.
For instance, there is a function that counts number of all divisors and is going through all numbers from 1 to N and count. But using an efficient formula would make it run much master.
Let's say that one need to sum divisors of number a^b where a is prime number.
Instead of calculating a^b and going form 1 to a^b, one can see that it is better going
a^1, a^2, a^3, ..., a^n, because only these numbers are divisors. But you can go even further and observe that the sum of these numbers are the sum of geometric progression so the number of divisors become:
sum divisors, a^b = (a^(b+1)-1) / (a-1)