Happy New Year everyone! :)
I'm writing a Gauss-Seidel function in Matlab and I'm encountering some problems.
The iteration must stop when we reach 6 decimal digits of precision. It means that the infinite norm (asked to use it) of x-xprevious must be less than 0.5*10^(-6).
Firstly, here's my function:
function [x] = ex1_3(A,b)
format long
sizeA=size(A,1);
x=zeros(sizeA,1);
%Just a check for the conditions of the Gauss-Seidel Method
for i=1:sizeA
sum=0;
for j=1:sizeA
if i~=j
sum=sum+A(i,j);
end
end
if A(i,i)<sum
fprintf('\nGauss-Seidel''s conditions not met!\n');
return
end
end
%Actual Gauss-Seidel Method
max_temp=10^(-6); %Pass first iteration
while max_temp>(0.5*10^(-6))
xprevious=x;
for i=1:sizeA
x(i,1)=b(i,1);
for j=1:sizeA
if i~=j
x(i,1)=x(i,1)-A(i,j)*x(j,1);
end
end
x(i,1)=x(i,1)/A(i,i);
end
x
%Calculating infinite norm of vector x-xprevious
temp=x-xprevious;
max_temp=temp(1,1);
for i=2:sizeA
if abs(temp(i,1))>max_temp
max_temp=abs(temp(i,1));
end
end
end
And now the problems! When I call the function for a 3x3 array, I think it works. However, when I call it for a 10x10 array x becomes Inf (I guess it's out of machine numbers limits). Is there anything I can do to prevent this, except for changing the infinite norm and the 6 decimal digits precision (I must use these two, because my tutor told me so) ?
In the array I use (which was given to me) the entries outside the diagonal are -1 and the ones on the diagonal are 3. b is like this b=[2;1;1;1;1;1;1;1;1;2] (for n=10)
Your condition of the Gauss-Seidel Method is not correct:
D=diag(diag(A));
L=-tril(A,-1);U=-triu(A,1);
B=(D-L)\U;
R = max(abs(eig(B)));
if R>=1
fprintf('\nGauss-Seidel''s conditions not met!\n');
return
end
R is called spectral radius of iterative matrix B. It has to be less than 1 that Gauss-Seidel converges. Actually the matrix A in your test case has the R=1.8092, thus Gauss-Seidel method won't converge.
Check this slide from page 18 for more details.
EDIT
According to #LutzL's comment, you may use Gershgorin circle theorem to estimate the eigenvalue rather than calculate them with computational cost.
Related
I am trying to write a program that does integration with monte carlo method. One of its features is to place dots on the graph with different colours, blue or red depending on the if statement. The if statement is put in a "for" loop and i dont know why but it seems like the first option is ignored after first iteration. The whole thing looks like this :
but it should look like this :
In addition i dont know why but it looks like the plot makes some additional empty space at the top
The whole code is not finished yet, its just a matter of a few lines but these dots are so annoying that I want to figure out whats wrong first. Heres the code.
function p=montecarlo(f, a, b, n, t)
%f is a function provided by user
%a and b is a range
%n is the amount of random points
%t is a t=a:01:b vector to draw a plot
upper=max(f(t));
lower=min(f(t));
x=a+(b-a).*(rand(n,1)) %generates vector of random numbers from a to b
y=lower+(upper-lower).*(rand(n,1)) %generates vector of ranom numbers from min to max
hold on
for i=1:n
if y(i)>=f(i)
plot(x(i),y(i),'bo')
else
plot(x(i),y(i),'ro')
end
plot(t,f(t),'k')
end
end
Arguments provided to the function : f= x.^2+3*x+5, a= -4 , b= 2, n= 1000 .
The problem is simple. The statement:
if y(i)>=f(i)
is wrong. What you want to do is compare the random value y(i) with the function value at the corresponding point x(i), so it should be:
if y(i)>=f(x(i))
plot(x(i),y(i),'bo')
else
plot(x(i),y(i),'ro')
end
I have a function
b=2.02478;
g=3.45581;
s=0.6;
R=1;
p =#(r) 1 - (b./r).^2 - (g^-2)*((2/15)*(s/R)^9 *(1./(r - 1).^9 - 1./(r + 1).^9 - 9./(8*r).*(1./(r - 1).^8 - 1./(r + 1).^8)) -(s/R)^3 *(1./(r-1).^3 - 1./(r+1).^3 - 3./(2*r).*(1./(r-1).^2 - 1./(r+1).^2)));
options = optimset('Display','off');
tic
r2 = fzero(p,[1.001,100])
toc
tic
r3 = fsolve(p,[1.001,100],options)
toc
and the answer
r2 =
2.0198
Elapsed time is 0.002342 seconds.
r3 =
2.1648 2.2745
Elapsed time is 0.048991 seconds.
which is more confiable ? fzero returns different values than fsolve
You should always look at the exit flag (or output struct) of a function, especially when your result is not as expected.
This is what I get:
fzero(func,[1.00001,100]):
X = 4.9969
FVAL
EXITFLAG = 1 % fzero found a zero X.
OUTPUT.message = 'Zero found in the interval [1.00001, 100]'
fzero(func,1.1):
X = 1
FVAL = 8.2304e+136
EXITFLAG = -5 % fzero may have converged to a singular point.
OUTPUT.message = 'Current point x may be near a singular point. The interval [0.975549, 1.188] reduced to the requested tolerance and the function changes sign in the interval, but f(x) increased in magnitude as the interval reduced.'
The meaning of the exit flag is explained in the matlab documentation:
1 Function converged to a solution x.
-5 Algorithm might have converged to a singular point.
-6 fzero did not detect a sign change.
So, based on this information it is clear that the first one gives you the correct result.
Why does fzero fails
As documented in the manual, fzero calculates the zero by finding a sign change:
tries to find a point x where fun(x) = 0. This solution is where fun(x) changes sign—fzero cannot find a root of a function such as x^2.
Therefore, X = 1 is also a solution of your formulation as the sign changes at this location from +inf to -inf as can be seen on a plot:
Note that it is always a good idea to provide a search range if possible as mentioned in the manual:
Calling fzero with a finite interval guarantees fzero will return a value near a point where FUN changes sign.
Tip: Calling fzero with an interval (x0 with two elements) is often faster than calling it with a scalar x0.
Alternative: fsolve
Note that this method is developed for solving a system of multiple nonlinear equations. Therefore, it is not as efficient as fzero (~20x slower in your case). fzero uses gradient based methods (check the manual for more information), which may work better in certain situations, but may get stuck in a local extrema. In this case, the gradient of your function gives the correct direction as long as your initial value is larger than 1. So, for this specific function fsolve is somewhat more robust than fzero with a single initial value, i.e. fsolve(func, 1.1) returns the expected value.
Conclusion: In general, use fzero with a search range instead of an initial value if possible for a single variable and fsolve for multiple variables. If one method fails, you can try another method or another starting point.
As you can read in documentation:
The algorithm, which was originated by T. Dekker, uses a combination of bisection, secant, and inverse quadratic interpolation methods.
So, it is sensitive into the initial point and area which it is seeking for the solution. Hence, you have gotten different result for different initial value and scope.
Im implementing the k-means algorithm on matlab without using the k-means built-in function, The stopping criteria is when the new centroids doesn't change by new iterations, but i cannot implement it in matlab , can anybody help?
Thanks
Setting no change as a stopping criteria is a bad idea. There are a few main reasons you shouldn't use a 0 change condition
even for a well behaved function the difference between 0 change and a very small change (say 1e-5 perhaps)could be 1000+ iterations, so you are wasting time trying to get them to be exactly the same. Especially because computers usually keep far more digits than we are interested in. IF you only need 1 digit accuracy, why wait for the computer to find an answer within 1e-31?
computers have floating point errors everywhere. Try doing some easily reversible matrix operations like a = rand(3,3); b = a*a*inv(a); a-b theoretically this should be 0 but you will see it isn't. So these errors alone could prevent your program from ever stopping
dithering. lets say we have a 1d k means problem with 3 numbers and we want to split them into 2 groups. One iteration the grouping can be a,b vs c. the next iteration could be a vs b,c the next could be a,b vs c the next.... This is of course a simplified example, but there can be instances where a few data points can dither between clusters, and you will end up with a never ending algorithm. Since those few points are reassigned, the change will never be 0
the solution is to use a delta threshold. basically you subtract the current values from the previous and if they are less than a threshold you are done. This on its own is powerful, but as with any loop, you need a backup escape plan. And that is setting a max_iterations variable. Look at matlabs documentation for kmeans, even they have a MaxIter variable (default is 100) so even if your kmeans doesn't converge, at least it wont run endlessly. Something like this might work
%problem specific
max_iter = 100;
%choose a small number appropriate to your problem
thresh = 1e-3;
%ensures it runs the first time
delta_mu = thresh + 1;
num_iter = 0;
%do your kmeans in the loop
while (delta_mu > thresh && num_iter < max_iter)
%save these right away
old_mu = curr_mu;
%calculate new means and variances, this is the standard kmeans iteration
%then store the values in a variable called curr_mu
curr_mu = newly_calculate_values;
%use the two norm to find the delta as a single number. no matter what
%the original dimensionality of mu was. If old_mu -new_mu was
% 0 the norm is still 0. so it behaves well as a distance measure.
delta_mu = norm(old_mu - curr_mu,2);
num_ter = num_iter + 1;
end
edit
if you don't know the 2 norm is essentially the euclidean distance
I'm trying to implement the Jacobi iteration in MATLAB but am unable to get it to converge. I have looked online and elsewhere for working code for comparison but am unable to find any that is something similar to my code and still works. Here is what I have:
function x = Jacobi(A,b,tol,maxiter)
n = size(A,1);
xp = zeros(n,1);
x = zeros(n,1);
k=0; % number of steps
while(k<=maxiter)
k=k+1;
for i=1:n
xp(i) = 1/A(i,i)*(b(i) - A(i,1:i-1)*x(1:i-1) - A(i,i+1:n)*x(i+1:n));
end
err = norm(A*xp-b);
if(err<tol)
x=xp;
break;
end
x=xp;
end
This just blows up no matter what A and b I use. It's probably a small error I'm overlooking but I would be very grateful if anyone could explain what's wrong because this should be correct but is not so in practice.
Your code is correct. The reason why it may not seem to work is because you are specifying systems that may not converge when you are using Jacobi iterations.
To be specific (thanks to #Saraubh), this method will converge if your matrix A is strictly diagonally dominant. In other words, for each row i in your matrix, the absolute summation of all of the columns j at row i without the diagonal coefficient at i must be less than the diagonal itself. In other words:
However, there are some systems that will converge with Jacobi, even if this condition isn't satisfied, but you should use this as a general rule before trying to use Jacobi for your system. It's actually more stable if you use Gauss-Seidel. The only difference is that you are re-using the solution of x and feeding it into the other variables as you progress down the rows. To make this Gauss-Seidel, all you have to do is change one character within your for loop. Change it from this:
xp(i) = 1/A(i,i)*(b(i) - A(i,1:i-1)*x(1:i-1) - A(i,i+1:n)*x(i+1:n));
To this:
xp(i) = 1/A(i,i)*(b(i) - A(i,1:i-1)*xp(1:i-1) - A(i,i+1:n)*x(i+1:n));
**HERE**
Here are two examples that I will show you:
Where we specify a system that does not converge by Jacobi, but there is a solution. This system is not diagonally dominant.
Where we specify a system that does converge by Jacobi. Specifically, this system is diagonally dominant.
Example #1
A = [1 2 2 3; -1 4 2 7; 3 1 6 0; 1 0 3 4];
b = [0;1;-1;2];
x = Jacobi(A, b, 0.001, 40)
xtrue = A \ b
x =
1.0e+09 *
4.1567
0.8382
1.2380
1.0983
xtrue =
-0.1979
-0.7187
0.0521
0.5104
Now, if I used the Gauss-Seidel solution, this is what I get:
x =
-0.1988
-0.7190
0.0526
0.5103
Woah! It converged for Gauss-Seidel and not Jacobi, even though the system isn't diagonally dominant, I may have an explanation for that, and I'll provide later.
Example #2
A = [10 -1 2 0; -1 -11 -1 3; 2 -1 10 -1; 0 3 -1 8];
b = [6;25;-11;15];
x = Jacobi(A, b, 0.001, 40);
xtrue = A \ b
x =
0.6729
-1.5936
-1.1612
2.3275
xtrue =
0.6729
-1.5936
-1.1612
2.3274
This is what I get with Gauss-Seidel:
x =
0.6729
-1.5936
-1.1612
2.3274
This certainly converged for both, and the system is diagonally dominant.
As such, there is nothing wrong with your code. You are just specifying a system that can't be solved using Jacobi. It's better to use Gauss-Seidel for iterative methods that revolve around this kind of solving. The reason why is because you are immediately using information from the current iteration and spreading this to the rest of the variables. Jacobi does not do this, which is the reason why it diverges more quickly. For Jacobi, you can see that Example #1 failed to converge, while Example #2 did. Gauss-Seidel converged for both. In fact, when they both converge, they're quite close to the true solution.
Again, you need to make sure that your systems are diagonally dominant so you are guaranteed to have convergence. Not enforcing this rule... well... you'll be taking a risk as it may or may not converge.
Good luck!
Though this does not point out the problem in your code, I believe that you are looking for the Numerical Methods: Jacobi File Exchange Submission.
%JACOBI Jacobi iteration for solving a linear system.
% Sample call
% [X,dX] = jacobi(A,B,P,delta,max1)
% [X,dX,Z] = jacobi(A,B,P,delta,max1)
It seems to do exactly what you describe.
As others have pointed out that not all systems are convergent using Jacobi method, but they do not point out why? Actually only a small sub-set of systems converge with Jacobi method.
The convergence criteria is that the "sum of all the coefficients (non-diagonal) in a row" must be lesser than the "coefficient at the diagonal position in that row". This criteria must be satisfied by all the rows. You can read more at: Jacobi Method Convergence
Before you decide to use Jacobi method, you must see whether this criteria is satisfied by the numerical method or not. The Gauss-Seidel method has a slightly more relaxed convergence criteria which allows you to use it for most of the Finite Difference type numerical methods.
I have already asked a similar question here, I will try this time to be a lot more elaborate in consideration that what puzzles me the most about Matlab still is how to handle/control output as in plotting/listing.
I have the following function
function e = calcEulerSum2(n)
for i=1:n % step size still one, although left blank
e = 1;
e = e + sum(1./factorial(1:n)); % computes a vector of length n and takes
end % the factorial of its entries, then piecewise division
end
To approximate e using the Taylor Summation method. I am now asked to compute the absolute error using
abserror(n):=abs(calcEulerSum2(n)-exp(1))
for values of n= 10^0,10^1,...,10^16.
My idea was to make the above function sensitive for vector input (if you call it like that, I have read vectorization on here as well). I thought this would be the best thing I could do, because I obviously want to evaluate the above function for a couple of values of n and then maybe plot the result in a graph to see the results.
However, I believe that my understanding of MATLAB is too rudimentary as of now to find a simple solution to that problem.
Additional: With the help of this website I already managed to solve a similar problem, using a recursive definition of e to get a vector output for each successful iteration using:
function e = calcEulerSum3(n)
for i = n % here the idea is to let n be a continuous vector
e(1)=1; % base case
e(i+1)=e(i)+1/factorial(i); % recursive definition to approx e
end
This function now understands vector input, but only if the vector is what I would call continuous on the integer line, for example n'=[ 1 2 3 4 5 ] and so on to make the recursive iteration through a vector work. This time however, my vector n as above would destroy this concept.
Question(s):
Are there simple ways to handle the above output abserror(n) ?, because I have a feeling that I am overachieving by trying to vectorize my functions
Update: As suggested by #vish the following function does a much better job for my purpose:
function e = calcEulerSum2(n) %Updated
for i=1:n % step size still one, although left blank
e = 1;
e = e + cumsum(1./factorial(1:n)); % computes a vector of length n and takes
end % the factorial of its entries, then piecewise division
end
the output of this program for n=3 would be
calcEulerSum2(3)
ans =
2.0000 2.5000 2.6667
This is great and indeed what I am looking for, but lets say I want to plot the absolute error now for several values of n, for the sake of the argument I will choose smaller values (because I understand that 10^16! is really blunt to input), but still many. So if I decide I want to print out the absolute error abserror(n) for the following values of n
n = [1 4 9 11 17 18 19 22 27 29 31 33 ]
how would I do that?
Not sure you are computing what you think you are computing.
The for loop is totally not needed since you keep overwriting e at every iteration and you only return the last one.
factorial(1:3) == [1,2,6]
then you sum the reciprocals.
The first iteration of the loop factorial(1:1) goes nowhere.
Try
cumsum(1./factorial(1:7))
l think that's what you wanted
Edit: , factorial(10^5) is too big to be meaningful, i think going from 1 to n=16 will give you good enough results. (not 10^16)
2nd edit
run this
N=0:16;
res = exp(1) - cumsum(1./factorial(N))
see what happens
you can also plot it on a log scale.
semilogy(res)