I wanna solve #n linear equations AX=bi(for #n b's) in Matlab which b changes in a loop and A is constant.
One way which is fast, is to compute the inverse of A before the loop and in the loop body just get X from inv(A)*b, but because the matrix A is singular, I get an awful answer!
Of course, the numerical solution A/b gives a good answer, but the point is that it takes a long time to compute #n different X's in #n loops.
What I want is a solution which can be both accurate and fast.
I actually think this is a good question, typos and issues of matrix singularity aside. There are a few good ways to handle this, and Tim Davis' factorize submission on MATLAB Central covers all the angles.
However, just for reference, let's do it on our own in native MATLAB, starting with the case where A is square. First, there are the two methods you suggested (inv and \,mldivide):
% inv, slow and inacurate
xinvsol = inv(A)*b;
norm(A*xinvsol - b ,'fro')
% mldivide, faster and accurate
xref = A\b;
norm(A*xref - b ,'fro')
But if like you said A does not change, just factorize A and solve for new b! Say A is symmetric positive definite:
L = chol(A,'lower'); % Cholesky factorization
% mldivide, much faster (not counting the chol factorization) and most accurate
xcholbs= L'\(L\b); %'
norm(A*xcholbs - b ,'fro')
% linsolve, fastest (omits checks for matrix configuration) and most accurate
sol1 = linsolve(L, b, struct('LT',true));
xcholsolv = linsolve(L, sol1, struct('LT',true,'TRANSA',true));
norm(A*xcholsolv - b ,'fro')
If A is not symmetric positive definite, then you'd use LU decomposition for a square matrix or QR otherwise. Again, you can do it all yourself, or you can just use Tim Davis' awesome factorize functions.
Related
I have a matrix that is a function of some parameter A=A(x). I would like to find the points x where this matrix becomes singular. Example (I have a large matrix though):
syms x
A=[x sin(x); cos(x^2) 2.5];
So far I have been symbolically computing the determinant of the matrix and then used fzero or newtzero to find the roots of that characteristic equation. I.e.
detA = det(A);
fzero(matlabFunction(detA),startingGuess)
Then I found this: How to find out if a matrix is singular?, where it is advocated to not use the determinant under any circumstances.
Indeed the symbolic determinant calculation is terribly slow. However I tried to use rank(A) instead as suggested in the link and it does not seem to work for symbolic matrices.
Is there any way to implement the suggestions in the link for finding the roots of a characteristic equation of a matrix that is given symbolically?
A possible approach would be the following: a square matrix A is singular if and only if the homogeneous linear (with respect to the vector y) system A*y = 0 has nontrivial solutions y <> 0 (which is equivalent to det(A) = 0 and rank(A) = 0 among others. So a more or less standard, as I recall from the past, technique to compute such points x is to solve the nonlinear system
A(x)*y = 0 (1)
||y|| = 1 (2)
This way you can compute a point x* and a vector y* such that A(x*) is singular and y* is an eigenvector corresponding to the zero eigenvalue of A(x*).
If I remember correctly, you can also solve the somewhat easier system
A(x)*y = 0 (1)
<y,c> = 1 (2a)
where c is "almost" any nonzero random vector (normalize it to 1 to avoid numerical problems).
As a matter of fact there is an enormous bibliography on the subject - you can look for saddle-node bifurcation computations (in case A(x) is the Jacobian of a vector field), or for "distance to instability".
From a discussion with Ander Biguri it seems that the determinant is actually a perfectly fine method of approaching this problem. The problem seems to be to solve the final equation in a stable manner, which would be a different question.
I am working on a MR-physic simulation written in Matlab which simulates bloch's equations on an defined object. The magnetisation in the object is updated every time-step with the following functions.
function Mt = evolveMtrans(gamma, delta_B, G, T2, Mt0, delta_t)
% this function calculates precession and relaxation of the
% transversal component, Mt, of M
delta_phi = gamma*(delta_B + G)*delta_t;
Mt = Mt0 .* exp(-delta_t*1./T2 - 1i*delta_phi);
end
This function is a very small part of the entire code but is called upon up to 250.000 times and thus slows down the code and the performance of the entire simulation. I have thought about how I can speed up the calculation but haven't come up with a good solution. There is one line that is VERY time consuming and stands for approximately 50% - 60% of the overall simulation time. This is the line,
Mt = Mt0 .* exp(-delta_t*1./T2 - 1i*delta_phi);
where
Mt0 = 512x512 matrix
delta_t = a scalar
T2 = 512x512 matrix
delta_phi = 512x512 matrix
I would be very grateful for any suggestion to speed up this calculation.
More info below,
The function evovleMtrans is called every timestep during the simulation.
The parameters that are used for calling the function are,
gamma = a constant. (gyramagnetic constant)
delta_B = the magnetic field value
G = gradientstrength
T2 = a 512x512 matrix with T2-values for the object
Mstart.r = a 512x512 matrix with the values M.r had the last timestep
delta_t = a scalar with the difference in time since the last calculated M.r
The only parameters of these that changed during the simulation are,
G, Mstart.r and delta_t. The rest do not change their values during the simulation.
The part below is the part in the main code that calls the function.
% update phase and relaxation to calcTime
delta_t = calcTime - Mstart_t;
delta_B = (d-d0)*B0;
G = Sq.Gx*Sq.xGxref + Sq.Gz*Sq.zGzref;
% Precession around B0 (z-axis) and B1 (+-x-axis or +-y-axis)
% is defined clock-wise in a right hand system x, y, z and
% x', y', z (see the Bloch equation, Bloch 1946 and Levitt
% 1997). The x-axis has angle zero and the y-axis has angle 90.
% For flipping/precession around B1 in the xy-plane, z-axis has
% angle zero.
% For testing of precession direction:
% delta_phi = gamma*((ones(size(d)))*1e-6*B0)*delta_t;
M.r = evolveMtrans(gamma, delta_B, G, T2, Mstart.r, delta_t);
M.l = evolveMlong(T1, M0.l, Mstart.l, delta_t);
This is not a surprise.
That "single line" is a matrix equation. It's really 1,024 simultaneous equations.
Per Jannick, that first term means element-wise division, so "delta_t/T[i,j]". Multiplying a matrix by a scalar is O(N^2). Matrix addition is O(N^2). Evaluating exponential of a matrix will be O(N^2).
I'm not sure if I saw a complex argument in there as well. Does that mean complex matricies with real and imaginary entries? Does your equation simplify to real and imaginary parts? That means twice the number of computations.
Your best hope is to exploit symmetry as much as possible. If all your matricies are symmetric, you cut your calculations roughly in half.
Use parallelization if you can.
Algorithm choice can make a big difference, too. If you're using explicit Euler integration, you may have time step limitations due to stability concerns. Is that why you have 250,000 steps? Maybe a larger time step is possible with a more stable integration schema. Think about a higher order adaptive scheme with error correction, like 5th order Runge Kutta.
There are several possibilities to improve the speed of the code but all that I see come with a caveat.
Numerical ode integration
The first possibility would be to change your analytical solution by numerical differential equation solver. This has several advantages
The analytical solution includes the complex exponential function, which is costly to calculate, while the differential equation contains only multiplication and addition. (d/dt u = -a u => u=exp(-at))
There are plenty of built-in solvers for matlab available and they are typically pretty fast (e.g. ode45). The built-ins however all use a variable step size. This improves speed and accuracy but would be a problem if you really need a fixed equally spaced grid of time points. Here are unofficial fixed step solvers.
As a start you could also try to use just an euler step by replacing
M.r = evolveMtrans(gamma, delta_B, G, T2, Mstart.r, delta_t);
by
delta_phi = gamma*(delta_B + G)*t_step;
M.r += M.r .* (1-t_step*1./T2 - 1i*delta_phi);
You can then further improve that by precalculating all constant values, e.g. one_over_T1=1/T1, moving delta_phi out of the loop.
Caveat:
You are bound to a minimum step size or the accuracy suffers. Therefore this is only a good idea if you time-spacing is quite fine.
Less points in time
You should carfully analyze whether you really need so many points in time. It seems somewhat puzzling to me that you need so many points. As you know the full analytical solution you can freely choose how to sample the time and maybe use this to your advantage.
Going fortran
This might seem like a grand step but in my experience basic (simple loops, matrix operations etc.) matlab code can be relatively easily translated to fortran line-by-line. This would be especially helpful in addition to my first point. If you still want to use the full analytical solution probably there is not much to gain here because exp is already pretty fast in matlab.
I have the following differential equation which I'm not able to solve.
We know the following about the equation:
D(r) is a third grade polynom
D'(1)=D'(2)=0
D(2)=2D(1)
u(1)=450
u'(2)=-K * (u(2)-Te)
Where K and Te are constants.
I want to approximate the problem using a matrix and I managed to solve
the similiar equation: with the same limit conditions for u(1) and u'(2).
On this equation I approximated u' and u'' with central differences and used a finite difference method between r=1 to r=2. I then placed the results in a matrix A in matlab and the limit conditions in the vector Y in matlab and ran u=A\Y to get how the u value changes. Heres my matlab code for the equation I managed to solve:
clear
a=1;
b=2;
N=100;
h = (b-a)/N;
K=3.20;
Ti=450;
Te=20;
A = zeros(N+2);
A(1,1)=1;
A(end,end)=1/(2*h*K);
A(end,end-1)=1;
A(end,end-2)=-1/(2*h*K);
r=a+h:h:b;
%y(i)
for i=1:1:length(r)
yi(i)=-r(i)*(2/(h^2));
end
A(2:end-1,2:end-1)=A(2:end-1,2:end-1)+diag(yi);
%y(i-1)
for i=1:1:length(r)-1
ymin(i)=r(i+1)*(1/(h^2))-1/(2*h);
end
A(3:end-1,2:end-2) = A(3:end-1,2:end-2)+diag(ymin);
%y(i+1)
for i=1:1:length(r)
ymax(i)=r(i)*(1/(h^2))+1/(2*h);
end
A(2:end-1,3:end)=A(2:end-1,3:end)+diag(ymax);
Y=zeros(N+2,1);
Y(1) =Ti;
Y(2)=-(Ti*(r(1)/(h^2)-(1/(2*h))));
Y(end) = Te;
r=[1,r];
u=A\Y;
plot(r,u(1:end-1));
My question is, how do I solve the first differential equation?
As TroyHaskin pointed out in comments, one can determine D up to a constant factor, and that constant factor cancels out in D'/D anyway. Put another way: we can assume that D(1)=1 (a convenient number), since D can be multiplied by any constant. Now it's easy to find the coefficients (done with Wolfram Alpha), and the polynomial turns out to be
D(r) = -2r^3+9r^2-12r+6
with derivative D'(r) = -6r^2+18r-12. (There is also a smarter way to find the polynomial by starting with D', which is quadratic with known roots.)
I would probably use this information right away, computing the coefficient k of the first derivative:
r = a+h:h:b;
k = 1+r.*(-6*r.^2+18*r-12)./(-2*r.^3+9*r.^2-12*r+6);
It seems that k is always positive on the interval [1,2], so if you want to minimize the changes to existing code, just replace r(i) by r(i)/k(i) in it.
By the way, instead of loops like
for i=1:1:length(r)
yi(i)=-r(i)*(2/(h^2));
end
one usually does simply
yi=-r*(2/(h^2));
This vectorization makes the code more compact and can benefit the performance too (not so much in your example, where solving the linear system is the bottleneck). Another benefit is that yi is properly initialized, while with your loop construction, if yi happened to already exist and have length greater than length(r), the resulting array would have extraneous entries. (This is a potential source of hard-to-track bugs.)
I have to solve in MATLAB a linear system of equations A*x=B where A is symmetric and its elements depend on the difference of the indices: Aij=f(i-j).
I use iterative solvers because the size of A is say 40000x40000. The iterative solvers require to determine the product A*x where x is the test solution. The evaluation of this product turns out to be a convolution and therefore can be done dy means of fast fourier transforms (cputime ~ Nlog(N) instead of N^2). I have the following questions to this problem:
is this convolution circular? Because if it is circular I think that I have to use a specific indexing for the new matrices to take the fft. Is that right?
I find difficult to program the routine for the fft because I cannot understand the indexing I should use. Is there any ready routine which I can use to evaluate by fft directly the product A*x and not the convolution? Actually, the matrix A is constructed of 3x3 blocks and is symmetric. A ready routine for the product A*x would be the best solution for me.
In case that there is no ready routine, could you give me an idea by example how I could construct this routine to evaluate a matrix-vector product by fft?
Thank you in advance,
Panos
Very good and interesting question! :)
For certain special matrix structures, the Ax = b problem can be solved very quickly.
Circulant matrices.
Matrices corresponding to cyclic convolution Ax = h*x (* - is convolution symbol) are diagonalized in
the Fourier domain, and can be solved by:
x = ifft(fft(b)./fft(h));
Triangular and banded.
Triangular matrices and diagonally-dominant banded matrices are solved
efficiently by sparse LU factorization:
[L,U] = lu(sparse(A)); x = U\(L\b);
Poisson problem.
If A is a finite difference approximation of the Laplacian, the problem is efficiently solved by multigrid methods (e.g., web search for "matlab multigrid").
Interesting question!
The convolution is not circular in your case, unless you impose additional conditions. For example, A(1,3) should equal A(2,1), etc.
You could do it with conv (retaining only the non-zero-padded part with option valid), which probably is also N*log(N). For example, let
A = [a b c d
e a b c
f e a b
g f e a];
Then A*x is the same as
conv(fliplr([g f e a b c d]),x,'valid').'
Or more generally, A*x is the same as
conv(fliplr([A(end,1:end-1) A(1,:)]),x,'valid').'
I'd like to add some comments on Pio_Koon's answer.
First of all, I wouldn't advise to follow the suggestion for triangular and banded matrices. The time taken by a call to Matlab's lu() procedure on a large sparse matrix massively overshadows any benefits gained by solving the linear system as x=U\(L\b).
Second, in the Poisson problem you end up with a circulant matrix, therefore you can solve it using the FFT as described. In this specific case, your convolution mask h is a Laplacian, i.e., h=[0 -0.25 0; -0.25 1 -0.25; 0 -0.25 0].
I implement the LU decomposition algorithm in Matlab for some large sparse Matrices to solve the linear system. When I got the L,U matrix, I used the backward substitution and forward substitution algorithm to solve the triangular linear system:
%x = U\y;
for i = n : -1 : 1
x(i,:) = (y(i,:)-U(i,:)*x)/U(i,i);
end
but I found this code is the bottleneck. Although I can use the A\b to get the solution, but I want to know how can I implement a efficient algorithm to solve this problem in Matlab, For example, Can I write the matrix product to simulate the following action without for loop?
(I got some reference books and paper, but all of the code is not in Matlab, just for C++ or C code)
First off: correctness goes before speed; the loop you posted produces results different from U\y, so you might want to check that first :)
AFAIK, the backslash does some checks on the input matrix, and calls the fastest algorithm accordingly. When those checks indicate A is lower triangular, it will do exactly what you did (but then probably more efficient).
Anyway, to speed up your code: you should pre-allocate x, otherwise Matlab is forced to grow the vector at each iteration. Also, call your loop variable ii -- i is the imaginary unit, and the name resolution at each iteration takes some time. So, in summary:
x = zeros(size(y));
for ii = n : -1 : 1
x(ii,:) = (y(ii,:)-U(ii,:)*x)/U(ii,ii);
end
Note that there is no 'vectorized' solution, as the next result depends on the previous one.