Solve *sparse* upper triangular system - matlab

If I want to solve a full upper triangular system, I can call linsolve(A,b,'UT'). However this currently is not supported for sparse matrices. How can I overcome this?

UT and LT systems are amongst the easiest systems to solve. Have a read on the wiki about it. Knowing this, it is easy to write your own UT or LT solver:
%# some example data
A = sparse( triu(rand(100)) );
b = rand(100,1);
%# solve UT system by back substitution
x = zeros(size(b));
for n = size(A,1):-1:1
x(n) = (b(n) - A(n,n+1:end)*x(n+1:end) ) / A(n,n);
end
The procedure is quite similar for LT systems.
Having said that, it is generally much easier and faster to use Matlab's backslash operator:
x = A\b
which also works for spares matrices, as nate already indicated.
Note that this operator also solves UT systems which have non-square A or if A has some elements equal to zero (or < eps) on the main diagonal. It solves these cases in a least-squares sense, which may or may not be desirable for you. You could check for these cases before carrying out the solve:
if size(A,1)==size(A,2) && all(abs(diag(A)) > eps)
x = A\b;
else
%# error, warning, whatever you want
end
Read more about the (back)slash operator by typing
>> help \
or
>> help slash
on the Matlab command prompt.

Edit Since what you need is a triangular solve procedure, also called backward/forward substitution, you can use ordinary MATLAB backslash \ operator for that:
x = U\b
As mentioned in the original answer, MATLAB will recognise the fact that your matrix is triangular. To be sure of that, you can compare the performance to cs_usolve procedure found in SuiteSparse. It is a mex function implemented in C that computes sparse triangular solve for upper-triangular sparse matrix (there are similar functions there too: cs_lsolve, cs_utsolve and cs_ltsolve).
You can have a look at a performance comparison of native MATLAB and cs_l(t)solve in the context of sparse Cholesky factorization. Essentially, MATLAB performance is good. The only pitfall is if you want to solve a transposed system
x = U'\b
MATLAB does not recognize that and explicitly creates a transpose of U. In that case you should call cs_utsolve explicitly.
Original answer If your system is symmetric and you only store the upper triangular matrix part (that is how I understood full in your question), and if Cholesky decomposition is suitable for you, chol handles symmetric matrices, if your matrix is positive definite. For indefinite matrices you can use ldl. Both handle sparse storage and work on the symmetric matrix parts.
Newer matlab versions use cholmod and suitesparse for that. That is by far the best performing Cholesky factorization I know of. In matlab it is also parallelised usin parallel BALS.
The factor you obtain from the above functions is upper triangular matrix L such that
A=LL'
All you need to do now is perform forward and backward substitution, which is simple and cheap. In matlab this is automatically done in tha backslash operator
x=L'\(L\b)
the matrix can be sparse, and matlab will recognise that it is upper/lower triangular. You would also use this call together with forward substitution for factors obtained using the cholesky factorization.

you can use MLDIVIDE( \ ) or MRDIVIDE( / ) operators on your sparse matrices...

Related

How to compute inverse of a matrix accurately?

I'm trying to compute an inverse of a matrix P, but if I multiply inv(P)*P, the MATLAB does not return the identity matrix. It's almost the identity (non diagonal values in the order of 10^(-12)). However, in my application I need more precision.
What can I do in this situation?
Only if you explicitly need the inverse of a matrix you use inv(), otherwise you just use the backslash operator \.
The documentation on inv() explicitly states:
x = A\b is computed differently than x = inv(A)*b and is recommended for solving systems of linear equations.
This is because the backslash operator, or mldivide() uses whatever method is most suited for your specific matrix:
x = A\B solves the system of linear equations A*x = B. The matrices A and B must have the same number of rows. MATLABĀ® displays a warning message if A is badly scaled or nearly singular, but performs the calculation regardless.
Just so you know what algorithm MATLAB chooses depending on your input matrices, here's the full algorithm flowchart as provided in their documentation
The versatility of mldivide in solving linear systems stems from its ability to take advantage of symmetries in the problem by dispatching to an appropriate solver. This approach aims to minimize computation time. The first distinction the function makes is between full (also called "dense") and sparse input arrays.
As a side-note about error of order of magnitude 10^(-12), besides the above mentioned inaccuracy of the inv() function, there's floating point accuracy. This post on MATLAB issues on it is rather insightful, with a more general computer science post on it here. Basically, if you are computing numerics, don't worry (too much at least) about errors 12 orders of magnitude smaller.
You have what's called an ill-conditioned matrix. It's risky to try to take the inverse of such a matrix. In general, taking the inverse of anything but the smallest matrices (such as those you see in an introduction to linear algebra textbook) is risky. If you must, you could try taking the Moore-Penrose pseudoinverse (see Wikipedia), but even that is not foolproof.

Sparse diagonal matrix solver

I want to solve, in MatLab, a linear system (corresponding to a PDE system of two equations written in finite difference scheme). The action of the system matrix (corresponding to one of the diffusive terms of the PDE system) reads, symbolically (u is one of the unknown fields, n is the time step, j is the grid point):
and fully:
The above matrix has to be intended as A, where A*U^n+1 = B is the system. U contains the 'u' and the 'v' (second unknown field of the PDE system) alternatively: U = [u_1,v_1,u_2,v_2,...,u_J,v_J].
So far I have been filling this matrix using spdiags and diag in the following expensive way:
E=zeros(2*J,1);
E(1:2:2*J) = 1;
E(2:2:2*J) = 0;
Dvec=zeros(2*J,1);
for i=3:2:2*J-3
Dvec(i)=D_11((i+1)/2);
end
for i=4:2:2*J-2
Dvec(i)=D_21(i/2);
end
A = diag(Dvec)*spdiags([-E,-E,2*E,2*E,-E,-E],[-3,-2,-1,0,1,2],2*J,2*J)/(dx^2);`
and for the solution
[L,U]=lu(A);
y = L\B;
U(:) =U\y;
where B is the right hand side vector.
This is obviously unreasonably expensive because it needs to build a JxJ matrix, do a JxJ matrix multiplication, etc.
Then comes my question: is there a way to solve the system without passing MatLab a matrix, e.g., by passing the vector Dvec or alternatively directly D_11 and D_22?
This would spare me a lot of memory and processing time!
Matlab doesn't store sparse matrices as JxJ arrays but as lists of size O(J). See
http://au.mathworks.com/help/matlab/math/constructing-sparse-matrices.html
Since you are using the spdiags function to construct A, Matlab should already recognize A as sparse and you should indeed see such a list if you display A in console view.
For a tridiagonal matrix like yours, the L and U matrices should already be sparse.
So you just need to ensure that the \ operator uses the appropriate sparse algorithm according to the rules in http://au.mathworks.com/help/matlab/ref/mldivide.html. It's not clear whether the vector B will already be considered sparse, but you could recast it as a diagonal matrix which should certainly be considered sparse.

Solve Ax = b using MATLAB

I have a linear system of equations AX = B to solve in MATLAB. What I have known is A is sparse, positive-definite and symmetric. I know the command x = A \ b works yet I am not sure MATLAB takes full advantage of A's good properties so as to maximize the efficiency. Is there any way to specify the algorithm to solve it, for example Conjugate Gradient algorithm in MATLAB?
If your matrix is sparse, you can use all these iterative functions, for example bicg for a biconjugate gradients method.
MATLAB's mldivide operator does indeed take advantage of properties of A. See the documentation for details - expand the "Algorithm" section.

Solve the sparse triangular linear system in Matlab

I implement the LU decomposition algorithm in Matlab for some large sparse Matrices to solve the linear system. When I got the L,U matrix, I used the backward substitution and forward substitution algorithm to solve the triangular linear system:
%x = U\y;
for i = n : -1 : 1
x(i,:) = (y(i,:)-U(i,:)*x)/U(i,i);
end
but I found this code is the bottleneck. Although I can use the A\b to get the solution, but I want to know how can I implement a efficient algorithm to solve this problem in Matlab, For example, Can I write the matrix product to simulate the following action without for loop?
(I got some reference books and paper, but all of the code is not in Matlab, just for C++ or C code)
First off: correctness goes before speed; the loop you posted produces results different from U\y, so you might want to check that first :)
AFAIK, the backslash does some checks on the input matrix, and calls the fastest algorithm accordingly. When those checks indicate A is lower triangular, it will do exactly what you did (but then probably more efficient).
Anyway, to speed up your code: you should pre-allocate x, otherwise Matlab is forced to grow the vector at each iteration. Also, call your loop variable ii -- i is the imaginary unit, and the name resolution at each iteration takes some time. So, in summary:
x = zeros(size(y));
for ii = n : -1 : 1
x(ii,:) = (y(ii,:)-U(ii,:)*x)/U(ii,ii);
end
Note that there is no 'vectorized' solution, as the next result depends on the previous one.

Issues with backslash operator in matlab

I have this huge matrix A of dimension 900000x900000. And I have to solve this linear equation
Ax=b where b is a column matrix of size 900000x1.
I used matlab's backslash operator like A\b to try to get x. However, it freezes and I couldn't get x. Mostly I get out of memory issue. Even though I ran it in a computer with higher memory it makes the system very slow and I have to wait to get the answer.
How can I solve this equation. My matrix is pretty sparse. However, it's band is wider but most of the elements are zero. b is a full matrix. Any suggestions?
I did a project, where we also operated with such large but fortunately very sparse matrices.
Using such large matrices, you are pretty lost with direct methods: You can never compute the inverse because it will be a dense matrix, which you can never store. Also methods such as LU or Cholesky factorization are quite expensive because they again create a significant fill-in, i.e. they destroy zeros.
A viable alternative is to use iterative methods. If you know that your matrix is symmetric and positive-definite, try the Conjugate gradient method:
x = pcg(A, b); %# Computes a solution to Ax = b, with A symm. pos-def.
I would just give it a try and have a look, if the method converges. Proofing the assumption of positive-definiteness is not easy, I'm afraid.
If you do not get a solution, there are many more iterative methods. For example:
bicg - BiConjugate Gradient Method
bicgstab - BiConjugate Gradient Method (stabilized)
lsqr - Least Squares QR Method
gmres - Generalized Minimum Residual Method (I like this a lot)