Perform LU decomposition without pivoting in MATLAB - matlab

How can I implement the function lu(A) in MATLAB so that L*U is directly A and I also get the real L matrix?
When I use [L,U] = lu(A), MATLAB doesn't give me the right L matrix. When I use [L,U,P] = lu(A), I need to implement P*A = L*U, but I only want to multiply L*U to receive A.

MATLAB's lu always performs pivoting by default. If you had for example a diagonal coefficient that was equal to 0 when you tried to do the conventional LU decomposition algorithm, it will not work as the diagonal coefficients are required when performing the Gaussian elimination to create the upper triangular matrix U so you would get a divide by zero error. Pivoting is required to ensure that the decomposition is stable.
However, if you can guarantee that the diagonal coefficients of your matrix are non-zero, it is very simple but you will have to write this on your own. All you have to do is perform Gaussian elimination on the matrix and reduce the matrix into reduced echelon form. The result reduced echelon form matrix is U while the coefficients required to remove the lower triangular part of L in Gaussian elimination would be placed in the lower triangular half to make U.
Something like this could work, assuming your matrix is stored in A. Remember that I'm assuming a square matrix here. The implementation of the non-pivoting LU decomposition algorithm is placed in a MATLAB function file called lu_nopivot:
function [L, U] = lu_nopivot(A)
n = size(A, 1); % Obtain number of rows (should equal number of columns)
L = eye(n); % Start L off as identity and populate the lower triangular half slowly
for k = 1 : n
% For each row k, access columns from k+1 to the end and divide by
% the diagonal coefficient at A(k ,k)
L(k + 1 : n, k) = A(k + 1 : n, k) / A(k, k);
% For each row k+1 to the end, perform Gaussian elimination
% In the end, A will contain U
for l = k + 1 : n
A(l, :) = A(l, :) - L(l, k) * A(k, :);
end
end
U = A;
end
As a running example, suppose we have the following 3 x 3 matrix:
>> rng(123)
>> A = randi(10, 3, 3)
A =
7 6 10
3 8 7
3 5 5
Running the algorithm gives us:
>> [L,U] = lu_nopivot(A)
L =
1.0000 0 0
0.4286 1.0000 0
0.4286 0.4474 1.0000
U =
7.0000 6.0000 10.0000
0 5.4286 2.7143
0 0 -0.5000
Multiplying L and U together gives:
>> L*U
ans =
7 6 10
3 8 7
3 5 5
... which is the original matrix A.

You could use this hack (though as already mentioned, you might lose numerical stability):
[L, U] = lu(sparse(A), 0)

You might want to consider doing LDU decomposition instead of unpivoted LU. See, LU without pivoting is numerically unstable - even for matrices that are full rank and invertible. The simple algorithm provided above shows why - there is division by each diagonal element of the matrix involved. Thus, if there is a zero anywhere on the diagonal, decomposition fails, even though the matrix could still be non-singular.
Wikipedia talks a little about LDU decomposition here:
https://en.wikipedia.org/wiki/LU_decomposition#LDU_decomposition
without citing an algorithm. It cites the following textbook for proof of existence:
Horn, Roger A.; Johnson, Charles R. (1985), Matrix Analysis, Cambridge University Press, ISBN 978-0-521-38632-6. See Section 3.5.
LDU is guaranteed to exist (at least for an invertible matrix), it is numerically stable, and it is also unique (provided that both L and U are constrained to have unit elements on the diagonal).
Then, if for any reason "D" gets in your way, you can absorb the diagonal matrix D into either L (L:=LD) or U (U:=DU), or split it symmetrically between L and U (such as L:=L*sqrt(D) and U:=sqrt(D)*U), or however you want to do it. There is an infinite number of ways to split LDU into LU, and this is why LU decomposition is not unique.

Related

What does the operation A'\B' do if A and B are both row vectors of the same size?

[1 2 1]'\[1 2 3]' This is a numerical example. This example gives an answer of 1.333
From the documentation:
x = A\B
If A is a rectangular m-by-n matrix with m ~= n, and B is a matrix with m rows, then A\B returns a least-squares solution to the system of equations A*x= B.
Furthermore the ' compute the conjugate transposed of a matrix. In your case you have two real matrices so you just get the transposed each time.

LU decomposition using matlab's linsolve

So i'm trying to solve a system of linear equations using LU decomposition. I made a code in matlab that i'm comparing with the output of matlab's linsolve. The thing that's confussing me is this:
As far as i understand (from what i learned from this site: http://www.cs.mtu.edu/~shene/COURSES/cs3621/NOTES/INT-APP/CURVE-linear-system.html) Lu decomposition works by decomposing A into L(lower triangular) and U (upper triangular). And then it calculates x solving two equations:
B = LY;
Y = UX;
So my confussion comes here.
If i do x_solutions=linsolve(A,B), i get a different result that if i do x=linsolve(U,y) (of course doing first y = linsolve(L,B)).
Does someone know why this happens? shouldn't x be equal to x_solutions in this case or am i missing something?
Just for the sake of giving away all the information this is how i'm doing it:
A=[1 2 6; 1 2 2; 2 2 1];
B=[1 0 1]';
G=linsolve(A,B);
UPP = triu(A);
LOW= tril(A);
y=linsolve(LOW,B);
x=linsolve(UPP,y);
Thank you in advance!
triu and tril do not give you the L and U in LU decomposition.
They just give entries lower/upper part of matrix, i.e.,
A == LOW + UPP - diag(A)
diag(A) is subtracted because both LOW and UPP have diagonal part of A
In LU decomposition, L and U should satisfy
A == L*U
If you want to get such L and U, use
[L,U] = lu(A);

singular value decomposition and low rank tensorial approximation

according this article
http://www.wseas.us/e-library/conferences/2012/Vouliagmeni/MMAS/MMAS-07.pdf
matrix can be approximated by one rank matrices using tensorial approximation,i know that in matlab kronecker product plays same role as tensorial product,function is kron,now let us suppose that we have following matrix
a=[2 1 3;4 3 5]
a =
2 1 3
4 3 5
SVD of this matrix is
[U E V]=svd(a)
U =
-0.4641 -0.8858
-0.8858 0.4641
E =
7.9764 0 0
0 0.6142 0
V =
-0.5606 0.1382 -0.8165
-0.3913 0.8247 0.4082
-0.7298 -0.5484 0.4082
please help me to implement algorithm with using tensorial approximation reconstructs original matrix in matlab languages,how can i apply tensorial product?like this
X=kron(U(:,1),V(:,1));
or?thanks in advance
I'm not quite sure about the Tensorial interpretation but the closest rank-1 approximation to the matrix is essentially the outer-product of the two dominant singular vectors amplified by the singular value.
In simple words, if [U E V] = svd(X), then the closest rank-1 approximation to X is the outer-product of the first singular vectors multiplied by the first singular value.
In MATLAB, you could do this as:
U(:,1)*E(1,1)*V(:,1)'
Which yields:
ans =
2.0752 1.4487 2.7017
3.9606 2.7649 5.1563
Also, mathematically speaking, the kronecker product of a row vector and a column vector is essentially their outer product. So, you could do the same thing using Kronecker products as:
(kron(U(:,1)',V(:,1))*E(1,1))'
Which yields the same answer.

Find a matrix that gives same result when multiplied by a constant or another matrix

I have got a problem like A*x=lambda*x, where A is of order d*d, x is of order d*c and lambda is a constant. A and lambda are known and the matrix x is unknown.
Is there any way to solve this problem in matlab?? (Like eigen values but x is a d*c matrix instead of being a vector).
If I've understood you correctly, there will not necessarily be any solutions for x. If A*x=lambda*x, then any column y of x satisfies A*y=lambda*y, so the columns of x are simply eigenvectors of A corresponding to the eigenvalue lambda, and there will only be any solutions if lambda is in fact an eigenvalue.
From the documentation:
[V,D] = eig(A) produces matrices of eigenvalues (D) and eigenvectors
(V) of matrix A, so that A*V = V*D. Matrix D is the canonical form of
A — a diagonal matrix with A's eigenvalues on the main diagonal.
Matrix V is the modal matrix — its columns are the eigenvectors of A.
You can use this to check if lambda is an eigenvalue, and find any corresponding eigenvectors.
You can transform this problem. Write x as vector by by using x(:) (has size d*c x 1). Then A can be rewritten to a d*c x d*c matrix which has c versions of A along the diagonal.
Now it's a simple eigenvalue problem.
Its actually trivial. Your requirement is that A*X = lambda*X, where X is an array. Effectively, look at what happens for a single column of X. If An array X exists, then it is true that
A*X(:,i) = lambda*X(:,i)
And this must be true for the SAME value of lambda for all columns of X. Essentially, this means that X(:,i) is an eigenvector of A, with corresponding eigenvalue lambda. More importantly, it means that EVERY column of X has the same eigenvalue as every other column.
So a trivial solution to this problem is to simply have a matrix X with identical columns, as long as that column is an eigenvector of A. If an eigenvalue has multiplicity greater than one (therefore there are multiple eigenvectors with the same eigenvalue) then the columns of X may be any linear combination of those eigenvectors.
Try it in practice. I'll pick some simple matrix A.
>> A = [2 3;3 2];
>> [V,D] = eig(A)
V =
-0.70711 0.70711
0.70711 0.70711
D =
-1 0
0 5
The second column of V is an eigenvector, with eigenvalue of 5. We can arbitrarily scale an eigenvector by any constant. So now pick the vector vec, and create a matrix with replicated columns.
>> vec = [1;1];
>> A*[vec,vec,vec]
ans =
5 5 5
5 5 5
This should surprise nobody.

Efficient low-rank appoximation in MATLAB

I'd like to compute a low-rank approximation to a matrix which is optimal under the Frobenius norm. The trivial way to do this is to compute the SVD decomposition of the matrix, set the smallest singular values to zero and compute the low-rank matrix by multiplying the factors. Is there a simple and more efficient way to do this in MATLAB?
If your matrix is sparse, use svds.
Assuming it is not sparse but it's large, you can use random projections for fast low-rank approximation.
From a tutorial:
An optimal low rank approximation can be easily computed using the SVD of A in O(mn^2
). Using random projections we show how to achieve an ”almost optimal” low rank pproximation in O(mn log(n)).
Matlab code from a blog:
clear
% preparing the problem
% trying to find a low approximation to A, an m x n matrix
% where m >= n
m = 1000;
n = 900;
%// first let's produce example A
A = rand(m,n);
%
% beginning of the algorithm designed to find alow rank matrix of A
% let us define that rank to be equal to k
k = 50;
% R is an m x l matrix drawn from a N(0,1)
% where l is such that l > c log(n)/ epsilon^2
%
l = 100;
% timing the random algorithm
trand =cputime;
R = randn(m,l);
B = 1/sqrt(l)* R' * A;
[a,s,b]=svd(B);
Ak = A*b(:,1:k)*b(:,1:k)';
trandend = cputime-trand;
% now timing the normal SVD algorithm
tsvd = cputime;
% doing it the normal SVD way
[U,S,V] = svd(A,0);
Aksvd= U(1:m,1:k)*S(1:k,1:k)*V(1:n,1:k)';
tsvdend = cputime -tsvd;
Also, remember the econ parameter of svd.
You can rapidly compute a low-rank approximation based on SVD, using the svds function.
[U,S,V] = svds(A,r); %# only first r singular values are computed
svds uses eigs to compute a subset of the singular values - it will be especially fast for large, sparse matrices. See the documentation; you can set tolerance and maximum number of iterations or choose to calculate small singular values instead of large.
I thought svds and eigs could be faster than svd and eig for dense matrices, but then I did some benchmarking. They are only faster for large matrices when sufficiently few values are requested:
n k svds svd eigs eig comment
10 1 4.6941e-03 8.8188e-05 2.8311e-03 7.1699e-05 random matrices
100 1 8.9591e-03 7.5931e-03 4.7711e-03 1.5964e-02 (uniform dist)
1000 1 3.6464e-01 1.8024e+00 3.9019e-02 3.4057e+00
2 1.7184e+00 1.8302e+00 2.3294e+00 3.4592e+00
3 1.4665e+00 1.8429e+00 2.3943e+00 3.5064e+00
4 1.5920e+00 1.8208e+00 1.0100e+00 3.4189e+00
4000 1 7.5255e+00 8.5846e+01 5.1709e-01 1.2287e+02
2 3.8368e+01 8.6006e+01 1.0966e+02 1.2243e+02
3 4.1639e+01 8.4399e+01 6.0963e+01 1.2297e+02
4 4.2523e+01 8.4211e+01 8.3964e+01 1.2251e+02
10 1 4.4501e-03 1.2028e-04 2.8001e-03 8.0108e-05 random pos. def.
100 1 3.0927e-02 7.1261e-03 1.7364e-02 1.2342e-02 (uniform dist)
1000 1 3.3647e+00 1.8096e+00 4.5111e-01 3.2644e+00
2 4.2939e+00 1.8379e+00 2.6098e+00 3.4405e+00
3 4.3249e+00 1.8245e+00 6.9845e-01 3.7606e+00
4 3.1962e+00 1.9782e+00 7.8082e-01 3.3626e+00
4000 1 1.4272e+02 8.5545e+01 1.1795e+01 1.4214e+02
2 1.7096e+02 8.4905e+01 1.0411e+02 1.4322e+02
3 2.7061e+02 8.5045e+01 4.6654e+01 1.4283e+02
4 1.7161e+02 8.5358e+01 3.0066e+01 1.4262e+02
With size-n square matrices, k singular/eigen values and runtimes in seconds. I used Steve Eddins' timeit file exchange function for benchmarking, which tries to account for overhead and runtime variations.
svds and eigs are faster if you want a few values from a very large matrix. It also depends on the properties of the matrix in question (edit svds should give you some idea why).