So i'm trying to solve a system of linear equations using LU decomposition. I made a code in matlab that i'm comparing with the output of matlab's linsolve. The thing that's confussing me is this:
As far as i understand (from what i learned from this site: http://www.cs.mtu.edu/~shene/COURSES/cs3621/NOTES/INT-APP/CURVE-linear-system.html) Lu decomposition works by decomposing A into L(lower triangular) and U (upper triangular). And then it calculates x solving two equations:
B = LY;
Y = UX;
So my confussion comes here.
If i do x_solutions=linsolve(A,B), i get a different result that if i do x=linsolve(U,y) (of course doing first y = linsolve(L,B)).
Does someone know why this happens? shouldn't x be equal to x_solutions in this case or am i missing something?
Just for the sake of giving away all the information this is how i'm doing it:
A=[1 2 6; 1 2 2; 2 2 1];
B=[1 0 1]';
G=linsolve(A,B);
UPP = triu(A);
LOW= tril(A);
y=linsolve(LOW,B);
x=linsolve(UPP,y);
Thank you in advance!
triu and tril do not give you the L and U in LU decomposition.
They just give entries lower/upper part of matrix, i.e.,
A == LOW + UPP - diag(A)
diag(A) is subtracted because both LOW and UPP have diagonal part of A
In LU decomposition, L and U should satisfy
A == L*U
If you want to get such L and U, use
[L,U] = lu(A);
Related
How can I implement the function lu(A) in MATLAB so that L*U is directly A and I also get the real L matrix?
When I use [L,U] = lu(A), MATLAB doesn't give me the right L matrix. When I use [L,U,P] = lu(A), I need to implement P*A = L*U, but I only want to multiply L*U to receive A.
MATLAB's lu always performs pivoting by default. If you had for example a diagonal coefficient that was equal to 0 when you tried to do the conventional LU decomposition algorithm, it will not work as the diagonal coefficients are required when performing the Gaussian elimination to create the upper triangular matrix U so you would get a divide by zero error. Pivoting is required to ensure that the decomposition is stable.
However, if you can guarantee that the diagonal coefficients of your matrix are non-zero, it is very simple but you will have to write this on your own. All you have to do is perform Gaussian elimination on the matrix and reduce the matrix into reduced echelon form. The result reduced echelon form matrix is U while the coefficients required to remove the lower triangular part of L in Gaussian elimination would be placed in the lower triangular half to make U.
Something like this could work, assuming your matrix is stored in A. Remember that I'm assuming a square matrix here. The implementation of the non-pivoting LU decomposition algorithm is placed in a MATLAB function file called lu_nopivot:
function [L, U] = lu_nopivot(A)
n = size(A, 1); % Obtain number of rows (should equal number of columns)
L = eye(n); % Start L off as identity and populate the lower triangular half slowly
for k = 1 : n
% For each row k, access columns from k+1 to the end and divide by
% the diagonal coefficient at A(k ,k)
L(k + 1 : n, k) = A(k + 1 : n, k) / A(k, k);
% For each row k+1 to the end, perform Gaussian elimination
% In the end, A will contain U
for l = k + 1 : n
A(l, :) = A(l, :) - L(l, k) * A(k, :);
end
end
U = A;
end
As a running example, suppose we have the following 3 x 3 matrix:
>> rng(123)
>> A = randi(10, 3, 3)
A =
7 6 10
3 8 7
3 5 5
Running the algorithm gives us:
>> [L,U] = lu_nopivot(A)
L =
1.0000 0 0
0.4286 1.0000 0
0.4286 0.4474 1.0000
U =
7.0000 6.0000 10.0000
0 5.4286 2.7143
0 0 -0.5000
Multiplying L and U together gives:
>> L*U
ans =
7 6 10
3 8 7
3 5 5
... which is the original matrix A.
You could use this hack (though as already mentioned, you might lose numerical stability):
[L, U] = lu(sparse(A), 0)
You might want to consider doing LDU decomposition instead of unpivoted LU. See, LU without pivoting is numerically unstable - even for matrices that are full rank and invertible. The simple algorithm provided above shows why - there is division by each diagonal element of the matrix involved. Thus, if there is a zero anywhere on the diagonal, decomposition fails, even though the matrix could still be non-singular.
Wikipedia talks a little about LDU decomposition here:
https://en.wikipedia.org/wiki/LU_decomposition#LDU_decomposition
without citing an algorithm. It cites the following textbook for proof of existence:
Horn, Roger A.; Johnson, Charles R. (1985), Matrix Analysis, Cambridge University Press, ISBN 978-0-521-38632-6. See Section 3.5.
LDU is guaranteed to exist (at least for an invertible matrix), it is numerically stable, and it is also unique (provided that both L and U are constrained to have unit elements on the diagonal).
Then, if for any reason "D" gets in your way, you can absorb the diagonal matrix D into either L (L:=LD) or U (U:=DU), or split it symmetrically between L and U (such as L:=L*sqrt(D) and U:=sqrt(D)*U), or however you want to do it. There is an infinite number of ways to split LDU into LU, and this is why LU decomposition is not unique.
I have an unconstrained quadratic optimization problem- I have to find u that will minimize norm (u^H * A_k *u - d_k)) where A_k is 4x4 matrix (k=1,2...180) and u is 4x1 vector(H denotes hermitian).
There are several functions in MATLAB to solve optimization problems but I am not able to figure out the method I need to use for my OP. I would highly appreciate if anyone provides me some hint or suggestion to solve this problem in MATLAB.
Math preliminaries:
If A is symmetric, positive definite, then the the matrix A defines a norm. This norm, called the A norm, is written ||x||_A = x'Ax. Your problem, minimize (over x) |x'*A*x - d| is equivalent to searching for a vector x whose 'A norm' is equal to d. Such a vector is trivial to find by simply scaling up or down any non-zero vector until it has the appropriate magnitude.
Pick arbitrary y (can't be all zero). eg. y = [1; zeros(n-1, 1)]; or y = rand(n, 1);
Solve for some scalar c such that x = c*y and x'Ax=d. Substituting we get c^2 y'Ay=d hence:.
Trivial procedure to find an answer:
y = [1; zeros(n-1)]; %Any arbitrary y will do as long as its not all zero
c = sqrt(d / (y'*A*y))
x = c * y`
x now solves your problem. No toolbox required!
options = optimoptions('fminunc','GradObj','on','TolFun',1e-9,'TolX',1e-9);
x = fminunc(#(x)qfun(x,A,d),zeros(4,1),options);
with
function [y,g]=qfun(x,A,d)
y=(x'*A*x-d);
g=2*y*(A+A')*x;
y=y*y;
Seems to work.
according this article
http://www.wseas.us/e-library/conferences/2012/Vouliagmeni/MMAS/MMAS-07.pdf
matrix can be approximated by one rank matrices using tensorial approximation,i know that in matlab kronecker product plays same role as tensorial product,function is kron,now let us suppose that we have following matrix
a=[2 1 3;4 3 5]
a =
2 1 3
4 3 5
SVD of this matrix is
[U E V]=svd(a)
U =
-0.4641 -0.8858
-0.8858 0.4641
E =
7.9764 0 0
0 0.6142 0
V =
-0.5606 0.1382 -0.8165
-0.3913 0.8247 0.4082
-0.7298 -0.5484 0.4082
please help me to implement algorithm with using tensorial approximation reconstructs original matrix in matlab languages,how can i apply tensorial product?like this
X=kron(U(:,1),V(:,1));
or?thanks in advance
I'm not quite sure about the Tensorial interpretation but the closest rank-1 approximation to the matrix is essentially the outer-product of the two dominant singular vectors amplified by the singular value.
In simple words, if [U E V] = svd(X), then the closest rank-1 approximation to X is the outer-product of the first singular vectors multiplied by the first singular value.
In MATLAB, you could do this as:
U(:,1)*E(1,1)*V(:,1)'
Which yields:
ans =
2.0752 1.4487 2.7017
3.9606 2.7649 5.1563
Also, mathematically speaking, the kronecker product of a row vector and a column vector is essentially their outer product. So, you could do the same thing using Kronecker products as:
(kron(U(:,1)',V(:,1))*E(1,1))'
Which yields the same answer.
I follow the formula on wiki:
http://en.wikipedia.org/wiki/Pseudo_inverse
to compute pseudoinverse but i can not receive the right result. for example:
I want to find theta of the equation: Y=R*theta, i write on matlab:
R = -[1/sqrt(2) 1 1/sqrt(2) 0;0 1/sqrt(2) 1 1/sqrt(2);-1/sqrt(2) 0 1/sqrt(2) 1];
% R is 3x4 matrix
Y = [0; -1/sqrt(2);-1]; %Y is 3x1 matrix
B1 = pinv(R);
theta1 = B1*Y;
result1 = R*theta1 - Y
B2 = R'*inv(R*R');
theta2 = B2*Y;
result2 = R*theta2 - Y
and this is the result:
result1 =
1.0e-15 *
-0.1110
-0.2220
-0.2220
Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 1.904842e-17.
> In pseudoinverse at 14
result2 =
0.1036
-0.1768
-0.3536
Cleary, theta2 is the wrong answer, but i don't know how and why. I read many book and they give me the same formula as wiki.
Can anybody help me to do pseudo inverse by hand ? thanks !
The algebra tells you that a pseudo-inverse can be used to solve such equations, but the algebra isn't accounting for finite precision computation.
In fact computation of a pseudo-inverse using the matrix multiplication method is not suitable because it is numerically unstable. Use the \ operator for matrix division, as in
theta = R \ Y;
Algebraically, matrix division is the same as multiplication by pseudo-inverse. But MATLAB's implementation is far more stable.
For more information, including stable methods, see
Moore–Penrose pseudoinverse on Wikipedia
As Ben has already said, matrix inversion is numerically unstable. The function inv is not recommended unless you want to have the actual inversion of a matrix, see for example this link. The misuse of inv is the mistake a new student of numerical linear algebra most often makes.
In most linear algebra computations, you can circumvent inv by using a numerically-stable algorithm. For example, we have LU factorization for linear solvers, and QR or SVD method for ordinary least squares.
You're not calculating B1 correctly. In your case
B1 = inv(R'*R)*R';
Because R is leading (traditionally it is the other way around). However, that doesn't solve your singularity problem.
pinv used SVD to calculate the pseudo-inverse, which you can read about here.
So basically it does in a more elegant fashion:
[U,S,V] = svd(R);
S(abs(S)<(sum(sum(S))*1e-8)) = 0; % removes near-singular values.
Stemp = 1./S;
Stemp(isinf(Stemp)) = 0; % This take the inverse of non-zero terms... I'm sure there is faster way
B1 = V * Stemp' * U';
And then you can continue on your way...
Given L and U LU decomposition and vector of constants b such that LU*x=b , is there any built in function which find the x ? Mean something like -
X = functionName(L,U,b)
Note that in both L and U we are dealing with triangular matrices which can be solved directly by forward and backward substitution without using the Gaussian elimination process.
Edit :
Solving this linear equation system should be according to the following steps -
1. define y - s.t Ux=y
2. solve Ly=b by forward substitution
3. solve Ux=y by backward substitution
4. return y
Edit 2 :
I found linalg::matlinsolveLU but I didn't try it cause I have too old version (R2010a) . Is it working for anyone ?
If you have:
A = rand(3);
b = rand(3,1);
then the solution to the system can be simply computed as:
x = A\b
Or if you already have an LU decomposition of A, then:
[L,U] = lu(A);
xx = U\(L\b)
the mldivide function is smart enough to detect that the matrix is triangular and chose an algorithm accordingly (forward/backward substitution)
I think this is what you're looking for:
A = rand(3,3); % Random 3-by-3 matrix
b = rand(3,1); % Random 3-by-1 vector
[L,U] = lu(A); % LU decomposition
x = U\(L\b) % Solve system of equations via mldivide (same as x = A\b or x = (L*U)\b)
err = L*U*x-b % Numerical error
The system of equations is solved using mldivide. You might also look at qr which implements QR decomposition instead of using LU decomposition. qr can directly solve A*x = b type problems and is more efficient. Also look at linsolve. For symbolic systems you may still be able to use mldivide, or try linalg::matlinsolveLU in MuPAD.