When I try to calculate a matrix inverse using Matlab's inv() operation:
A = rand(10,10);
b = rand(10,1);
C = inv(A);
D = C*b;
I get the following warning at the last row: INV is slow and inaccurate. Use A\b for INV(A)*b , and b/A for b*INV(A).
I can change the code above into:
A = rand(10,10);
b = rand(10,1);
C = inv(A);
D = A\b;
Now I don't get the warning, but I don't think this solution is better.
Note: I need to store both the inverse of matrix A as well as inv(A)*c. Also, in my real file the size of matrix A can be 5000 x 5000 or even bigger.
Are there any better solutions in terms of efficiency and accuracy or is the first method fine?
You should listen to Matlab and use the second option. inv(A)*b and A\b are computed with different algorithms under the hood, and \ is indeed more accurate.
The documentation for inv states:
In practice, it is seldom necessary to form the explicit inverse of a matrix. A frequent misuse of inv arises when solving the system of linear equations Ax = b. One way to solve this is with x = inv(A)*b. A better way, from both an execution time and numerical accuracy standpoint, is to use the matrix division operator x = A\b. This produces the solution using Gaussian elimination, without forming the inverse. See mldivide () for further information.
Some additional information:
If you are to calculate
Ax = b
For many different b's, but with a constant A, you might want to pre-factorize A. That is:
[L U P] = lu(A);
x = (U \ (L \ ( P * b)));
Don't know about other fields, but this occurs frequently in power system engineering at least.
If you absolutely need the inverse later on, then you have to compute it. If you can use the backslash operator (\) instead of an inverse again later on, I would stay away from the inverse and listen to MATLAB's suggestion. For numerical reasons, it is always better to use a slash operator when you can, so the second approach is better even though it is slower.
Related
I am calculating the solution of a constrained linear least-squares problem as follows:
lb = zeros(7,1);
ub = ones(7,1);
for i = 1:size(b,2)
x(:,i) = lsqlin(C,b(:,i),[],[],[],[],lb,ub);
end
where C is m x 7 and b is m x n. n is quite large leading to a slow computation time. Is there any way to speed up this procedure and get rid of the slow for loop. I am using lsqlin instead of pinv or \ because I need to constrain my solution to the boundaries of 0–1 (lb and ub).
The for loop is not necessarily the reason for any slowness – you're not pre-allocating and lsqlin is probably printing out a lot of stuff on each iteration. However, you may be able to speed this up by turning your C matrix into a sparse block diagonal matrix, C2, with n identical blocks (see here). This solves all n problems in one go. If the new C2 is not sparse you may use a lot more memory and the computation may take much longer than with the for loop.
n = size(b,2);
C2 = kron(speye(n),C);
b2 = b(:);
lb2 = repmat(lb,n,1); % or zeros(7*n,1);
ub2 = repmat(ub,n,1); % or ones(7*n,1);
opts = optimoptions(#lsqlin,'Algorithm','interior-point','Display','off');
x = lsqlin(C2,b2,[],[],[],[],lb2,ub2,[],opts);
Using optimoptions, I've specified the algorithm and set 'Display' to 'off' to make sure any outputs and warnings don't slow down the calculations.
On my machine this is 6–10 times faster than using a for loop (with proper pre-allocation and setting options). This approach assumes that the sparse C2 matrix with m*n*7 elements can fit in memory. If not, a for loop based approach will be the only option (other than writing your own specialized version of lsqlin or taking advantage any other spareness in the problem).
I have to solve in MATLAB a linear system of equations A*x=B where A is symmetric and its elements depend on the difference of the indices: Aij=f(i-j).
I use iterative solvers because the size of A is say 40000x40000. The iterative solvers require to determine the product A*x where x is the test solution. The evaluation of this product turns out to be a convolution and therefore can be done dy means of fast fourier transforms (cputime ~ Nlog(N) instead of N^2). I have the following questions to this problem:
is this convolution circular? Because if it is circular I think that I have to use a specific indexing for the new matrices to take the fft. Is that right?
I find difficult to program the routine for the fft because I cannot understand the indexing I should use. Is there any ready routine which I can use to evaluate by fft directly the product A*x and not the convolution? Actually, the matrix A is constructed of 3x3 blocks and is symmetric. A ready routine for the product A*x would be the best solution for me.
In case that there is no ready routine, could you give me an idea by example how I could construct this routine to evaluate a matrix-vector product by fft?
Thank you in advance,
Panos
Very good and interesting question! :)
For certain special matrix structures, the Ax = b problem can be solved very quickly.
Circulant matrices.
Matrices corresponding to cyclic convolution Ax = h*x (* - is convolution symbol) are diagonalized in
the Fourier domain, and can be solved by:
x = ifft(fft(b)./fft(h));
Triangular and banded.
Triangular matrices and diagonally-dominant banded matrices are solved
efficiently by sparse LU factorization:
[L,U] = lu(sparse(A)); x = U\(L\b);
Poisson problem.
If A is a finite difference approximation of the Laplacian, the problem is efficiently solved by multigrid methods (e.g., web search for "matlab multigrid").
Interesting question!
The convolution is not circular in your case, unless you impose additional conditions. For example, A(1,3) should equal A(2,1), etc.
You could do it with conv (retaining only the non-zero-padded part with option valid), which probably is also N*log(N). For example, let
A = [a b c d
e a b c
f e a b
g f e a];
Then A*x is the same as
conv(fliplr([g f e a b c d]),x,'valid').'
Or more generally, A*x is the same as
conv(fliplr([A(end,1:end-1) A(1,:)]),x,'valid').'
I'd like to add some comments on Pio_Koon's answer.
First of all, I wouldn't advise to follow the suggestion for triangular and banded matrices. The time taken by a call to Matlab's lu() procedure on a large sparse matrix massively overshadows any benefits gained by solving the linear system as x=U\(L\b).
Second, in the Poisson problem you end up with a circulant matrix, therefore you can solve it using the FFT as described. In this specific case, your convolution mask h is a Laplacian, i.e., h=[0 -0.25 0; -0.25 1 -0.25; 0 -0.25 0].
I wanna solve #n linear equations AX=bi(for #n b's) in Matlab which b changes in a loop and A is constant.
One way which is fast, is to compute the inverse of A before the loop and in the loop body just get X from inv(A)*b, but because the matrix A is singular, I get an awful answer!
Of course, the numerical solution A/b gives a good answer, but the point is that it takes a long time to compute #n different X's in #n loops.
What I want is a solution which can be both accurate and fast.
I actually think this is a good question, typos and issues of matrix singularity aside. There are a few good ways to handle this, and Tim Davis' factorize submission on MATLAB Central covers all the angles.
However, just for reference, let's do it on our own in native MATLAB, starting with the case where A is square. First, there are the two methods you suggested (inv and \,mldivide):
% inv, slow and inacurate
xinvsol = inv(A)*b;
norm(A*xinvsol - b ,'fro')
% mldivide, faster and accurate
xref = A\b;
norm(A*xref - b ,'fro')
But if like you said A does not change, just factorize A and solve for new b! Say A is symmetric positive definite:
L = chol(A,'lower'); % Cholesky factorization
% mldivide, much faster (not counting the chol factorization) and most accurate
xcholbs= L'\(L\b); %'
norm(A*xcholbs - b ,'fro')
% linsolve, fastest (omits checks for matrix configuration) and most accurate
sol1 = linsolve(L, b, struct('LT',true));
xcholsolv = linsolve(L, sol1, struct('LT',true,'TRANSA',true));
norm(A*xcholsolv - b ,'fro')
If A is not symmetric positive definite, then you'd use LU decomposition for a square matrix or QR otherwise. Again, you can do it all yourself, or you can just use Tim Davis' awesome factorize functions.
This is the equation I am trying to solve:
h = (X'*X)^-1*X'*y
where X is a matrix and y is a vector ((X'X)^-1 is the inverse of X-transpose times X). I have coded this in Matlab as:
h = (X'*X)\X'*y
which I believe is correct. The problem is that X is around 10000x10000, and trying to calculate that inverse is crashing Matlab on even the most powerful computer I can find (16 cores, 24GB RAM). Is there any way to split this up, or a library designed for doing such large inversions?
Thank you.
That looks like a pseudo inverse. Are you perhaps looking for just
h = X \ y;
I generated a random 10,000 by 10,000 matrix X and a random 10,000 by 1 vector y.
I just broke up my computation step by step. (Code shown below)
Computed the transpose and held it in matrix K
Then I computed Matrix A by multiplying K by X
Computed vector b by multiplying K by vector y
Lastly, I used the backslash operator on A and b to solve
I didn't have a problem with the computation. It took a while, but breaking up the operations into the smallest groups possible helped to prevent the computer from being overwhelmed. However, it could be the composition of the matrix that you are using (ie. Sparse, decimals, etc.).
X = randi(2000, [10000, 10000]);
y = randi(2000, 10000, 1);
K = X';
A = K*X;
b = K*y;
S = A\b;
If you have multiple machines at your disposal, and you can recast your problem into the form h = X\y as proposed by #Ben, then you could use distributed arrays. This demo shows how you can do that.
Jordan,
Your equation is exactly the definition for "Moore-Penrose Matrix Inverse".
Check: http://mathworld.wolfram.com/Moore-PenroseMatrixInverse.html
Directly using h = X \ y; should help.
Or check Matlab pinv(X)*y
Suppose I have an arbitrary transformation matrix A such as,
A =
0.9966 0.0007 -6.5625
0.0027 0.9938 1.0598
0 0 1.0000
And a set of points such that their x and y coordinates are represented by X and Y respectively.
And suppose,
[Xf Yf] = tformfwd(maketform('projective',A),X,Y);
Now,
[Xff Yff] = tformfwd(maketform('projective',inv(A)),Xf,Yf);
[Xfi Yfi] = tforminv(maketform('projective',A),Xf,Yf);
[Xff Yff] and [Xfi Yfi] seem to be exactly the same (and they should).
Is tforminv just there for convenience or am I missing something here?
I'll preface this by saying it is my best guess...
It's possible that tforminv may perform the transformation without actually forming the inverse matrix. For example, you can solve a system of linear equations Ax = b in two ways:
x = inv(A)*b;
x = A\b;
According to the documentation for inv, the second option (using the matrix division operator) can perform better "from both an execution time and numerical accuracy standpoint" since it "produces the solution using Gaussian elimination, without forming the inverse". tforminv may do something similar and thus show better overall behavior compared with passing the inverse matrix to tformfwd.
If you were so inclined, you could probably try a number of different transformation matrices and test the two approaches (tforminv or tformfwd and inv) to see how accurate the results are and how fast they are each computed.