I have 37 linear equations and 36 variables in the form of a matrix equation; A*X=B . The equations don't have an exact answer. I want to use Matlab least square method to find the answers with the least error. I am new to Matlab so any comments will help. Thank you
If A is of full rank, i.e. the columns of A are linearly independent, the least-squares solution of an overdetermined system of linear equations
A * x = b
can be found by inverting the normal equations (see Linear Least Squares):
x = inv(A' * A) * A' * b
If A is not of full rank, A' * A is not invertible. Instead, one can use the pseudoinverse of A
x = pinv(A) * b
or Matlab's left-division operator
x = A \ b
Both give the same solution, but the left division is more computationally efficient.
The two latter computation methods can also deal with underdetermined systems of linear equations, but they give different solutions in that case: The pseudoinverse gives the solution where x has the smallest sum of squares, while the left-division operator gives a solution with as many 0 coefficients as possible.
The most general way to solve this is to use the pseudoinverse:
X = pinv(A) * B;
You can calculate x by:
x = (A'*A)\A'*B
Related
I have a matrix that is a function of some parameter A=A(x). I would like to find the points x where this matrix becomes singular. Example (I have a large matrix though):
syms x
A=[x sin(x); cos(x^2) 2.5];
So far I have been symbolically computing the determinant of the matrix and then used fzero or newtzero to find the roots of that characteristic equation. I.e.
detA = det(A);
fzero(matlabFunction(detA),startingGuess)
Then I found this: How to find out if a matrix is singular?, where it is advocated to not use the determinant under any circumstances.
Indeed the symbolic determinant calculation is terribly slow. However I tried to use rank(A) instead as suggested in the link and it does not seem to work for symbolic matrices.
Is there any way to implement the suggestions in the link for finding the roots of a characteristic equation of a matrix that is given symbolically?
A possible approach would be the following: a square matrix A is singular if and only if the homogeneous linear (with respect to the vector y) system A*y = 0 has nontrivial solutions y <> 0 (which is equivalent to det(A) = 0 and rank(A) = 0 among others. So a more or less standard, as I recall from the past, technique to compute such points x is to solve the nonlinear system
A(x)*y = 0 (1)
||y|| = 1 (2)
This way you can compute a point x* and a vector y* such that A(x*) is singular and y* is an eigenvector corresponding to the zero eigenvalue of A(x*).
If I remember correctly, you can also solve the somewhat easier system
A(x)*y = 0 (1)
<y,c> = 1 (2a)
where c is "almost" any nonzero random vector (normalize it to 1 to avoid numerical problems).
As a matter of fact there is an enormous bibliography on the subject - you can look for saddle-node bifurcation computations (in case A(x) is the Jacobian of a vector field), or for "distance to instability".
From a discussion with Ander Biguri it seems that the determinant is actually a perfectly fine method of approaching this problem. The problem seems to be to solve the final equation in a stable manner, which would be a different question.
I have an equation of the form A*X = X*A', where A and X are real, square matrices (3x3 in that case) and A is known and A' represent the transpose of A. How to solve for X using MATLAB? (up to a scale factor)
This is a Sylvester equation. However, it is singular because the eigenvalues of A and A' are the same. But you can use the formulae
[I⊗A-A'⊗I]X(:)=C(:):
m=kron(eye(3),a)+kron(-a,eye(3))
v=null(m)
x1=reshape(v(:,1),[3 3])
x2=reshape(v(:,2),[3 3])
x3=reshape(v(:,3),[3 3])
Now the solution is span{x1,x2,x2}, i.e. any matrix of the form
b x1 + c x2 +d x3, where b,c,d are any real numbers
I don't think Matlab has facilities for symbolic algebra.
If you expand A and X, and work through the expression, you obtain an 3x3 matrix with equation in several unknowns, all of which are zero. You then solve that.
But I don't think Matlab allows you to set a matrix to a symbol, rather than an value and expand it for you. For this simple case, you could easily write such a function, that multiples a string matrix by a numerical matrix. The snag is it's hard to scale it up to the general case without throwing the entire Maple / Mathematica engine at it.
I wanna to try calculate multiply of three matrix in matlab.
The formation of matrices described below:
L = D^(-1/2) * A * D^(-1/2);
D, A and L are a n*n matrices. A and L are not diagonal or sparse but D is diagonal. In this case n = 16900. When I calculate L in matlab, it takes a long time, about 4 hours!
My question is: Is there a more efficient way to calculate L?
You can use bsxfun twice. I'm not sure it will be faster or not:
v = diag(D).^(-1/2); %// this is the same as diag(D.^(-1/2))
L = bsxfun(#times, v.', bsxfun(#times, A, v));
Instead of using naive matrix multiplication, you can specialised asymptotically faster ones. Strassen's algorithm comes to mind but if I recall correctly it typically has a high constant, despite it's better asymptotic complexity. If you have a very limited set of possible values in your matrices, you can use a variation of the "four Russians" method.
I have to solve in MATLAB a linear system of equations A*x=B where A is symmetric and its elements depend on the difference of the indices: Aij=f(i-j).
I use iterative solvers because the size of A is say 40000x40000. The iterative solvers require to determine the product A*x where x is the test solution. The evaluation of this product turns out to be a convolution and therefore can be done dy means of fast fourier transforms (cputime ~ Nlog(N) instead of N^2). I have the following questions to this problem:
is this convolution circular? Because if it is circular I think that I have to use a specific indexing for the new matrices to take the fft. Is that right?
I find difficult to program the routine for the fft because I cannot understand the indexing I should use. Is there any ready routine which I can use to evaluate by fft directly the product A*x and not the convolution? Actually, the matrix A is constructed of 3x3 blocks and is symmetric. A ready routine for the product A*x would be the best solution for me.
In case that there is no ready routine, could you give me an idea by example how I could construct this routine to evaluate a matrix-vector product by fft?
Thank you in advance,
Panos
Very good and interesting question! :)
For certain special matrix structures, the Ax = b problem can be solved very quickly.
Circulant matrices.
Matrices corresponding to cyclic convolution Ax = h*x (* - is convolution symbol) are diagonalized in
the Fourier domain, and can be solved by:
x = ifft(fft(b)./fft(h));
Triangular and banded.
Triangular matrices and diagonally-dominant banded matrices are solved
efficiently by sparse LU factorization:
[L,U] = lu(sparse(A)); x = U\(L\b);
Poisson problem.
If A is a finite difference approximation of the Laplacian, the problem is efficiently solved by multigrid methods (e.g., web search for "matlab multigrid").
Interesting question!
The convolution is not circular in your case, unless you impose additional conditions. For example, A(1,3) should equal A(2,1), etc.
You could do it with conv (retaining only the non-zero-padded part with option valid), which probably is also N*log(N). For example, let
A = [a b c d
e a b c
f e a b
g f e a];
Then A*x is the same as
conv(fliplr([g f e a b c d]),x,'valid').'
Or more generally, A*x is the same as
conv(fliplr([A(end,1:end-1) A(1,:)]),x,'valid').'
I'd like to add some comments on Pio_Koon's answer.
First of all, I wouldn't advise to follow the suggestion for triangular and banded matrices. The time taken by a call to Matlab's lu() procedure on a large sparse matrix massively overshadows any benefits gained by solving the linear system as x=U\(L\b).
Second, in the Poisson problem you end up with a circulant matrix, therefore you can solve it using the FFT as described. In this specific case, your convolution mask h is a Laplacian, i.e., h=[0 -0.25 0; -0.25 1 -0.25; 0 -0.25 0].
The problem i am trying to understand is easy but i cant seem to get the correct result in matlab. The actual problem is that i want to get the weight vectors of a 2 hidden layer input RBF using just the plain distance as a function, i.e. no Baysian or Gaussian function as my φ. I will use the function with 2 centres let's say 0,0 and 1,1. So this will give me a Matrix φ of:
[0 sqrt(2) ; 1 1; 1 1; sqrt(2) 0] *[w1; w2] = [0;1;1;0] As defined my the XOR function.
When i apply the pseudoinverse of the Φ in matlab * [0;1;1;0] though i get [0.33 ; 0.33] which is not the correct value which would allow me to get the correct output values [0;1;1;0].
i.e. .33 * sqrt(2) != 0 .
Can someone explain to me why this is the case?
I'll take a swag at this. The matrix, I'll call A, A = [0 sqrt(2) ; 1 1; 1 1; sqrt(2) 0] has full column rank, but not full row rank, i.e. rank(A) = 2. Then you essentially solve the system Ax = b, where x is your weighting vector. You could also just do x = A\b in Matlab, which is supposedly a much more accurate answer. I get the same answer as you. This is a very rough explanation, when your system can not be solved for a certain solution vector, it means that there exists no such vector x that can be solved for Ax = b. What Matlab does is try to estimate the answer as close as possible. I'm guessing you used pinv, if you look at the Matlab help it says:
If A has more rows than columns and is not of full rank, then the overdetermined least squares problem
minimize norm(A*x-b)
does not have a unique solution. Two of the infinitely many solutions are
x = pinv(A)*b
and
y = A\b
So, this appears to be your problem. I would recommend looking at your φ matrix if possible to come up with a more robust system. Hope this is useful.