I encountered some numerical questions when running simulation on MatLab. Here please find the questions:
I found that A*A' (a matrix times its transpose) is not guaranteed to be symmetric in MatLab. Can I know what is the reason? And because I will have A*C*A', where C is a symmetric matrix, and I would like to keep A*C*A' as symmetric. Is there any method to fix the numerical difference created by the transpose operation?
I implemented a for loop in Matlab to compute a set of matrices. Small numerical difference (around 10^(-10)) in each round accumulates to the next run, and it finally diverges after around 30 rounds. Is there any method to fix small error in each run and do not affect the result at the same time.
Thank you for reading my questions!
"I found that A*A' (a matrix times its transpose) is not guaranteed to be symmetric in MatLab."
I would dispute that statement as written. The MATLAB parser is smart enough to recognize that the operands of A*A' are the same and call a symmetric BLAS routine in the background to do the work, and then manually copy one triangle into the other resulting in an exactly symmetric result. Where one usually gets into trouble is by coding something that the parser cannot recognize. E.g.,
A = whatever;
B = whatever;
X = A + B;
(A+B) * (A+B)' <-- MATLAB parser will call generic BLAS routine
X * X' <-- MATLAB parser will call symmetric BLAS routine
In the first matrix multiply above, the MATLAB parser may not be not smart enough to recognize the symmetry so a generic matrix multiply BLAS routine (e.g., dgemm) could be called to do the work and the result is not guaranteed to be exactly symmetric. But in the second matrix multiply above the MATLAB parser does recognize the symmetry and calls a symmetric BLAS matrix multiply routine.
For the ACA' case, I don't know of any method to force MATLAB to generate an exact symmetric result. You could manually copy one resulting triangle into the other after the fact. I suppose you could also factor C into two parts X*X' and then regroup but that seems like too much work for what you are trying to do.
Related
I have the following problem. I have a N x N real matrix called Z(x; t), where x and t might be vectors in general. I have N_s observations (x_k, Z_k), k=1,..., N_s and I'd like to find the vector of parameters t that better approximates the data in the least square sense, which means I want t that minimizes
S(t) = \sum_{k=1}^{N_s} \sum_{i=1}^{N} \sum_{j=1}^N (Z_{k, i j} - Z(x_k; t))^2
This is in general a non-linear fitting of a matrix function. I'm only finding examples in which one has to fit scalar functions which are not immediately generalizable to a matrix function (nor a vector function). I tried using the scipy.optimize.leastsq function, the package symfit and lmfit, but still I don't manage to find a solution. Eventually, I'm ending up writing my own code...any help is appreciated!
You can do curve-fitting with multi-dimensional data. As far as I am aware, none of the low-level algorithms explicitly support multidimensional data, but they do minimize a one-dimensional array in the least-squares sense. And the fitting methods do not really care about the "independent variable(s)" x except in that they help you calculate the array to be minimized - perhaps to calculate a model function to match to y data.
That is to say: if you can write a function that would take the parameter values and calculate the matrix to be minimized, just flatten that 2-d (on n-d) array to one dimension. The fit will not mind.
Solve Ax = b. Real double. A is overdetermined Mx2 with M >> 2. b is Mx1. I've run a ton of data against mldivide, and the results are excellent. I wrote a mex routine with MKL LAPACKE_dgels and it's nowhere near as good. The results have a ton of noise and the underlying signal is barely there. I checked the routine against the MKL example results first. I've searched through the mldivide doc (flowchart) and the SO questions. All I found is Matlab uses QR factorization for overdetermined rectangular.
What should I try next? Am I using the wrong LAPACK routine? Please help guide me in the right direction.
Update:
To within E-15 floating point difference on the solution vector, Intel MKL LAPACKE_dgels has the same result as Matlab mldivide for real double overdetermined (rectangular) problems. As far as I can tell, this is the QR method used.
Beware the residuals returned from this dgels. They do not equate to b - Ax. Many of them are close to this value, whereas some are far from it.
The problem was not the solution x, rather the returned residuals from DGELS. This routine's outputs are modify-in-place on the input array pointers. The MKL doc says the input array b is overwritten with the output vector x for the first N rows, then the residuals in N+1 to M. I confirmed this with my code.
The mistake was in aligning the b[N+1] residuals to original inputs b[1], and making further algorithmic decisions on that. The correct alignment of residual to original input is b[1] to b[1]. The first N residuals are not available; you have to compute those afterwards.
The doc doesn't say they are residuals per se, rather specifically
the residual sum of squares for the solution in each column is given by the sum of squares of modulus of elements n+1 to m in that column.
I'm trying to compute an inverse of a matrix P, but if I multiply inv(P)*P, the MATLAB does not return the identity matrix. It's almost the identity (non diagonal values in the order of 10^(-12)). However, in my application I need more precision.
What can I do in this situation?
Only if you explicitly need the inverse of a matrix you use inv(), otherwise you just use the backslash operator \.
The documentation on inv() explicitly states:
x = A\b is computed differently than x = inv(A)*b and is recommended for solving systems of linear equations.
This is because the backslash operator, or mldivide() uses whatever method is most suited for your specific matrix:
x = A\B solves the system of linear equations A*x = B. The matrices A and B must have the same number of rows. MATLAB® displays a warning message if A is badly scaled or nearly singular, but performs the calculation regardless.
Just so you know what algorithm MATLAB chooses depending on your input matrices, here's the full algorithm flowchart as provided in their documentation
The versatility of mldivide in solving linear systems stems from its ability to take advantage of symmetries in the problem by dispatching to an appropriate solver. This approach aims to minimize computation time. The first distinction the function makes is between full (also called "dense") and sparse input arrays.
As a side-note about error of order of magnitude 10^(-12), besides the above mentioned inaccuracy of the inv() function, there's floating point accuracy. This post on MATLAB issues on it is rather insightful, with a more general computer science post on it here. Basically, if you are computing numerics, don't worry (too much at least) about errors 12 orders of magnitude smaller.
You have what's called an ill-conditioned matrix. It's risky to try to take the inverse of such a matrix. In general, taking the inverse of anything but the smallest matrices (such as those you see in an introduction to linear algebra textbook) is risky. If you must, you could try taking the Moore-Penrose pseudoinverse (see Wikipedia), but even that is not foolproof.
I am trying to use preconditioned conjugate gradient in matlab to speed things up. I am using this iterative method because backslash operator was too time consuming. However, I am having some issues. For e.g. I want to solve this system of linear equations given by
Ax=B
where A is a sparse positive definite matrix and B is a matrix as well. In matlab I can do that simply by
x= A\B
However, if I use pcg function, then I would have to loop over all the columns of B and solve individual
x(:,i)=pcg(A,B(:,i))
This loop will take more time than x=A\B. If I consider just a single column as b instead of matrix B, then pcg works faster than the backslash operator. However, if I consider the whole matrix B, then pcg is slower than backslash operator. So there is no point of using pcg.
Any suggestions guys?
When using the method as suggested by Mattj, it shows the following error
Error using iterapp (line 60)
user supplied function ==>
#(x)reshape(repmat(A*x,1,nb),[],1)
failed with the following error:
Inner matrix dimensions must agree.
Error in pcg (line 161)
r = b -
iterapp('mtimes',afun,atype,afcnstr,x,varargin{:});
I think we need to see more data on your timing tests and the dimensions/sparsities of A and B, and better understand why pcg is faster than mldivide. However, you can implement what you're after this way,
[ma,na]=size(A);
[mb,nb]=size(B);
afun=#(x) reshape(A*reshape(x,na,[]),[],1);
X=pcg(afun,B(:));
X=reshape(X,na,nb);
However, if I consider the whole matrix B, then pcg is slower than
backslash operator. So there is no point of using pcg.
That does make a certain amount of sense. When backslash solves the first set of equations A*x=B(:,1), it can recycle pieces of its analysis to the later columns B(:,i), e.g., if it performs an LU decomposition of A.
Conversely, all the work that PCG applies to the different B(:,i) are independent. So, it very well might not make sense to use PCG. The one exception to this is if each B(:,i+1) is similar to B(:,i). In other words, if the columns of B change in a gradual continuous manner. If so, then you should run PCG in a loop as you've been doing, but use the i-th solution x(:,i) to initialize PCG in the next iteration of the loop. That will cut down on the total amount of work PCG must perform.
I have this huge matrix A of dimension 900000x900000. And I have to solve this linear equation
Ax=b where b is a column matrix of size 900000x1.
I used matlab's backslash operator like A\b to try to get x. However, it freezes and I couldn't get x. Mostly I get out of memory issue. Even though I ran it in a computer with higher memory it makes the system very slow and I have to wait to get the answer.
How can I solve this equation. My matrix is pretty sparse. However, it's band is wider but most of the elements are zero. b is a full matrix. Any suggestions?
I did a project, where we also operated with such large but fortunately very sparse matrices.
Using such large matrices, you are pretty lost with direct methods: You can never compute the inverse because it will be a dense matrix, which you can never store. Also methods such as LU or Cholesky factorization are quite expensive because they again create a significant fill-in, i.e. they destroy zeros.
A viable alternative is to use iterative methods. If you know that your matrix is symmetric and positive-definite, try the Conjugate gradient method:
x = pcg(A, b); %# Computes a solution to Ax = b, with A symm. pos-def.
I would just give it a try and have a look, if the method converges. Proofing the assumption of positive-definiteness is not easy, I'm afraid.
If you do not get a solution, there are many more iterative methods. For example:
bicg - BiConjugate Gradient Method
bicgstab - BiConjugate Gradient Method (stabilized)
lsqr - Least Squares QR Method
gmres - Generalized Minimum Residual Method (I like this a lot)