If we consider a matrix R of size pxp. If we want to multiply A'RA where A is equal to (I+Givens rotation). Here I is an identity matrix and ' denotes the transpose operator.
We know that a Givens rotation is a sparse matrix written as:
To perform the multiplication A'RA in matlab, we can do this fast implementation:
%Fast implementation
ci = R(:,ik)*(cos(theta))+R(:,jk)*(sin(theta)); % R*A
cj = R(:,jk)*(cos(theta)) - R(:,ik)*(sin(theta));
R(:,ik) = ci;
R(:,jk) = cj;
ri = R(ik,:)*(cos(theta))+R(jk,:)*(sin(theta)); % A'*R*A
rj = R(jk,:)*(cos(theta)) - R(ik,:)*(sin(theta));
R(ik,:) = ri;
R(jk,:) = rj;
But I didn't understand how they wrote this Matlab code. In other terms, I am not understanding how this Matlab code apply the multiplication A'RA. Kindly, can someone help me to understand this code?
One possible source of confusion is that either the signs in the Givens rotation matrix, or the side on which we need to transpose, is wrong in your example. I'll assume the latter: I'll use the same A matrix as you defined, but transform with A*R*A' (changing the A to transpose is equivalent to taking the rotation angle with opposite sign).
The algorithm is relatively straightforward. For starters, as the comments in the code suggest, the transformation is performed in two steps:
Rnew = A * R * A' = A * (R * A')
First, we compute R*A'. For this, imagine the transformation matrix A = I + M with the Givens rotation matrix M. The formula which you showed basically says "Take a unit matrix, except for 2 specified dimensions in which you rotate by a given angle". Here's how the full A matrix looks like for a small problem (6d matrix, ik=2, jk=4, both in full and sparse form):
You can see that except for the (ik,jk) 2d subspace, this matrix is a unit matrix, leaving every other dimension intact. So the action of R*A' will result in R for every dimension except for columns ik and jk.
In these two columns the result of R*A' is the linear combination of R(:,ik) and R(:,jk) with these trigonometric coefficients:
[R*A'](:,ik) = R(:,ik)*cos(theta) + R(:,jk)*sin(theta)
[R*A'](:,jk) = -R(:,ik)*sin(theta) + R(:,jk)*cos(theta)
while the rest of the columns are left unchanged. If you look at the code you cited: this is exactly what it's doing. This is, by definition, what R*A' means with the A matrix shown above. All of this is the implication of that the A matrix is a unit matrix except for a 2d subspace.
The next step is then quite similar: using this new R*A' matrix we multiply from the left with A. Again, the effect along most of the dimensions (rows, this time) will be identity, but in rows ik and jk we again get a linear combination:
[A*[R*A']](ik,:) = cos(theta)*[R*A'](ik,:) + sin(theta)*[R*A'](jk,:)
[A*[R*A']](jk,:) = -sin(theta)*[R*A'](ik,:) + cos(theta)*[R*A'](jk,:)
By noting that the code overwrites the R matrix with R*A' after the first step, it's again clear that the same is performed in the "fast implementation" code.
Disclaimer: A' is the adjoint (conjugate transpose) in matlab, so you should use A.' to refer to the transpose. For complex matrices there's a huge difference, and people often forget to use the proper transpose when eventually encountering complex matrices.
Related
So I'm trying to implement an EM-Algorithm to train a Gaussian Class Conditional model for classifying data. I'm stuck in the M-step at the moment because I can't figure out how to calculate the covariance matrix.
The problem is I have a big data set and using a for loop to go through each point would be way to slow. I also can't use the covariance function cov(), because I need to use a mean which I calculated using this formula(mu symbol one)
Is there a way to adjust cov() to use the mean I want? Or is there another way I could do this without for loops?
Edit: Forgot to explain what the data matrix is like. Its an nx3 where each row is a data point.
It technically needs to work for the general case nxm but n is usually really big(1000 or more) while m is relatively small.
You can calculate your covariance matrix manually. Let data be the matrix containing all your variables (for example, [x y]) and mu your custom mean, proceed as follows:
n = size(data,1);
data_dem = data - (ones(n,1) * mu);
cov_mat = (data_dem.' * data_dem) ./ (n - 1);
Notice that I used the Bessel's Correction (n-1 instead of n) because the Matlab cov function uses it, unless you specify the third argument as 1:
cov_mat = cov(x,y,1);
C = cov(___,w) specifies the normalization weight for any of the
previous syntaxes. When w = 0 (default), C is normalized by the number
of observations-1. When w = 1, it is normalized by the number of
observations.
I am looking to find 'half' of an affine transformation matrix using MATLAB. Yes I understand, 'half' a matrix isn't really correct, but exactly what I'm looking for was actually explained very well here: stackexchange mathematics
So I'm looking for an affine transformation matrix (B) which when applied twice to my image, will give the same result as when applying my initial matrix (A) once.
Reflection will not be part of A, otherwise it would be impossible to find B.
My initial matrix (A) is calculated using A = estimateGeometricTransform(movingPoints,fixedPoints,'affine'), which gives me an affine2d object.
If there is no way to find the 'half' matrix from the initial matrix, maybe the arrays of matched points can be manipulated in a way to find B from them.
Cheers
I think there is a possibility to find the half matrix that you speak of. It is called the matrix square root. Suppose you have the matrixa A. In Matlab you can just do B=sqrtm(A), where the m stands for matrix. Then you get a matrix B, where norm(B*B - A) is very small, if the matrix A was well behaved.
If I understand correctly you want to have half of an affine transformation aff = #(x) A*x + b. This can be done using homogenious coordinates. Every transformation aff can be represented by a the matrix
M = [A b; zeros(1,length(b)) 1], where
normalize = #(y) y(1:end-1)/y(end);
affhom = #(x) normalize(M*[x; 1]);
Note that aff and affhom do exactly the same thing. Here we can use what I was talking about earlier. Half of affhom can be represented using
affhomhalf = #(x) normalize(sqrtm(M)*[x; 1])
where
affhomhalf(affhomhalf(y)) - aff(y)
is small for all y, if A and b were well behaved.
I'm not sure about this, but I think you can even decompose sqrtm(M) into an linear and translatory part.
I have a matrix M of size NxP. Every P columns are orthogonal (M is a basis). I also have a vector V of size N.
My objective is to transform the first vector of M into V and to update the others in order to conservate their orthogonality. I know that the origins of V and M are the same, so it is basically a rotation from a certain angle. I assume we can find a matrix T such that T*M = M'. However, I can't figure out the details of how to do it (with MATLAB).
Also, I know there might be an infinite number of transforms doing that, but I'd like to get the simplest one (in which others vectors of M approximately remain the same, i.e no rotation around the first vector).
A small picture to illustrate. In my actual case, N and P can be large integers (not necessarily 3):
Thanks in advance for your help!
[EDIT] Alternative solution to Gram-Schmidt (accepted answer)
I managed to get a correct solution by retrieving a rotation matrix R by solving an optimization problem minimizing the 2-norm between M and R*M, under the constraints:
V is orthogonal to R*M[1] ... R*M[P-1] (i.e V'*(R*M[i]) = 0)
R*M[0] = V
Due to the solver constraints, I couldn't indicate that R*M[0] ... R*M[P-1] are all pairwise orthogonal (i.e (R*M)' * (R*M) = I).
Luckily, it seems that with this problem and with my solver (CVX using SDPT3), the resulting R*M[0] ... R*M[P-1] are also pairwise orthogonal.
I believe you want to use the Gram-Schmidt process here, which finds an orthogonal basis for a set of vectors. If V is not orthogonal to M[0], you can simply change M[0] to V and run Gram-Schmidt, to arrive at an orthogonal basis. If it is orthogonal to M[0], instead change another, non-orthogonal vector such as M[1] to V and swap the columns to make it first.
Mind you, the vector V needs to be in the column space of M, or you will always have a different basis than you had before.
Matlab doesn't have a built-in Gram-Schmidt command, although you can use the qr command to get an orthogonal basis. However, this won't work if you need V to be one of the vectors.
Option # 1 : if you have some vector and after some changes you want to rotate matrix to restore its orthogonality then, I believe, this method should work for you in Matlab
http://www.mathworks.com/help/symbolic/mupad_ref/numeric-rotationmatrix.html
(edit by another user: above link is broken, possible redirect: Matrix Rotations and Transformations)
If it does not, then ...
Option # 2 : I did not do this in Matlab but a part of another task was to find Eigenvalues and Eigenvectors of the matrix. To achieve this I used SVD. Part of SVD algorithm was Jacobi Rotation. It says to rotate the matrix until it is almost diagonalizable with some precision and invertible.
https://math.stackexchange.com/questions/222171/what-is-the-difference-between-diagonalization-and-orthogonal-diagonalization
Approximate algorithm of Jacobi rotation in your case should be similar to this one. I may be wrong at some point so you will need to double check this in relevant docs :
1) change values in existing vector
2) compute angle between actual and new vector
3) create rotation matrix and ...
put Cosine(angle) to diagonal of rotation matrix
put Sin(angle) to the top left corner of the matric
put minus -Sin(angle) to the right bottom corner of the matrix
4) multiple vector or matrix of vectors by rotation matrix in a loop until your vector matrix is invertible and diagonalizable, ability to invert can be calculated by determinant (check for singularity) and orthogonality (matrix is diagonalized) can be tested with this check - if Max value in LU matrix is less then some constant then stop rotation, at this point new matrix should contain only orthogonal vectors.
Unfortunately, I am not able to find exact pseudo code that I was referring to in the past but these links may help you to understand Jacobi Rotation :
http://www.physik.uni-freiburg.de/~severin/fulltext.pdf
http://web.stanford.edu/class/cme335/lecture7.pdf
https://www.nada.kth.se/utbildning/grukth/exjobb/rapportlistor/2003/rapporter03/maleko_mercy_03003.pdf
What's the best way to do the following (in Matlab) if I have two matrices A and B, let'say both of size m-by-n:
C = zeros(m,m);
for t=1:n
C=C+A(:,t)*B(:,t)';
end
This is nothing more than
C = A*B';
where A and B are each m-by-n. I'm not sure that you're going to get more efficient than that unless the matrices have special properties.
One place where you might get a benefit from using bsxfun for matrix multiplication is when the dimensions are sufficiently large (probably 100-by-100 or more) and one matrix is diagonal, e.g.:
A = rand(1e2);
B = diag(rand(1,1e2));
C = bsxfun(#times,A,diag(B).');
This occurs in many matrix transforms – see the code for sqrtm for example (edit sqrtm).
Here is my testing function:
function diff = svdtester()
y = rand(500,20);
[U,S,V] = svd(y);
%{
y = sprand(500,20,.1);
[U,S,V] = svds(y);
%}
diff_mat = y - U*S*V';
diff = mean(abs(diff_mat(:)));
end
There are two very similar parts: one finds the SVD of a random matrix, the other finds the SVD of a random sparse matrix. Regardless of which one you choose to comment (right now the second one is commented-out), we compute the difference between the original matrix and the product of its SVD components and return that average absolute difference.
When using rand/svd, the typical return (mean error) value is around 8.8e-16, basically zero. When using sprand/svds, the typical return values is around 0.07, which is fairly terrible considering the sparse matrix is 90% 0's to start with.
Am I misunderstanding how SVD should work for sparse matrices, or is something wrong with these functions?
Yes, the behavior of svds is a little bit different from svd. According to MATLAB's documentation:
[U,S,V] = svds(A,...) returns three output arguments, and if A is m-by-n:
U is m-by-k with orthonormal columns
S is k-by-k diagonal
V is n-by-k with orthonormal columns
U*S*V' is the closest rank k approximation to A
In fact, usually k will be somethings about 6, so you will get rather "rude" approximation. To get more exact approximation specify k to be min(size(y)):
[U, S, V] = svds(y, min(size(y)))
and you will get error of the same order of magnitude as in case of svd.
P.S. Also, MATLAB's documentations says:
Note svds is best used to find a few singular values of a large, sparse matrix. To find all the singular values of such a matrix, svd(full(A)) will usually perform better than svds(A,min(size(A))).