How can I generate a random, invertible, symmetric, positive semidefinite matrix using MATLAB?
I found this Python code:
matrixSize = 10
A = random.rand(matrixSize,matrixSize)
B = numpy.dot(A,A.transpose())
But I am not sure if this generates random positive semi-define matrix B.
The MATLAB equivalent of your code is:
matrixSize = 10;
A = rand(matrixSize);
B = A * A.';
This does produce a symmetric, positive-semidefinite matrix. But this matrix is not necessarily invertible, it is possible (though very unlikely) that the matrix is singular. More likely is that it is almost singular, meaning that the inverse will get very large values. This inverse is imprecise, and B*inv(B) will be different from the identity matrix by an amount larger than your tolerance.
One simple way of ensuring that B*inv(B) is within tolerance of the identity matrix is to repeatedly generate a random matrix until you find one that is OK:
tol = 1e-12;
while true
A = rand(matrixSize);
B = A*A.';
err = abs(B*inv(B) - eye(matrixSize));
if all(err(:)<tol)
break
end
end
The loop above will run only once most of the time, only occasionally will it need to generate a second matrix.
For any eps > 0 and any nxk (for any k) matrix B the matrix
P = eps*I + B*B'
is positive definite and invertible.
If k < n and eps is small then P will be nearly singular, in the sense that it will have eps as an eigenvalue. When generating these matrices to test something, it can be handy to be able to generate something nearly singular.
MATLAB code to obtain P:
n = 10;
k = 1;
B = rand(n,k);
B = B * B.';
P = B + eye(size(B)) * eps(max(B(:)));
Related
I am working on MATLAB problems from my textbook and one of the problems asks me to use the eig command in MATLAB, compute the matrices V and D such that A = V * D * inv(V). Knowing that the first column of V corresponds to the first eigenvalue D(1,1) and so on, I need to reorder the diagonal entries of D so that the real part is increasing down the diagonal and reorder the columns of V accordingly so that A = V * D * inv(V) still holds. Here's what I have written so far:
r = RandStream('mt19937ar','Seed',1234);
A = r.randn(10,10)+1j*r.randn(10,10);
[V,D] = eig(A);
for tt = 1:9
if (real(D(tt,tt)) > real(D(tt+1,tt+1)))
temp = D(tt,tt);
D(tt,tt) = D(tt+1,tt+1);
D(tt+1,tt+1) = temp;
tempV = V(1,tt);
V(1,tt) = V(1,tt+1);
V(1,tt+1) = tempV;
if (A == V*D*inv(V))
break
end
end
end
When I tested it, the diagonal elements of D did not change from the original order, I know it might be due to the conditionals I set, but I am not sure what specifically is causing it to not do anything. I also think there might be issues in the way I am reordering the diagonal elements and corresponding eigenvectors. Any feedback or suggestions is appreciated, thank you in advance.
Your code has multiple problems:
You need two for-loops for sorting.
You are only swapping first element of eigenvectors, use V(:, tt) for whole column.
V*D*inv(V) will never be exactly equal to A (see this).
To sort eigenvalues by their real parts, try this:
clc;
r = RandStream('mt19937ar','Seed',1234);
n = 10;
A = r.randn(n)+1j*r.randn(n);
[V,D] = eig(A);
d = diag(D, 0); % get eigenvalues in a vector
[~, I] = sort(real(d)); % get index of eigenvalues after sorting
D2 = diag(d(I)); % get sorted eigenvalues as diagonal matrix
V2 = V(:, I); % reorder eigenvectors to match sorted eigenvalues
any(any(abs(A - V2*D2*inv(V2)) > 1e-14)) % test sorted eigenvalues and eigenvectors
Hi everybody I have this problem:
I have Dataset of n vectors each has D dimensions.
I also have a covariance matrix of size D*D, Let It be C.
I perform the following action:
I choose K vectors from the dataset, and also choose E dimensions randomly. Let M be the sample covariance of the selected data on the selected dimensions.so M is a E*E matrix.
let P be the partial covariance matrix corresponding to the dimensions E of C, ie. C(E,E) in matlab
is the following matrix positive semi definite?:
X = (1-a)P + aM
where a is a constant like 0.2.
I sometimes get the following error when using mvnrnd(mean,X) :
SIGMA must be a symmetric positive semi-definite matrix
My code is:
%%%Dims are randomly choosen dimensions
%%%Inds are randomly choosen Indexes form {1, 2, ...,n}
%%% PP are n D dimensional vectors, composing my data set PP is n*D
%%% Sigmaa is a D*D covariance matrix
co = cov(PP(Inds,Dims));
me = mean(PP(Inds,Dims));
Bettaa = 0.2;
sigmaaDims = sigmaa(Dims,Dims);
sigmaaDims = (1-Bettaa)*sigmaaDims + (co)*Bettaa;
Tem = mvnrnd(me,sigmaaDims);
Simply looking at the matrix dimensions It is not possible to tell if a matrix is positive semi-definite.
To find out if a given matrix is positive semi-definite, you must check if It's eigenvalues are non-negative and it's symmetry:
symmetry = issymmetric(X);
[~,D]=eig(X);
eigenvalues = diag(D);
if all(eigenvalues>0) & symmetry
disp('Positive semi-definite matrix.')
else
disp('Non positive semi-definite matrix.')
end
Where X is the matrix you are interested in.
Note that if you use the weaker definition of a positive definite matrix (see Extention for non symmetric matrices section), X does not need to be symmetric and you would end up with:
[~,D]=eig(X);
eigenvalues = diag(D);
if all(eigenvalues>=0)
disp('Positive semi-definite matrix.')
else
disp('Non positive semi-definite matrix.')
end
This question is about the use of the covariance matrix in the multidimensional normal distribution:
I want to generate multi-dimensional random numbers x in Matlab with a given mean mu and covariance matrix Sigma. Assuming Z is a standard normally distributed random number (e.g. generated using randn), what is the correct code:
x = mu + chol(Sigma) * Z
or
x = mu + Sigma ^ 0.5 * Z
?
I am not sure about the use of the covariance matrix in the definition of the multidimensional normal distribution – whether the determinant in the denominator is of the square root or the Cholesky factor...
If by definition you refer to the density of the multivariate normal distribution:
it contains neither the Cholesky decomposition nor the matrix square root of Σ, but its inverse and the scalar square root of its determinant.
But for numerically generating random numbers from this distribution, the density is not helpful. It is not even the most general description of the multivariate normal distribution, since the density formula makes only sense for positive definite matrices Σ, while the distribution is also defined if there are zero eigenvalues – that just means that the variance is 0 in the direction of the respective eigenvector.
Your question follows the approach to start from standard multivariate normally distributed random numbers Z as produced by randn, and then apply a linear transformation. Assuming that mu is a p-dimensional row vector we want an nxp-dimensional random matrix (each row one observation, each column one variable):
Z = randn(n, p);
x = mu + Z * A;
We need a matrix A such that the covariance of x is Sigma. Since the covariance of Z is the identity matrix, the covariance of x is given by A' * A. A solution to this is given by the Cholesky decomposition, so the natural choice is
A = chol(Sigma);
where A is an upper triangular matrix.
However, we can also search for a Hermitian solution, A' = A, and then A' * A becomes A^2, the matrix square. A solution to this is given by a matrix square root, which is computed by replacing each eigenvalue of Sigma by its square root (or its negative); in general there are 2ⁿ possible solutions for n positive eigenvalues. The Matlab function sqrtm returns the principal matrix square root, which is the unique nonnegative-definite solution. Therefore,
A = sqrtm(Sigma)
works also. A ^ 0.5 should in principle do the same.
Simulations using this code
p = 10;
n = 1000;
nr = 1000;
cp = nan(nr, 1);
sp = nan(nr, 1);
pp = nan(nr, 1);
for i = 1 : nr
x = randn(n, p);
Sigma = cov(x);
cS = chol(Sigma);
cp(i) = norm(cS' * cS - Sigma);
sS = sqrtm(Sigma);
sp(i) = norm(sS' * sS - Sigma);
pS = Sigma ^ 0.5;
pp(i) = norm(pS' * pS - Sigma);
end
mean([cp sp pp])
yield that chol is more precise than the two other methods, and profiling shows that it is also much faster, for both p = 10 and p = 100.
The Cholesky decomposition does however have the disadvantage that it is only defined for positive-definite Σ, while the requirement of the matrix square root is merely that Σ is nonnegative-definite (sqrtm returns a warning for a singular input, but returns a valid result).
I'm trying to write a program that gets a matrix A of any size, and SVD decomposes it:
A = U * S * V'
Where A is the matrix the user enters, U is an orthogonal matrix composes of the eigenvectors of A * A', S is a diagonal matrix of the singular values, and V is an orthogonal matrix of the eigenvectors of A' * A.
Problem is: the MATLAB function eig sometimes returns the wrong eigenvectors.
This is my code:
function [U,S,V]=badsvd(A)
W=A*A';
[U,S]=eig(W);
max=0;
for i=1:size(W,1) %%sort
for j=i:size(W,1)
if(S(j,j)>max)
max=S(j,j);
temp_index=j;
end
end
max=0;
temp=S(temp_index,temp_index);
S(temp_index,temp_index)=S(i,i);
S(i,i)=temp;
temp=U(:,temp_index);
U(:,temp_index)=U(:,i);
U(:,i)=temp;
end
W=A'*A;
[V,s]=eig(W);
max=0;
for i=1:size(W,1) %%sort
for j=i:size(W,1)
if(s(j,j)>max)
max=s(j,j);
temp_index=j;
end
end
max=0;
temp=s(temp_index,temp_index);
s(temp_index,temp_index)=s(i,i);
s(i,i)=temp;
temp=V(:,temp_index);
V(:,temp_index)=V(:,i);
V(:,i)=temp;
end
s=sqrt(s);
end
My code returns the correct s matrix, and also "nearly" correct U and V matrices. But some of the columns are multiplied by -1. obviously if t is an eigenvector, then also -t is an eigenvector, but with the signs inverted (for some of the columns, not all) I don't get A = U * S * V'.
Is there any way to fix this?
Example: for the matrix A=[1,2;3,4] my function returns:
U=[0.4046,-0.9145;0.9145,0.4046]
and the built-in MATLAB svd function returns:
u=[-0.4046,-0.9145;-0.9145,0.4046]
Note that eigenvectors are not unique. Multiplying by any constant, including -1 (which simply changes the sign), gives another valid eigenvector. This is clear given the definition of an eigenvector:
A·v = λ·v
MATLAB chooses to normalize the eigenvectors to have a norm of 1.0, the sign is arbitrary:
For eig(A), the eigenvectors are scaled so that the norm of each is 1.0.
For eig(A,B), eig(A,'nobalance'), and eig(A,B,flag), the eigenvectors are not normalized
Now as you know, SVD and eigendecomposition are related. Below is some code to test this fact. Note that svd and eig return results in different order (one sorted high to low, the other in reverse):
% some random matrix
A = rand(5);
% singular value decomposition
[U,S,V] = svd(A);
% eigenvectors of A'*A are the same as the right-singular vectors
[V2,D2] = eig(A'*A);
[D2,ord] = sort(diag(D2), 'descend');
S2 = diag(sqrt(D2));
V2 = V2(:,ord);
% eigenvectors of A*A' are the same as the left-singular vectors
[U2,D2] = eig(A*A');
[D2,ord] = sort(diag(D2), 'descend');
S3 = diag(sqrt(D2));
U2 = U2(:,ord);
% check results
A
U*S*V'
U2*S2*V2'
I get very similar results (ignoring minor floating-point errors):
>> norm(A - U*S*V')
ans =
7.5771e-16
>> norm(A - U2*S2*V2')
ans =
3.2841e-14
EDIT:
To get consistent results, one usually adopts a convention of requiring that the first element in each eigenvector be of a certain sign. That way if you get an eigenvector that does not follow this rule, you multiply it by -1 to flip the sign...
I have a large dense matrix, say matrix A of size 10000 by 10000 and I need to extract a banded matrix of bandwidth say 10 from it, i.e.,
B(i,j) = A(i,j) if |i-j| <=10
B(i,j) = 0 otherwise
What is the most efficient way to go about doing this in MATLAB?
I don't know that this is the most efficient way, but here is a way to create a matrix banded about the main diagonal via masking using the toeplitz() function:
r = zeros(1,size(A,2));
r(1 : ceil(bandwidth/2)) = 1;
bandedMask = toeplitz(r); %Create a banded toeplitz matrix of 1s and 0s
bandedMat = bandedMask.*A;
Note: This method assumes your bandwidth is odd.
As it is a huge matrix it might be a useful option not to copy it a second time to the memory. In that case
N = 10;
M = ...
for lin = 1:size(M,1)
M(lin, lin+N:end) = 0;
M(lin, 1:lin-N) = 0;
end
could be useful (depends whether you need the original matrix or not afterwards).
In the case you have to keep the original matrix you could think about representing your matrix diagonal by diagonal or as sparse matrix. In the case you have to copy the matrix you shouldn't touch all the elements that you don't need.
You should evaluate the different ways and tell us your results :-)
Suppose you have a matrix B and bandwidth n:
B = rand(16,7);
n = 4;
% Index main diagonal
szB = size(B);
idx = abs(bsxfun(#minus, (1:szB(1))',1:szB(2))) <= n;
% Build sparse
[r,c] = find(idx);
sparse(r,c,B(idx))