Suppose I have a matrix A. I want to calculate its 2-norm/spectral norm. How can I calculate this efficiently?
I know 2-norm of a matrix is equal to its largest singular value. So, result of the following MATLAB code will be zero
>> [u,s,v]=svd(A,'econ');
norm(A,2)-s(1,1)
But to know 2-norm I have to calculate SVD of full matrix A, is there any efficient way to calculate 2-norm? Answer in form of MATLAB code will be much appereciated.
This example with norm and random data
A = randn(2000,2000);
tic;
n1 = norm(A)
toc;
gives
n1 = 89.298
Elapsed time is 2.16777 seconds.
You can try eigs to find only one (the largest) eigenvalue of the symmetric matrix A'*A (or A*A' if it is smaller for A rectangular). It uses a Lanczos iteration method.
tic;
B = A'*A; % symmetric positive-definite. B = A*A' if it is smaller
n2 = sqrt(eigs(B, 1)),
toc
it outputs:
n2 = 89.298
Elapsed time is 0.311942 seconds.
If you don't want to use norm or eigs, and your matrix A has good properties (singular values properly separated), you can try to approximate it with a power iteration method:
tic;
B = A'*A; % or B = A*A' if it is smaller
x = B(:,1); % example of starting point, x will have the largest eigenvector
x = x/norm(x);
for i = 1:200
y = B*x;
y = y/norm(y);
% norm(x - y); % <- residual, you can try to use it to stop iteration
x = y;
end;
n3 = sqrt(mean(B*x./x)) % translate eigenvalue of B to singular value of A
toc
which for the same random matrix (not particularly good properties) gives a ~0.1% accurate solution:
n3 = 89.420
Elapsed time is 0.428032 seconds.
Related
I only know of the following power iteration. But it needs to create a huge matrix A'*A when both of rows and columns are pretty large. And A is a dense matrix as well. Is there any alternative to power iteration method below? I have heard of krylov subspace method, but I am not familiar with it. In anycase I am looking for any faster method than the one mentioned below:
B = A'*A; % or B = A*A' if it is smaller
x = B(:,1); % example of starting point, x will have the largest eigenvector
x = x/norm(x);
for i = 1:200
y = B*x;
y = y/norm(y);
% norm(x - y); % <- residual, you can try to use it to stop iteration
x = y;
end;
n3 = sqrt(mean(B*x./x)) % translate eigenvalue of B to singular value of A
I checked 'svd' command of matlab with a 100*100 randomly generated matrix. It is almost 5 times faster than your code.
s = svd(A);
n3 = s(1);
Repost with additional details that greatly change the scope of my first question. Here is the original code:
K = zeros(N*N)
for a=1:N
for i=1:I
for j=1:J
M = kron(X(:,:,a).',Y(:,:,a,i,j));
pr = real(trace(E*M));
K = K+H(i,j,a)*M/pr;
end
end
end
Where E is a boolean mask, H is 3D matrix containing N IxJ histograms. K is the output
The goal is to vectorize the kroniker multiplication calls. My intuition is to think of X and Y as containers of matrices (for reference, the slices of X and Y being fed to kron are square matrices of the order 7x7). Under this container scheme, X appears a 1-D container and Y as a 3-D container. My next guess was to reshape Y into a 2-D container or better yet a 1-D container and then do element wise multiplication of X and Y. Questions are: how would do this reshaping in a way that preserves the trace of M and can matlab even handle this idea in this container idea or do the containers need to be further reshaped to expose the inner matrix elements further?
Matrix multiplication with 7D permute
% Get sizes
[m1,m2,~] = size(X);
[n1,n2,N,n4,n5] = size(Y);
% Perform kron format elementwise multiplication betwen the first two dims
% of X and Y, keeping the third dim aligned and "pushing out" leftover dims
% from Y to the back
mults = bsxfun(#times,permute(X,[4,2,5,1,3]),permute(Y,[1,6,2,7,3,4,5]));
mults3D = reshape(mults,m1*n1,m2*n2,[]);
Emults3D = reshape(E*reshape(mults3D,size(mults3D,1),[]),size(mults3D));
% Trace summations by using linear indices of diagonal on 3D slices in Emults3D
MN = m1*n1;
idx = 1:MN+1:MN^2;
idx2D = bsxfun(#plus,idx(:),MN^2*(0:size(Emults3D,3)-1));
pr_sums = sum(Emults3D(idx2D),1);
% Perform "M/pr" equivalent elementwise divisions and then use
% matrix-multiplication to reduce the iterative summations
Mp = bsxfun(#rdivide,mults3D,reshape(pr_sums,1,1,[]));
out = reshape(Mp,[],size(Mp,3))*reshape(permute(H,[3,1,2]),[],1);
out = reshape(out,m1*n1,m2*n2);
Benchmarking
The inputs were setup like so -
% Size parameter
n = 5;
% Setup inputs
X = rand(n,n,n);
Y = rand(n,n,n,n,n);
E = rand(n*n,n*n)>0.5;
H = rand(n,n,n);
num_iter = 500; % Number of iterations to run the approaches for
The runtime results were -
----------------------------- With Loop
Elapsed time is 8.806286 seconds.
----------------------------- With Vectorization
Elapsed time is 1.471877 seconds.
With the size parameter n set as 10, the runtimes were -
----------------------------- With Loop
Elapsed time is 5.068872 seconds.
----------------------------- With Vectorization
Elapsed time is 4.399783 seconds.
Here is the original code:
K = zeros(N*N)
for a=1:N
for i=1:I
for j=1:J
M = kron(X(:,:,a).',Y(:,:,a,i,j));
%A function that essentially adds M to K.
end
end
end
The goal is to vectorize the kroniker multiplication calls. My intuition is to think of X and Y as containers of matrices (for reference, the slices of X and Y being fed to kron are square matrices of the order 7x7). Under this container scheme, X appears a 1-D container and Y as a 3-D container. My next guess was to reshape Y into a 2-D container or better yet a 1-D container and then do element wise multiplication of X and Y. Questions are: how would do this reshaping in a way that preserves the trace of M and can matlab even handle this idea in this container idea or do the containers need to be further reshaped to expose the inner matrix elements further?
Approach #1: Matrix multiplication with 6D permute
% Get sizes
[m1,m2,~] = size(X);
[n1,n2,N,n4,n5] = size(Y);
% Lose the third dim from X and Y with matrix-multiplication
parte1 = reshape(permute(Y,[1,2,4,5,3]),[],N)*reshape(X,[],N).';
% Rearrange the leftover dims to bring kron format
parte2 = reshape(parte1,[n1,n2,I,J,m1,m2]);
% Lose dims correspinding to last two dims coming in from Y corresponding
% to the iterative summation as suggested in the question
out = reshape(permute(sum(sum(parte2,3),4),[1,6,2,5,3,4]),m1*n1,m2*n2)
Approach #2: Simple 7D permute
% Get sizes
[m1,m2,~] = size(X);
[n1,n2,N,n4,n5] = size(Y);
% Perform kron format elementwise multiplication betwen the first two dims
% of X and Y, keeping the third dim aligned and "pushing out" leftover dims
% from Y to the back
mults = bsxfun(#times,permute(X,[4,2,5,1,3]),permute(Y,[1,6,2,7,3,4,5]));
% Lose the two dims with summation reduction for final output
out = sum(reshape(mults,m1*n1,m2*n2,[]),3);
Verification
Here's a setup for running the original and the proposed approaches -
% Setup inputs
X = rand(10,10,10);
Y = rand(10,10,10,10,10);
% Original approach
[n1,n2,N,I,J] = size(Y);
K = zeros(100);
for a=1:N
for i=1:I
for j=1:J
M = kron(X(:,:,a).',Y(:,:,a,i,j));
K = K + M;
end
end
end
% Approach #1
[m1,m2,~] = size(X);
[n1,n2,N,n4,n5] = size(Y);
mults = bsxfun(#times,permute(X,[4,2,5,1,3]),permute(Y,[1,6,2,7,3,4,5]));
out1 = sum(reshape(mults,m1*n1,m2*n2,[]),3);
% Approach #2
[m1,m2,~] = size(X);
[n1,n2,N,n4,n5] = size(Y);
parte1 = reshape(permute(Y,[1,2,4,5,3]),[],N)*reshape(X,[],N).';
parte2 = reshape(parte1,[n1,n2,I,J,m1,m2]);
out2 = reshape(permute(sum(sum(parte2,3),4),[1,6,2,5,3,4]),m1*n1,m2*n2);
After running, we see the max. absolute deviation with the proposed approaches against the original one -
>> error_app1 = max(abs(K(:)-out1(:)))
error_app1 =
1.1369e-12
>> error_app2 = max(abs(K(:)-out2(:)))
error_app2 =
1.1937e-12
Values look good to me!
Benchmarking
Timing these three approaches using the same big dataset as used for verification, we get something like this -
----------------------------- With Loop
Elapsed time is 1.541443 seconds.
----------------------------- With BSXFUN
Elapsed time is 1.283935 seconds.
----------------------------- With MATRIX-MULTIPLICATION
Elapsed time is 0.164312 seconds.
Seems like matrix-multiplication is doing fairly good for dataset of these sizes!
In the question, A is a square matrix and x is a vector. The non-zero entries of A can only be 1 or -1. Then how to speed up A*x in Matlab?
I asked this question, because A*x contains multiplication operations that are computation-expensive.
However, if A only contains some +1 and -1, and assume y is a row vector from A, then y*x can avoid multiplication operations. You just need group the entries of x into 2, one is corresponding to the entries +1 of y, and the other one is corresponding to the entries -1 of y. Then do a summation of these 2 groups respectively, and do a subtraction.
One way to speed this up in MATLAB is to take advantage of the sparse structure of A (rather than the +1, -1 structure). You can realize huge speedups this way if A is sparse enough:
N = 1e4;
k = .0001;
% setup
x = rand(N, 1);
inds = find(rand(1,N^2) <= k);
values = 2*(randi(2, 1, length(inds))-1)-1;
% dense
fprintf('dense\n')
A = zeros(N,N);
A(inds) = values;
for i = 1:5
tic;
y = A*x;
toc;
end
% sparse
fprintf('sparse\n')
A = sparse(N,N);
A(inds) = values;
for i = 1:5
tic;
y = A*x;
toc;
end
I've got function values in a vector f and also the vector containing values of the argument x. I need to find the define integral value of f. But the argument vector x is not uniform. Is there any function in Matlab that deals with integration over non-uniform grids?
Taken from help :
Z = trapz(X,Y) computes the integral of Y with respect to X using
the trapezoidal method. X and Y must be vectors of the same
length, or X must be a column vector and Y an array whose first
non-singleton dimension is length(X). trapz operates along this
dimension.
As you can see x does not have to be uniform.
For instance:
x = sort(rand(100,1)); %# Create random values of x in [0,1]
y = x;
trapz( x, y)
Returns:
ans =
0.4990
Another example:
x = sort(rand(100,1)); %# Create random values of x in [0,1]
y = x.^2;
trapz( x, y)
returns:
ans =
0.3030
Depending on your function (and how x is distributed), you might get more accuracy by doing a spline interpolation through your data first:
pp = spline(x,y);
quadgk(#(t) ppval(pp,t), [range])
That's the quick-n-dirty way. Ther is a faster and more direct approach, but that is fugly and much less transparent:
result = sum(sum(...
bsxfun(#times, pp.coefs, 1./(4:-1:1)) .*... % coefficients of primitive
bsxfun(#power, diff(pp.breaks).', 4:-1:1)... % all 4 powers of shifted x-values
));
As an example why all this could be useful, I borrow the example from here. The exact answer should be
>> pi/2/sqrt(2)*(17-40^(3/4))
ans =
1.215778726893561e+00
Defining
>> x = [0 sort(3*rand(1,5)) 3];
>> y = (x.^3.*(3-x)).^(1/4)./(5-x);
we find
>> trapz(x,y)
ans =
1.142392438652055e+00
>> pp = spline(x,y);
>> tic; quadgk(#(t) ppval(pp,t), 0, 3), toc
ans =
1.213866446458034e+00
Elapsed time is 0.017472 seconds.
>> tic; result = sum(sum(...
bsxfun(#times, pp.coefs, 1./(4:-1:1)) .*... % coefficients of primitive
bsxfun(#power, diff(pp.breaks).', 4:-1:1)... % all 4 powers of shifted x-values
)), toc
result =
1.213866467945575e+00
Elapsed time is 0.002887 seconds.
So trapz underestimates the value by more than 0.07. With the latter two methods, the error is an order of magnitude less. Also, the less-readable version of the spline approach is an order of magnitude faster.
So, armed with this knowledge: choose wisely :)
You can do Gaussian quadrature over each piecewise pair of x and sum them up to get the complete integral.