I'm using LAPACKE to solve generalized eigenvalue problem A*V = lambda*B*V and I need both eigenvalues and right eigenvectors. But the problem is I can only get correct eigenvalues but wrong eigenvectors. Actually I've used my code to test the example in this link http://www.nag.com/numeric/FL/nagdoc_fl22/examples/baseresults/f08ykfe.r. It's successful. So I'm wondering why it doesn't work for this case. I'd post my code and results below. Can anybody tell me give me a hint? Thanks a lot.
double AAtest[4] = {-1.999783, 0, 0, 1.999783};
double BBtest[4] = {2.167327e-4, 1.999783, 1.999783, 0};
int i, j, N = 2;
int matrix_order = LAPACK_COL_MAJOR;
double *RR, *QQ, *ZZ, *alphar, *alphai, *beta;
int *select, m;
int info1, info2, info3, info4, info5, info6, info7;
int ilo, ihi;
double *lscale, *rscale, *tau;
RR = (double *)calloc(N*N, sizeof(double));
QQ = (double *)calloc(N*N, sizeof(double));
ZZ = (double *)calloc(N*N, sizeof(double));
alphar = (double *)calloc(N, sizeof(double));
alphai = (double *)calloc(N, sizeof(double));
beta = (double *)calloc(N, sizeof(double));
select = (int *)calloc(N, sizeof(int));
lscale = (double *)calloc(N, sizeof(double));
rscale = (double *)calloc(N, sizeof(double));
tau = (double *)calloc(N, sizeof(double));
for (i = 0; i < N; i++)
select[i] = 1;
info1 = LAPACKE_dggbal(matrix_order, 'B', N, AA, N, BB, N, &ilo, &ihi, lscale, rscale);
info2 = LAPACKE_dgeqrf(matrix_order, N, N, BB, N, tau);
for (i = 0; i < N; i++){
for (j = i; j < N; j++){
RR[j*N+i] = BB[j*N+i];
BB[j*N+i] = 0;
}
}
info3 = LAPACKE_dormqr(matrix_order, 'L', 'T', N, N, N, BB, N, tau, AA, N);
info4 = LAPACKE_dgghrd(matrix_order, 'I', 'I', N, ilo, ihi, AA, N, RR, N, QQ, N, ZZ, N);
info5 = LAPACKE_dhgeqz(matrix_order, 'S', 'V', 'V', N, ilo, ihi, AA, N, RR, N, alphar, alphai, beta, QQ, N, ZZ, N);
info6 = LAPACKE_dtgevc(matrix_order, 'R', 'B', select, N, AA, N, RR, N, QQ, N, ZZ, N, N, &m);
info7 = LAPACKE_dggbak(matrix_order, 'B', 'R', N, ilo, ihi, lscale, rscale, m, ZZ, N);
Results:
Input matrix AA....................................
-1.999783e+00 0.000000e+00
0.000000e+00 1.999783e+00
Input matrix BB....................................
2.167327e-04 1.999783e+00
1.999783e+00 0.000000e+00
After balancing AA...................
-1.999783e+02 0.000000e+00
0.000000e+00 1.999783e+00
After balancing BB...................
2.167327e-02 1.999783e+01
1.999783e+01 0.000000e+00
R factor.............................
-1.999784e+01 -2.167326e-02
0.000000e+00 -1.999782e+01
Q'*A.................................
2.167326e-01 -1.999782e+00
1.999782e+02 2.167326e-03
Hessenberg form of AA................
2.167326e-01 -1.999782e+00
1.999782e+02 2.167326e-03
triangular form of BB................
-1.999784e+01 -2.167326e-02
0.000000e+00 -1.999782e+01
Eigenvalues..........................
-0.000054 + j1.000000
-0.000054 + j-1.000000
Schur form of AA................
-9.904398e+01 1.009890e+02
-1.009890e+02 9.893453e+01
Schur form of BB................
2.000867e+01 0.000000e+00
0.000000e+00 1.998700e+01
Right eigenvectors.....................
-9.090507e-02 9.090506e-01
-9.090457e-01 -9.095433e-02
But when I use matlab to solve this problem, the eigenvectors I got is
-0.500000000733645 - 0.499999999266355i -0.500000000733645 + 0.499999999266355i
0.500027093059538 - 0.499972905472315i 0.500027093059538 + 0.499972905472315i
pls help...Thanks!
Solved this problem by only calling LAPACKE_dggev. This function would call those routines automatically. Not sure about whether it would balance the matrix or not. At least now the results match with matlab results.
Related
I'm new to Octave and Matlab and I have a problem. I need to make a program, which solves a Triangular system of linear equations and making a triangular factorization with pivoting.
For example I need to do next exercise
That is my lufact function
function X = lufact (A, B)
[N, N] = size(A);
X = zeros(N, 1);
Y = zeros(N, 1);
C = zeros(1, N);
R = 1:N;
for p=1:N-1
[max1,j]=max(abs(A(p:N,p)));
C = A(p, :);
A(p, :) = A(j + p - 1, :);
A(j + p - 1, :) = C;
d = R(p);
R(p) = R(j + p - 1);
R(j + p - 1) = d;
if A(p,p) == 0
'A is singular. No unique solution'
break
endif
endfor
for k = p + 1: N
mult = A(k,p)/A(p,p);
A(k,p) = mult;
A(k, p + 1: N) = A(k, p + 1: N) - mult*A(p, p+1: N);
endfor
endfunction
Y(1) = B(R(1));
for k = 2: N
Y(k) = B(R(k)) - A(k, 1:k - 1)*Y(1:k - 1);
endfor
X(N) = Y(N)/A(N,N);
for k = N-1: -1: 1
X(k) = (Y(k) - A(k, k + 1: N)*X(k + 1: N))/A(k, k);
endfor
And that is my main function
A = [2 4 -6; 1 5 3; 1 3 2];
BA = [-4; 10; 5]
BB = [20; 49; 32]
XA = lufact(A, BA);
XB = lufact(A, BB);
disp(XA);
disp(XB);
The output of my program
What am I doing wrong and what should I do to fix that?
You don't seem to be attempting to answer the question you're asking, so your question doesn't make much sense...
But, in any case. The exercise asks you to show that given the following matrices / vectors:
A = [ 2, 4, -6; 1, 5, 3; 1, 3, 2 ];
L = [ 1, 0, 0; 1/2, 1, 0; 1/2, 1/3, 1 ];
U = [ 2, 4, -6; 0, 3, 6; 0, 0, 3 ];
show that LY = B, UX = Y, AX = B for a given B.
Ok. In octave, the best way to solve a matrix equation of the form aX = b w.r.t X for a given a and b, is to use the "matrix left-division" operator, \, i.e. x = a\b. Type help mldivide in your octave console for details.
So,
Y = L\B;
X = U\Y;
A*X % the result should be the same as B. QED.
Then there is the separate question of how can I perform LU decomposition in octave? Well, octave provides a function for this: lu.
[L, U] = lu(A)
% L = 1.0000 0 0
% 0.5000 1.0000 0
% 0.5000 0.3333 1.0000
%
% U = 2 4 -6
% 0 3 6
% 0 0 3
Then there's the implied further question of "I would like to perform LU decomposition by hand."
Great. Good luck. Your code here is a bit messy, no comments, no self-explanatory variable names ... I won't attempt to debug it here in detail. One thing to note though is, LU decomposition only takes a matrix as an input. I'm not sure why you are trying to pass the 'B' matrix to it as an input. Plus it doesn't seem like you're using it anywhere in the function. Or creating L or U matrices for that matter. Also, if the whole top part is inside an lufact.m file, then you should know that your function terminates well before the for loops; these get ignored completely. What were you trying to do exactly?
I want to realize component-wise matrix multiplication in MATLAB, which can be done using numpy.einsum in Python as below:
import numpy as np
M = 2
N = 4
I = 2000
J = 300
A = np.random.randn(M, M, I)
B = np.random.randn(M, M, N, J, I)
C = np.random.randn(M, J, I)
# using einsum
D = np.einsum('mki, klnji, lji -> mnji', A, B, C)
# naive for-loop
E = np.zeros(M, N, J, I)
for i in range(I):
for j in range(J):
for n in range(N):
E[:,n,j,i] = B[:,:,i] # A[:,:,n,j,i] # C[:,j,i]
print(np.sum(np.abs(D-E))) # expected small enough
So far I use for-loop of i, j, and n, but I don't want to, at least for-loop of n.
Option 1: Calling numpy from MATLAB
Assuming your system is set up according to the documentation, and you have the numpy package installed, you could do (in MATLAB):
np = py.importlib.import_module('numpy');
M = 2;
N = 4;
I = 2000;
J = 300;
A = matpy.mat2nparray( randn(M, M, I) );
B = matpy.mat2nparray( randn(M, M, N, J, I) );
C = matpy.mat2nparray( randn(M, J, I) );
D = matpy.nparray2mat( np.einsum('mki, klnji, lji -> mnji', A, B, C) );
Where matpy can be found here.
Option 2: Native MATLAB
Here the most important part is to get the permutations right, so we need to keep track of our dimensions. We'll be using the following order:
I(1) J(2) K(3) L(4) M(5) N(6)
Now, I'll explain how I got the correct permute order (let's take the example of A): einsum expects the dimension order to be mki, which according to our numbering is 5 3 1. This tells us that the 1st dimension of A needs to be the 5th, the 2nd needs to be 3rd and the 3rd needs to be 1st (in short 1->5, 2->3, 3->1). This also means that the "sourceless dimensions" (meaning those that have no original dimensions becoming them; in this case 2 4 6) should be singleton. Using ipermute this is really simple to write:
pA = ipermute(A, [5,3,1,2,4,6]);
In the above example, 1->5 means we write 5 first, and the same goes for the other two dimensions (yielding [5,3,1]). Then we just add the singletons (2,4,6) at the end to get [5,3,1,2,4,6]. Finally:
A = randn(M, M, I);
B = randn(M, M, N, J, I);
C = randn(M, J, I);
% Reference dim order: I(1) J(2) K(3) L(4) M(5) N(6)
pA = ipermute(A, [5,3,1,2,4,6]); % 1->5, 2->3, 3->1; 2nd, 4th & 6th are singletons
pB = ipermute(B, [3,4,6,2,1,5]); % 1->3, 2->4, 3->6, 4->2, 5->1; 5th is singleton
pC = ipermute(C, [4,2,1,3,5,6]); % 1->4, 2->2, 3->1; 3rd, 5th & 6th are singletons
pD = sum( ...
permute(pA .* pB .* pC, [5,6,2,1,3,4]), ... 1->5, 2->6, 3->2, 4->1; 3rd & 4th are singletons
[5,6]);
(see note regarding sum at the bottom of the post.)
Another way to do it in MATLAB, as mentioned by #AndrasDeak, is the following:
rD = squeeze(sum(reshape(A, [M, M, 1, 1, 1, I]) .* ...
reshape(B, [1, M, M, N, J, I]) .* ...
... % same as: reshape(B, [1, size(B)]) .* ...
... % same as: shiftdim(B,-1) .* ...
reshape(C, [1, 1, M, 1, J, I]), [2, 3]));
See also: squeeze, reshape, permute, ipermute, shiftdim.
Here's a full example that shows that tests whether these methods are equivalent:
function q55913093
M = 2;
N = 4;
I = 2000;
J = 300;
mA = randn(M, M, I);
mB = randn(M, M, N, J, I);
mC = randn(M, J, I);
%% Option 1 - using numpy:
np = py.importlib.import_module('numpy');
A = matpy.mat2nparray( mA );
B = matpy.mat2nparray( mB );
C = matpy.mat2nparray( mC );
D = matpy.nparray2mat( np.einsum('mki, klnji, lji -> mnji', A, B, C) );
%% Option 2 - native MATLAB:
%%% Reference dim order: I(1) J(2) K(3) L(4) M(5) N(6)
pA = ipermute(mA, [5,3,1,2,4,6]); % 1->5, 2->3, 3->1; 2nd, 4th & 6th are singletons
pB = ipermute(mB, [3,4,6,2,1,5]); % 1->3, 2->4, 3->6, 4->2, 5->1; 5th is singleton
pC = ipermute(mC, [4,2,1,3,5,6]); % 1->4, 2->2, 3->1; 3rd, 5th & 6th are singletons
pD = sum( permute( ...
pA .* pB .* pC, [5,6,2,1,3,4]), ... % 1->5, 2->6, 3->2, 4->1; 3rd & 4th are singletons
[5,6]);
rD = squeeze(sum(reshape(mA, [M, M, 1, 1, 1, I]) .* ...
reshape(mB, [1, M, M, N, J, I]) .* ...
reshape(mC, [1, 1, M, 1, J, I]), [2, 3]));
%% Comparisons:
sum(abs(pD-D), 'all')
isequal(pD,rD)
Running the above we get that the results are indeed equivalent:
>> q55913093
ans =
2.1816e-10
ans =
logical
1
Note that these two methods of calling sum were introduced in recent releases, so you might need to replace them if your MATLAB is relatively old:
S = sum(A,'all') % can be replaced by ` sum(A(:)) `
S = sum(A,vecdim) % can be replaced by ` sum( sum(A, dim1), dim2) `
As requested in the comments, here's a benchmark comparing the methods:
function t = q55913093_benchmark(M,N,I,J)
if nargin == 0
M = 2;
N = 4;
I = 2000;
J = 300;
end
% Define the arrays in MATLAB
mA = randn(M, M, I);
mB = randn(M, M, N, J, I);
mC = randn(M, J, I);
% Define the arrays in numpy
np = py.importlib.import_module('numpy');
pA = matpy.mat2nparray( mA );
pB = matpy.mat2nparray( mB );
pC = matpy.mat2nparray( mC );
% Test for equivalence
D = cat(5, M1(), M2(), M3());
assert( sum(abs(D(:,:,:,:,1) - D(:,:,:,:,2)), 'all') < 1E-8 );
assert( isequal (D(:,:,:,:,2), D(:,:,:,:,3)));
% Time
t = [ timeit(#M1,1), timeit(#M2,1), timeit(#M3,1)];
function out = M1()
out = matpy.nparray2mat( np.einsum('mki, klnji, lji -> mnji', pA, pB, pC) );
end
function out = M2()
out = permute( ...
sum( ...
ipermute(mA, [5,3,1,2,4,6]) .* ...
ipermute(mB, [3,4,6,2,1,5]) .* ...
ipermute(mC, [4,2,1,3,5,6]), [3,4]...
), [5,6,2,1,3,4]...
);
end
function out = M3()
out = squeeze(sum(reshape(mA, [M, M, 1, 1, 1, I]) .* ...
reshape(mB, [1, M, M, N, J, I]) .* ...
reshape(mC, [1, 1, M, 1, J, I]), [2, 3]));
end
end
On my system this results in:
>> q55913093_benchmark
ans =
1.3964 0.1864 0.2428
Which means that the 2nd method is preferable (at least for the default input sizes).
I work primarily in MATLAB but I think the answer should not be too hard to carry over from one language to another.
I have a multi-dimensional array X with dimensions [n, p, 3].
I would like to calculate the following multi-dimensional array.
T = zeros(p, p, p)
for i = 1:p
for j = 1:p
for k = 1:p
T(i, j, k) = sum(X(:, i, 1) .* X(:, j, 2) .* X(:, k, 3));
end
end
end
The sum is of the elements of a length-n vector. Any help is appreciated!
You only need some permuting of dimensions and multiplication with singleton expansion:
T = sum(bsxfun(#times, bsxfun(#times, permute(X(:,:,1), [2 4 5 3 1]), permute(X(:,:,2), [4 2 5 3 1])), permute(X(:,:,3), [4 5 2 3 1])), 5);
From R2016b onwards, this can be written more easily as
T = sum(permute(X(:,:,1), [2 4 5 3 1]) .* permute(X(:,:,2), [4 2 5 3 1]) .* permute(X(:,:,3), [4 5 2 3 1]), 5);
As I mentioned in a comment, vectorization is not always a huge advantage any more. Therefore there are vectorization methods that slow down the code rather than speed it up. You must always time your solutions. Vectorization often involves the creation of large temporary arrays, or the copy of large amounts of data, which are avoided in loop code. It depends on the architecture, the size of the input, and many other factors if such a solution is going to be faster.
Nonetheless, in this case it seems vectorization approaches can yield a large speedup.
The first thing to notice about the original code is that X(:, i, 1) .* X(:, j, 2) gets re-computed in the inner loop, though it is a constant value there. Rewriting the inner loop as this will save time:
Y = X(:, i, 1) .* X(:, j, 2);
for k = 1:p
T(i, j, k) = sum(Y .* X(:, k, 3));
end
Now we notice that the inner loop is a dot product, and can be written as follows:
Y = X(:, i, 1) .* X(:, j, 2);
T(i, j, :) = Y.' * X(:, :, 3);
The .' transpose on Y does not copy the data, as Y is a vector. Next, we notice that X(:, :, 3) is indexed repeatedly. Let's move this out of the outer loop. Now I'm left with the following code:
T = zeros(p, p, p);
X1 = X(:, :, 1);
X2 = X(:, :, 2);
X3 = X(:, :, 3);
for i = 1:p
for j = 1:p
Y = X1(:, i) .* X2(:, j);
T(i, j, :) = Y.' * X3;
end
end
It is likely that removing the loop over j is equally easy, which would leave a single loop over i. But this is where I stop.
This is the timings I see (R2017a, 3-year old iMac with 4 cores). For n=10, p=20:
original: 0.0206
moving Y out the inner loop: 0.0100
removing inner loop: 0.0016
moving indexing out of loops: 7.6294e-04
Luis' answer: 1.9196e-04
For a larger array with n=50, p=100:
original: 2.9107
moving Y out the inner loop: 1.3488
removing inner loop: 0.0910
moving indexing out of loops: 0.0361
Luis' answer: 0.1417
"Luis' answer" is this one. It is by far fastest for small arrays, but for larger arrays it shows the cost of the permutation. Moving the computation of the first product out of the inner loop saves a bit over half the computation cost. But removing the inner loop reduces the cost quite dramatically (which I hadn't expected, I presume the single matrix product can use parallelism better than the many small element-wise products). We then get a further time reduction by reducing the amount of indexing operations within the loop.
This is the timing code:
function so()
n = 10; p = 20;
%n = 50; p = 100;
X = randn(n,p,3);
T1 = method1(X);
T2 = method2(X);
T3 = method3(X);
T4 = method4(X);
T5 = method5(X);
assert(max(abs(T1(:)-T2(:)))<1e-13)
assert(max(abs(T1(:)-T3(:)))<1e-13)
assert(max(abs(T1(:)-T4(:)))<1e-13)
assert(max(abs(T1(:)-T5(:)))<1e-13)
timeit(#()method1(X))
timeit(#()method2(X))
timeit(#()method3(X))
timeit(#()method4(X))
timeit(#()method5(X))
function T = method1(X)
p = size(X,2);
T = zeros(p, p, p);
for i = 1:p
for j = 1:p
for k = 1:p
T(i, j, k) = sum(X(:, i, 1) .* X(:, j, 2) .* X(:, k, 3));
end
end
end
function T = method2(X)
p = size(X,2);
T = zeros(p, p, p);
for i = 1:p
for j = 1:p
Y = X(:, i, 1) .* X(:, j, 2);
for k = 1:p
T(i, j, k) = sum(Y .* X(:, k, 3));
end
end
end
function T = method3(X)
p = size(X,2);
T = zeros(p, p, p);
for i = 1:p
for j = 1:p
Y = X(:, i, 1) .* X(:, j, 2);
T(i, j, :) = Y.' * X(:, :, 3);
end
end
function T = method4(X)
p = size(X,2);
T = zeros(p, p, p);
X1 = X(:, :, 1);
X2 = X(:, :, 2);
X3 = X(:, :, 3);
for i = 1:p
for j = 1:p
Y = X1(:, i) .* X2(:, j);
T(i, j, :) = Y.' * X3;
end
end
function T = method5(X)
T = sum(permute(X(:,:,1), [2 4 5 3 1]) .* permute(X(:,:,2), [4 2 5 3 1]) .* permute(X(:,:,3), [4 5 2 3 1]), 5);
You have mentioned you are open to other languages and NumPy by its syntax is very close to MATLAB, so we will try to have a NumPy based solution on this.
Now, these tensor related sum-reductions, specially matrix multiplications ones are easily expressed as einstein-notation and NumPy luckily has one function on the same as np.einsum. Under the hoods, it's implemented in C and is pretty efficient. Recently it's been optimized further to leverage BLAS based matrix-multiplication implementations.
So, a translation of the stated code onto NumPy territory keeping in mind that it follows 0-based indexing and the axes are visuallized differently than the dimensions with MATLAB, would be -
import numpy as np
# X is a NumPy array of shape : (n,p,3). So, a random one could be
# generated with : `X = np.random.rand(n,p,3)`.
T = np.zeros((p, p, p))
for i in range(p):
for j in range(p):
for k in range(p):
T[i, j, k] = np.sum(X[:, i, 0] * X[:, j, 1] * X[:, k, 2])
The einsum way to solve it would be -
np.einsum('ia,ib,ic->abc',X[...,0],X[...,1],X[...,2])
To leverage matrix-multiplication, use optimize flag -
np.einsum('ia,ib,ic->abc',X[...,0],X[...,1],X[...,2],optimize=True)
Timings (with large sizes)
In [27]: n,p = 100,100
...: X = np.random.rand(n,p,3)
In [28]: %%timeit
...: T = np.zeros((p, p, p))
...: for i in range(p):
...: for j in range(p):
...: for k in range(p):
...: T[i, j, k] = np.sum(X[:, i, 0] * X[:, j, 1] * X[:, k, 2])
1 loop, best of 3: 6.23 s per loop
In [29]: %timeit np.einsum('ia,ib,ic->abc',X[...,0],X[...,1],X[...,2])
1 loop, best of 3: 353 ms per loop
In [31]: %timeit np.einsum('ia,ib,ic->abc',X[...,0],X[...,1],X[...,2],optimize=True)
100 loops, best of 3: 10.5 ms per loop
In [32]: 6230.0/10.5
Out[32]: 593.3333333333334
Around 600x speedup there!
Suppose I have a m-by-n-by-p matrix "A", each indices stores a real number, now I want to create another matrix "B" and B(i, j, k) = f(A(i, j, k), i, j, k, otherVars), is there a faster way to do it in matlab rather than looping through all the elements? (notice the function requires the index number (i, j, k))
An example is as follows(The actual function f could be more complex):
A = rand(3, 4, 5);
B = zeros(size(A));
C = 10;
for x = 1:size(A, 1)
for y = 1:size(A, 2)
for z = 1:size(A, 3)
B(x, y, z) = A(x,y,z) + x - y * z + C;
end
end
end
I've tried creating a cell "B", and
B{i, j, k} = [A(i, j, k), i, j, k];
I then applied cellfun() to do the parallel computing, but it's even slower than a for-loop over each elements in A.
In my real implementation, function f is much more complex than B = A + X - Y.*Z + C; it takes four scaler values and I don't want to modify it since it's a function written in an external package. Any suggestions?
Vectorize it by building an ndgrid of the appropriate values:
[X,Y,Z] = ndgrid(1:size(A,1), 1:size(A,2), 1:size(A,3));
B = A + X - Y.*Z + C;
this is my first time here so I hope that someone can help me.
I'm trying to implementing the Gauss-Seidel method and the power method using a matrix with the storage CSR or called Morse storage. Unfortunately I can't manage to do better then the following codes:
GS-MORSE:
function [y] = gs_morse(aa, diag, col, row, nmax, tol)
[n, n] = size(A);
y = [1, 1, 1, 1];
m = 1;
while m < nmax,
for i = 1: n,
k1 = row(i);
k2 = row(i + 1) - 1;
for k = k1: k2,
y(i) = y(i) + aa(k) * x(col(k));
y(col(k)) = y(col(k)) + aa(k) * diag(i);
end
k2 = k2 + 1;
y(i) = y(i) + aa(k) * diag(i);
end
if (norm(y - x)) < tol
disp(y);
end
m = m + 1;
for i = 1: n,
x(i) = y(i);
end
end
POWER-MORSE:
I was able only to implement the power method but I don't understand how to use the former matrix... so my code for power method is:
function [y, l] = potencia_iterada(A, v)
numiter=100;
eps=1e-10;
x = v(:);
y = x/norm(x);
l = 0;
for k = 1: numiter,
x = A * y;
y = x / norm(x);
l0 = x.' * y;
if abs(l0) < eps
return
end
l = l0;
end
Please anyone can help me for completing these codes or can explain me how can I do that? I really don't understand how to do. Thank you very much