G'day,
I'm trying to find a way to use a vector of [x,y] points to index from a large matrix in MATLAB.
Usually, I would convert the subscript points to the linear index of the matrix.(for eg. Use a vector as an index to a matrix) However, the matrix is 4-dimensional, and I want to take all of the elements of the 3rd and 4th dimensions that have the same 1st and 2nd dimension. Let me hopefully demonstrate with an example:
Matrix = nan(4,4,2,2); % where the dimensions are (x,y,depth,time)
Matrix(1,2,:,:) = 999; % note that this value could change in depth (3rd dim) and time (4th time)
Matrix(3,4,:,:) = 888; % note that this value could change in depth (3rd dim) and time (4th time)
Matrix(4,4,:,:) = 124;
Now, I want to be able to index with the subscripts (1,2) and (3,4), etc and return not only the 999 and 888 which exist in Matrix(:,:,1,1) but the contents which exist at Matrix(:,:,1,2),Matrix(:,:,2,1) and Matrix(:,:,2,2), and so on (IRL, the dimensions of Matrix might be more like size(Matrix) = (300 250 30 200)
I don't want to use linear indices because I would like the results to be in a similar vector fashion. For example, I would like a result which is something like:
ans(time=1)
999 888 124
999 888 124
ans(time=2)
etc etc etc
etc etc etc
I'd also like to add that due to the size of the matrix I'm dealing with, speed is an issue here - thus why I'd like to use subscript indices to index to the data.
I should also mention that (unlike this question: Accessing values using subscripts without using sub2ind) since I want all the information stored in the extra dimensions, 3 and 4, of the i and jth indices, I don't think that a slightly faster version of sub2ind still would not cut it..
I can think of three ways to go about this
Simple loop
Just loop over all the 2D indices you have, and use colons to access the remaining dimensions:
for jj = 1:size(twoDinds,1)
M(twoDinds(jj,1),twoDinds(jj,2),:,:) = rand;
end
Vectorized calculation of Linear indices
Skip sub2ind and vectorize the computation of linear indices:
% generalized for arbitrary dimensions of M
sz = size(M);
nd = ndims(M);
arg = arrayfun(#(x)1:x, sz(3:nd), 'UniformOutput', false);
[argout{1:nd-2}] = ndgrid(arg{:});
argout = cellfun(...
#(x) repmat(x(:), size(twoDinds,1),1), ...
argout, 'Uniformoutput', false);
twoDinds = kron(twoDinds, ones(prod(sz(3:nd)),1));
% the linear indices
inds = twoDinds(:,1) + ([twoDinds(:,2) [argout{:}]]-1) * cumprod(sz(1:3)).';
Sub2ind
Just use the ready-made tool that ships with Matlab:
inds = sub2ind(size(M), twoDinds(:,1), twoDinds(:,2), argout{:});
Speed
So which one's the fastest? Let's find out:
clc
M = nan(4,4,2,2);
sz = size(M);
nd = ndims(M);
twoDinds = [...
1 2
4 3
3 4
4 4
2 1];
tic
for ii = 1:1e3
for jj = 1:size(twoDinds,1)
M(twoDinds(jj,1),twoDinds(jj,2),:,:) = rand;
end
end
toc
tic
twoDinds_prev = twoDinds;
for ii = 1:1e3
twoDinds = twoDinds_prev;
arg = arrayfun(#(x)1:x, sz(3:nd), 'UniformOutput', false);
[argout{1:nd-2}] = ndgrid(arg{:});
argout = cellfun(...
#(x) repmat(x(:), size(twoDinds,1),1), ...
argout, 'Uniformoutput', false);
twoDinds = kron(twoDinds, ones(prod(sz(3:nd)),1));
inds = twoDinds(:,1) + ([twoDinds(:,2) [argout{:}]]-1) * cumprod(sz(1:3)).';
M(inds) = rand;
end
toc
tic
for ii = 1:1e3
twoDinds = twoDinds_prev;
arg = arrayfun(#(x)1:x, sz(3:nd), 'UniformOutput', false);
[argout{1:nd-2}] = ndgrid(arg{:});
argout = cellfun(...
#(x) repmat(x(:), size(twoDinds,1),1), ...
argout, 'Uniformoutput', false);
twoDinds = kron(twoDinds, ones(prod(sz(3:nd)),1));
inds = sub2ind(size(M), twoDinds(:,1), twoDinds(:,2), argout{:});
M(inds) = rand;
end
toc
Results:
Elapsed time is 0.004778 seconds. % loop
Elapsed time is 0.807236 seconds. % vectorized linear inds
Elapsed time is 0.839970 seconds. % linear inds with sub2ind
Conclusion: use the loop.
Granted, the tests above are largely influenced by JIT's failure to compile the two last loops, and the non-specificity to 4D arrays (the last two method also work on ND arrays). Making a specialized version for 4D will undoubtedly be much faster.
Nevertheless, the indexing with simple loop is, well, simplest to do, easiest on the eyes and very fast too, thanks to JIT.
So, here is a possible answer... but it is messy. I suspect it would more computationally expensive then a more direct method... And this would definitely not be my preferred answer. It would be great if we could get the answer without any for loops!
Matrix = rand(100,200,30,400);
grabthese_x = (1 30 50 90);
grabthese_y = (61 9 180 189);
result=nan(size(length(grabthese_x),size(Matrix,3),size(Matrix,4));
for tt = 1:size(Matrix,4)
subset = squeeze(Matrix(grabthese_x,grabthese_y,:,tt));
for NN=1:size(Matrix,3)
result(:,NN,tt) = diag(subset(:,:,NN));
end
end
The resulting matrix, result should have size size(result) = (4 N tt).
I think this should work, even if Matrix isn't square. However, it is not ideal, as I said above.
Related
So, I need to vectorize some for loops into a single line. I understand how vectorize one and two for-loops, but am really struggling to do more than that. Essentially, I am computing a "blur" matrix M2 of size (n-2)x(m-2) of an original matrix M of size nxm, where s = size(M):
for x = 0:1
for y = 0:1
m = zeros(1, 9);
k = 1;
for i = 1:(s(1) - 1)
for j = 1:(s(2) - 1)
m(1, k) = M(i+x,j+y);
k = k+1;
end
end
M2(x+1,y+1) = mean(m);
end
end
This is the closest I've gotten:
for x=0:1
for y=0:1
M2(x+1, y+1) = mean(mean(M((x+1):(3+x),(y+1):(3+y))))
end
end
To get any closer to a one-line solution, it seems like there has to be some kind of "communication" where I assign two variables (x,y) to index over M2 and index over M; I just don't see how it can be done otherwise, but I am assured there is a solution.
Is there a reason why you are not using MATLAB's convolution function to help you do this? You are performing a blur with a 3 x 3 averaging kernel with overlapping neighbourhoods. This is exactly what convolution is doing. You can perform this using conv2:
M2 = conv2(M, ones(3) / 9, 'valid');
The 'valid' flag ensures that you return a size(M) - 2 matrix in both dimensions as you have requested.
In your code, you have hardcoded this for a 4 x 4 matrix. To double-check to see if we have the right results, let's generate a random 4 x 4 matrix:
rng(123);
M = rand(4, 4);
s = size(M);
If we run this with your code, we get:
>> M2
M2 =
0.5054 0.4707
0.5130 0.5276
Doing this with conv2:
>> M2 = conv2(M, ones(3) / 9, 'valid')
M2 =
0.5054 0.4707
0.5130 0.5276
However, if you want to do this from first principles, the overlapping of the pixel neighbourhoods is very difficult to escape using loops. The two for loop approach you have is good enough and it tackles the problem appropriately. I would make the size of the input instead of being hard coded. Therefore, write a function that does something like this:
function M2 = blur_fp(M)
s = size(M);
M2 = zeros(s(1) - 2, s(2) - 2);
for ii = 2 : s(1) - 1
for jj = 2 : s(2) - 1
p = M(ii - 1 : ii + 1, jj - 1 : jj + 1);
M2(ii - 1, jj - 1) = mean(p(:));
end
end
The first line of code defines the function, which we will call blur_fp. The next couple lines of code determine the size of the input matrix as well as initialising a blank matrix to store out output. We then loop through each pixel location in the matrix that is possible without the kernel going outside of the boundaries of the image, we grab a 3 x 3 neighbourhood with each pixel location serving as the centre, we then unroll the matrix into a single column vector, find the average and store it in the appropriate output. For small kernels and relatively large matrices, this should perform OK.
To take this a little further, you can use user Divakar's im2col_sliding function which takes overlapping neighbourhoods and unrolls them into columns. Therefore, each column represents a neighbourhood which you can then blur the input using vector-matrix multiplication. You would then use reshape to reshape the result back into a matrix:
T = im2col_sliding(M, [3 3]);
V = ones(1, 9) / 9;
s = size(M);
M2 = reshape(V * T, s(1) - 2, s(2) - 2);
This unfortunately cannot be done in a single line unless you use built-in functions. I'm not sure what your intention is, but hopefully the gamut of approaches you have seen here have given you some insight on how to do this efficiently. BTW, using loops for small matrices (i.e. 4 x 4) may be better in efficiency. You will start to notice performance changes when you increase the size of the input... then again, I would argue that using loops are competitive as of R2015b when the JIT has significantly improved.
I have a1 a2 a3. They are constants. I have a matrix A. What I want to do is to get a1*A, a2*A, a3*A three matrices. Then I want transfer them into a diagonal block matrix. For three constants case, this is easy. I can let b1 = a1*A, b2=a2*A, b3=a3*A, then use blkdiag(b1, b2, b3) in matlab.
What if I have n constants, a1 ... an. How could I do this without any looping?I know this can be done by kronecker product but this is very time-consuming and you need do a lot of unnecessary 0 * constant.
Thank you.
Discussion and code
This could be one approach with bsxfun(#plus that facilitates in linear indexing as coded in a function format -
function out = bsxfun_linidx(A,a)
%// Get sizes
[A_nrows,A_ncols] = size(A);
N_a = numel(a);
%// Linear indexing offsets between 2 columns in a block & between 2 blocks
off1 = A_nrows*N_a;
off2 = off1*A_ncols+A_nrows;
%// Get the matrix multiplication results
vals = bsxfun(#times,A,permute(a,[1 3 2])); %// OR vals = A(:)*a_arr;
%// Get linear indices for the first block
block1_idx = bsxfun(#plus,[1:A_nrows]',[0:A_ncols-1]*off1); %//'
%// Initialize output array base on fast pre-allocation inspired by -
%// http://undocumentedmatlab.com/blog/preallocation-performance
out(A_nrows*N_a,A_ncols*N_a) = 0;
%// Get linear indices for all blocks and place vals in out indexed by them
out(bsxfun(#plus,block1_idx(:),(0:N_a-1)*off2)) = vals;
return;
How to use: To use the above listed function code, let's suppose you have the a1, a2, a3, ...., an stored in a vector a, then do something like this out = bsxfun_linidx(A,a) to have the desired output in out.
Benchmarking
This section compares or benchmarks the approach listed in this answer against the other two approaches listed in the other answers for runtime performances.
Other answers were converted to function forms, like so -
function B = bsxfun_blkdiag(A,a)
B = bsxfun(#times, A, reshape(a,1,1,[])); %// step 1: compute products as a 3D array
B = mat2cell(B,size(A,1),size(A,2),ones(1,numel(a))); %// step 2: convert to cell array
B = blkdiag(B{:}); %// step 3: call blkdiag with comma-separated list from cell array
and,
function out = kron_diag(A,a_arr)
out = kron(diag(a_arr),A);
For the comparison, four combinations of sizes of A and a were tested, which are -
A as 500 x 500 and a as 1 x 10
A as 200 x 200 and a as 1 x 50
A as 100 x 100 and a as 1 x 100
A as 50 x 50 and a as 1 x 200
The benchmarking code used is listed next -
%// Datasizes
N_a = [10 50 100 200];
N_A = [500 200 100 50];
timeall = zeros(3,numel(N_a)); %// Array to store runtimes
for iter = 1:numel(N_a)
%// Create random inputs
a = randi(9,1,N_a(iter));
A = rand(N_A(iter),N_A(iter));
%// Time the approaches
func1 = #() kron_diag(A,a);
timeall(1,iter) = timeit(func1); clear func1
func2 = #() bsxfun_blkdiag(A,a);
timeall(2,iter) = timeit(func2); clear func2
func3 = #() bsxfun_linidx(A,a);
timeall(3,iter) = timeit(func3); clear func3
end
%// Plot runtimes against size of A
figure,hold on,grid on
plot(N_A,timeall(1,:),'-ro'),
plot(N_A,timeall(2,:),'-kx'),
plot(N_A,timeall(3,:),'-b+'),
legend('KRON + DIAG','BSXFUN + BLKDIAG','BSXFUN + LINEAR INDEXING'),
xlabel('Datasize (Size of A) ->'),ylabel('Runtimes (sec)'),title('Runtime Plot')
%// Plot runtimes against size of a
figure,hold on,grid on
plot(N_a,timeall(1,:),'-ro'),
plot(N_a,timeall(2,:),'-kx'),
plot(N_a,timeall(3,:),'-b+'),
legend('KRON + DIAG','BSXFUN + BLKDIAG','BSXFUN + LINEAR INDEXING'),
xlabel('Datasize (Size of a) ->'),ylabel('Runtimes (sec)'),title('Runtime Plot')
Runtime plots thus obtained at my end were -
Conclusions: As you can see, either one of the bsxfun based methods could be looked into, depending on what kind of datasizes you are dealing with!
Here's another approach:
Compute the products as a 3D array using bsxfun;
Convert into a cell array with one product (matrix) in each cell;
Call blkdiag with a comma-separated list generated from the cell array.
Let A denote your matrix, and a denote a vector with your constants. Then the desired result B is obtained as
B = bsxfun(#times, A, reshape(a,1,1,[])); %// step 1: compute products as a 3D array
B = mat2cell(B,size(A,1),size(A,2),ones(1,numel(a))); %// step 2: convert to cell array
B = blkdiag(B{:}); %// step 3: call blkdiag with comma-separated list from cell array
Here's a method using kron which seems to be faster and more memory efficient than Divakar's bsxfun based solution. I'm not sure if this is different to your method, but the timing seems pretty good. It might be worth doing some testing between the different methods to work out which is more efficient for you problem.
A=magic(4);
a1=1;
a2=2;
a3=3;
kron(diag([a1 a2 a3]),A)
I have a block matrix [A B C...] and a matrix D (all 2-dimensional). D has dimensions y-by-y, and A, B, C, etc are each z-by-y. Basically, what I want to compute is the matrix [D*(A'); D*(B'); D*(C');...], where X' refers to the transpose of X. However, I want to accomplish this without loops for speed considerations.
I have been playing with the reshape command for several hours now, and I know how to use it in other cases, but this use case is different from the other ones and I cannot figure it out. I also would like to avoid using multi-dimensional matrices if at all possible.
Honestly, a loop is probably the best way to do it. In my image-processing work I found a well-written loop that takes advantage of Matlab's JIT compiler is often faster than all the extra overhead of manipulating the data to be able to use a vectorised operation. A loop like this:
[m n] = size(A);
T = zeros(m, n);
AT = A';
for ii=1:m:n
T(:, ii:ii+m-1) = D * AT(ii:ii+m-1, :);
end
contains only built-in operators and the bare minimum of copying, and given the JIT is going to be hard to beat. Even if you want to factor in interpreter overhead it's still only a single statement with no functions to consider.
The "loop-free" version with extra faffing around and memory copying, is to split the matrix and iterate over the blocks with a hidden loop:
blksize = size(D, 1);
blkcnt = size(A, 2) / blksize;
blocks = mat2cell(A, blksize, repmat(blksize,1,blkcnt));
blocks = cellfun(#(x) D*x', blocks, 'UniformOutput', false);
T = cell2mat(blocks);
Of course, if you have access to the Image Processing Toolbox, you can also cheat horribly:
T = blockproc(A, size(D), #(x) D*x.data');
Prospective approach & Solution Code
Given:
M is the block matrix [A B C...], where each A, B, C etc. are of size z x y. Let the number of such matrices be num_mat for easy reference later on.
If those matrices are concatenated along the columns, then M would be of size z x num_mat*y.
D is the matrix to be multiplied with each of those matrices A, B, C etc. and is of size y x y.
Now, as stated in the problem, the output you are after is [D*(A'); D*(B'); D*(C');...], i.e. the multiplication results being concatenated along the rows.
If you are okay with those multiplication results to be concatenated along the columns instead i.e. [D*(A') D*(B') D*(C') ...],
you can achieve the same with some reshaping and then performing the
matrix multiplications for the entire M with D and thus have a vectorized no-loop approach. Thus, to get such a matrix multiplication result, you can do -
mults = D*reshape(permute(reshape(M,z,y,[]),[2 1 3]),y,[]);
But, if you HAVE to get an output with the multiplication results being concatenated along the rows, you need to do some more reshaping like so -
out = reshape(permute(reshape(mults,y,z,[]),[1 3 2]),[],z);
Benchmarking
This section covers benchmarking codes comparing the proposed vectorized approach against a naive JIT powered loopy approach to get the desired output. As discussed earlier, depending on how the output array must hold the multiplication results, you can have two cases.
Case I: Multiplication results concatenated along the columns
%// Define size paramters and then define random inputs with those
z = 500; y = 500; num_mat = 500;
M = rand(z,num_mat*y);
D = rand(y,y);
%// Warm up tic/toc.
for k = 1:100000
tic(); elapsed = toc();
end
disp('---------------------------- With loopy approach')
tic
out1 = zeros(z,y*num_mat);
for k1 = 1:y:y*num_mat
out1(:,k1:k1+y-1) = D*M(:,k1:k1+y-1).'; %//'
end
toc, clear out1 k1
disp('---------------------------- With proposed approach')
tic
mults = D*reshape(permute(reshape(M,z,y,[]),[2 1 3]),y,[]);
toc
Case II: Multiplication results concatenated along the rows
%// Define size paramters and then define random inputs with those
z = 500; y = 500; num_mat = 500;
M = rand(z,num_mat*y);
D = rand(y,y);
%// Warm up tic/toc.
for k = 1:100000
tic(); elapsed = toc();
end
disp('---------------------------- With loopy approach')
tic
out1 = zeros(y*num_mat,z);
for k1 = 1:y:y*num_mat
out1(k1:k1+y-1,:) = D*M(:,k1:k1+y-1).'; %//'
end
toc, clear out1 k1
disp('---------------------------- With proposed approach')
tic
mults = D*reshape(permute(reshape(M,z,y,[]),[2 1 3]),y,[]);
out2 = reshape(permute(reshape(mults,y,z,[]),[1 3 2]),[],z);
toc
Runtimes
Case I:
---------------------------- With loopy approach
Elapsed time is 3.889852 seconds.
---------------------------- With proposed approach
Elapsed time is 3.051376 seconds.
Case II:
---------------------------- With loopy approach
Elapsed time is 3.798058 seconds.
---------------------------- With proposed approach
Elapsed time is 3.292559 seconds.
Conclusions
The runtimes suggest about a good 25% speedup with the proposed vectorized approach! So, hopefully this works out for you!
If you want to get A, B, and C from a bigger matrix you can do this, assuming the bigger matrix is called X:
A = X(:,1:y)
B = X(:,y+1:2*y)
C = X(:,2*y+1:3*y)
If there are N such matrices, the best way is to use reshape like:
F = reshape(X, x,y,N)
Then use a loop to generate a new matrix I call it F1 as:
F1=[];
for n=1:N
F1 = [F1 F(:,:,n)'];
end
Then compute F2 as:
F2 = D*F1;
and finally get your result as:
R = reshape(F2,N*y,x)
Note: this for loop does not slow you down as it is just to reformat the matrix and the multiplication is done in matrix form.
I need to make a scilab / MATLAB program that averages the values of a 3D matrix in cubes of a given size(N x N x N).I am eternally grateful to anyone who can help me.
Thanks in advance
In MATLAB, mat2cell and cellfun make a great team for working on N-dimensional non-overlapping blocks, as I think is the case in the question. An example scenario:
[IN]: A = [30x30x30] array
[IN]: bd = [5 5 5], size of cube
[OUT]: B = [6x6x6] array of block means
To accomplish the above, the solution is:
dims = [30 30 30]; bd = [5 5 5];
A = rand(dims);
f = floor(dims./bd);
remDims = mod(dims,bd); % handle dims that are not a multiple of block size
Ac = mat2cell(A,...
[bd(1)*ones(f(1),1); remDims(1)*ones(remDims(1)>0)], ....
[bd(2)*ones(f(2),1); remDims(2)*ones(remDims(2)>0)], ....
[bd(3)*ones(f(3),1); remDims(3)*ones(remDims(3)>0)] );
B = cellfun(#(x) mean(x(:)),Ac);
If you need a full size output with the mean values replicated, there is a straightforward solution involving the 'UniformOutput' option of cellfun followed by cell2mat.
If you want overlapping cubes and the same size output as input, you can simply do convn(A,ones(blockDims)/prod(blockDims),'same').
EDIT: Simplifications, clarity, generality and fixes.
N = 10; %Same as OP's parameter
M = 10*N;%The input matrix's size in each dimensiona, assumes M is an integer multiple of N
Mat = rand(M,M,M); % A random input matrix
avgs = zeros((M/N)^3,1); %Initializing output vector
l=1; %indexing
for i=1:M/N %indexing 1st coord
for j=1:M/N %indexing 2nd coord
for k=1:M/N % indexing third coord
temp = Mat((i-1)*N+1:i*N,(j-1)*N+1:j*N,(k-1)*N+1:k*N); %temporary copy
avg(l) = mean(temp(:)); %averaging operation on the N*N*N copy
l = l+1; %increment indexing
end
end
end
The for loops and copying can be eliminated once you get the gist of indexing.
I have a non-fixed dimensional matrix M, from which I want to access a single element.
The element's indices are contained in a vector J.
So for example:
M = rand(6,4,8,2);
J = [5 2 7 1];
output = M(5,2,7,1)
This time M has 4 dimensions, but this is not known in advance. This is dependent on the setup of the algorithm I'm writing. It could likewise be that
M = rand(6,4);
J = [3 1];
output = M(3,1)
so I can't simply use
output=M(J(1),J(2))
I was thinking of using sub2ind, but this also needs its variables comma separated..
#gnovice
this works, but I intend to use this kind of element extraction from the matrix M quite a lot. So if I have to create a temporary variable cellJ every time I access M, wouldn't this tremendously slow down the computation??
I could also write a separate function
function x= getM(M,J)
x=M(J(1),J(2));
% M doesn't change in this function, so no mem copy needed = passed by reference
end
and adapt this for different configurations of the algorithm. This is of course a speed vs flexibility consideration which I hadn't included in my question..
BUT: this is only available for getting the element, for setting there is no other way than actually using the indices (and preferably the linear index). I still think sub2ind is an option. The final result I had intended was something like:
function idx = getLinearIdx(J, size_M)
idx = ...
end
RESULTS:
function lin_idx = Lidx_ml( J, M )%#eml
%LIDX_ML converts an array of indices J for a multidimensional array M to
%linear indices, directly useable on M
%
% INPUT
% J NxP matrix containing P sets of N indices
% M A example matrix, with same size as on which the indices in J
% will be applicable.
%
% OUTPUT
% lin_idx Px1 array of linear indices
%
% method 1
%lin_idx = zeros(size(J,2),1);
%for ii = 1:size(J,2)
% cellJ = num2cell(J(:,ii));
% lin_idx(ii) = sub2ind(size(M),cellJ{:});
%end
% method 2
sizeM = size(M);
J(2:end,:) = J(2:end,:)-1;
lin_idx = cumprod([1 sizeM(1:end-1)])*J;
end
method 2 is 20 (small number of index sets (=P) to convert) to 80 (large number of index sets (=P)) times faster than method 1. easy choice
For the general case where J can be any length (which I assume always matches the number of dimensions in M), there are a couple options you have:
You can place each entry of J in a cell of a cell array using the num2cell function, then create a comma-separated list from this cell array using the colon operator:
cellJ = num2cell(J);
output = M(cellJ{:});
You can sidestep the sub2ind function and compute the linear index yourself with a little bit of math:
sizeM = size(M);
index = cumprod([1 sizeM(1:end-1)]) * (J(:) - [0; ones(numel(J)-1, 1)]);
output = M(index);
Here is a version of gnovices option 2) which allows to process a whole matrix of subscripts, where each row contains one subscript. E.g for 3 subscripts:
J = [5 2 7 1
1 5 2 7
4 3 9 2];
sizeM = size(M);
idx = cumprod([1 sizeX(1:end-1)])*(J - [zeros(size(J,1),1) ones(size(J,1),size(J,2)-1)]).';