So we are given with some w and we want to so something like such pseudocode:
u = zeros(size(w));
for o=1:length(size(w))
for i=1:size(w)(1),
for j=1:size(w)(2),
...
for k=1:size(w)(length(size(w))),
u(i, j, ..., k )=1/(exp((-w(i,j, ..., k )))+25);
end
...
end
end
end
is such thing possible with Matlab and how to do it?
This is certainly possible, but also not recommended, since it's very non-idiomatic.
For simplicity, I assume that there is a typo in your question and that it should say exp(-w(i,j,...)).
Then, u can be calculated as
u = exp(-w);
You can use recursion - write a function, that iterates over the desired variables, and calls itself each time.
#Jonas's answer is the best option, however for a general-purpose code (one that #Jonas's 'matricized' answer does not apply), you can use a code-generation approach, for example:
fcode = fopen('manyForLoopsScript.m','w');
w = rand(2,3,4);
numLoops = length(size(w));
u = zeros(size(w));
% open the 'for' loops
for m = 1 : numLoops
fprintf(fcode, 'for a%d = 1:size(w,%d)\n',m,m);
end
% the actuall 'stuff'
indStr = [];
for m = 1 : numLoops
indStr = [indStr,sprintf('a%d,',m)];
end
indStr = indStr(1:end-1);
fprintf(fcode, ['u(' indStr ') = exp(-w(' indStr '));\n']);
% close the 'for' loops
for m = 1 : numLoops
fprintf(fcode, 'end\n');
end
fclose(fcode);
manyForLoopsScript
One option is to rewrite the function you want to apply to be vectorized, so that it is computed element-wise (the same way EXP built-in function works for scalar/vector/matrix input):
u = 1./(exp(-w)+25);
Otherwise, if you prefer a for-loop, all you need to do is to traverse the multi-dimensional input matrix using linear indices (in a column-major order), apply the function to each element, then RESHAPE the result back into the expected shape:
u = zeros(numel(w),1);
for i=1:numel(w)
u(i) = 1 ./ ( exp(-w(i)) + 25 );
end
u = reshape(u, size(w));
This is basically what the ARRAYFUN functions does for you:
u = arrayfun(#(x) 1./(exp(-x)+25), w);
In the chance that you also need access to the actual indices while looping over the elements of the matrix, you could always get them with IND2SUB (which converts linear indices to subscripts), or even generate all of them with functions like MESHGRID and NDGRID...
Related
I am using this command in Matlab:
grazAng = grazingang(H,R)
If i fix H, I can treat R as a vector:
z=[];
for i=1:1000
z(i)=abs(grazingang(1,i));
end
Now I would like to have both H and R to by dynamic. For example:
H=[0,0.25,0.5]
R=[1,2,3]
And I would like my loop to run three times, each time selecting a pair of (H,R) values with the same indexes, i.e. (0,1),(0.25,2),(0.5,3) and then store the result in z. Could anyone help me out with this?
Remember, everything in MATLAB is an array. To do this with a loop, you need to index into the arrays:
H = [0,0.25,0.5];
R = [1,2,3];
z = zeros(size(H)); % Pre-allocation is generally advised
for i = 1:1000
z(i) = abs(grazingang(H(i),R(i)));
end
But MATLAB functions generally accept vectors and do this for you, so all you need to do is:
H=[0,0.25,0.5];
R=[1,2,3];
z = abs(grazingang(H,R));
Consider the nxn matrix C with elements
Cij = 0 if i/j < 2
Cij = ij^2 otherwise
with 1 <= i,j <= n
Write a Matlab function matSetup that takes as input n and returns the matrix C. Use your function to create C for n = 6. function [Cij]= matSetup(n)
I have written this but it seems not to be correct
function Cij=matSetup(n)
for n=1:n
% whatever you write here is done with i=1, then i=2, then i=3, etc.
Cij(3,j)=i+7;
if (i/j)<2
Cij=0
else
Cij=i*(j)^2
end
end
end
Unfortunately, you can't just write something math-like and have a computer understand it. Things like 1<=i<=n have to be written instead with something like an explicit loop. For Matlab, here's one way to write a loop:
for i=1:n
% whatever you write here is done with i=1, then i=2, then i=3, etc.
end
To assign a value to an element of an array in Matlab, do something like this:
Cij(3,j)=i+7;
To test for a condition in Matlab, do this:
if i+3>2*j
% What you write here is done if the condition is true
else
% What you write here is done if the condition is false
end
If you put all of those things together correctly, you should be able to write your desired function.
Like the other answer, you should first learn to use loops to write a naive program to achieve your goal. When you are comfortable enough, you can try vectorizing your program with functions like meshgrid. Below is an example:
n = 20;
eps = 1/(n+1);
[x, y] = meshgrid(1:n, 1:n);
r = y./x;
z = heaviside(r - 2 + eps) .* y .* x.^2;
I have the following piece of code, which runs through two nested for-loops and has an if condition in the middle:
N=1e4;
cond_array = [0:(N/2-1) -N/2+(0:(N/2-1))]/(N);
condition = 0.1;
arr_one = zeros(N, 1);
arr_two = zeros(N, 1);
for m=1:N
for k=1:N
if(abs(cond_array(k)) <= condition)
arr_one(m) = arr_one(m) + m*k;
else
arr_two(m) = arr_two(m) + m*k;
end
end
end
I'd like to optimize this code as I potentially need to use very large N (>1e4) and from my own experience, for-loops in MATLAB are often very CPU-consuming and not very efficient.
Is there a way to optimize this piece of code, maybe by using vectorized functions that work on entire arrays?
Here is a much faster (and readable) way to get the same result:
N=1e4;
cond_array = [0:(N/2-1) -N/2+(0:(N/2-1))]/(N);
condition = 0.1;
% this is so you don't check the same condition multiple times:
cond = abs(cond_array)<=condition;
% a vector of the positions in 'arr_one' and 'arr_two':
pos = (1:N).';
% instead of m*k(1)+m*k(2)... use: m*sum(k):
k1 = sum(pos(cond)); % for arr_one
k2 = sum(pos(~cond));% for arr_two
% the final result:
arr_one = pos.*k1;
arr_two = pos.*k2;
For the second case you mention in the comments, where m*k becomes exp((m-1)*(k-1)), we can again compute the sum of exp((m(1)-1)*(k(1)-1)) +...+ exp((m(1)-1)*(k(N)-1))... using vectorization, and then use some minimal loop to go over all ms:
% we define new vectors of constants:
k1 = pos(cond);
k2 = pos(~cond);
% define new functions for each of the arrays:
ek1 = #(m) sum(exp((m-1).*(k1-1)));
ek2 = #(m) sum(exp((m-1).*(k2-1)));
% and we use 'arrayfun' for the final result:
arr_one = arrayfun(ek1,1:N).';
arr_two = arrayfun(ek2,1:N).';
arrayfun is not faster than a for loop, just more compact. Using a for loop will be like this:
arr_one = zeros(N,1);
arr_two = zeros(N,1);
for k = 1:N
arr_one(k) = ek1(k);
arr_two(k) = ek2(k);
end
And here is another, more general, option using bsxfun:
ek = #(m,k) exp((m-1).*(k-1));
arr_one = sum(bsxfun(ek,1:N,k1)).';
arr_two = sum(bsxfun(ek,1:N,k2)).';
Look into the use of logical indexing.
Without understanding your code too well, I'd imagine that you can remove the loops through array functions similar to the below.
arr_one(abs(cond_array)<condition)
This will remove the need for the comparison function, which will allow you to use an anonymous array function in order to create your computation.
However, if I understand your code correctly, you're just adding the product of the position of the condition index and the position of the array index to the array.
If you do that, it will be much easier to just do something similar to the below.
position=(1:N)'
arr_one(abs(cond_array)<condition)=arr_one(abs(cond_array)<condition)+position(abs(cond_array)<condition)
In this statement, you're looking for all the items where the cond_array is less than the condition, and adding the position (effectively a proxy for the m and k in your previous example).
In terms of runtime, I think you have to be 1 to 3 orders of magnitude larger than 1e4 for this code to make a significant difference in runtimes.
I have a function A(k) which returns a matrix dependent on k when I type for example A(1). I want to automatically create the matrix:
[A(3)*A(2)*A(1) A(3)*A(2) A(3)]
In Mathematica I can do for example:
Table[Apply[Dot,Table[A(k),{k,3,i,-1}]],{i,1,3}]
Assume A(k) is a function which returns a 3x3 matrix. For example typing A(1) may return:
[1,2,3;4,5,6;7,8,9]
Explanation of Mathematica code: Table[A(k),{k,3,i,-1}] is a "loop" from k=3 to k=i with each iteration decrementing k by -1. Therefore the output would be a list {A(3),A(2),...,A(i)}... obviously k starting from 3 then for i=2 the output list will be {A(3),A(2)}. The function Apply[Dot,Table[A(k),{k,3,i,-1}]] multiplied the elements of the list together. For i=2 this produces A(3)*A(2). Finally, Table[Apply[Dot,Table[A(k),{k,3,i,-1}]],{i,1,3}] applies the same logic as the first statement, looping i from 1 to 3. Because the inner table depends on i, this creates a list of elements {A(3)*A(2)*A(1),A(3)*A(2),A(3)}. A list is Mathematica's version of a matrix.
How can I achieve the same effect in MATLAB, i.e. not use a for loop to achieve the result? Thanks!
For the record, here's a looping version, mostly for subsequent timing checks. This is not the answer you want, but the answer you need;)
N = 10; %size of A
old = eye(N);
M2 = [];
for i=3:-1:1
new = old*A(i);
M2 = [new M2];
old = new;
end
Or if you want to be really efficient (which is probably not the case):
N = 10; %size of A
M2 = A(3);
old = M2;
for i=2:-1:1
new = old*A(i);
M2 = [new M2];
old = new;
end
This answer basically is the solution to the problem, but as we are discussing efficient use of matlab I want to leave my thoughts here how to use Matlab efficient. Instead of the large 2D-Matrix it is much simpler to create a 3D-Matrix where the results of each multiplication are stacked. If a 3D-Matrix is acceptable use the code as it is, otherwise comment in the last line to get a 2D-matrix.
M3=[];
n=3;
M3(:,:,n) = A(n);
for ix=n-1:-1:1
M3(:,:,ix) = M3(:,:,ix+1)*A(i);
end
%M3=reshape(M3,size(M3,1),[]);
You can use arrayfun() with indexes as parameters, eg:
%// Compute the product A(3)*A(2)*...*A(idx)
F = #(idx) prod(reshape(cell2mat(arrayfun(A,3:-1:idx, 'UniformOutput', false)),10,10,4-idx),3);
%// Use the last function with idx=1:3
M = cell2mat(arrayfun(F,1:3, 'UniformOutput', false));
Say I have a long list A of values (say of length 1000) for which I want to compute the std in pairs of 100, i.e. I want to compute std(A(1:100)), std(A(2:101)), std(A(3:102)), ..., std(A(901:1000)).
In Excel/VBA one can easily accomplish this by writing e.g. =STDEV(A1:A100) in one cell and then filling down in one go. Now my question is, how could one accomplish this efficiently in Matlab without having to use any expensive for-loops.
edit: Is it also possible to do this for a list of time series, e.g. when A has dimensions 1000 x 4 (i.e. 4 time series of length 1000)? The output matrix should then have dimensions 901 x 4.
Note: For the fastest solution see Luis Mendo's answer
So firstly using a for loop for this (especially if those are your actual dimensions) really isn't going to be expensive. Unless you're using a very old version of Matlab, the JIT compiler (together with pre-allocation of course) makes for loops inexpensive.
Secondly - have you tried for loops yet? Because you should really try out the naive implementation first before you start optimizing prematurely.
Thirdly - arrayfun can make this a one liner but it is basically just a for loop with extra overhead and very likely to be slower than a for loop if speed really is your concern.
Finally some code:
n = 1000;
A = rand(n,1);
l = 100;
for loop (hardly bulky, likely to be efficient):
S = zeros(n-l+1,1); %//Pre-allocation of memory like this is essential for efficiency!
for t = 1:(n-l+1)
S(t) = std(A(t:(t+l-1)));
end
A vectorized (memory in-efficient!) solution:
[X,Y] = meshgrid(1:l)
S = std(A(X+Y-1))
A probably better vectorized solution (and a one-liner) but still memory in-efficient:
S = std(A(bsxfun(#plus, 0:l-1, (1:l)')))
Note that with all these methods you can replace std with any function so long as it is applies itself to the columns of the matrix (which is the standard in Matlab)
Going 2D:
To go 2D we need to go 3D
n = 1000;
k = 4;
A = rand(n,k);
l = 100;
ind = bsxfun(#plus, permute(o:n:(k-1)*n, [3,1,2]), bsxfun(#plus, 0:l-1, (1:l)')); %'
S = squeeze(std(A(ind)));
M = squeeze(mean(A(ind)));
%// etc...
OR
[X,Y,Z] = meshgrid(1:l, 1:l, o:n:(k-1)*n);
ind = X+Y+Z-1;
S = squeeze(std(A(ind)))
M = squeeze(mean(A(ind)))
%// etc...
OR
ind = bsxfun(#plus, 0:l-1, (1:l)'); %'
for t = 1:k
S = std(A(ind));
M = mean(A(ind));
%// etc...
end
OR (taken from Luis Mendo's answer - note in his answer he shows a faster alternative to this simple loop)
S = zeros(n-l+1,k);
M = zeros(n-l+1,k);
for t = 1:(n-l+1)
S(t,:) = std(A(k:(k+l-1),:));
M(t,:) = mean(A(k:(k+l-1),:));
%// etc...
end
What you're doing is basically a filter operation.
If you have access to the image processing toolbox,
stdfilt(A,ones(101,1)) %# assumes that data series are in columns
will do the trick (no matter the dimensionality of A). Note that if you also have access to the parallel computing toolbox, you can let filter operations like these run on a GPU, although your problem might be too small to generate noticeable speedups.
To minimize number of operations, you can exploit the fact that the standard deviation can be computed as a difference involving second and first moments,
and moments over a rolling window are obtained efficiently with a cumulative sum (using cumsum):
A = randn(1000,4); %// random data
N = 100; %// window size
c = size(A,2);
A1 = [zeros(1,c); cumsum(A)];
A2 = [zeros(1,c); cumsum(A.^2)];
S = sqrt( (A2(1+N:end,:)-A2(1:end-N,:) ...
- (A1(1+N:end,:)-A1(1:end-N,:)).^2/N) / (N-1) ); %// result
Benchmarking
Here's a comparison against a loop based solution, using timeit. The loop approach is as in Dan's solution but adapted to the 2D case, exploting the fact that std works along each column in a vectorized manner.
%// File loop_approach.m
function S = loop_approach(A,N);
[n, p] = size(A);
S = zeros(n-N+1,p);
for k = 1:(n-N+1)
S(k,:) = std(A(k:(k+N-1),:));
end
%// File bsxfun_approach.m
function S = bsxfun_approach(A,N);
[n, p] = size(A);
ind = bsxfun(#plus, permute(0:n:(p-1)*n, [3,1,2]), bsxfun(#plus, 0:n-N, (1:N).')); %'
S = squeeze(std(A(ind)));
%// File cumsum_approach.m
function S = cumsum_approach(A,N);
c = size(A,2);
A1 = [zeros(1,c); cumsum(A)];
A2 = [zeros(1,c); cumsum(A.^2)];
S = sqrt( (A2(1+N:end,:)-A2(1:end-N,:) ...
- (A1(1+N:end,:)-A1(1:end-N,:)).^2/N) / (N-1) );
%// Benchmarking code
clear all
A = randn(1000,4); %// Or A = randn(1000,1);
N = 100;
t_loop = timeit(#() loop_approach(A,N));
t_bsxfun = timeit(#() bsxfun_approach(A,N));
t_cumsum = timeit(#() cumsum_approach(A,N));
disp(' ')
disp(['loop approach: ' num2str(t_loop)])
disp(['bsxfun approach: ' num2str(t_bsxfun)])
disp(['cumsum approach: ' num2str(t_cumsum)])
disp(' ')
disp(['bsxfun/loop gain factor: ' num2str(t_loop/t_bsxfun)])
disp(['cumsum/loop gain factor: ' num2str(t_loop/t_cumsum)])
Results
I'm using Matlab R2014b, Windows 7 64 bits, dual core processor, 4 GB RAM:
4-column case:
loop approach: 0.092035
bsxfun approach: 0.023535
cumsum approach: 0.0002338
bsxfun/loop gain factor: 3.9106
cumsum/loop gain factor: 393.6526
Single-column case:
loop approach: 0.085618
bsxfun approach: 0.0040495
cumsum approach: 8.3642e-05
bsxfun/loop gain factor: 21.1431
cumsum/loop gain factor: 1023.6236
So the cumsum-based approach seems to be the fastest: about 400 times faster than the loop in the 4-column case, and 1000 times faster in the single-column case.
Several functions can do the job efficiently in Matlab.
On one side, you can use functions such as colfilt or nlfilter, which performs computations on sliding blocks. colfilt is way more efficient than nlfilter, but can be used only if the order of the elements inside a block does not matter. Here is how to use it on your data:
S = colfilt(A, [100,1], 'sliding', #std);
or
S = nlfilter(A, [100,1], #std);
On your example, you can clearly see the difference of performance. But there is a trick : both functions pad the input array so that the output vector has the same size as the input array. To get only the relevant part of the output vector, you need to skip the first floor((100-1)/2) = 49 first elements, and take 1000-100+1 values.
S(50:end-50)
But there is also another solution, close to colfilt, more efficient. colfilt calls col2im to reshape the input vector into a matrix on which it applies the given function on each distinct column. This transforms your input vector of size [1000,1] into a matrix of size [100,901]. But colfilt pads the input array with 0 or 1, and you don't need it. So you can run colfilt without the padding step, then apply std on each column and this is easy because std applied on a matrix returns a row vector of the stds of the columns. Finally, transpose it to get a column vector if you want. In brief and in one line:
S = std(im2col(X,[100 1],'sliding')).';
Remark: if you want to apply a more complex function, see the code of colfilt, line 144 and 147 (for v2013b).
If your concern is speed of the for loop, you can greatly reduce the number of loop iteration by folding your vector into an array (using reshape) with the columns having the number of element you want to apply your function on.
This will let Matlab and the JIT perform the optimization (and in most case they do that way better than us) by calculating your function on each column of your array.
You then reshape an offseted version of your array and do the same. You will still need a loop but the number of iteration will only be l (so 100 in your example case), instead of n-l+1=901 in a classic for loop (one window at a time).
When you're done, you reshape the array of result in a vector, then you still need to calculate manually the last window, but overall it is still much faster.
Taking the same input notation than Dan:
n = 1000;
A = rand(n,1);
l = 100;
It will take this shape:
width = (n/l)-1 ; %// width of each line in the temporary result array
tmp = zeros( l , width ) ; %// preallocation never hurts
for k = 1:l
tmp(k,:) = std( reshape( A(k:end-l+k-1) , l , [] ) ) ; %// calculate your stat on the array (reshaped vector)
end
S2 = [tmp(:) ; std( A(end-l+1:end) ) ] ; %// "unfold" your results then add the last window calculation
If I tic ... toc the complete loop version and the folded one, I obtain this averaged results:
Elapsed time is 0.057190 seconds. %// windows by window FOR loop
Elapsed time is 0.016345 seconds. %// "Folded" FOR loop
I know tic/toc is not the way to go for perfect timing but I don't have the timeit function on my matlab version. Besides, the difference is significant enough to show that there is an improvement (albeit not precisely quantifiable by this method). I removed the first run of course and I checked that the results are consistent with different matrix sizes.
Now regarding your "one liner" request, I suggest your wrap this code into a function like so:
function out = foldfunction( func , vec , nPts )
n = length( vec ) ;
width = (n/nPts)-1 ;
tmp = zeros( nPts , width ) ;
for k = 1:nPts
tmp(k,:) = func( reshape( vec(k:end-nPts+k-1) , nPts , [] ) ) ;
end
out = [tmp(:) ; func( vec(end-nPts+1:end) ) ] ;
Which in your main code allows you to call it in one line:
S = foldfunction( #std , A , l ) ;
The other great benefit of this format, is that you can use the very same sub function for other statistical function. For example, if you want the "mean" of your windows, you call the same just changing the func argument:
S = foldfunction( #mean , A , l ) ;
Only restriction, as it is it only works for vector as input, but with a bit of rework it could be made to take arrays as input too.