In MatLab I have to call the cdf of the t distribution (tcdf) iteratively (since the next input value depends on the previous output of tcdf), which unfortunately slows down my code massively.
tic
z = NaN(1e5,1);
z(1) = 1;
x = 2;
for ii = 2:1e5
x = tcdf(z(ii-1),x);
z(ii) = z(ii-1)*x;
end
toc
Elapsed time is 4.717087 seconds.
Is there a way to speed this up somehow?
For comparison:
tic
z = randn(1e5,1);
tcdf(z,5);
toc
Elapsed time is 0.091353 seconds.
Move the random number generation outside the loop as suggested below
numVal = 1e5
z = randn(numVal,1);
for ii = 2:numVal
z(ii) = z(ii-1) + z(ii);
end
tcdf(z,5);
Related
I'm trying to generate a new matrix, based on index values stored in another matrix.
This is trivial to do with a for loop, but this is currently the slowest line in some code I'm trying to optimise, and so I'm looking for a way to do it without the loop, and pulling my hair out.
I'm sure this has been answered before, and that I just don't know the right search terms.
n1 = 10;
n2 = 100;
a = randi(n2,[1,n1]);
b = randi(n2,[4,n1]);
c = rand(100,100);
for i = 1:n1
d(:,i) = c(a(i),b(:,i));
end
I'm assuming the value of n1 in your code is way bigger than in the example you provide, which would explain why it is "slow".
In order to do this without a loop, you can use Linear indexing:
n1 = 1e6;
n2 = 100;
a = randi(n2,[1,n1]);
b = randi(n2,[4,n1]);
c = rand(n2,n2);
% With a loop
d = zeros(4,n1);
tic
for i = 1:n1
d(:,i) = c(a(i),b(:,i));
end
toc
% A faster way for big values of `n1`
d2 = zeros(4,n1);
tic
a_rep = repmat(a,4,1); % Repeat row indexes to match the elements in b
idx_Lin = sub2ind([n2,n2],a_rep(:),b(:)); % Get linear indexes
d2(:) = c(idx_Lin); % Fill
toc
isequal(d,d2)
Elapsed time is 1.309654 seconds.
Elapsed time is 0.062549 seconds.
ans =
logical
1
Try This:
n1 = 10;
n2 = 100;
a = randi(n2,[1,n1]);
b = randi(n2,[4,n1]);
c = rand(100,100);
idx = (1:n1);
tic
d1=(c(a(idx),b(:,idx)))';
[idx,idy]=meshgrid(0:44:400,1:4);
d1=d1(idy+idx);
toc
this is the timeing:
Elapsed time is 0.000517 seconds.
I have the following data:
N = 10^3;
x = randn(N,1);
y = randn(N,1);
z = randn(N,1);
f = x.^2+y.^2+z.^2;
Now I want to split this continuous 3D space into nB bins.
nB = 20;
[~,~,x_bins] = histcounts(x,nB);
[~,~,y_bins] = histcounts(y,nB);
[~,~,z_bins] = histcounts(z,nB);
And put in each cube average f or nan if no observations happen in the cube:
F = nan(50,50,50);
for iX = 1:20
for iY = 1:20
for iZ = 1:20
idx = (x_bins==iX)&(y_bins==iY)&(z_bins==iZ);
F(iX,iY,iZ) = mean(f(idx));
end
end
end
isosurface(F,0.5)
This code does what I want. My problem is the speed. This code is extremely slow when N > 10^5 and nB = 100.
How can I speed up this code?
I also tried the accumarray() function:
subs=([x_bins,y_bins,z_bins]);
F2 = accumarray(subs,f,[],#mean);
all(F(:) == F2(:)) % false
However, this code produces a different result.
The problem with the code in the OP is that it tests all elements of the data for each element in the output array. The output array has nB^3 elements, the data has N elements, so the algorithm is O(N*nB^3). Instead, one can loop over the N elements of the input, and set the corresponding element in the output array, which is an operation O(N) (2nd code block below).
The accumarray solution in the OP needs to use the fillvals parameter, set it to NaN (3rd code block below).
To compare the results, one needs to explicitly test that both arrays have NaN in the same locations, and have equal non-NaN values elsewhere:
all( ( isnan(F(:)) & isnan(F2(:)) ) | ( F(:) == F2(:) ) )
% \-------same NaN values------/ \--same values--/
Here is code. All three versions produce identical results. Timings in Octave 4.4.1 (no JIT), in MATLAB the loop code should be faster. (Using input data from OP, with N=10^3 and nB=20).
%% OP's code, O(N*nB^3)
tic
F = nan(nB,nB,nB);
for iX = 1:nB
for iY = 1:nB
for iZ = 1:nB
idx = (x_bins==iX)&(y_bins==iY)&(z_bins==iZ);
F(iX,iY,iZ) = mean(f(idx));
end
end
end
toc
% Elapsed time is 1.61736 seconds.
%% Looping over input, O(N)
tic
s = zeros(nB,nB,nB);
c = zeros(nB,nB,nB);
ind = sub2ind([nB,nB,nB],x_bins,y_bins,z_bins);
for ii=1:N
s(ind(ii)) = s(ind(ii)) + f(ii);
c(ind(ii)) = c(ind(ii)) + 1;
end
F2 = s ./ c;
toc
% Elapsed time is 0.0606539 seconds.
%% Other alternative, using accumarray
tic
ind = sub2ind([nB,nB,nB],x_bins,y_bins,z_bins);
F3 = accumarray(ind,f,[nB,nB,nB],#mean,NaN);
toc
% Elapsed time is 0.14113 seconds.
I have a big square matrix (refConnect) with approx 500000 elements.
I need to perform this operation:
tmp = find(referenceCluster == 67);
for j=1:length(tmp)
refConnect(tmp(j),tmp)=1;
end
I wonder if there is a simple way to vectorise this so I can avoid the for loop which is taking forever.
Thanks for any help.
Cheers
Seems you can't significantly decrease execution time.
Try evaluate the execution time with this test function.
function test_spped(N, M)
if nargin < 1
N = 707;
end
if nargin < 2
M = 2;
end
refConnectIn = rand(N, N);
referenceCluster = randi(M, N, 1);
refConnectA = refConnectIn;
tic
tmpA = find(referenceCluster == 1);
for j=1:length(tmpA)
refConnectA(tmpA(j),tmpA) = 1;
end
toc
refConnectB = refConnectIn;
tic
tmpB = referenceCluster == 1;
refConnectB(tmpB, tmpB) = 1;
toc
if isequal(refConnectA, refConnectB)
disp('Result are equals');
else
disp('Result are UNEQUALS!');
end
With default parameters you get:
>> test_speed
Elapsed time is 0.002865 seconds.
Elapsed time is 0.001575 seconds.
Result are equals
Note, the execution time of the vectorized code (B case) can be worse for large M:
>> test_speed(707,1000)
Elapsed time is 0.001623 seconds.
Elapsed time is 0.002219 seconds.
Result are equals
I am trying to derive a function for calculating a moving/rolling correlation for two vectors and speed is a high priority, since I need to apply this function in an array function. What I have (which is too slow) is this:
Data1 = rand(3000,1);
Data2 = rand(3000,1);
function y = MovCorr(Data1,Data2)
[N,~] = size(Data1);
correlationTS = nan(N, 1);
for t = 20+1:N
correlationTS(t, :) = corr(Data1(t-20:t, 1),Data2(t-20:t,1),'rows','complete');
end
y = correlationTS;
end
I am thinking that the for loop could be done more efficiently if I knew how to generate the roling window indices and then applying accumarray. Any suggestions?
Following the advice from #knedlsepp, and using filter as in the movingstd, I found the following solution, which is quite fast:
function Cor = MovCorr1(Data1,Data2,k)
y = zscore(Data2);
n = size(y,1);
if (n<k)
Cor = NaN(n,1);
else
x = zscore(Data1);
x2 = x.^2;
y2 = y.^2;
xy = x .* y;
A=1;
B = ones(1,k);
Stdx = sqrt((filter(B,A,x2) - (filter(B,A,x).^2)*(1/k))/(k-1));
Stdy = sqrt((filter(B,A,y2) - (filter(B,A,y).^2)*(1/k))/(k-1));
Cor = (filter(B,A,xy) - filter(B,A,x).*filter(B,A,y)/k)./((k-1)*Stdx.*Stdy);
Cor(1:(k-1)) = NaN;
end
end
Comparing with my original solution the execution times are:
tic
MovCorr(Data1,Data2);
toc
Elapsed time is 1.017552 seconds.
tic
MovCorr1(Data1,Data2,21);
toc
Elapsed time is 0.019400 seconds.
I am very new in Matlab. I just try to implement sum of series 1+x+x^2/2!+x^3/3!..... . But I could not find out how to do it. So far I did just sum of numbers. Help please.
for ii = 1:length(a)
sum_a = sum_a + a(ii)
sum_a
end
n = 0 : 10; % elements of the series
x = 2; % value of x
s = sum(x .^ n ./ factorial(n)); % sum
The second part of your answer is:
n = 0:input('variable?')
Cheery's approach is perfectly valid when the number of terms of the series is small. For large values, a faster approach is as follows. This is more efficient because it avoids repeating multiplications:
m = 10;
x = 2;
result = 1+sum(cumprod(x./[1:m]));
Example running time for m = 1000; x = 1;
tic
for k = 1:1e4
result = 1+sum(cumprod(x./[1:m]));
end
toc
tic
for k = 1:1e4
result = sum(x.^(0:m)./factorial(0:m));
end
toc
gives
Elapsed time is 1.572464 seconds.
Elapsed time is 2.999566 seconds.