Deleting matrix elements by = [] vs reassigning matrix - matlab

Is there any difference between these two methods for deleting elements in Matlab:
ElementsToDelete = [0 0 1 0 1 0 0 1 1 0]
A = 1:10
A(ElementsToDelete) = []
%Versus
A = 1:10
A = A(~ElementsToDelete)
Are there times when one method is more appropriate than the other? Is there a difference in efficiency? Or are they completely interchangeable?

Try this:
A = rand(1e3, 1);
b = A<0.5;
tic;
for ii = 1:1e5
a = A;
a(b) = [];
end
toc
tic;
for ii = 1:1e5
a = A;
a = a(~b);
end
toc
Results:
Elapsed time is 1.654146 seconds
Elapsed time is 1.126325 seconds
So the difference is a speed factor of 1.5 in favour of re-assigning. This however, is worse:
A = rand(1e4, 1);
stop = 0;
for jj = 1:10
a = A;
start = tic;
for ii = 1:1e5
a(a < rand) = [];
end
stop = stop + toc(start);
end
avg1 = stop/10
stop = 0;
for jj = 1:10
a = A;
start = tic;
for ii = 1:1e5
a = a(a > rand);
end
stop = stop + toc(start);
end
avg2 = stop/10
avg1/avg2
Results:
avg1 = 1.1740235 seconds
avg2 = 0.1850463 seconds
avg1/avg2 = 6.344485136963019
So, the factor's increased to well over 6.
My guess is that deletion (i.e., assigning with []) re-writes the entire array on each and every occurrence of a true in the internal loop through the logical indices. This is hopelessly inefficient, as becomes apparent when testing it like this. Re-assigning on the other hand can determine the size of the new array beforehand and initialize it accordingly; no re-writes needed.
Why the JIT does not compile the one into the other is a mystery to me, because deletion is a far more intuitive notation IMHO. But, as you see, it is inefficient compared to alternatives, and should thus be used sparingly. Never use it inside loops!

Related

MATLAB indexing by indexes in another matrix

I'm trying to generate a new matrix, based on index values stored in another matrix.
This is trivial to do with a for loop, but this is currently the slowest line in some code I'm trying to optimise, and so I'm looking for a way to do it without the loop, and pulling my hair out.
I'm sure this has been answered before, and that I just don't know the right search terms.
n1 = 10;
n2 = 100;
a = randi(n2,[1,n1]);
b = randi(n2,[4,n1]);
c = rand(100,100);
for i = 1:n1
d(:,i) = c(a(i),b(:,i));
end
I'm assuming the value of n1 in your code is way bigger than in the example you provide, which would explain why it is "slow".
In order to do this without a loop, you can use Linear indexing:
n1 = 1e6;
n2 = 100;
a = randi(n2,[1,n1]);
b = randi(n2,[4,n1]);
c = rand(n2,n2);
% With a loop
d = zeros(4,n1);
tic
for i = 1:n1
d(:,i) = c(a(i),b(:,i));
end
toc
% A faster way for big values of `n1`
d2 = zeros(4,n1);
tic
a_rep = repmat(a,4,1); % Repeat row indexes to match the elements in b
idx_Lin = sub2ind([n2,n2],a_rep(:),b(:)); % Get linear indexes
d2(:) = c(idx_Lin); % Fill
toc
isequal(d,d2)
Elapsed time is 1.309654 seconds.
Elapsed time is 0.062549 seconds.
ans =
logical
1
Try This:
n1 = 10;
n2 = 100;
a = randi(n2,[1,n1]);
b = randi(n2,[4,n1]);
c = rand(100,100);
idx = (1:n1);
tic
d1=(c(a(idx),b(:,idx)))';
[idx,idy]=meshgrid(0:44:400,1:4);
d1=d1(idy+idx);
toc
this is the timeing:
Elapsed time is 0.000517 seconds.

How can I speed up this MATLAB code with a whileloop?

I'm using a code that calculates expectation value of probabilities. This code contains a while-loop that finds all possible combinations and adds up products of probability combinations. However, when the number of elements becomes large(over 40) it takes too much time, and I want to make the code faster.
The code is as follow-
function pcs = combsum(N,K,prbv)
nprbv=1-prbv; %prbv: probability vector
WV = 1:K; % Working vector.
lim = K; % Sets the limit for working index.
inc = 0; % Controls which element of WV is being worked on.
pcs = 0;
stopp=0;
while stopp==0
if logical((inc+lim)-N)
stp = inc; % This is where the for loop below stops.
flg = 0; % Used for resetting inc.
else
stp = 1;
flg = 1;
end
for jj = 1:stp
WV(K + jj - inc) = lim + jj; % Faster than a vector assignment.
end
PV=nprbv;
PV(WV)=prbv(WV);
pcs=prod(PV)+pcs;
inc = inc*flg + 1; % Increment the counter.
lim = WV(K - inc + 1 ); % lim for next run.
if (inc==K)&&(lim==N-K)
stopp=1;
WV = (N-K+1):N;
PV=nprbv;
PV(WV)=prbv(WV);
pcs=prod(PV)+pcs;
end
end
Is there a way to reduce calculation time? I wonder if parallel computing using GPU would help.
I tried to remove dependent variables in a loop for parallel computing, and I made a matrix of possible combinations using 'combnk' function. This worked faster.
nprbv=1-prbv; %prbv : a probability vector
N = 40;
K = 4;
n_combnk = size(combnk(1:N,K),1);
PV_mat = repmat(nprbv,n_combnk,1);
cnt = 0;
tic;
for i = 1:N-K+1
for j = i+1:N-K+2
for k = j+1:N-K+3
for l = k+1:N-K+4
cnt = cnt+1;
PV_mat(cnt,i) = prbv(i);
PV_mat(cnt,j) = prbv(j);
PV_mat(cnt,k) = prbv(k);
PV_mat(cnt,l) = prbv(l);
end
end
end
end
toc;
tic;
pcs_rr = sum(prod(PV_mat,2));
toc;
However, when K value gets larger, an out-of-memory problem happens in building a combination matrix(PV_mat). How can I break up the big matrix into small ones to avoid memory problem?

Matlab 2016b: looping with structures is much slower than with variables

I am computing a list of error statistics of a variable u. Whenever I put the structures in the loop, Matlab becomes superslow, while it is pretty fast with standard variables. I have heard that with Matlab 2016b one can either use loops or use vectorized notation, the speed should be the same. It looks like it does not work for structures. Do you know why? Best suggestion here? This is a minimal example ( I just used trivial "if" clauses, mine are more complex and that's the main reason why I don't vectorize):
n = 1000,
m = 1000;
uMODEL = rand(n,m);
uOBS = rand(n,m);
%
%
%
tic
ERRORS = struct('Emedio',0,'MAE',0,'sigma',0,'Emax',0,'Emin',0);
Nerr(1:max(n,1),1) = 0; %Nerr at most it contains n
fields = fieldnames(ERRORS);
for i = 1:numel(fields)
ERRORS.(fields{i}) = zeros(max(n,1),1);
end
for i=1:m
for j=1:n
if (1000>2 && 2000<343532)
diff = uMODEL(j,i)-uOBS(j,i);
ERRORS.Emedio(j) = ERRORS.Emedio(j) + diff;
ERRORS.MAE(j) = ERRORS.MAE(j) + abs(diff);
ERRORS.sigma(j) = ERRORS.sigma(j) + diff^2;
ERRORS.Emax(j) = max(ERRORS.Emax(j),diff);
ERRORS.Emin(j) = min(ERRORS.Emin(j),diff);
Nerr(n) = Nerr(n) + 1;
end
end
end
ERRORS.Emedio(:) = ERRORS.Emedio(:)./Nerr(:);
ERRORS.MAE(:) = ERRORS.MAE(:)./Nerr(:);
ERRORS.sigma(:) = sqrt(ERRORS.sigma(:)./(Nerr(:)-1));
toc
clear ERRORS
tic
%
% here instead I define variables, I fill them up and then I throw them in structures, cause loops with structures are strangely slow...
Emedio = zeros(max(n,1),1);
MAE = zeros(max(n,1),1);
sigma = zeros(max(n,1),1);
Nerr = zeros(max(n,1),1);
Emax = zeros(max(n,1),1);
Emin = zeros(max(n,1),1);
for i=1:m
for j=1:n
if (1000>2 && 2000<343532)
diff = uMODEL(j,i)-uOBS(j,i);
Emedio(j) = Emedio(j) + diff;
MAE(j) = MAE(j) + abs(diff);
sigma(j) = sigma(j) + diff^2;
Emax(j) = max(Emax(j),diff);
Emin(j) = min(Emin(j),diff);
Nerr(n) = Nerr(n) + 1;
end
end
end
Emedio(:) = Emedio(:)./Nerr(:);
MAE(:) = MAE(:)./Nerr(:);
sigma(:) = sqrt(sigma(:)./(Nerr(:)-1));
ERRORS.Emedio(:) = Emedio(:);
ERRORS.MAE(:) = MAE(:);
ERRORS.sigma(:) = sigma(:);
ERRORS.Emax(:) = Emax(:);
ERRORS.Emin(:) = Emin(:);
toc
the output is:
Elapsed time is 2.372765 seconds.
Elapsed time is 0.057719 seconds.

What is the fastest way of appending an element to an array?

This is a follow-up question to How to append an element to an array in MATLAB? That question addressed how to append an element to an array. Two approaches are discussed there:
A = [A elem] % for a row array
A = [A; elem] % for a column array
and
A(end+1) = elem;
The second approach has the obvious advantage of being compatible with both row and column arrays.
However, this question is: which of the two approaches is fastest? My intuition tells me that the second one is, but I'd like some evidence for or against that. Any idea?
The second approach (A(end+1) = elem) is faster
According to the benchmarks below (run with the timeit benchmarking function from File Exchange), the second approach (A(end+1) = elem) is faster and should therefore be preferred.
Interestingly, though, the performance gap between the two approaches is much narrower in older versions of MATLAB than it is in more recent versions.
R2008a
R2013a
Benchmark code
function benchmark
n = logspace(2, 5, 40);
% n = logspace(2, 4, 40);
tf = zeros(size(n));
tg = tf;
for k = 1 : numel(n)
x = rand(round(n(k)), 1);
f = #() append(x);
tf(k) = timeit(f);
g = #() addtoend(x);
tg(k) = timeit(g);
end
figure
hold on
plot(n, tf, 'bo')
plot(n, tg, 'ro')
hold off
xlabel('input size')
ylabel('time (s)')
leg = legend('y = [y, x(k)]', 'y(end + 1) = x(k)');
set(leg, 'Location', 'NorthWest');
end
% Approach 1: y = [y, x(k)];
function y = append(x)
y = [];
for k = 1 : numel(x);
y = [y, x(k)];
end
end
% Approach 2: y(end + 1) = x(k);
function y = addtoend(x)
y = [];
for k = 1 : numel(x);
y(end + 1) = x(k);
end
end
How about this?
function somescript
RStime = timeit(#RowSlow)
CStime = timeit(#ColSlow)
RFtime = timeit(#RowFast)
CFtime = timeit(#ColFast)
function RowSlow
rng(1)
A = zeros(1,2);
for i = 1:1e5
A = [A rand(1,1)];
end
end
function ColSlow
rng(1)
A = zeros(2,1);
for i = 1:1e5
A = [A; rand(1,1)];
end
end
function RowFast
rng(1)
A = zeros(1,2);
for i = 1:1e5
A(end+1) = rand(1,1);
end
end
function ColFast
rng(1)
A = zeros(2,1);
for i = 1:1e5
A(end+1) = rand(1,1);
end
end
end
For my machine, this yields the following timings:
RStime =
30.4064
CStime =
29.1075
RFtime =
0.3318
CFtime =
0.3351
The orientation of the vector does not seem to matter that much, but the second approach is about a factor 100 faster on my machine.
In addition to the fast growing method pointing out above (i.e., A(k+1)), you can also get a speed increase from increasing the array size by some multiple, so that allocations become less as the size increases.
On my laptop using R2014b, a conditional doubling of size results in about a factor of 6 speed increase:
>> SO
GATime =
0.0288
DWNTime =
0.0048
In a real application, the size of A would needed to be limited to the needed size or the unfilled results filtered out in some way.
The Code for the SO function is below. I note that I switched to cos(k) since, for some unknown reason, there is a large difference in performance between rand() and rand(1,1) on my machine. But I don't think this affects the outcome too much.
function [] = SO()
GATime = timeit(#GrowAlways)
DWNTime = timeit(#DoubleWhenNeeded)
end
function [] = DoubleWhenNeeded()
A = 0;
sizeA = 1;
for k = 1:1E5
if ((k+1) > sizeA)
A(2*sizeA) = 0;
sizeA = 2*sizeA;
end
A(k+1) = cos(k);
end
end
function [] = GrowAlways()
A = 0;
for k = 1:1E5
A(k+1) = cos(k);
end
end

Matlab index to logic indexing

I have given a list of indices, e.g. i = [3 5] and a vector v = 1:6. I need a function f which returns the logical map for the vector v given the indices i, e.g.:
f(i, length(v)) = [0 0 1 0 1 0]
Since I will call this function several million times, I would like to make it as fast as possible. Is there a builtin function which performs this task?
I know I'm late in the game, but I really wanted to find a faster solution which is just as elegant as ismember. And indeed there is one, that employs the undocumented ismembc function:
ismembc(v, i)
Benchmark
N = 7;
i = [3 5];
%// slayton's solution
tic
for ii = 1:1e5
clear idx;
idx(N) = false;
idx(i) = true;
end
toc
%// H.Muster's solution
tic
for ii = 1:1e5
v = 1:N;
idx = ismember(v, i);
end
toc
%// Jonas' solution
tic
for ii = 1:1e5
idx = sparse(i, 1, true, N, 1);
end
toc
%// ismembc solution
tic
for ii = 1:1e5
v = 1:N;
idx = ismembc(v, i);
end
toc
Here's what I got:
Elapsed time is 1.482971 seconds.
Elapsed time is 6.369626 seconds.
Elapsed time is 2.039481 seconds.
Elapsed time is 0.776234 seconds.
Amazingly, ismembc is indeed the fastest!
Edit:
For very large values of N (i.e. when v is a large array), the faster solution is actually slayton's (and HebeleHododo's, for that matter). You have quite a variety of strategies to choose from, pick carefully :)
Edit by H.Muster:
Here's are benchmark results including _ismemberoneoutput:
Slayton's solution:
Elapsed time is 1.075650 seconds.
ismember:
Elapsed time is 3.163412 seconds.
ismembc:
Elapsed time is 0.390953 seconds.
_ismemberoneoutput:
Elapsed time is 0.477098 seconds.
Interestingly, Jonas' solution does not run for me, as I get an Index exceeds matrix dimensions. error...
Edit by hoogamaphone:
It's worth noting that ismembc requires both inputs to be numerical, sorted, non-sparse, non-NaN values, which is a detail that could be easily missed in the source documentation.
You can use ismember
i = [3 5];
v = 1:6;
ismember(v,i)
will return
ans =
0 0 1 0 1 0
For a probably faster version, you can try
builtin('_ismemberoneoutput', v, i)
Note that I tested this only for row vectors like specified by you.
Simply create a vector of logical indices and set the desired locations to true/false
idx = false( size( v) );
idx( i ) = true;
This can be wrapped in a function like so:
function idx = getLogicalIdx(size, i)
idx = false(size);
idx(i) = true;
end
If you need a indexing vector of the same size for each of your million operations allocated the vector once and then operate on it each iteration:
idx = false(size(v)); % allocate the vector
while( keepGoing)
idx(i) = true; % set the desired values to true for this iteration
doSomethingWithIndecies(idx);
idx(i) = false; % set indices back to false for next iteration
end
If you really need performance than you can write a mex function to do this for you. Here is a very basic, untested function that I wrote that is about 2x faster than the other methods:
#include <math.h>
#include <matrix.h>
#include <mex.h>
void mexFunction(int nlhs, mxArray *plhs[],
int nrhs, const mxArray *prhs[])
{
double M;
double *in;
M = mxGetScalar(prhs[0]);
in = mxGetPr(prhs[1]);
size_t N = mxGetNumberOfElements(prhs[1]);
plhs[0] = mxCreateLogicalMatrix( M,1 );
mxLogical *out= mxGetLogicals( plhs[0] );
int i, ind;
for (i=0; i<N; i++){
out[ (int)in[i] ] = 1;
}
}
There are several different ways to allocate a vector in matlab. Some are faster than others, see this Undocumented Matlab post for a good summary:
Here are some quick benchmarks comparing the different methods. The last method is by far the fastest but it requires you to use the same size logical indexing vector for each operation.
N = 1000;
ITER = 1e5;
i = randi(5000,100,1);
sz = [N, 1];
fprintf('Create using false()\n');
tic;
for j = 1:ITER
clear idx;
idx = false( N, 1 );
idx(i) = true;
end
toc;
fprintf('Create using indexing\n');
tic;
for j = 1:ITER
clear idx;
idx(N) = false;
idx(i) = true;
end
toc;
fprintf('Create once, update as needed\n');
tic;
idx = false(N,1);
for j = 1:ITER
idx(i) = true;
idx(i) = false;
end
toc;
fprintf('Create using ismembc\n');
a = ones(N,1);
tic;
for j = 1:ITER
idx = ismembc(1:N, i);
end
toc;
I expect that #slayton's solution is fastest. However, here's a one-liner alternative, that may at least save you some memory if the vectors are large.
vecLen = 6;
logicalIdx = sparse(idx,1,true,vecLen,1);
Just address a new variable with the idx matrix, it wil fill in the zeros for you:
idx = [3 5];
a(idx) = true
No need for a function, nor for passing the length in unless you want trailing zeros too.
You can write a function like this:
function logicalIdx = getLogicalIdx(idx, v)
logicalIdx = zeros(1,size(v,2));
logicalIdx(idx) = 1;
end
When you call the function:
v = 1:6;
idx = [3 5];
getLogicalIdx(idx,v)
The output will be:
ans =
0 0 1 0 1 0
Can you simply do v(i) =1 ?
for example if you say x = zeros(1,10);
and a = [1 3 4];
x(a) = 1 will return 1
0
1
1
0
0
0
0
0
0