I'm trying to write a simple generic parallel code for minimizing a function in MATLAB. The idea is very simple, essentially:
parfor k = 1:N
(...find a good solution xcurrent with cost fcurrent ... )
% keep best current value
fmin = min(fmin,fxcurrent)
end
This works fine, because fmin is a reduction variable, and thus I can use this construction to update the current best value.
However, I couldn't find a nice elegant way of keeping (or storing) the best current solution ("xcurrent").
How do I keep track of the best solution found so far?
In other words, if the current value is strictly smaller than fmin, how can I save xcurrent (subject to the constraints that parallel loops impose in MATLAB)?
[Of course, the serial version is trivial, just prepend
if fxcurrent < fmin;
xbest = xcurrent;
end;
but this does not work on a parfor loop.]
A few approaches that come to mind:
I could just store all solutions and costs (using sliced variables), but this is hugely memory inefficient (the number of iterations N is very large, and the solutions themselves are very big).
Similarly, I could use a (set or matrix) reduction variable and do:
solutionset = [solutionset,xcurrent]
but this is almost as bad in terms of memory requirement.
I could also save xcurrent to disk every time the solution is improved.
I tried to look around for a simpler solution, but nothing was very useful.
The question seems to be well-defined (so it's not like in other problems, where the output could depend on iteration order), but I couldn't find an elegant way of doing this.
Apologies in advance if I'm missing something obvious, and thanks a lot in advance!
Thanks so I copy the suggestion down here.
Just an idea- what if you write your own reduction function - basically just containing the if block and a save or output?
You will presumably need to maintain multiple xcurrent structures in memory anyway, since there will have to be a separate copy for each worker executing the loop-body. I would try splitting your loop into an outer parallel part and an inner serial part -- this will allow you to adjust the number of copies of xcurrent separately to the total iteration count.
The inner (serial) loop can use the normal if fxcurrent < fmin; xmin = xcurrent; end construct to update its best solution, and the outer (parallel) loop can just store all solutions using slicing. As a final step you select the best solution from your (small) set.
Related
Say I have many (around 1000) large matrices (about 1000 by 1000) and I want to add them together element-wise. The very naive way is using a temp variable and accumulates in a loop. For example,
summ=0;
for ii=1:20
for jj=1:20
summ=summ+ rand(400);
end
end
After searching on the Internet for some while, someone said it's better to do with the help of sum(). For example,
sump=zeros(400,400,400);
count=0;
for ii=1:20
for j=1:20
count=count+1;
sump(:,:,count)=rand(400);
end
end
sum(sump,3);
However, after I tested two ways, the result is
Elapsed time is 0.780819 seconds.
Elapsed time is 1.085279 seconds.
which means the second method is even worse.
So I am just wondering if there any effective way to do addition? Assume that I am working on a computer with very large memory and a GTX 1080 (CUDA might be helpful but I don't know whether it's worthy to do so since communication also takes time.)
Thanks for your time! Any reply will be highly appreciated!.
The fastes way is to not use any loops in matlab at all.
In many cases, the internal functions of matlab all well optimized to use SIMD or other acceleration techniques.
An example for using the build in functionalities to create matrices of the desired size is X = rand(sz1,...,szN).
In your explicit case sum(rand(400,400,400),3) should give you then the fastest result.
I have this part of code which is very long to run and I would like to know if it is possible to do an optimization or a vectorization for a faster running?
if intersect(pt, coord,'rows')
for t=1:size(pt,1)
for u=1:size(Mbb,1)
if pt(t,1)==Mbb(u,1)
img(pt(t,1),Mbb(u,2))=1;
end
end
end
end
Try multi-threading. Even on a single core multi-threading may increase the efficiency of your core. If you have a multi-core system then multi-threading will yield even more benefit. In MATLAB this is done using parfor. Note that this can only be done when there is no dependencies between loop iteration. Your code will have to look something like this. Sometimes the MATLAB interpreter will be over conservative in detecting dependencies and hence you have to write your loops in such a way that the interpreter doesnt see dependencies in iterations
if intersect(pt, coord,'rows')
loopsize=size(pt,1);
parfor t=1:loopsize
for u=1:size(Mbb,1)
if pt(t,1)==Mbb(u,1)
img(pt(t,1),Mbb(u,2))=1;
end
end
end
end
You spend much time comparing pt(t,1) and Mbb(u,1) to find matches, in a double loop. If the respective sizes are large, this can be costly (O(NM)).
What you can do is to pre-sort these arrays and search for equal values by a merge-like process, taking only O(N+M) operations.
Anyway, note that if the arrays pt and Mbb include many equal elements, which are also equal between the arrays, the problem can degenerate to NM matches. In this case, the sorting trick can't help.
I have a, b and A.
a = some expression 1
b = some expression 2
A = a + b
vs
A = some expression 1 + some expression 2
In my code, there are not just a and b but a lot of those. By using the later method without creating variables at first, i.e. by just summing all the expressions in A, I get 1s faster in my program, total is about 11s. This is confirmed after a long of tests. So it reduces from 11s to about 10s. Is this due to just not creating variables at first? Does not creating variables at first lead to faster computation?
I need to run a lot of for loop and run ode solver and for long computation. Variables are calculated and created inside the loop. If i can get a about 10% decrease this is good.
In general (not just MATLAB).
Your first scenario these additional steps are required, which do not apply to the second scenario.
When variable is created, memory needs to be allocated where the value for the variable can be stored.
When a value is assigned to that variable, that value needs to be written to the variable's space in memory.
When the calculation is requested, the value for each variable needs to be retrieved from memory.
Many compilers optimize away these additional overheads by using various techniques, but many interpreted languages do not. (This is not a hard and fast rule though, there are smart interpreted languages and stupid compiled ones).
I do not know exactly how the internals of MATLAB works, but I do think it is
interpreted, which means that the additional steps likely will incur additional overhead.
The problem with your second scenario is that is less readable and maintainable in the long run though. It is easier to read computations and intermediate steps when variable names are used. The trick is balance performance and readability.
I'm not sure how much of a difference it would make in terms of performance, but I don't think it would be a sizeable difference. Maybe a few hundredths of a second.
You can test it for yourself by using the tic toc function.
tic
a = some expression 1
b = some expression 2
A = a + b
toc
VS
tic
A = some expression 1 + some expression 2
toc
As mentioned in the other answer, readability is the main difference. You want to keep your code as simple as possible so that if there is a problem you know exactly where it is and hopefully why there was a problem!
I am in the process of coding a simple Genetic Algorithm (GA). There are probably countless areas where I have unnecessarily used a for loop. I would like some tips on how to be more MATLAB efficient as well as an answer to my question. As far as I can tell I have succeeded but I am not sure. The area which this code defines is single-point crossover
Here is what I have tried...
crossPoints=randi([1 24],popSize/2,1);
for popNo=2:2:popSize
isolate=chromoParent(popNo-1:popNo,crossPoints(popNo/2,1)+1:end);
isolate([1 2],:)=isolate([2 1],:);
chromoParent(popNo-1:popNo,crossPoints(popNo/2,1)+1:end)=isolate;
end
chromoChild=chromoParent;
where, 'crossPoints' is the point at which single point crossover
between two binary encoded chromosomes is required.
'popSize' is the size of the population, required by my code to
be an even number
'isolate' defines the sections of 2 rows which are required to be swapped
with each other
'chromoParent' is the initial population which is required to be
changed by single-point crossover
'chromoChild' is the resulting population
Both 'chromoParent' and 'chromoChild' are represented by an array of
size, popSize x 25 binary characters
Can you spot an error in the way I am thinking about this problem? What's the most efficient way (in computational time) to achieve the same thing? It would help if you could be as broad as possible so that I could begin applying the principles I learn here to the rest of my code.
Thank you.
Your code looks fine. If you want, you can reduce the instructions in the loop to a single line by some very simple indexing:
chromoParent( popNo-1:popNo, crossPoints(popNo/2,1)+1:end) = ...
chromoParent(popNo:-1:popNo-1,crossPoints(popNo/2,1)+1:end);
This may be marginally faster, but as with any optimization, you should profile it first (My guess is that these line contribute very little to the overall CPU time).
I have an m x n matrix of integers and where n is a fairly big number m and n ~1000. I want to iterate through all of these and perform a some operations, like accessing a particular cell and assigning a value of a particular cells.
However, at least in my implementation, this is rather inefficient as I have two for loops with Matrix(a,b) = Matrix(a,b+1) or something along these lines. Is there any other way to do this seeing as my current implementation takes a long time to traverse through about 100,000 cells and perform some operations.
Thank you
In matlab, it's almost always possible to avoid loops.
If you want to do Matrix(a,b)=Matrix(a,b+1), you should just do Matrix2=Matrix(:,2:end);
If you are more precise about what you do inside the loop, I can help you more.
Matlab uses column major ordering of matrixes in memory (unlike C). Are you sure you are iterating the indexes in the correct order? If not, try switching them and see if performance improves..
If you can't get rid of the for loops, one possibility would be to rewrite the expensive operations in C and create a MEX file as described here.