In a MATLAB-function the following code is used:
function stuff()
if a == 2
do1();
else
do2();
end
end
This code is placed inside a simulation-loop and gets called 1000 times or more per second. The if-statement does only matter in the first call of the function, after that either do1 or do2 are used, the variable a will not change any more.
How do I prevent to waste processing time with this if-statement? Basically, how do I tell Matlab, to not check the if-statement any more, and just call the one function, that gets selected in the first call to stuff?
Contrary to your beliefs this is not a problem, the compiler (should) automatically does this optimization for you. See e.g. Loop-invariant code motion.
What you can do to help the compiler is to move the calculation of the check outside as a flag, e.g.
flag = a==2;
for i = 1:100
stuff(flag)
end
Then you only have to do the calculation once and it is clear to the compiler that the value does not change.
NOTE: Obviously, if your check really is a==2, this would not make much of a difference.
EDIT: I have not been able to definitely verify that MATLAB does this automatically. However, this is only the first layer of optimization that is done for you. All modern processors use what is called a Branch predictor, see e.g. this brilliant answer Why is processing a sorted array faster than processing an unsorted array?, or this wiki page. In short, the processor guesses the result of the if-statement, if it is correct, everything goes faster. I think it is fair to say that the processor guesses correctly in all of your cases.
TLDR: Do not wory about it.
Given the comments above, it seems what you are actually looking for is a way to dynamically chose the function to be run in your simulation. This choice should be dynamic (you do not know which function to use at runtime) but the choice should only be done once. This is easily achievable using function handles: https://www.mathworks.com/help/matlab/function-handles.html
Here is an example:
function dynamicSimulation()
if ( rand() > 0.5 ) % determine which function should be called dynamically
sim=#func1;
else
sim=#func2;
end
other_params = [];
for k = 1:5 % run the simulation
sim( k, other_params );
end
end
function func1( index, other_params )
fprintf( 'Index=%d: Simulating using function 1\n', index );
end
function func2( index, other_params )
fprintf( 'Index=%d: Simulating using function 2\n', index );
end
If you run this several times you will notice that the (random) choice of func1 or func2 will mean you do not get the same function being run each time, although the same one is used for the entire simulation.
I reckon you don't waste much time on checking the validity that if statement. However, since you specifically mention it only checks for the first iteration: why not get that out? So instead of:
for ii = 1:10
if ii == 1
k = 1;
else
k = k + 1;
end
end
You could do
k = 1;
for ii = 2:10
k = k + 1;
end
Thus eliminating the check.
NB: this scales badly of course, but as it is only a single iteration here I consider it a good option.
Related
I have been asked to obtain the first 15 triplets according to this series and this code ought to work. However, it does only produce a table (15*3) filled with zero rather than the 15 Pythagorean triplets? Any help will be welcome.
A = zeros(15, 3);
ii = 1;
for c = 5:120
c2=c^2;
for a=1:c-1
a2=a^2;
for b=a:c-1
if c2-(a2+b^2) == 0
A(ii,1) = a;
A(ii,2) = b;
A(ii,3) = c;
ii=ii+1;
if A(15, 1) ~= 0
flag = 1;
break
end
end
end
if flag == 1
break
end
end
if flag == 1
break
end
end
T1 = array2table(A);
disp(T1)
So, the code generated a correct table on application-restart before failing on all subsequent attempts. And, now I notice that the code runs successfully only for the first time after every relaunch of the application. (Resolved, thanks Dan Pollard.)
Also, interested in knowing if there is any way to not write an upper limit (120) into the code.
I don't think your if statement is ever satisfied. For example, for c=5, you'd expect a=3, b=4 to be a triplet. But you're only letting a and b go up to floor(sqrt(c-1)), which is 2.
Do you mean to let a and b go up to floor(sqrt(c2-1))?
Edit As the question has changed.
When you run the code, Matlab creates all the variables which you assign, and stores them in the workspace. This can be useful, but here it's hurting you as you have the variable flag which is stored as 1. This means that when the code runs, it checks if flag==1 after the first run through b, which it is, so the code ends. Resolve this by placing clear; at the beginning of your script.
There isn't a practical way to remove the upper limit on c. Matlab has the built-in variable Inf but at best Matlab won't let you use it in that context. Realistically you could just replace the 120 with a really large number, but this will take more time and more memory as the number gets bigger. Computers have a finite RAM to store matlab arrays in though, and there are infinitely many pythagorean triples, so doing the calculation without an upper limit will fail in some way.
When I have to display the variable value every n iterations of a for loop I always do something along these lines:
for ii=1:1000
if mod(ii,100)==0
display(num2str(ii))
end
end
I was wondering if there is a way to move the if condition outside the loop in order to speed up the code. Or also if there is something different I could do.
You can use nested loops:
N = 1000;
n = 100;
for ii = n:n:N
for k = ii-n+1:ii-1
thingsToDo(k);
end
disp(ii)
thingsToDo(ii);
end
where thingsToDo() get the relevant counter (if needed). This a little more messy, but can save a lot of if testing.
Unless the number of tested values is much larger than the number of printed values, I would not blame the if-statement. It may not seem this way at first, but printing is indeed a fairly complex task. A variable needs to be converted and sent to an output stream which is then printing in the terminal. In case you need to speed the code up, then reduce the amount of printed data.
Normally Matlab function takes vector inputs as well. This is the case for disp and display and does only take a single function call. Further, conversion to string is unnecessary before printing. Matlab should send the data to some kind of stream anyway (which may indeed take argument of type char but this is not the same char as Matlab uses), so this is probably just a waste of time. In addition to that num2str do a lot of things to ensure typesafe conversion. You already know that display is typesafe, so all these checks are redundant.
Try this instead,
q = (1:1000)'; % assuming q is some real data in your case
disp(q(mod(q,100)==0)) % this requires a single call to disp
The MATLAB documentation describes the break keyword thus:
break terminates the execution of a for or while loop. Statements in the loop after the break statement do not execute.
In nested loops, break exits only from the loop in which it occurs. Control passes to the statement that follows the end of that loop.
(my emphasis)
What if you want to exit from multiple nested loops? Other languages, such as Java, offer labelled breaks, which allow you to specify where the flow of control is to be transferred, but MATLAB lacks such a mechanism.
Consider the following example:
% assume A to be a 2D array
% nested 'for' loops
for j = 1 : n
for i = 1 : m
if f(A(i, j)) % where f is a predicate
break; % if want to break from both loops, not just the inner one
else
% do something interesting with A
end
end
% <--- the break transfers control to here...
end
% <--- ... but I want to transfer control to here
What is an idiomatic way (in MATLAB) of exiting from both loops?
I would say for your original specific example, rather use linear indexing and a single loop:
%// sample-data generation
m = 4;
n = 5;
A = rand(m, n);
temp = 0;
for k = 1:numel(A)
if A(k) > 0.8 %// note that if you had switched your inner and outer loops you would have had to transpose A first as Matlab uses column-major indexing
break;
else
temp = temp + A(k);
end
end
Or the practically identical (but with less branching):
for k = 1:numel(A)
if A(k) <= 0.8 %// note that if you had switched your inner and outer loops you would have had to transpose A first as Matlab uses column-major indexing
temp = temp + A(k);
end
end
I would think that this answer will vary from case to case and there is no general one size fits all idiomatically correct solution but I would approach it in the following way depending on your problem (note these all assumes that a vectorized solution is not practical as that is the obvious first choice)
Reduce the dimensions of the nesting and use either no breaks or just one single break (i.e. as shown above).
Don't use break at all because unless the calculation of your predicate is expensive and your loop has very many iterations, those extra iterations at the end should be practically free.
Failing that set a flag and break at each level.
Or finally wrap your loop into a function and call return instead of break.
As far as I know there are no such functionality built in. However, in most cases matlab does not need nested loops due to its support for vectorization. In those cases where vectorization does not work, the loops are mostly long and complicated and thus multiple breaks would not hinder readability significantly. As noted in a comment, you would not really need a nested loop here. Vectorization would do the trick,
m = 5;
n=4;
x = rand(m,n);
tmp = find(x>0.8, 1, 'first');
if (isempty(tmp))
tmp = m*n+1;
end
tmp = tmp-1;
tot = sum(x(1:tmp));
There might of course be people claiming that for loops are not necessarily slow anymore, but the fact remains that Matlab is column heavy and using more than one loop will in most cases include looping over non optimal dimensions. Vectorized solutions does not require that since they can use smart methods avoiding such loops (which of course does not hold if the input is a row vector, so avoiding this is also good).
The best idiomatic way to use Python (or poison of your choice) and forget all this but that's another story. Also I don't agree with the vectorization claims of the other answers anymore. Recent matlab versions handle for loops pretty quickly. You might be surprised.
My personal preference goes to raising an exception deliberately and cradling it within a try and catch block.
% assume A to be a 2D array
A = rand(10) - 0.5;
A(3,2) = 0;
wreaker = MException('Loop:breaker','Breaking the law');
try
for j = 1 : size(A,1)
% forloop number 1
for i = 1 : size(A,2)
% forloop number 2
for k = 1:10
% forloop number 3
if k == 5 && j == 3 && i == 6
mycurrentval = 5;
throw(wreaker)
end
end
end
end
catch
return % I don't remember the do nothing keyword for matlab apparently
end
You can change the location of your try catch indentation to fall back to the loop of your choice. Also by slaying kittens, you can write your own exceptions such that they label the exception depending on the nest count and then you can listen for them. There is no end to ugliness though still prettier than having counters or custom variables with if clauses in my opinion.
Note that, this is exactly why matlab drives many people crazy. It silently throws exceptions in a pretty similar way and you get a nonsensical error for the last randomly chosen function while passing by, such as size mismatch in some differential equation solver so on. I actually learned all this stuff after reading a lot matlab toolbox source codes.
I parallelised the code below but the simulation time is actually 400-500 times longer than the serial code. The only reason i can think of that can cause this is the message 'variable x is indexed but not sliced in parfor loop and 'variable p is indexed but not sliced in parfor loop. Can anyone verify whether this is the reason for the huge increase in simulation time or the way i parallelised the code.
p=(1,i) and x(1,i) are matrix with values set before hand.
nt=1;
nc=32;
time(1,1) = 0.0;
for t=dt:dt:0.1
nt=nt+1;
time(1,nt) = t;
disp(t);
for ii=2:nc
mytemp=zeros(1,ii);
dummy=0.0;
parfor jj=1:nc+1
if ii==jj % skipped
continue;
end
dxx = x(1,jj) - x(1,ii);
rr=abs(dxx);
if rr < re
dummy(jj) = (p(nt-1,jj)-p(nt-1,ii))*kernel(rr,re,ktype)*rr;
mytemp(jj) = kernel(rr,re,ktype)*rr;
%sumw(1,ii) = sumw(1,ii) + kernel(rr,re,1);
end
end
mysum = sum(dummy);
zeta(1,ii)=sum(mytemp);
lapp(1,ii) = 2.0*dim*mysum/zeta(1,ii);
p(nt,ii) = p(nt-1,ii) + dt*lapp(1,ii);
end
% update boundary value
p(nt,1) = function_phi(0,t);
p(nt,nc+1) = function_phi(1,t);
end
Can't be sure that is the reason, but if some parts of the code end up being parallelized while others cannot, it will create a lot of overhead without any speedup. See for example the Q&A here for a more detailed discussion of slicing.
Basically, if you have a parfor with a variable jj, then every statement in which jj is used on the right hand side should also use jj on the left hand side - in that way, the "job" can be divided between different processors, each of which tackles part of the array in parallel. As soon as that doesn't happen, for example in your lines
dxx = x(1,jj) - x(1,ii);
rr=abs(dxx);
you break the paradigm. 400x slower? I don't know about that - but the warning is pretty clear.
The first two lines could be consolidated, by the way, by computing rr(jj) (although you don't need an array):
rr(jj) = abs(x(1,jj) - x(1,ii));
You then use that value rather than rr later in the loop. This is a bit like having a private variable for each copy of the loop (a concept that I don't think Matlab has - but exists in OMP ).
I don't see where p is indexed in the parfor loop … it seems to be update outside of the inner loop, where it ought not to matter.
You might find it helpful to profile your code with the parallel profiler http://www.mathworks.com/help/distcomp/profiling-parallel-code.html - it will be instructive.
I am running a very large meta-simulation where I go through two hyperparameters (lets say x and y) and for each set of hyperparameters (x_i & y_j) I run a modest sized subsimulation. Thus:
for x=1:I
for y=1:j
subsimulation(x,y)
end
end
For each subsimulation however, about 50% of the data is common to every other subsimulation, or subsimulation(x_1,y_1).commondata=subsimulation(x_2,y_2).commondata.
This is very relevant since so far the total simulation results file size is ~10Gb! Obviously, I want to save the common subsimulation data 1 time to save space. However, the obvious solution, being to save it in one place would screw up my plotting function, since it directly calls subsimulation(x,y).commondata.
I was wondering whether I could do something like
subsimulation(x,y).commondata=% pointer to 1 location in memory %
If that cant work, what about this less elegant solution:
subsimulation(x,y).commondata='variable name' %string
and then adding
if(~isstruct(subsimulation(x,y).commondata)),
subsimulation(x,y).commondata=eval(subsimulation(x,y).commondata)
end
What solution do you guys think is best?
Thanks
DankMasterDan
You could do this fairly easily by defining a handle class. See also the documentation.
An example:
classdef SimulationCommonData < handle
properties
someData
end
methods
function this = SimulationCommonData(someData)
% Constructor
this.someData = someData;
end
end
end
Then use like this,
commonData = SimulationCommonData(something);
subsimulation(x, y).commondata = commonData;
subsimulation(x, y+1).commondata = commonData;
% These now point to the same reference (handle)
As per my comment, as long as you do not modify the common data, you can pass it as third input and still not copy the array in memory on each iteration (a very good read is Internal Matlab memory optimizations). This image will clarify:
As you can see, the first jump in memory is due to the creation of common and the second one to the allocation of the output c. If the data were copied on each iteration, you would have seen many more memory fluctuations. For instance, a third jump, then a decrease, then back up again and so on...
Follows the code (I added a pause in between each iteration to make it clearer that no big jumps occur during the loop):
function out = foo(a,b,common)
out = a+b+common;
end
for ii = 1:10; c = foo(ii,ii+1,common); pause(2); end