This question already has an answer here:
Why is this simple parallel Matlab program much slower than the non-parallel version?
(1 answer)
Closed 12 months ago.
When I call function normally, execution time is much faster than parfeval.
tic
f = parfeval(#magic,1,10000);
value = fetchOutputs(f);
toc
Elapsed time is 2.244390 seconds.
magic function works on parfeval with 2.24 seconds.
tic
magic(10000);
toc
Elapsed time is 0.592743 seconds.
But when i call normally, it works fastly. What is the reason of this and How to speed up parfeval function?
In general, there is some overhead that needs to be considered when setting up threads (which parfeval does). This is the main reason for the time discrepancy.
When using any kind of parallel processing you have to first determine if the process runs long enough that the overhead from spawning the processes is negligible. In this case, it isn't.
Testing a longer run case:
tic
test(1E10)
toc
tic
f = parfeval(#test, 1, 1E10)
value = fetchOutputs(f);
toc
function x = test(n)
x = 1;
for i = 1:n
x = x * 1;
end
end
Which gives the time (on my computer) 5.51 and 5.49 seconds.
Related
I received a computer with 4xGPU's Tesla K80 and I am trying the parfor loops from Matlab PCT to speed up FFT's calculation and it is yet slower.
Here is what I am trying:
% Pupil is based on a 512x512 array
parfor zz = 1:4
gd = gpuDevice;
d{zz} = gd.Index;
probe{zz} = gpuArray(pupil);
Essai{zz} = gpuArray(pupil);
end
tic;
parfor ii = 1:4
gd2 = gpuDevice;
d2{ii} = gd2.Index;
for i = 1:100
[Essai{ii}] = fftn(probe{ii});
end
end
toc
%%
Starting parallel pool (parpool) using the 'local' profile ... connected to 4 workers.
Elapsed time is 1.805763 seconds.
Elapsed time is 1.412928 seconds.
Elapsed time is 1.409559 seconds.
Starting parallel pool (parpool) using the 'local' profile ... connected to 8 workers.
Elapsed time is 0.606602 seconds.
Elapsed time is 0.297850 seconds.
Elapsed time is 0.294365 seconds.
%%
tic; for i = 1:400; Essai{1} = fftn( probe{1} ); end; toc
Elapsed time is 0.193579 seconds !!!
Why is opening 8 workers faster as in principle I stored my variables into 4gpu's only (out of 8)?
Also, how to use a Tesla K80 as a single GPU?
Merci, Nicolas
I doubt that parfor works for multi-GPU systems. If speed is critical and you want to take full advantage of your GPUs, I suggest to write your own little CUDA script using the cuFFT library:
http://docs.nvidia.com/cuda/cufft/#multiple-GPU-cufft-transforms
Here is how to write your mex file containing CUDA code:
http://www.mathworks.com/help/distcomp/run-mex-functions-containing-cuda-code.html
many thanks for your quick reply and for the links ! It is true that I was trying to avoid CUDA but it seems like the best option to spread FFTs.
Although I thought that parfor and spmd were great tools for multiple GPUs..
This question already has answers here:
Faster way to initialize arrays via empty matrix multiplication? (Matlab)
(4 answers)
Closed 9 years ago.
/edit: See here for an interesting discussion of the topic. Thanks #Dan
Using a(m,n) = 0 appears to be faster, depending of the size of matrix a, than a = zeros(m,n). Are both variants the same when it comes to pre-allocation before a loop?
They are definately not the same.
Though there are ways to beat the performance of a=zeros(m,n), simply doing a(m,n) = 0 is not a safe way to do it. If any entries in a already exist they will keep existing.
See this for some nice options, also consider doing the loop backwards if you don't mind the risk.
I think it depends on your m and n. You can check the time for yourself
tic; b(2000,2000) = 0; toc;
Elapsed time is 0.004719 seconds.
tic; a = zeros(2000,2000); toc;
Elapsed time is 0.004399 seconds.
tic; a = zeros(2,2); toc;
Elapsed time is 0.000030 seconds.
tic; b(2,2) = 0; toc;
Elapsed time is 0.000023 seconds.
I'm checking the running time of a function using tic/toc. I write the following in the command window (and execute it simultaneously):
tic
res = checkFunc('case2736sp',1:3000);
toc
Elapsed time is 0.080491 seconds.
where checkFunc returns a 2736x2500 full matrix.
What puzzles me is that I have to wait almost 20 seconds for the output saying the run time is only 80 ms.
Does anyone have a clue why this is?
It's possible that tic/toc's internal counter is getting reset somehow during execution. Try it like this:
t = tic
res = checkFunc('case2736sp',1:3000);
toc(t)
(correctly and instructively asnwered, see below)
I'm beginning to do experiments with matlab and gpu (nvidia gtx660).
Now, I wrote this simple monte carlo algorithm to calculate PI. The following is the CPU version:
function pig = mc1vecnocuda(n)
countr=0;
A=rand(n,2);
for i=1:n
if norm(A(i,:))<1
countr=countr+1;
end
end
pig=(countr/n)*4;
end
This takes very little time to be executed on CPU with 100000 points "thrown" into the unit circle:
>> tic; mc1vecnocuda(100000);toc;
Elapsed time is 0.092473 seconds.
See, instead, what happens with gpu-ized version of the algorithm:
function pig = mc1veccuda(n)
countr=0;
gpucountr=gpuArray(countr);
A=gpuArray.rand(n,2);
parfor (i=1:n,1024)
if norm(A(i,:))<1
gpucountr=gpucountr+1;
end
end
pig=(gpucountr/n)*4;
end
Now, this takes a LONG time to be executed:
>> tic; mc1veccuda(100000);toc;
Elapsed time is 21.137954 seconds.
I don't understand why. I used parfor loop with 1024 workers, because querying my nvidia card with gpuDevice, 1024 is the maximum number of simultaneous threads allowed on the gtx660.
Can someone help me? Thanks.
Edit: this is the updated version that avoids IF:
function pig = mc2veccuda(n)
countr=0;
gpucountr=gpuArray(countr);
A=gpuArray.rand(n,2);
parfor (i=1:n,1024)
gpucountr = gpucountr+nnz(norm(A(i,:))<1);
end
pig=(gpucountr/n)*4;
end
And this is the code written following Bichoy's guidelines (the
right code to achieve result):
function pig = mc3veccuda(n)
countr=0;
gpucountr=gpuArray(countr);
A=gpuArray.rand(n,2);
Asq = A.^2;
Asqsum_big_column = Asq(:,1)+Asq(:,2);
Anorms=Asqsum_big_column.^(1/2);
gpucountr=gpucountr+nnz(Anorms<1);
pig=(gpucountr/n)*4;
end
Please note execution time with n=10 millions:
>> tic; mc3veccuda(10000000); toc;
Elapsed time is 0.131348 seconds.
>> tic; mc1vecnocuda(10000000); toc;
Elapsed time is 8.108907 seconds.
I didn't test my original cuda version (for/parfor), for its execution would require hours with n=10000000.
Great Bichoy! ;)
I guess the problem is with parfor!
parfor is supposed to run on MATLAB workers, that is your host not the GPU!
I guess what is actually happening is that you are starting 1024 threads on your host (not on your GPU) and each of them is trying to call the GPU. This result in the tremendous time your code is taking.
Try to re-write your code to use matrix and array operations, not for-loops! This will show some speed-up. Also, remember that you should have much more calculations to do in the GPU otherwise, memory transfer will just dominate your code.
Code:
This is the final code after including all corrections and suggestions from several people:
function pig = mc2veccuda(n)
A=gpuArray.rand(n,2); % An nx2 random matrix
Asq = A.^2; % Get the square value of each element
Anormsq = Asq(:,1)+Asq(:,2); % Get the norm squared of each point
gpucountr = nnz(Anorm<1); % Check the number of elements < 1
pig=(gpucountr/n)*4;
Many reasons like:
Movement of data between host & device
Computation within each loop is very small
Call to rand on GPU may not be parallel
if condition within the loop can cause divergence
Accumulation to a common variable may run in serial, with overhead
It is difficult to profile Matlab+CUDA code. You should probably try in native C++/CUDA and use parallel Nsight to find the bottleneck.
As Bichoy said, CUDA code should always be done vectorized. In MATLAB, unless you're writing a CUDA Kernal, the only large speedup that you're getting is that the vectorized operations are called on the GPU which has thousands of (slow) cores. If you don't have large vectors and vectorized code, it won't help.
Another thing that hasn't been mentioned is that for highly parallel architectures like GPUs you want to use different random number generating algorithms than the "standard" ones. So to add to Bichoy's answer, adding the parameter 'Threefry4x64' (64-bit) or 'Philox4x32-10' (32-bit and a lot faster! Super fast!) can lead to large speedups in CUDA code. MATLAB explains this here: http://www.mathworks.com/help/distcomp/examples/generating-random-numbers-on-a-gpu.html
I have a program which I copied from a textbook, and which times the difference in program execution runtime when calculating the same thing with uninitialized, initialized array and vectors.
However, although the program runs somewhat as expected, if running several times every once in a while it will give out a crazy result. See below for program and an example of crazy result.
clear all; clc;
% Purpose:
% This program calculates the time required to calculate the squares of
% all integers from 1 to 10000 in three different ways:
% 1. using a for loop with an uninitialized output array
% 2. Using a for loop with a pre-allocated output array
% 3. Using vectors
% PERFORM CALCULATION WITH AN UNINITIALIZED ARRAY
% (done only once because it is so slow)
maxcount = 1;
tic;
for jj = 1:maxcount
clear square
for ii = 1:10000
square(ii) = ii^2;
end
end
average1 = (toc)/maxcount;
% PERFORM CALCULATION WITH A PRE-ALLOCATED ARRAY
% (averaged over 10 loops)
maxcount = 10;
tic;
for jj = 1:maxcount
clear square
square = zeros(1,10000);
for ii = 1:10000
square(ii) = ii^2;
end
end
average2 = (toc)/maxcount;
% PERFORM CALCULATION WITH VECTORS
% (averaged over 100 executions)
maxcount = 100;
tic;
for jj = 1:maxcount
clear square
ii = 1:10000;
square = ii.^2;
end
average3 = (toc)/maxcount;
% Display results
fprintf('Loop / uninitialized array = %8.6f\n', average1)
fprintf('Loop / initialized array = %8.6f\n', average2)
fprintf('Vectorized = %8.6f\n', average3)
Result - normal:
Loop / uninitialized array = 0.195286
Loop / initialized array = 0.000339
Vectorized = 0.000079
Result - crazy:
Loop / uninitialized array = 0.203350
Loop / initialized array = 973258065.680879
Vectorized = 0.000102
Why is this happening ?
(sometimes the crazy number is on vectorized, sometimes on loop initialized)
Where did MATLAB "find" that number?
That is indeed crazy. Don't know what could cause it, and was unable to reproduce on my own Matlab R2010a copy over several runs, invoked by name or via F5.
Here's an idea for debugging it.
When using tic/toc inside a script or function, use the "tstart = tic" form that captures the output. This makes it safe to use nested tic/toc calls (e.g. inside called functions), and lets you hold on to multiple start and elapsed times and examine them programmatically.
t0 = tic;
% ... do some work ...
te = toc(t0); % "te" for "time elapsed"
You can use different "t0_label" suffixes for each of the tic and toc returns, or store them in a vector, so you preserve them until the end of your script.
t0_uninit = tic;
% ... do the uninitialized-array test ...
te_uninit = toc(t0_uninit);
t0_prealloc = tic;
% ... test the preallocated array ...
te_prealloc = toc(t0_prealloc);
Have the script break in to the debugger when it finds one of the large values.
if any([te_uninit te_prealloc te_vector] > 5)
keyboard
end
Then you can examine the workspace and the return values from tic, which might provide some clues.
EDIT: You could also try testing tic() on its own to see if there's something odd with your system clock, or whatever tic/toc is calling. tic()'s return value looks like a native timestamp of some sort. Try calling it many times in a row and comparing the subsequent values. If it ever goes backwards, that would be surprising.
function test_tic
t0 = tic;
for i = 1:1000000
t1 = tic;
if t1 <= t0
fprintf('tic went backwards: %s to %s\n', num2str(t0), num2str(t1));
end
t0 = t1;
end
On Matlab R2010b (prerelease), which has int64 math, you can reproduce a similar ridiculous toc result by jiggering the reference tic value to be "in the future". Looks like an int rollover effect, as suggested by gary comtois.
>> t0 = tic; toc(t0+999999)
Elapsed time is 6148914691.236258 seconds.
This suggests that if there were some jitter in the timer that toc were using, you might get rollover if it occurs while you're timing very short operations. (I assume toc() internally does something like tic() to get a value to compare the input to.) Increasing the number of iterations could make the effect go away because a small amount of clock jitter would be less significant as part of longer tic/toc periods. Would also explain why you don't see this in your non-preallocated test, which takes longer.
UPDATE: I was able to reproduce this behavior. I was working on some unrelated code and found that on one particular desktop with a CPU model we haven't used before, a Core 2 Q8400 2.66GHz quad core, tic was giving inaccurate results. Looks like a system-dependent bug in tic/toc.
On this particular machine, tic/toc will regularly report bizarrely high values like yours.
>> for i = 1:50000; t0 = tic; te = toc(t0); if te > 1; fprintf('elapsed: %.9f\n', te); end; end
elapsed: 6934787980.471930500
elapsed: 6934787980.471931500
elapsed: 6934787980.471899000
>> for i = 1:50000; t0 = tic; te = toc(t0); if te > 1; fprintf('elapsed: %.9f\n', te); end; end
>> for i = 1:50000; t0 = tic; te = toc(t0); if te > 1; fprintf('elapsed: %.9f\n', te); end; end
elapsed: 6934787980.471928600
elapsed: 6934787980.471913300
>>
It goes past that. On this machine, tic/toc will regularly under-report elapsed time for operations, especially for low CPU usage tasks.
>> t0 = tic; c0 = clock; pause(4); toc(t0); fprintf('Wall time is %.6f seconds.\n', etime(clock, c0));
Elapsed time is 0.183467 seconds.
Wall time is 4.000000 seconds.
So it looks like this is a bug in tic/toc that is related to particular CPU models (or something else specific to the system configuration). I've reported the bug to MathWorks.
This means that tic/toc is probably giving you inaccurate results even when it doesn't produce those insanely large numbers. As a workaround, on this machine, use etime() instead, and time only longer chunks of work to compensate for etime's lower resolution. You could wrap it in your own tick/tock functions that use the for i=1:50000 test to detect when tic is broken on the current machine, use tic/toc normally, and have them warn and fall back to using etime() on broken-tic systems.
UPDATE 2012-03-28: I've seen this in the wild for a while now, and it's highly likely due to an interaction with the CPU's high resolution performance timer and speed scaling, and (on Windows) QueryPerformanceCounter, as described here: http://support.microsoft.com/kb/895980/. It is not a bug in tic/toc, the issue is in the OS features that tic/toc is calling. Setting a boot parameter can work around it.
Here's my theory about what might be happening, based on these two pieces of data I found:
There is a function maxNumCompThreads which controls the maximum number of computational threads used by MATLAB to perform tasks. Quoting the documentation:
By default, MATLAB makes use of the
multithreading capabilities of the
computer on which it is running.
Which leads me to think that perhaps multiple copies of your script are running at the same time.
This newsgroup thread discusses a bug in an older version of MATLAB (R14) "in the way that MATLAB accelerates M-code with global structure variables", which it appears the TIC/TOC functions may use. The solution there was to disable the accelerator using the undocumented FEATURE function:
feature accel off
Putting these two things together, I'm wondering if the multiple versions of your script that are running in the workspace may be simultaneously resetting global variables used by the TIC/TOC functions and screwing one another up. Maybe this isn't a problem when converting your script to a function as Amro did since this would separate the workspaces that the two programs are running in (i.e. they wouldn't both be running in the main workspace).
This could also explain the exceedingly large numbers you get. As gary and Andrew have pointed out, these numbers appear to be due to an integer roll-over effect (i.e. an integer overflow) whereby the starting time (from TIC) is larger than the ending time (from TOC). This would result in a huge number that is still positive because TIC/TOC are internally using unsigned 64-bit integers as time measures. Consider the following possible scenario with two scripts running at the same time on different threads:
The first thread calls TIC, initializing a global variable to a starting time measure (i.e. the current time).
The first thread then calls TOC, and the immediate action the TOC function is likely to make is to get the current time measure.
The second thread calls TIC, resetting the global starting time measure to the current time, which is later than the time just measured by the TOC function for the first thread.
The TOC function for the first thread accesses the global starting time measure to get the difference between it and the measure it previously took. This difference would result in a negative number, except that the time measures are unsigned integers. This results in integer overflow, giving a huge positive number for the time difference.
So, how might you avoid this problem? Changing your scripts to functions like Amro did is probably the best choice, as that seems to circumvent the problem and keeps the workspace from becoming cluttered. An alternative work-around you could try is to set the maximum number of computational threads to one:
maxNumCompThreads(1);
This should keep multiple copies of your script from running at the same time in the main workspace.
There are at least two possible error sources. Can you try to differentiate between 'tic/toc' and 'fprintf' by just looking at the computed values without formatting them.
I don't understand the braces around 'toc' but they shouldn't do any harm.
Here is a hypothesis which is testable. Matlab's tic()/toc() have to be using some high-resolution timer. On Windows, because their return value looks like clock cycles, I think they're using the Win32 QueryPerformanceCounter() call, or maybe something else hitting the CPU's RDTSC time stamp counter. These apparently have glitches on some multiprocessor systems, mentioned in the linked articles. Perhaps your machine is one of those, getting different results if the Matlab process is moved from core to core by the process scheduler.
http://msdn.microsoft.com/en-us/library/ms644904(VS.85).aspx
http://www.virtualdub.org/blog/pivot/entry.php?id=106
This would be hardware and system configuration dependent, which would explain why other posters haven't been able to reproduce it.
Try using Windows Task Manager to set the affinity on your Matlab.exe process to a single CPU. (On the Processes tab, right-click MATLAB.exe, "Set affinity...", un-check all but CPU 0.) If the crazy timing goes away while affinity is set, looks like you found the cause.
Regardless, the workaround looks like to just increase maxcount so you're timing longer pieces of work, and the noise you're apparently getting in tic()/toc() is small compared to the measured value. (You don't want to have to muck around with CPU affinity; Matlab is supposed to be easy to run.) If there's a problem in there that's causing int overflow, the other small positive numbers are a bit suspect too. Besides, hi-res timing in a high level language like Matlab is a bit problematic. Timing workloads down to a couple hundred microseconds subjects them to noise from other transient conditions in your machine's state.