Suppose that I have a program that is to be run on a linux machine with 32 cores (64 threads), of which I'm only allowed to use 10 cores (20 threads). So I would like to specify this before I run the program.
I googled and found maxNumCompThreads but it doesn't appear to work when I test it on a machine with MATLAB 2016a, core i5 (2 cores, 4 threads). That is, I get the same output for feature('numCores') when I do any of the following
maxNumCompThreads(1)
maxNumCompThreads(2)
maxNumCompThreads(4)
maxNumCompThreads('Automatic')
Then I tried parpool (each time I closed the current parpool session with delete(gcp('nocreate'))). I got an error when running parpool(4) (I think I understand why: parpool takes in the number of cores and hyper-threading is enabled automatically and the testing machine has just 2 physical cores). So I tested with parpool(1) and parpool(2). Again, the output for feature('numCores') did not change.
Question: so what is a right tool for the job for the situation described in the first paragraph above? And is feature('numCores') the right monitoring tool to see if the appropriate specification is in effect?
The same feature('numCores') output I keep referring to above is:
MATLAB detected: 2 physical cores.
MATLAB detected: 4 logical cores.
MATLAB was assigned: 4 logical cores by the OS.
MATLAB is using: 2 logical cores.
MATLAB is not using all logical cores because hyper-threading is enabled.
Edit: when I run parpool(10) on the linux machine I got the following error
Starting parallel pool (parpool) using the 'local' profile ... Error using parpo ol (line 103)
Couldn't interpret output from psname.sh: ""
Error in parpool_test_2016_10_03 (line 3)
parpool(10);
No, it's not the right monitoring tool. Look at feature('numthreads') instead:
>> feature('numcores')
MATLAB detected: 4 physical cores.
MATLAB detected: 8 logical cores.
MATLAB was assigned: 8 logical cores by the OS.
MATLAB is using: 4 logical cores.
MATLAB is not using all logical cores because hyper-threading is enabled.
ans =
4
>> feature('numthreads')
ans =
4
>> maxNumCompThreads(1)
ans =
4
>> feature('numcores')
MATLAB detected: 4 physical cores.
MATLAB detected: 8 logical cores.
MATLAB was assigned: 8 logical cores by the OS.
MATLAB is using: 4 logical cores.
MATLAB is not using all logical cores because hyper-threading is enabled.
ans =
4
>> feature('numthreads')
ans =
1
In general, be careful using feature as it's undocumented and prone to change without warning. Take a look at this post and this StackOverflow question for more information about feature.
Related
I need to run a parfor loop in Matlab and assign the number of workers to the loop.
Total Number of Cores: 10 (8 performance and 2 efficiency).
What is the maximum number of cores that I can take up, without crashing the computer and still being able to use the computer for browsing and email? Is leaving only 1 core free enough?
Below is an example in which 3 cores are selected.
n = 3 % Select the number of workers that you need and set n to that integer.
parpool(n)
parfor
i=1:n, c(:,i) = eig(rand(100));
end
I want to use parallel computing on matlab, i have i7-960 CPU with 4 cores and 8 threads, when i run feature('numCores') command in matlab, i obtain this
feature('numCores')
MATLAB detected: 3 physical cores.
MATLAB detected: 6 logical cores.
MATLAB was assigned: 6 logical cores by the OS.
MATLAB is using: 3 logical cores.
MATLAB is not using all logical cores because hyper-threading is enabled.
ans =
3
why matlab detect just juste 3 physical cores ! and how can i use all logical and physical cores for parallel computing.
thanks.
To use all logical processes (number of threads) you need to change the NumWorkers in the matlab setting.
in matlab 2018 menu follow this:
Preferences >> Parallel Computing Toolbox>> Cluster Profile Manager >> click "Edit" on the bottom right >> Set "NumWorkers" to the number of logical process, 8 in your case. > Done >> close and apply
This question already has answers here:
Saving time and memory using parfor?
(2 answers)
Closed 7 years ago.
I am using Matlab R2011b version on Windows 7 64 bit, Core i7 CPU with 8 GB RAM. I am running Approximate Nearest Neighbor algorithm called the Locality Sensitive Hashing using Matlabpool. Upon starting Matlab pool, I get the output
Starting matlabpool using the 'local' configuration ... connected to 4 labs.
When the control reaches the for loop, Matlab throws errro
Error using parallel_function (line 598)
Out of memory. Type HELP MEMORY for your options.
Error stack:
remoteParallelFunction.m at 29
Error in Evaluate (line 19)
parfor i=1:query_num
I have no clue how to solve this problem. Please help. Thank you
That is because the parfor requires a lot more memory.
All the workers/labs in a parfor loop are independent so each of them needs his amount of memory. Also, there is overhead involved due to the fact that the pool must spread and collect data from/to the workers.
Try using a regular for or open a pool with 2 workers instead of 4.
Also, it depends on how optimized your some_function() is: try using as few variables as possible.
maxNumCompThreads is deprecated, but is it still working with R2014a?
I tried to force a script to use a single computational thread, but it uses 2 logical cores:
maxNumCompThreads(1); % limit MATLAB to a single computational thread.
signal = rand(1, 1000000);
for i=1:100
cwt(signal,1:10,'sym2');
i
end
Any idea why?
Setting the -singleCompThread option when starting MATLAB does work fine (the script then uses one core only).
Note that my computer has hyperthreading, so 2 logical cores is actually only 1 physical core but usually Matlab count with logical cores, not physical ones (e.g. when setting the number of cores in a parallel pool).
I am running a code on MATLAB. I'm using Parallel Computing Toolkit. I make use of SPMD. I I have 8 cores on my PC. I want to run my code first on 1 core, see the time and then run my code on 2 cores and see the time, and so on.
How can I disable/enable some cores of my machine?
Thanks
The command maxNumCompThreads returns the maximum number of computational threads that the copy of MATLAB executing the command might use. However, MATLAB workers, as created by matlabpool, are single-threaded by design. Try this to confirm:
matlabpool open 2
parfor i = 1:2
warning('off','MATLAB:maxNumCompThreads:Deprecated')
maxNumCompThreads
warning('on','MATLAB:maxNumCompThreads:Deprecated')
end
ans =
1
ans =
1
If you want to time your code while running on an increasing number of cores, try something like:
matlabpool open 1
runmycode
matlabpool close
matlabpool open 2
runmycode
matlabpool close
matlabpool open 3
runmycode
etc. up to matlabpool open 8.
You can use:
maxNumCompThreads(1)
It's deprectaed, but works fine one all matlab I tested untill matlab 2011a.