I run on my computer very often Matlab programs compiled using mcc, in which I execute parfor. Each program has slow startup time, I think because the parallel worker pool is created (it takes about 20 seconds just to startup the parallel pool). It would be more efficient for me if the pool could remain open all the time in the background. For example when opening a parpool in the matlab interface, it says that the parpool will remain open for 30 minutes and so there is no need to open a parpool for each matlab script. Is something like that possible also when the code is compiled, or are there other solutions?
You could increase the time for which the pool is opened. During testing, you can type
>> preferences
and choosing Parallel Computing Toolbox settings on left menu .
You can achieve the same result adding to the code
p = parpool
p.IdleTimeout = 120 %minutes
If you have the pool opened for a longer time you should be able to run multiple scripts without the need for opening and closing it multiple times.
I would avoid leaving it opened permanently.
Related
I have this problem:
I run some large calculations before going to sleep (or work).
When I return sometimes RAM is already filled and the program starts writing to Disk, which is a problem since then computer becomes almost non responsive, also the button "Interrupt the current operation" doesn't stop mserver.exe from executing a task.
This is what I saw 10 mins after I pressed the button "Interrupt the current operation":
Not to mention that calculations are probably like 100 or even 1000 times slower when it starts using Disk instead of RAM (so it's pointless anyway).
Another problem is that I was unable to save some variables to file since in Maple I couldn't type anything while mserver.exe was executing a task and after I killed the process mserver.exe I was still unable to save those variables since Maple commands don't work when connection to kernel is lost.
So, my question: can I make it so that mserver.exe won't use Disk at all (I mean from Maple alone, not by disabling page file in Windows) and just stop execution automatically when RAM is full (just like Classic Maple does when it hits 2GB limit)?
Also it would be nice to be able to limit Maple from using processor too much, for example up to 75% or so, so that I could work on that computer without problems.
You might experiment with a few of the options available for specifying limits on the Maple (kernel, mserver) engine.
In particular,
--init-reserve-mem=memorysize
(or, possible, the -T option). See here for some more detail:
https://www.maplesoft.com/support/help/MapleSim/view.aspx?path=maple
On Linux/OSX you could pass that in a call to the maple script that launches Maple. On MS-Windows you could add that to the command string/Property in the launcher (icon).
You might try setting it to a fraction of your total RAM, eg. 50-75%, and see how it goes. Presumably you'll have some other processes running.
As far as restricting the CPU use goes, that's more of a OS issue. On Linux/OSX you could use the system's nice facility. I don't know what's avaliable on MS-Windows (built-in or 3rd party). You might be able to set the Priority of the running mserver process from the Task Manager. Or you might look at something like the START facility:
https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/start
I want to have two MATLAB windows open on the same computer. The desired scenario is as follows: MATLAB window 1 is continuously running a script that has nothing to do with MATLAB window 2. At the same time, MATLAB window 2 is running a script that continuously checks for a certain condition, and if it is met, then it will terminate the script running on MATLAB window 1, and then terminate its own script as well. I want to have two MATLAB windows instead of one since I believe it will be more time efficient for what I am trying to do. I found an interesting "KeyInject" program at http://au.mathworks.com/matlabcentral/fileexchange/40001-keyinject , but I was wondering if there is a simpler way already built into MATLAB.
Do you want simple, or a flexible, infinitely expandable version 1.0? Simple would be to trigger System A via a file created by System B.
Simple would have System B create a file, then System A would check for the file with the command
if exist ( fileName, 'file' )
then do your shutdown commands. On startup, System A would delete the file with
delete ( fileName );
The second option is to use the udp command. UDP allows any data to be sent between processes, whether on the same computer or over a network. (See https://www.mathworks.com/help/instrument/udp.html for more info).
I see several ways:
Restructure to avoid this XY problem
Use (mat) files (as Hoki suggested), possibly using the parallel computing toolbox to keep everything in one MATLAB session.
Write some MEX functions that communicate with each other via a global pipe.
Write an Auto(Hot)key script.
Option 2 is probably easiest. Take a look at events and listeners if you write in OOP, otherwise, you'd have to poll inside a loop
Option 3 is harder and way more time consuming to implement, but allows for much faster detection of the condition, and much faster data transfer between the sessions. Use only if speed is essential...but I guess that doesn't apply :)
Option 4: the AutoHotkey solution is probably the most Horrible Thing® you could do on an already Horrible Construction®, but oh what fun!! In both MATLAB sessions, you create a (hidden) figure with the name Window1 or Window2, respectively. These window names are something that AutoHotkey can easily track. If the conditions are met, you update the corresponding window name, triggering the remainder of the AutoHotkey script: press a button in the other window! If you need to transfer data between the windows: you can create basic edit boxes in both GUIs, and copy-paste the data between them. If you're on Linux: you can use Autokey for the same purpose, but by then you're basically writing Python code doing the heavy lifting, so just use Python.
Or, you know, use KeyInject. Less fun.
I've searched to try and find the answer to this question, but unfortunately, information is few and far between.
I need to be able to open multiple instances of GNU Octave to run a script which will convert a file. Each file takes approximately 5 minutes to convert, so for 10 files I'm waiting nearly 50 minutes.
However, I can (and have) manually open 10 instances of Octave and run the script manually in each instance. Is there a way to do this programmatically? So if I had 10 files, 10 different instances of Octave would open and each would run the script to convert 1 file. These conversions aren't dependent on each other, but if it's simple enough to communicate between the different instances of Octave, that would be a "nice to know".
I believe you can use some of the high-level commands, like system and popen which Octave include for starting subprocesses. These subprocesses allow you to start new tasks or start another program. After performing the tasks it can observe the output, which is a way to communicate with the main program that starts the subprocesses.
You should take a look at the following reference page, which covers the different approaches.
I am running a parallelized code courtesy the MATLAB Parallel Computing Toolbox using the spmd command. Specifically, the code is like this:
spmd
out = function(data,labindex);
end
Now the function involves a library (libsvm) which gives me a trained classifier for each iteration. During the training process, there are several debug messages being printed out to the standard output by the library and somehow these are not appearing on my standard terminal - I think this is because the workers are actually on a cluster and hence the debug messages are not visible to me.
Is there anyway to reroute the debug messages ? (possibly other than writing to a file on a shared disk)
One option may be to try the Parallel Command Window. This opens a new special Command Window with one pane per lab. You'll need to run commands from the "P>>" pmode prompt in this window. More here.
I'm running a long simulation in MATLAB that I've realized I need to stop and rerun. However, MATLAB is really into this calculation, and it's stopped responding. How can I interrupt this run without killing MATLAB?
(I realize this is a problem with many Windows programs, but it's really acute with MATLAB.)
Go to the command window, and hit Ctrl-C a lot. From my experience, on a single-core machine you do not have a chance, unless you do lots of output. On a multi-core or multi-processor machine, you'll probably stop it eventually, but it takes time.
See also http://www.mathworks.com/support/solutions/en/data/1-188VX/index.html
Added: it is a good practice to (1) save a snapshot of your workspace before running anything really long and (2) within a very long calculation, write some of the variables to a file from time to time, so that you can resume the calculation if it was interrupted (by power failure, e.g.).
How well MATLAB responds to CTRL-C rather depends on what it's doing. If it's in the middle of a BLAS or LAPACK call for example, it will not respond until that call returns. If you're in a block of code where lots of lines of MATLAB are being executed, you can expect CTRL-C to be more responsive.
I have got a very simple trick to pause (or stop) a non-responsive execution.
If my simulation is running a long loop I always do the following:
for ii = 1:N
do_stuff();
clear empty_script;
empty_script;
end
And then create a file empty_script.m containing the following:
%keyboard
Whenever I want to pause execution I open an external text editor and uncomment the line saying keyboard in empty_script.m. That leaves me in debugging mode where I can watch variables, modify stuff or even stop the program.
Another strategy for dealing with this problem is to introduce a very short pause somewhere in the calculation (especially in a FOR or WHILE loop), as in:
for ii = 1:N
do_stuff();
pause(0.1);
end
This increases the chances that your maniacal Ctrl-C'ing will actually stop it.
you can find the MATLAB process in the windows task manager and set the priority as high or low and let other program to have lower or higher priority. In my experience, it is an efficient way.
if you wont to stop and rerun then killing is not bad choise
Go to windows task manager-> Processes then fined MATLAB.exe and push the End Process button