I have this problem:
I run some large calculations before going to sleep (or work).
When I return sometimes RAM is already filled and the program starts writing to Disk, which is a problem since then computer becomes almost non responsive, also the button "Interrupt the current operation" doesn't stop mserver.exe from executing a task.
This is what I saw 10 mins after I pressed the button "Interrupt the current operation":
Not to mention that calculations are probably like 100 or even 1000 times slower when it starts using Disk instead of RAM (so it's pointless anyway).
Another problem is that I was unable to save some variables to file since in Maple I couldn't type anything while mserver.exe was executing a task and after I killed the process mserver.exe I was still unable to save those variables since Maple commands don't work when connection to kernel is lost.
So, my question: can I make it so that mserver.exe won't use Disk at all (I mean from Maple alone, not by disabling page file in Windows) and just stop execution automatically when RAM is full (just like Classic Maple does when it hits 2GB limit)?
Also it would be nice to be able to limit Maple from using processor too much, for example up to 75% or so, so that I could work on that computer without problems.
You might experiment with a few of the options available for specifying limits on the Maple (kernel, mserver) engine.
In particular,
--init-reserve-mem=memorysize
(or, possible, the -T option). See here for some more detail:
https://www.maplesoft.com/support/help/MapleSim/view.aspx?path=maple
On Linux/OSX you could pass that in a call to the maple script that launches Maple. On MS-Windows you could add that to the command string/Property in the launcher (icon).
You might try setting it to a fraction of your total RAM, eg. 50-75%, and see how it goes. Presumably you'll have some other processes running.
As far as restricting the CPU use goes, that's more of a OS issue. On Linux/OSX you could use the system's nice facility. I don't know what's avaliable on MS-Windows (built-in or 3rd party). You might be able to set the Priority of the running mserver process from the Task Manager. Or you might look at something like the START facility:
https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/start
Related
I'm trying to run a Netlogo model in behaviorspace, in headless mode, on a linux server.
My netlogo version is 5.3.1 (the 64b version).
The server has 32 cores with 64gigs of RAM.
I'm setting Xmx to 3072m.
After a few runs (~300) the memory usage is so high that I get a Java heap space error.
Surprisingly, the memory usage grows regularly, as if there were no flush-like function called between runs. And it gets to a point it shouldn't reach if I understand things well (for example, for 15 parallel threads it reaches 64000MB and beyond when it should stay around 15 * 3072 = 46080.
I'm using ca at setup so I thought everything was supposed to be flushed out between runs. I'm not opening any file from the code (I use the standard behaviorspace output, in table format, not spreadsheet), and I'm not using any extension.
I'm kind oh puzzled here. Is there something I should look at into behaviorspace specific parameterization that says to keeps track of variables, turtles, etc. between runs ? I couldn't find such a thing.
Could someone help me ?
Thanks a lot !
Thomas
I am having recurring intermittent problems with "losing" my environment variables, most onerously %windir% and %path%. The problem occurs when I have locked the keyboard and log back in. Rebooting the system (cold- and warm-boot) does not reliably bring them back, but eventually multiple iterations of booting has (so far) brought everything back.
If I open a command window and type echo %windir% and echo %path% and find that the variables exist and are properly defined, and if I leave that command window open, I can leave my system running for days without a problem.
I have captured the results of set to list all envars, both when the system is broken, and when it is fixed. The broken list is much shorter (%windir% is not even defined, %path% contains the definition from registry HKCU\Environment, but not from HKLM\SYSTEM\CurrentControlSet\Control\SessionManager\Environment).
I am guessing that the boot-up process is getting sidetracked.
Spent all morning with Geek Squad but they had no concrete suggestions. (They did suggest "taking the computer back to a previous restore point", but I fear that could cause more problems... and they didn't have confidence it would help.)
Do I have any options beyond possibly reinstalling everything?
I did finally find the answer, and although I understand this subject was closed by the commenter, I thought others might want to know. This link explains it pretty well:
https://superuser.com/questions/355594/windows-7s-path-and-environment-variables-are-corrupted
The short version is this: My system PATH exceeded a Windows maximum of 2048 bytes (it was more than 2200 bytes long). When that happens, the boot process fails to instantiate PATH and WINDIR.
The "fix" was to run c:\windows\system32\systempropertiesadvanced.exe from a command prompt (because without WINDIR, you can't open the Control Panel app for environment variables), and manually extract anything from the PATH I thought I could live without, until I whittled the PATH string down to under 2048 bytes.
DOS is always given as an example of single tasking operating system. However when a command is issued through command-line, control switches from the shell to the command and then switches back to shell when the command completes.Thus there are two processes executing simultaneously. Is there something wrong in my understanding ?
No, they weren't executing simultaneously.
COMMAND.COM had a resident portion that was in memory all the time and a transient portion that could be tossed out at will.
When you ran a program, it typically got loaded in place of the transient portion and then run. When the program exited, it did so by calling code in the resident portion which would then reload the transient portion if necessary and continue.
The fact that some of the code remained resident in no way means that it was "running". In a similar way, vast tracts of MS-DOS (the kernel) stayed continuously in memory yet they weren't "running", unless called explicitly by a non-kernel program.
Now there were things can could be said to run concurrently, DOS had plenty of TSR (terminate and stay resident) programs that would run, hook into an interrupt or DOS in some way, then exit but leaving some memory allocated (where its code was).
Then, in response to certain events, that code would be run. Perhaps one of the famous ones was Borland Sidekick which was a personal information manager that would pop up instantly with a keypress.
While the other process is running, the command line processor is not running: it is suspended. The only "multitasking" facility that was available in DOS was "Terminate and Stay Resident".
It doesnt matter whether you are running DOS or Windows or Linux or BSD or whatever on that processor it is all the same. At that period of time you for purposes of this discussion had a single execution unit, a single core executing the instructions, mostly in order. Makes no difference if those instructions wear the name DOS or Linux or windows. Just instructions.
Just like now as then when a windows program decides to terminate it tries to do it nicely with some flavor of exit call. When a linux program terminates it tries to do so nicely with some flavor of exit call to the system. And when a dos program terminates it tries to do so nicely with some flavor of exit call to the system. In a shell, command prompt, etc linux, windows, dos, that shell, which is a program itself, loads and branches to the program you have loaded and your program runs for a while and as mentioned tries to return to the prior program nicely with some flavor of exit. Just like when the shell you were running wants to return when it is done running it tries to do so nicely.
As with linux or windows, easier to see back then, you dont run anything "at the same time" or "in parallel" one instruction stream at a time. (today we have multiple execution units and/or cores that are designed to each be doing something in parallel with something managing them, so today you can actually say "in parallel") You want to switch "tasks" or "threads" or "processes" you needed an interrupt, that switched to you different code, an interrupt handler, and that handler could return to the same program that was interrupted or switch to another. You can put whatever name on it you want it is how you make things look like they are running at the same time. dos, linux, windows, etc, this is typically how you switch from one "program" or bit of code to another. linux and windows have their kernels and operating system behind them that was called during the interrupts, and dos had that as well (dos HAS that, dos is still alive you touch a dos machine every few days most likely (gas pump, atm machine, etc), dos is also still used in the development and testing of x86 motherboards/computers, nothing can compete with it as an embedded x86 platform, nothing has the freedom that dos has to do what you want, this is why bios upgrades are still distributed as a dos program). The interrupt handlers would give time slices to the various bios handlers and dos handlers. task/process/thread switching was not as designed or planned as an operating system like linux or windows, but it was there, for each version of dos there were rules you followed and you could switch tasks (tsrs are a popular term). Just talking to a floppy, hard disk, etc there was code involved in the whole process, it wasnt buried in the hardware, lots of things happened in parallel. no different than a hard disk controller driver in something more complicated like linux or windows. At least one, maybe some, non-microsoft dos clones could multitask.
The short answer, When you have a function bob() that calls a function ted().
int bob ( int something )
{
...some code
...more code
ted();
...some code
...more code
}
is bob() still running? Are they running in parallel? No, the bob() code is still there, somewhere, waiting for the ted() code to finish what it was doing and return. So long as ted() doesnt crash it will return and bob() can continue to execute. bob is suspended while ted executes. Not much different with a shell or command line in an more complicated operating system. There is some function somewhere that has loaded your program into memory and called it, it might be a fork or clone of a command line that you were running so that that command line can continue "in parallel" or the clone can continue in parallel. but the concept is the same.
The difference from a trivial C program like the one above is that the code above can be thought of being resolved at compile time where loading and running a program is definitely runtime, basically self modifying code, the program modifies memory then jumps to it. When it returns that code, cleans up, unwinds, and exits itself or waits for another command depending on the design. DOS was just very very simple, a bunch of system calls, combined with a bunch of BIOS calls, and a very simple command line that could load programs and do a small number of other commands. It didnt have any rules you couldnt get around (windows is a dos program), if the program you launched didnt want to return (you could at least at the time launch linux from dos through an intermediate dos program) well it kind of messes up your question of what happens when the program completes, well linux didnt return, it took over the system.
I'm running a long simulation in MATLAB that I've realized I need to stop and rerun. However, MATLAB is really into this calculation, and it's stopped responding. How can I interrupt this run without killing MATLAB?
(I realize this is a problem with many Windows programs, but it's really acute with MATLAB.)
Go to the command window, and hit Ctrl-C a lot. From my experience, on a single-core machine you do not have a chance, unless you do lots of output. On a multi-core or multi-processor machine, you'll probably stop it eventually, but it takes time.
See also http://www.mathworks.com/support/solutions/en/data/1-188VX/index.html
Added: it is a good practice to (1) save a snapshot of your workspace before running anything really long and (2) within a very long calculation, write some of the variables to a file from time to time, so that you can resume the calculation if it was interrupted (by power failure, e.g.).
How well MATLAB responds to CTRL-C rather depends on what it's doing. If it's in the middle of a BLAS or LAPACK call for example, it will not respond until that call returns. If you're in a block of code where lots of lines of MATLAB are being executed, you can expect CTRL-C to be more responsive.
I have got a very simple trick to pause (or stop) a non-responsive execution.
If my simulation is running a long loop I always do the following:
for ii = 1:N
do_stuff();
clear empty_script;
empty_script;
end
And then create a file empty_script.m containing the following:
%keyboard
Whenever I want to pause execution I open an external text editor and uncomment the line saying keyboard in empty_script.m. That leaves me in debugging mode where I can watch variables, modify stuff or even stop the program.
Another strategy for dealing with this problem is to introduce a very short pause somewhere in the calculation (especially in a FOR or WHILE loop), as in:
for ii = 1:N
do_stuff();
pause(0.1);
end
This increases the chances that your maniacal Ctrl-C'ing will actually stop it.
you can find the MATLAB process in the windows task manager and set the priority as high or low and let other program to have lower or higher priority. In my experience, it is an efficient way.
if you wont to stop and rerun then killing is not bad choise
Go to windows task manager-> Processes then fined MATLAB.exe and push the End Process button
SOLVED see Edit 2
Hello,
I've been writing a Perl program to handle automatic upgrading of local (proprietary) programs (for the company I work for).
Basically, it runs via cron, and unfortunately has a memory leak (or something similar). The problem is that the leak only happens when I'm not looking (aka when run via cron, not via command line).
My code does not contain any circular (or other) references, so the commonly cited tools will not help me (Devel::Cycle, Devel::Peek).
How would I go about figuring out what is using so much memory that the kernel kills it?
Basically, the code SFTPs into a server (using ```sftp...`` `), calls OpenSSL to verify the file, and then SFTPs more if more files are needed, and installs them (untars them).
I have seen delays (~15 sec) before the first SFTP session, but it has never used so much memory as to be killed (in my presence).
If I can't sort this out, I'll need to re-write in a different language, and that will take precious time.
Edit: The following message is printed out by the kernel which led me to believe it was a memory leak:
[100023.123] Out of memory: kill process 9568 (update.pl) score 325406 or a child
[100023.123] Killed Process 9568 (update.pl)
I don't believe it is an issue with cron because of the stalling (for ~15 sec, sometimes) when running it via the command-line. Also, there are no environmental variables used (at least by what I've written, maybe underlying things do?)
Edit 2: I found the issue myself, with help from the below comment by mobrule (in response to this question). It turns out that the script was called from a crontab of a user (non-root) just once a day and that (non-root privs) caused a special infinite loop situation.
Sorry guys, I feel kinda stupid for not finding this before, but thanks.
mobrule, if you submit your comment as an answer, I will accept it as it lead to me finding the problem.
End Edits
Thanks,
Brian
P.S. I may be able to post small snippets of code, but not the whole thing due to company policy.
You could try using Devel::Size to profile some of your objects. e.g. in the main:: scope (the .pl file itself), do something like this:
use Devel::Size qw(total_size);
foreach my $varname (qw(varname1 varname2 ))
{
print "size used for variable $varname: " . total_size($$varname) . "\n";
}
Compare the actual size used to what you think is a reasonable value for each object. Something suspicious might pop out immediately (e.g. a cache that is massively bloated beyond anything that sounds reasonable).
Other things to try:
Eliminate bits of functionality one at a time to see if suddenly things get a lot better; I'd start with the use of any external libraries
Is the bad behaviour localized to just one particular machine, or one particular operating system? Move the program to other systems to see how its behaviour changes.
(In a separate installation) try upgrading to the latest Perl (5.10.1), and also upgrade all your CPAN modules
How do you know that it's a memory leak? I can think of many other reasons why the OS would kill a program.
The first question I would ask is "Does this program always work correctly from the command line?". If the answer is "No" then I'd fix these issues first.
On the other hand if the answer is "Yes", I would investigate all the differences between having the program executed under cron and from the command line to find out why it is misbehaving.
If it is run by cron, that shouldn't it die after iteration? If that is the case, hard for me to see how a memory leak would be a big deal...
Are you sure it is the script itself, and not the child processes that are using the memory? Perhaps it ends up creating a real lot of ssh sessions , instead of doing a bunch of stuff in one session?