If I run an external process via a perl program,the perl program will remain the parent of the process. Making process management easy.
system('sleep 3000'); # perl is still the parent
However if I try to run the process in the background so that the program does not have to wait for the process to exit...
system('sleep 3000 &');
The sleep process will be adopted by the systems init process and is no longer associated with the program that executed it.
What is the proper way to handle process management in this situation. How can I emulate running the process in the background but maintain process ancestry?
You can use threads,
use threads;
my $t = async { system('sleep 3000'); };
# do something in parallel ..
# wait for thread to finish
$t->join;
or fork
sub fasync(&) {
my ($worker) = #_;
my $pid = fork() // die "can't fork!";
if (!$pid) { $worker->(); exit(0); }
return sub {
my ($flags) = #_;
return waitpid($pid, $flags // 0);
}
}
my $t = fasync { system('sleep 3000'); };
# do something in parallel ..
# wait for fork to finish
$t->();
fork/exec and wait().
Fork creates the child process by creating a copy of the parent, the parent receives the process id of the child, and calls wait() on the child process id.
Meanwhile, the child process uses exec() to overlay itself (a copy of the parent) with the process that you wish to execute.
If you need more than one concurrent background job, I recommend Parallel::ForkManager.
Related
I have a Perl app that's been running largely untroubled on a RH system for a few years. In one place, I have to run a system command that can take take many minutes to complete, so I do this in a child process. The overall structure is like this:
$SIG{CHLD} = 'IGNORE'; # Ignore dead children, to avoid zombie processes
my $child = fork();
if ($child) { # Parent; return OK
$self->status_ok();
} else { # Child; run system command
# do a bunch of data retrieval, etc.
my $output;
my #command = # generate system command here
use IPC::System::Simple 'capture';
eval { $output = capture(#command); };
$self->log->error("Error running #command: $#") if $#;
# success: log $output, carry on
}
We recently changed some of our infrastructure, although not in ways that I expected would have any influence on this. (Still running on RH, still using nginx, etc.) However, now we find that almost every instance of running this code fails, logging 'Error running {command}: failed to start: "No child processes" at /path/to/code.pl'.
I've looked around and can't figure out what the right solution is for this. There was a suggestion to change $SIG{CHLD} from 'IGNORE' to 'DEFAULT', but then I have to worry about zombie processes.
What is causing the "No child processes" error, and how do we fix this?
There was a suggestion to change $SIG{CHLD} from 'IGNORE' to 'DEFAULT', but then I have to worry about zombie processes.
This isn't true.
A zombie process is a process that has ended, but hasn't been reaped by its parent yet. A parent reaps its children using wait(2), waitpid(2) or similar. capture waits for its child to end, so it doesn't leave any zombie behind.
In fact, the error you are getting is from waitpid. capture is waiting for the child to end to reap it and collect its error code, but the you instructed the OS to clean up the child as soon as it completes, leaving waitpid with no child to reap and no error code to collect.
To fix this problem, simply place local $SIG{CHLD} = 'DEFAULT'; before the call to capture.
I'm writing a debugging utility, and I want to fork a child while preventing that child's termination from triggering a SIGCHLD to its parent. I still want other children to normally cause a SIGCHLD upon termination.
I want to do this because I don't want the fork to trigger an existing $SIG{CHLD} handler, but I still want other children to trigger it. That is, I want to isolate my new child and I don't want it to interfere with management of existing children.
I'm wary of locally installing a custom $SIG{CHLD} handler since I don't think I can properly detect when I should call the original handler. For instance, if I install a local $SIG{CHLD} handler, then I'm guaranteed to receive a SIGCHLD signal once I spawn a child and have the parent waitpid for it to terminate. However, that SIGCHLD won't indicate whether or not other children have terminated as well, so I can't be sure whether to call the original handler.
I researched that a process cannot change its parent pid. I'm not sure if changing the child's process group id or session id would be useful.
Is it even possible to prevent a specific child from triggering the SIGCHLD on the parent? Am I forced to rely on the existing $SIG{CHLD} handler to Do The Right Thing when it receives the SIGCHLD signal for a child that it did not expect?
Although there may be a better way to implement that debugging utility (let me know if there is), I'm still wondering whether POSIX offers such fine-grained control over children, and am looking for a Perl solution to my dilemma.
You can't portably prevent the SIGCHLD entirely, but if the signal handler is written properly you can prevent the waitpid from returning your debugging tool's pid by orphaning it.
use Signal::Mask;
sub start_background_child {
local $Signal::Mask{CHLD} = 1;
my $first = fork;
croak "fork failed: $!" if not defined $first;
if ($first) {
waitpid $first, 0;
return;
}
my $second = fork;
croak "fork failed: $!" if not defined $second;
exit if $second;
your_stuff_here();
}
I think you could daemonize the special child -- fork twice -- to sever the parent-child relationship. You'd still receive a SIGCHLD when the process was created -- I don't know if that's acceptable for you.
sub start_special_child {
return if fork; # run rest of this routine in child process
fork && exit; # run rest of this routine in grandchild process
# the exit here triggers SIGCHLD in the parent
... # now run your special process
exit; # exit here does not trigger SIGCHLD
}
The other approach is to keep track of the process id's of your child processes, and use waitpid to figure out which process(es) triggered the SIGCHLD handler.
$SIG{CHLD} = \&sigchld_handler;
$pid1 = start_child_process();
$pid2 = start_child_process();
$pid3 = start_child_process();
$pidS = start_special_child_process();
sub sigchld_handler {
$pid = waitpid -1, WNOHANG; # use POSIX ':sys_wait_h'
return if $pid == $pidS;
}
Is there an option one can give to IPC::Run which kills the process upon the parent dying? Or alternatively a simple way to kill child processes when the parent dies? I know I can do this by catching signals in the parent, but I'd rather not reinvent the wheel if a simple way to do this already exists. I understand that this may not catch SIGKILL, but that's okay, I plan to kill the parent in a more reasonable manner.
Use an END block to clean up.
my #ipc_run_harnesses;
END { $_->kill_kill for #ipc_run_harnesses }
...
for my $start ( 1..2 ) {
push #ipc_run_harnesses, IPC::Run::start( "while true; do sleep 1; echo running $start; done" );
}
sleep 10;
exit;
Is it possible to implement some kind of timeout (time limit) for fork using Parallel::ForkManager ?
Basic Parallel::ForkManager script looks like this
use Parallel::ForkManager;
my $pm = Parallel::ForkManager->new( 10 );
for ( 1 .. 1000 ) {
$pm->start and next;
# some job for fork
$pm->finish;
}
$pm->wait_all_children();
I would like to limit time for "# some job for fork". For example, if its not finished in 90 secs. then it (fork) should be killed/terminated.
I thought about using this but I have to say, that I dont know how to use it with Parallel::ForkManager.
EDIT
Thanks hobbs and ikegami. Both your suggestions worked..... but only in this basic example, not in my actual script :(.
These forks will be there forever and - to be honest - I dont know why. I use this script for couple of months. Didnt change anything (although many things depends on outside variables).
Every fork has to download a page from a website, parse it and save results to a file. It should not take more than 30 secs per fork. Timeout is set to 180 secs. Those hanging forks are totally random so its very hard to trace the problem. Thats why I came up with a temporary, simple solution - timeout & kill.
What could possibly disable (interrupt) your methods of timeout in my code ? I dont have any other alarm() anywhere in my code.
EDIT 2
One of the forks, was hanging for 1h38m and returned "timeout PID" - which is what I type in die() for alarm(). So the timeout works... but its late about 1h36,5m ;). Do you have any ideas?
Update
Sorry to update after the close, but I'd be remiss if I didn't point out that Parallel::ForkManager also supports a run_on_start callback. This can be used to install a "child registration" function that takes care of the time()-stamping of PIDs for you.
E.g.,
$pm->run_on_start(sub { my $pid = shift; $workers{$pid} = time(); });
The upshot is that, in conjunction with run_on_wait as described below, the main loop of a P::FM doesn't have to do anything special. That is, it can remain a simple $pm->start and next, and the callbacks will take care of everything else.
Original Answer
Parallel::ForkManager's run_on_wait handler, and a bit of bookkeeping, can force hanging and ALRM-proof children to terminate.
The callback registered by that function can be run, periodically, while the $pm awaits child termination.
use strict; use warnings;
use Parallel::ForkManager;
use constant PATIENCE => 90; # seconds
our %workers;
sub dismiss_hung_workers {
while (my ($pid, $started_at) = each %workers) {
next unless time() - $started_at > PATIENCE;
kill TERM => $pid;
delete $workers{$pid};
}
}
...
sub main {
my $pm = Parallel::ForkManager->new(10);
$pm->run_on_wait(\&dismiss_hung_workers, 1); # 1 second between callback invocations
for (1 .. 1000) {
if (my $pid = $pm->start) {
$workers{$pid} = time();
next;
}
# Here we are child. Do some work.
# (Maybe install a $SIG{TERM} handler for graceful shutdown!)
...
$pm->finish;
}
$pm->wait_all_children;
}
(As others suggest, it's better to have the children regulate themselves via alarm(), but that appears intermittently unworkable for you. You could also resort to wasteful, gross hacks like having each child itself fork() or exec('bash', '-c', 'sleep 90; kill -TERM $PPID').)
All you need is one line:
use Parallel::ForkManager;
my $pm = Parallel::ForkManager->new( 10 );
for ( 1 .. 1000 ) {
$pm->start and next;
alarm 90; # <---
# some job for fork
$pm->finish;
}
$pm->wait_all_children();
You don't need to set up a signal handlers since you do mean for the process to die.
It even works if you exec in the child. It won't work on Windows, but using fork on Windows is questionable in the first place.
Just do what the answer you linked to suggests, inside the child process (i.e. between the $pm->start and next and the end of the loop. There's nothing special you need to do to make it interact with Parallel::ForkManager, other than make sure you don't accidentally kill the parent instead :)
I'm working on a mathematical model that uses data generated from XFOIL, a popular aerospace tool used to find the lift and drag coefficients on airfoils.
I have a Perl script that calls XFOIL repeatedly with different input parameters to generate the data I need. I need XFOIL to run 5,600 times, at around 100 seconds per run, soabout 6.5 days to complete.
I have a quad-core machine, but my experience as a programmer is limited, and I really only know how to use basic Perl.
I would like to run four instances of XFOIL at a time, all on their own core. Something like this:
while ( 1 ) {
for ( i = 1..4 ) {
if ( ! exists XFOIL_instance(i) ) {
start_new_XFOIL_instance(i, input_parameter_list);
}
}
}
So the program is checking (or preferably sleeping) until an XFOIL instance is free, when we can start a new instance with the new input parameter list.
Try Parallel::ForkManager. It's a module that provides a simple interface for forking off processes like this.
Here's some example code:
#!/usr/bin/perl
use strict;
use warnings;
use Parallel::ForkManager;
my #input_parameter_list =
map { join '_', ('param', $_) }
( 1 .. 15 );
my $n_processes = 4;
my $pm = Parallel::ForkManager->new( $n_processes );
for my $i ( 1 .. $n_processes ) {
$pm->start and next;
my $count = 0;
foreach my $param_set (#input_parameter_list) {
$count++;
if ( ( $count % $i ) == 0 ) {
if ( !output_exists($param_set) ) {
start_new_XFOIL_instance($param_set);
}
}
}
$pm->finish;
}
$pm->wait_all_children;
sub output_exists {
my $param_set = shift;
return ( -f "$param_set.out" );
}
sub start_new_XFOIL_instance {
my $param_set = shift;
print "starting XFOIL instance with parameters $param_set!\n";
sleep( 5 );
touch( "$param_set.out" );
print "finished run with parameters $param_set!\n";
}
sub touch {
my $fn = shift;
open FILE, ">$fn" or die $!;
close FILE or die $!;
}
You'll need to supply your own implementations for the start_new_XFOIL_instance and the output_exists functions, and you'll also want to define your own sets of parameters to pass to XFOIL.
This looks like you can use gearman for this project.
www.gearman.org
Gearman is a job queue. You can split your work flow into a lot of mini parts.
I would recommend using amazon.com or even their auction able servers to complete this project.
Spending 10cents per computing hour or less, can significantly spead up your project.
I would use gearman locally, make sure you have a "perfect" run for 5-10 of your subjobs before handing it off to an amazon compute farm.
Perl threads will take advantage of multiple cores and processors. The main pro of threads is its fairly easy to share data between the threads and coordinate their activities. A forked process cannot easily return data to the parent nor coordinate amongst themselves.
The main cons of Perl threads is they are relatively expensive to create compared to a fork, they must copy the entire program and all its data; you must have them compiled into your Perl; and they can be buggy, the older the Perl, the buggier the threads. If your work is expensive, the creation time should not matter.
Here's an example of how you might do it with threads. There's many ways to do it, this one uses Thread::Queue to create a big list of work your worker threads can share. When the queue is empty, the threads exit. The main advantages are that its easier to control how many threads are active, and you don't have to create a new, expensive thread for each bit of work.
This example shoves all the work into the queue at once, but there's no reason you can't add to the queue as you go. If you were to do that, you'd use dequeue instead of dequeue_nb which will wait around for more input.
use strict;
use warnings;
use threads;
use Thread::Queue;
# Dummy work routine
sub start_XFOIL_instance {
my $arg = shift;
print "$arg\n";
sleep 1;
}
# Read in dummy data
my #xfoil_args = <DATA>;
chomp #xfoil_args;
# Create a queue to push work onto and the threads to pull work from
# Populate it with all the data up front so threads can finish when
# the queue is exhausted. Makes things simpler.
# See https://rt.cpan.org/Ticket/Display.html?id=79733
my $queue = Thread::Queue->new(#xfoil_args);
# Create a bunch of threads to do the work
my #threads;
for(1..4) {
push #threads, threads->create( sub {
# Pull work from the queue, don't wait if its empty
while( my $xfoil_args = $queue->dequeue_nb ) {
# Do the work
start_XFOIL_instance($xfoil_args);
}
# Yell when the thread is done
print "Queue empty\n";
});
}
# Wait for threads to finish
$_->join for #threads;
__DATA__
blah
foo
bar
baz
biff
whatever
up
down
left
right
Did you consider gnu parallel parallel.
It will allow you to run several install instances of your program with different inputs and
fill your CPU cores as they begin available. It's often a very simple an efficient way to achieve parallelization of simple tasks.
This is quite old but if someone is still looking for suitable answers to this question, you might want to consider Perl Many-Core-Engine (MCE)