SIG error during global destruction - perl

I have an issue with a script I created. This is the first time I use signal as I would like my script to run like a daemon, so I set several signal Handler to properly end my script :
local $SIG{HUP} = \&StopSuperviser;
local $SIG{INT} = \&StopSuperviser;
local $SIG{QUIT} = \&StopSuperviser;
local $SIG{ILL} = \&StopSuperviser;
local $SIG{ABRT} = \&StopSuperviser;
local $SIG{TERM} = \&StopSuperviser;
This is working properly but when I send one of those signal to my script (Crt-C, kill -15, kill -1 ...) the StopSuperviser function is correctly called but I always receive the following error in my script's output:
Argument "HUP" isn't numeric in null operation during global destruction.
I have searched on google but I didn't find anything dealing with this behavior.
May someone put some light in this?
Thanks you very much for your help
Best Regards
Florent
#
Thanks four your replies, here is the StopSuperviser Function:
sub StopSuperviser
{
print "On quite\n";
$StopAlarm = 1;
&DeleteThreadOrder($AllProcess);
foreach my $Subprocess (#$AllProcess) {
foreach my $thread (#{$Subprocess->{Thread}}) {
$thread->kill('USR1');
$thread->join();
}
}
exit;
}
I also use the following package :
use Alarm::Concurrent;
This may important to know, or may not :)
Hope this help :)
Thanks again for your response and help
Best Regards
Florent

I had this problem earlier this year, and ended up finding the solution on PerlMonks, but I can't find the link now. I'll update if I come across it. Their explanation was much more detailed than what I present here.
It's an expression of a feature described in perlsub:
If a subroutine is called using the & form, the argument list is
optional, and if omitted, no #_ array is set up for the subroutine:
the #_ array at the time of the call is visible to subroutine instead.
If memory serves, you should be able to fix it by changing:
local $SIG{HUP} = \&StopSuperviser;
to
local $SIG{HUP} = sub { StopSuperviser() };

Related

Why am I getting this perl Net::Async::HTTP warning?

In this oversimplified script I'm doing a GET request with Net::Async::HTTP using IO::Async::Loop::EV:
use Modern::Perl '2017';
use IO::Async::Loop::EV;
use Net::Async::HTTP;
use Future;
my $loop = IO::Async::Loop::EV->new;
my $http = Net::Async::HTTP->new(max_redirects => 0);
$loop->add($http);
my $f = $http->GET('https://www.perl.org')
->then(sub {
my $response = shift;
printf STDERR "got resp code %d\n", $response->code;
return Future->done;
});
$http->adopt_future($f->else_done);
$loop->run;
I get this warning a couple of times:
EV: error in callback (ignoring): Can't call method "sysread" on an undefined value at .../IO/Async/Stream.pm line 974
I get this warning when using IO::Async::Loop::Event too (again in IO::Async::Stream, at line 974).
For non-secure (http) links, however, all looks good. So something's probably wrong with IO::Async::SSL. (I tried this on different machines, with different OS - still getting those warnings)
Why am I getting this warning multiple times? Does it occur on your machines too?
It seems this warning is specific to the IO::Async::Loop::EV implementation. If you just
use IO::Async::Loop;
my $loop = IO::Async::Loop->new;
then it appears to work just fine. Unless you're picking that for a specific purpose it's best to avoid it and just let the IO::Async::Loop->new magic constructor find a good one.
Additionally, rather than ending the script with
$loop->run
You could instead use
$f->get
so it performs a blocking wait until that Future is complete but then exits cleanly afterwards, so as not to have to <Ctrl-C> it to abort.
I've raised this as a bug against IO::Async::Loop::EV at https://rt.cpan.org/Ticket/Display.html?id=124030

Add parallelism to perl script

I have small perl script which gets services details from mongoDB, queries its statuses and gives html output
#...some stuff to get $token
my #cmd = ('/opt/mongo/bin/mongo', '127.0.0.1:27117/service_discovery', '--quiet', '-u', 'xxx', '-p', 'xxx', '--eval', "var environ='$env'; var action='status'", '/home/mongod/www/cgi/getstatus.js');
my $mongo_out;
run \#cmd, '>>', \$mongo_out;
$json->incr_parse ($mongo_out);
while (my $obj = $json->incr_parse) {
my $hostname = "$obj->{'hostname'}";
print "<tr><td colspan=4 align=\"center\"><h4>$hostname</h4></td></tr>";
foreach my $service (#{$obj->{'services'}}) {
my $name = "$service->{'name'}";
my $port = "$service->{'port'}";
my $proto = "$service->{'proto'}";
my $request = HTTP::Request->new(GET => "${proto}://$hostname:${port}/status/service");
$request->header(Authorization => "Bearer $token");
my $ua = LWP::UserAgent->new;
$ua->timeout(2);
my $response = $ua->request($request);
my $code = $response->code();
if ($code == 200) {
my $var = %$response->{'_content'};
my $coder = JSON::XS->new->ascii->pretty->allow_nonref;
my $out = try {my $output = $coder->decode($var)} catch {undef};
if(exists $out->{'name'} && exists $out->{'version'}) {
print "<tr><td align=\"center\">$port</td><td align=\"center\">$name</td><td align=\"center\">$out->{'name'}</td><td align=\"center\">$out->{'version'}</td></tr>";
} else {
print "<tr><td align=\"center\">$port</td><td align=\"center\">$name</td><td colspan=2 align=\"center\">auth failed</td></tr>";
}
} elsif ($code == 500) {
print "<tr><td align=\"center\">$port</td><td align=\"center\">$name</td><td colspan=2 align=\"center\">offline</td></tr>";
} elsif ($code == 404) {
print "<tr><td align=\"center\">$port</td><td align=\"center\">$name</td><td colspan=2 align=\"center\">page not found</td></tr>";
}
}
}
It executes for a while, especially when some services are offline. Is it possible to query services within same host simultaneously?
This is almost a question that's too broad to answer, because ... it depends.
But yes. You have two and a half mechanism for parallelising in perl:
thread
fork
Non blocking IO.
I say two and a half, because non-blocking IO isn't really parallel, as much as solving the same problem a different way.
Implementation of parallelism is a really good way to end up with some horrific and hard to trace bugs, and requires a bit of a shift of mind set, because your code is no longer executing in a well defined sequence - the whole point is that your code might hit different bits at different times, and that can cause utter chaos.
And not least because modules you import - might well not be "thread safe" (which means they may be fine, but occasionally will break in a very unpredictable way, and you'll tear your hair out trying to track down the bug).
So with that in mind
threads
Perhaps slightly counter intuitively, if you've used threads in another language - perl threads are NOT light weight. There is a significant cost to starting them, not least because you effectively end up multiplying your memory footprint by the number of threads you are running.
I would normally suggest as a result - look at a "worker threads" model, using Thread::Queue. You start up a number of threads, and use queues to serialise the input and output from the threads.
forking
fork() is a unix native system call. You use it a lot, and it's quite efficient. It splits your program into two identical copies - including position within the code - at the point at which it's called. The only difference initially is the return code of the fork() system call - the parent will get the process ID of the child, the child will get zero.
It's quite easy to do strange thing accidentally, as both piece of code at this point are at exactly the same point in terms of loop iterations, file handles, etc. but this rapidly diverges and you can again, end up with some very strange things happening if you interact with 'shared' resources.
I would normally suggest looking at Parallel::ForkManager module as an easy way to avoid tripping yourself up with fork().
non blocking IO
You can often use something like IO::Select and the can_read method, which detects which file handles will block if you read from them - you can skip that one, until it blocks. This would also work for your use case, although it's not always applicable.
I've got examples of both the above here: Perl daemonize with child daemons

Perl, Parallel::ForkManager - how to implement timeout for fork

Is it possible to implement some kind of timeout (time limit) for fork using Parallel::ForkManager ?
Basic Parallel::ForkManager script looks like this
use Parallel::ForkManager;
my $pm = Parallel::ForkManager->new( 10 );
for ( 1 .. 1000 ) {
$pm->start and next;
# some job for fork
$pm->finish;
}
$pm->wait_all_children();
I would like to limit time for "# some job for fork". For example, if its not finished in 90 secs. then it (fork) should be killed/terminated.
I thought about using this but I have to say, that I dont know how to use it with Parallel::ForkManager.
EDIT
Thanks hobbs and ikegami. Both your suggestions worked..... but only in this basic example, not in my actual script :(.
These forks will be there forever and - to be honest - I dont know why. I use this script for couple of months. Didnt change anything (although many things depends on outside variables).
Every fork has to download a page from a website, parse it and save results to a file. It should not take more than 30 secs per fork. Timeout is set to 180 secs. Those hanging forks are totally random so its very hard to trace the problem. Thats why I came up with a temporary, simple solution - timeout & kill.
What could possibly disable (interrupt) your methods of timeout in my code ? I dont have any other alarm() anywhere in my code.
EDIT 2
One of the forks, was hanging for 1h38m and returned "timeout PID" - which is what I type in die() for alarm(). So the timeout works... but its late about 1h36,5m ;). Do you have any ideas?
Update
Sorry to update after the close, but I'd be remiss if I didn't point out that Parallel::ForkManager also supports a run_on_start callback. This can be used to install a "child registration" function that takes care of the time()-stamping of PIDs for you.
E.g.,
$pm->run_on_start(sub { my $pid = shift; $workers{$pid} = time(); });
The upshot is that, in conjunction with run_on_wait as described below, the main loop of a P::FM doesn't have to do anything special. That is, it can remain a simple $pm->start and next, and the callbacks will take care of everything else.
Original Answer
Parallel::ForkManager's run_on_wait handler, and a bit of bookkeeping, can force hanging and ALRM-proof children to terminate.
The callback registered by that function can be run, periodically, while the $pm awaits child termination.
use strict; use warnings;
use Parallel::ForkManager;
use constant PATIENCE => 90; # seconds
our %workers;
sub dismiss_hung_workers {
while (my ($pid, $started_at) = each %workers) {
next unless time() - $started_at > PATIENCE;
kill TERM => $pid;
delete $workers{$pid};
}
}
...
sub main {
my $pm = Parallel::ForkManager->new(10);
$pm->run_on_wait(\&dismiss_hung_workers, 1); # 1 second between callback invocations
for (1 .. 1000) {
if (my $pid = $pm->start) {
$workers{$pid} = time();
next;
}
# Here we are child. Do some work.
# (Maybe install a $SIG{TERM} handler for graceful shutdown!)
...
$pm->finish;
}
$pm->wait_all_children;
}
(As others suggest, it's better to have the children regulate themselves via alarm(), but that appears intermittently unworkable for you. You could also resort to wasteful, gross hacks like having each child itself fork() or exec('bash', '-c', 'sleep 90; kill -TERM $PPID').)
All you need is one line:
use Parallel::ForkManager;
my $pm = Parallel::ForkManager->new( 10 );
for ( 1 .. 1000 ) {
$pm->start and next;
alarm 90; # <---
# some job for fork
$pm->finish;
}
$pm->wait_all_children();
You don't need to set up a signal handlers since you do mean for the process to die.
It even works if you exec in the child. It won't work on Windows, but using fork on Windows is questionable in the first place.
Just do what the answer you linked to suggests, inside the child process (i.e. between the $pm->start and next and the end of the loop. There's nothing special you need to do to make it interact with Parallel::ForkManager, other than make sure you don't accidentally kill the parent instead :)

Why are my shared variables getting cleared out between PerlChildInitHandler and PerlResponseHandler in mod_perl?

I am trying to load a configuration file into a hash during my PerlChildInitHandler and then access the values from PerlResponseHandler. However, even though the process number is the same, it seems that variables changed during the child_init() call revert back to their default values when handler() gets called.
The basic scenario is:
package StartupLog;
# the variable I'm testing
my $sticky = 0;
sub child_init {
$sticky = 1;
return 0;
}
sub handler {
warn __PACKAGE__ . " sticky = $sticky\n"; ### always says "0" but should say "1"
return 0;
}
1;
This was never answered, so eventually I moved on to using the PerlPostConfigHandler, which seemed to work acceptably. I can only assume it's something about the forking that happens in the PerlChildInitiHandler but, sorry to say, I gave up. Hope this helps someone in the future.
Generally, if you want to load something at childinit time, and access it in the response phase, you'd stuff it into a package global (like $My::variable = 'lols'). I've never tried to do it the way you are here. Did you try using our instead of my maybe?.

Calling clean up code in mod_perl environment

Some quote to pick from practical mod_perl
"Usually, a single process serves many requests before it exits, so END blocks cannot be used if they are expected to do something at the end of each request's processing."
So, in my a.cgi script :
my $flag = 1;
END {
# Value for $flag is undefined, if this script is run under mod_perl.
# END block code only executed when process to handle a.cgi exit.
# I wish to execute some code, just before process to handle a.cgi exit.
if ($flag) {
# clean up code.
}
}
The book recommences $r->register_cleanup(sub { #cleanup } );
However,
How I can obtain $r in a.cgi script?
Can the subroutine access the my scope flag variable?
Is this $r->register_cleanup shall be placed at a.cgi script? I only want the cleanup code to be executed for a.cgi script. Not the rest.
my $r = Apache->request;
Yes, but see http://modperlbook.org/html/6-2-Exposing-Apache-Registry-Secrets.html and the next couple of pages, regarding scoping of local variables and functions.
Yes, only register the function if you want it to run.
If I understand this correctly, you have a script you want to run both under mod_perl and as a plain CGI and it sounds like you are using Apache::Registry to do this.
You have cleanup code that you want run only when you are running as CGI script.
You need to detect whether or not you are running under mod_perl. That's fairly easy. The simplest way is to check your environment:
unless ($ENV{MOD_PERL})
{
#... cleanup code here.
}
You only to register a cleanup handler if you want something to run when your script terminates under Apache::Registry.
If you do want that, you should place your cleanup code into a sub and call that sub from your check in the CGI:
unless ($ENV{MOD_PERL})
{
cleanup_sub();
}
and from your cleanup handler:
my $r = Apache->request;
$r->register_cleanup(sub { cleanup_sub() } );