Can a system() call can ever die in Perl 5?
(in other words, in order to 100% crash-proof a program that does a system() call, does it need to be wrapped into an eval block, or is that wholly totally unnecessary?)
I haven't found a single mention to that possibility in perldoc system, but didn't quite find the precise "this call never dies" either.
NOTE: the question is about basic CORE Perl here, no autodie or any other custom module that would have similar effect. Also, assume no ALRM signal was set, or any other custom signal handler for that matter.
I'm assuming that all versions of Perl 5.* behave the same, but if not, an answer pertaining to 5.8 would be appreciated.
Unless my interpretation of the source is incorrect, this looks like a possibility:
Source: Perl 5.16.2 (checked 5.8.8 too), file: pp_sys.c, line: 4224 within the PP(pp_system) code block:
if (n != sizeof(int))
DIE(aTHX_ "panic: kid popen errno read, n=%u", n);
DIE is Perl_die(pTHX_ const* pat, ...) declared in util.c
According to the documentation, "panic: kid popen errno read" means "forked child returned an incomprehensible message about its errno".
Explanation of panic messages in Perl:
The convention is that when the interpreter dies with an internal
error, the message starts "panic: ". Historically, many panic messages
had been terse fixed strings, which means that the out-of-range values
that triggered the panic are lost. Now we try to report these values,
as such panics may not be repeatable, and the original error message
may be the only diagnostic we get when we try to find the cause.
You can call system() with the expectation that it will not throw an exception. There is no need to wrap it in an eval block.
system returns the exit status of the program. It means, if the program crashes, the calling Perl script continues (see system).
Nevertheless, the program itself can still kill the calling script or even crash the computer. For example, in Linux:
system 'killall', 'perl';
print "Alive\n";
I assume that you're talking about the implementation of the system function itself as opposed to whatever is invoked via the call. (Obviously, a child process can't call die in the parent's context, and even that assumes that the call is to Perl code.)
A definitive answer would require knowledge of the internals but given that attempting to invoke a non-existent program doesn't die, I can't imagine that anything else ever would, either:
system('abcd'); # 'abcd' is not recognized... [Win32 message]
say "I'm not dead."; # always prints
Related
I have written a perl program which internally calls three perl modules. My supervisor after reviewing the code asked me add global exception handling. I didn't understand what he meant by this. He also said use Eval to accomplish this.I am not sure how to use Eval so that it catches any exception in the enire perl module. Can anyone help me by providing links or by providing explanation?
Thanks in advance.
For each program he wants me to have an exception handling where in if something goes wrong it will be highlighted and it becomes easy for us to debug.
When an uncaught exception occurs, it is printed to STDERR. Your highlighting requirement is already being met.
Exceptions messages already include the line number at which they were thrown (unless specifically suppressed), so some information to help debug is already available.
$ perl -e'sub f { die "bar" } f("foo")'
bar at -e line 1.
Adding use Carp::Always; to your scripts will cause a stack backtrace to be provided as well, providing more information.
$ perl -e'use Carp::Always; sub f { die "bar" } f("foo")'
bar at -e line 1.
main::f("foo") called at -e line 1
The problem you are given seems imprecise. "Global" and eval are somewhat contradictory, as
Borodin explained in his comment. Another way to do things "global" is given in ikegami's answer. However, since mentioning eval is specific here is a rundown on a very basic use of that.
You use eval on a block of code (or an expression but that is not what you want here). If a die is thrown anywhere inside that block, the eval catches that in the sense that you get the control back, the program won't just die. The variable $# gets filled with the error message. Then you can interrogate what happened, print out diagnostics, and possibly recover from the error.
eval { run_some_code(#args) };
if ($#) {
carp "Error in `run_some_code()`: $# --";
# Do further investigation, print, recover ...
}
You can have any code in the eval block above, it needn't be a single function call. Note that eval itself returns values that convey what happened (aside from $# being set).
As for the "global" in your problem statement, one thing that comes to mind is that you use eval at the level of main:: -- wrap in it any subs that themselves invoke functions from the modules.
A crucial thing about exceptions is that they "bubble up". When a die (Perl's sole exception) is thrown in a sub and the caller doesn't eval it, it goes up the call chain ... eventually showing up in main::, and if it is not caught (eval-ed) there then the program dies. So you can eval the top-level call in main:: and get to know whether anything anywhere below went wrong.
eval { top_level_call(); };
if ($#) {
warn "Error from somewhere in `top_level_call(): $#";
}
# Functions
sub top_level_call {
# doing some work ...
another_sub();
# doing more ...
}
sub another_sub {
# do work, no eval checks
}
If an error triggering a die happens in another_sub() its processing stops immediately and the control is returned to the caller, the top_level_call(). Since that sub doesn't check (no eval there) its execution stops at that point as well, and the control returns to its caller (in this example the main:: itself). So it eventually hits main::, and eval-ing there lets you know about errors and your program won't just exit.
Perhaps that's what was meant by "global" exception handling using eval.
You can do far more along these lines, if this is what you need to be doing. See eval for starters.
Update your question with clarifications, so you get more discussion here.
In practical terms, I would say that you equip yourself with some understanding of eval use as well as some of "global" error reporting, and then ask your supervisor for clarification and/or examples, as suggested by Borodin.
We have a while(1) script that loops through its' various workings, then sleeps for 60 seconds and then goes through it all again. This script needs to run 24/7.
As we add new functionality within the while(1) loop, or just over the course of random issues, sometimes they fail and unfortunately crash the entire script. The solution has been wrapping any such functions in eval{}, but my question is...Is there anyway to globally set that all errors or fatals do NOT halt/kill the entire script so we don't have to wrap everything around the eval{} ?
What you are trying to do – ignore any errors, carry on at all costs – is a very questionable practice, may bring your program into undefined state, and makes actual bugs even harder to find.
You could in theory override the CORE::GLOBAL::die subroutine to catch exceptions from your Perl code, but a real die is still available as the CORE::die sub, and this doesn't trap errors from XS code or perl itself (unlike using eval). Note that some modules use die and warn for control flow. Consider the following code:
sub foo {
my ($x, $y) = #_;
croak "X must be smaller than Y" unless $x < $y;
return $y - $x;
}
Now if die becomes a warn, then that function could start to output negative numbers, wreaking all kind of havoc (e.g. when used for array indices).
Please, stay with the eval solution, or even better: migrate to Try::Tiny. Fatal errors exist for a reason.
If high reliability is a must, you may want to adopt an Erlang-like model: A pool of worker processes. If an error turns up, that process is killed, and a replacement process started.
That makes no sense. How would the program know where to resume? You must tell it where, and you do that using eval.
You would surely be better writing a wrapper for the script that logs failures and restarts the script.
Despite the other answers, the fact is that some errors may not be easily recoverable and just ignoring them and trudging along could easily cause unwanted behavior. Another option is to remove the while loop so that the script only executes once, and call it from cron, which allows you to run programs on a schedule. For example, you might open a shell and call crontab -e to edit the scheduler and add this line:
* * * * * perl /path/to/script.pl
which will run your script every minute and send you a mail with the output if there is any, including warnings and errors.
I want to create a simple IO object that represents a pipe opened to another program to that I can periodically write to another program's STDIN as my app runs. I want it to be bullet-proof (in that it catches all errors) and cross-platform. The best options I can find are:
open
sub io_read {
local $SIG{__WARN__} = sub { }; # Silence warning.
open my $pipe, '|-', #_ or die "Cannot exec $_[0]: $!\n";
return $pipe;
}
Advantages:
Cross-platform
Simple
Disadvantages
No $SIG{PIPE} to catch errors from the piped program
Are other errors caught?
IO::Pipe
sub io_read {
IO::Pipe->reader(#_);
}
Advantages:
Simple
Returns an IO::Handle object for OO interface
Supported by the Perl core.
Disadvantages
Still No $SIG{PIPE} to catch errors from the piped program
Not supported on Win32 (or, at least, its tests are skipped)
IPC::Run
There is no interface for writing to a file handle in IPC::Run, only appending to a scalar. This seems…weird.
IPC::Run3
No file handle interface here, either. I could use a code reference, which would be called repeatedly to spool to the child, but looking at the source code, it appears that it actually writes to a temporary file, and then opens it and spools its contents to the pipe'd command's STDIN. Wha?
IPC::Cmd
Still no file handle interface.
What am I missing here? It seems as if this should be a solved problem, and I'm kind of stunned that it's not. IO::Pipe comes closest to what I want, but the lack of $SIG{PIPE} error handling and the lack of support for Windows is distressing. Where is the piping module that will JDWIM?
Thanks to guidance from #ikegami, I have found that the best choice for interactively reading from and writing to another process in Perl is IPC::Run. However, it requires that the program you are reading from and writing to have a known output when it is done writing to its STDOUT, such as a prompt. Here's an example that executes bash, has it run ls -l, and then prints that output:
use v5.14;
use IPC::Run qw(start timeout new_appender new_chunker);
my #command = qw(bash);
# Connect to the other program.
my ($in, #out);
my $ipc = start \#command,
'<' => new_appender("echo __END__\n"), \$in,
'>' => new_chunker, sub { push #out, #_ },
timeout(10) or die "Error: $?\n";
# Send it a command and wait until it has received it.
$in .= "ls -l\n";
$ipc->pump while length $in;
# Wait until our end-of-output string appears.
$ipc->pump until #out && #out[-1] =~ /__END__\n/m;
pop #out;
say #out;
Because it is running as an IPC (I assume), bash does not emit a prompt when it is done writing to its STDOUT. So I use the new_appender() function to have it emit something I can match to find the end of the output (by calling echo __END__). I've also used an anonymous subroutine after a call to new_chunker to collect the output into an array, rather than a scalar (just pass a reference to a scalar to '>' if you want that).
So this works, but it sucks for a whole host of reasons, in my opinion:
There is no generally useful way to know that an IPC-controlled program is done printing to its STDOUT. Instead, you have to use a regular expression on its output to search for a string that usually means it's done.
If it doesn't emit one, you have to trick it into emitting one (as I have done here—god forbid if I should have a file named __END__, though). If I was controlling a database client, I might have to send something like SELECT 'IM OUTTA HERE';. Different applications would require different new_appender hacks.
The writing to the magic $in and $out scalars feels weird and action-at-a-distance-y. I dislike it.
One cannot do line-oriented processing on the scalars as one could if they were file handles. They are therefore less efficient.
The ability to use new_chunker to get line-oriented output is nice, if still a bit weird. That regains a bit of the efficiency on reading output from a program, though, assuming it is buffered efficiently by IPC::Run.
I now realize that, although the interface for IPC::Run could potentially be a bit nicer, overall the weaknesses of the IPC model in particular makes it tricky to deal with at all. There is no generally-useful IPC interface, because one has to know too much about the specifics of the particular program being run to get it to work. This is okay, maybe, if you know exactly how it will react to inputs, and can reliably recognize when it is done emitting output, and don't need to worry much about cross-platform compatibility. But that was far from sufficient for my need for a generally useful way to interact with various database command-line clients in a CPAN module that could be distributed to a whole host of operating systems.
In the end, thanks to packaging suggestions in comments on a blog post, I decided to abandon the use of IPC for controlling those clients, and to use the DBI, instead. It provides an excellent API, robust, stable, and mature, and suffers none of the drawbacks of IPC.
My recommendation for those who come after me is this:
If you just need to execute another program and wait for it to finish, or collect its output when it is done running, use IPC::System::Simple. Otherwise, if what you need to do is to interactively interface with something else, use an API whenever possible. And if it's not possible, then use something like IPC::Run and try to make the best of it—and be prepared to give up quite a bit of your time to get it "just right."
I've done something similar to this. Although it depends on the parent program and what you are trying to pipe. My solution was to spawn a child process (leaving $SIG{PIPE} to function) and writing that to the log, or handling the error in what way you see fit. I use POSIX to handle my child process and am able to utilize all the functionality of the parent. However if you're trying to have the child communicate back to the parent - then things get difficult. Do you have an example of the main program and what you're trying to PIPE?
I have a Plack/Starman application running with TryCatch statements that call 'confess' from the Carp module. However I notice that the confess output is not printing to STDOUT. I've tried routing STDERR output to STDOUT '2>&1', but still don't see anything. I have search for possible error log files with no luck. Where in the world is this printing to? I am sure it's probably a simple answer. Where are the log files located? I am running on a Ubuntu box if that matters.
Thanks
Some confusion here. First, confess (and all the other carps in the pond) don't print to STDOUT: they print to STDERR. Second, you're stopping the exception and hence the associated output using try/catch (glorified eval), so it's not printed unless you explicitly print it yourself. You'll see warnings, but you won't see messages of instructions that would terminate the program (well, not Plack, but your script) because they're swallowed by your try/catch code and it's up to you to decide whether any of it should be printed and where to.
https://metacpan.org/module/Carp
https://metacpan.org/module/TryCatch
https://metacpan.org/module/Plack
I have some Perl code that runs fine outside the debugger:
% perl somefile.pl
but when I run it inside the debugger:
% perl -d somefile.pl
it behaves differently.
The files in question (there are several) are part of the test suite for a large Perl module (~20K lines of code). The tests do a lot of setup work at compile time and use BEGIN blocks. Here's some minimal reproduction code:
BEGIN
{
package MyEx;
sub new { bless {}, shift }
package main;
eval { die MyEx->new };
if($#)
{
die "Really die" unless($#->isa('MyEx'));
}
}
print "OK\n";
If you put that in somefile.pl and run it, it prints "OK" as expected. If you run it in the debugger with perl -d somefile.pl, it dies with this error:
Can't call method "isa" without a package or object reference ...
The upshot is that $# is not an object when the code runs under the debugger. Instead, it's an unblessed scalar containing this string:
" at somefile.pl line 9
eval {...} called at somefile.pl line 9
main::BEGIN() called at somefile.pl line 16
eval {...} called at somefile.pl line 16
"
(Internal newlines and spacing preserved. That's the literal text, even the "..."s.)
I need code like this to run in the debugger. Using the debugger in the test suite is an important part of my workflow. The module uses exception objects and does a lot of stuff at compile time and expects an object thrown to be an object when caught.
My question (finally) is this: How can I get this to work? Is there a workaround? Is this a bug in the perl debugger module? What's the best way to go about getting this resolved? (I know that's several questions, but they're all related.)
I'm using perl 5.10.0 on Mac OS X 10.5.5.
The dieLevel thing suggested by Adam Bellaire looked promising, and indeed something (can't find out what) is setting it to 1 for me. But I set it to 0 using a ~/.perldb file and the problem persists. In fact, I set all three of the related settings to 0. My ~/.perldb file:
parse_options('dieLevel=0 warnLevel=0 signalLevel=0');
I confirmed that the settings are in effect by running the o command in the debugger. I see them all set to 0 when I run perl -de 0 and also when running the actual somefile.pl file.
Thanks, brian. I used perlbug to file a bug (RT 60890) and I've begun to sprinkle local $SIG{'__DIE__'} in all the appropriate places in my code. (I also noted in the bug that perldoc perldebug still seems to imply that the default dieLevel is 0.)
This is a problem with perl5db.pl creating __DIE__ handlers. If I localize $SIG{__DIE__} in your eval, things work as you expect.
eval {
local $SIG{__DIE__};
die MyEx->new
};
If you don't do that, you're getting the handler from DB::dbdie, which uses Carp::longmess. That shouldn't happen if dieLevel is 0, but by default it is 1, and it gets set to 1 if it is not defined. This was a patch to perl5db.pl back in 2001, and previously the default had been 0.
You're supposed to turn this off with:
PERLDB_OPT="dieLevel=0" perl5.10.0 -d program
But there is still a code reference in $SIG{__DIE__} after that, and it's a reference to dbdie. I think this is a bug in handling the global variable $prevdie in perl5db.pl's dieLevel. At the end of that subroutine, there is:
# perl5db.pl dieLevel, around line 7777
elsif ($prevdie) {
$SIG{__DIE__} = $prevdie;
print $OUT "Default die handler restored.\n";
}
But notice that after restoring $SIG{__DIE__}, it keeps the previous value in $prevdie, meaning whatever is in there leaks to another call. When I run that command line, there are two calls to dieLevel before it handles PERLDB_OPT, so $prevdie is probably dirty.
So, that's as far as I got before I didn't want to think about perl5db.pl anymore.
I consider it a bug any time code behaves differently in the debugger.
Your problem might be related to this: Debugger corrupts symbol table munging. Essentially, the debugger appears to play some tricks with local -- presumably as part of sandboxing things to provide interactivity. Obviously, messing with the symbol table can have unexpected side-effects. I'd guess that the debugger is localizing $# and thus obscuring your object. I can't think of a work-around.
Is it possible you have an RC file or environment variable (PERLDB_OPTS) that is modifying the dieLevel option of the debugger? I personally haven't used dieLevel but apparently when it's set to a value greater than zero it can force stack unwinding and "tends to hopelessly destroy any program that takes its exception handling seriously." (Quote from here).