Perl: After a successful system call, "or die" command still ends script - perl

I am using the following line to make a simple system call which works:
system ("mkdir -p Purged") or die "Failed to mkdir." ;
Executing the script does make the system call and I can find a directory called Purged, but the error message is still printed and the script dies. What is wrong with my syntax?

system returns the exit status of the command it calls. In shell, zero exit status means success. You have to invert the logic:
0 == system qw(mkdir -p Purged) or die "Failed to create the dir\n";

That would be a little confusing, wouldn't? - Leonardo Herrera on Ikegami's answer
Yes, it is confusing that the system command inverts true and false in Perl, and creates fun logic like this:
if ( system qw($command) ) {
die qq(Aw... If failed);
}
else {
say qq(Hooray! It worked!);
}
But, it's understandable why the system command does this. In Unix, an exit status of zero means the program worked, and a non-zero status could give you information why your system call failed. Maybe the program you were calling doesn't exist. Maybe the program sort of worked as expected. For example, grep returns an exit code of 1 when grep works, but there were no matching lines. You might want to distinguish when grep returns zero, one, or a return code greater than one. (Yes, I know it's silly to use a system call to grep in a Perl program, but that's the first example I could think of).
To prevent casual confusion, I create a variable that holds the exit status of my system command instead of testing the output of system directly:
my $error = system qw($command);
if ($error) {
die qq(Aw... It failed);
}
else {
say qq(Hooray! It worked!);
}
It's completely unnecessary, and people who work with Perl should know that system reverses Perl's definition of true and false, but if I hadn't had my coffee in the morning, I may miss it as I go over someone else's code. Doing this little step just makes the program look a bit more logical.
The Perldoc of system give you code that allows you to test the output of your system command to see exactly what happened. (If there was an error, or a system interrupt signal killed the system call). It's nice to know if you need to interrogate your system return value to figure out what went wrong.

system returns 0 on success, so you want and rather than or.
See also: use autodie qw( system );

To add what hasn't been mentioned but what comes with it and can quietly mess things up.
The system's return of 'zero-or-not' only tells whether the command itself executed successfully, by shell or execvp; the 0 does not mean that the command succeeded in what it was doing.†
In case of non-zero return you need to unpack $? for more information;† see system for how to do this. For one, the command's actual exit code is $? >> 8, what the executed program was designed to communicate at its exit.
Altogether you may want to do something to the effect of
sub run_system {
my ($cmd, #other_agrs) = #_; # may do more via #other_args
my ($cmdref, $sys_ret) = (ref $cmd, 0); # LIST or scalar invocation?
if ($cmdref eq 'ARRAY') {
$sys_ret = system(#$cmd);
}
elsif (not $cmdref) {
$sys_ret = system($cmd);
}
else { Carp::carp "Invocation error, got $cmdref" }
return 1 if $sys_ret == 0;
# Still here? The rest is error handling.
# (Or handling of particular non-zero returns; see text footnote)
Carp::carp "Trouble with 'system($cmd)': $?";
print "Got exit " . ($? >> 8) . " from $cmd\n";
return 0; # or Carp::croak (but then design changes)
}
This can be done merely as system == 0 or do { ... }; or such right in the code, but this way there's a little library, where we can refine error handling more easily, eventually decide to switch to a module to manage external commands, etc. Also, being a user-level facility now it makes sense that it returns true on success (we aren't changing system's design).
Additionally in this case, mkdir -p is meant to be quiet and it may not say anything when it cannot do its thing, in some cases. This is by design of course, but one should be aware of it.
† A program may return whatever it wants; nothing requires of it to return zero if it succeeded (completed without error) and it does not have to follow any conventions; what it returns is entirely up to its design.
Typical Unix programs generally follow the convention of returning non-zero only if there were problems, but some do return non-zero values merely to indicate a particular condition. See for example here and here.
One example: for at least some versions, zip returns non-zero (12) when it "has nothing to do" (nothing to update in a package for example), even though it completes successfully; so we get 3072 from system in that case (when that 12 is packed into high bits of a number which is the return code). This is documented on its man page.
A zero return, though, normally does imply a successful operation, what is the main point here.

Simplest
system ("mkdir -p Purged") && die "Failed to mkdir.";

The system command passes through the exit code of the program that it ran. It doesn't judge or interpret it. Raku, on the other hand, does interpret it and you have to fight the language to get around it. And, remember that Perl's system is basically C's system (and many things in Perl are just what C's version does).
You need to compare this to the value you expected based on your knowledge of that particular command's behavior, as shown in the system docs:
#args = ("command", "arg1", "arg2");
system(#args) == 0
or die "system #args failed: $?"
Usually, well made unix programs use 0 to indicate success and non-zero for everything else. That's a good rule of thumb, but it's a convention rather than a requirement. Some programs return non-zero values to indicate other sorts of success. system knows none of that. Not only that, many programmers have no idea what an exit code should be, and I've often had to deal with someone's reinvented idea of what that should be (and I've probably done that myself at some point).
There's been a growing trend of "trust and don't verify", which I think is the new generation of programmers not having systems and shell programming backgrounds. And, even then, there's how the world should be and how it actually is. Remember, your goal as the programmer is to know what happened and why, even if the world is messy.
Here are a few interesting links:
Well Behaved Non-Zero Exit Codes?
Non-zero exit status for clean exit
Are there any standard exit status codes in Linux?
I like this statement from Frederick the best, though:
More realistically, 0 means success or maybe failure, 1 means general failure or maybe success, 2 means general failure if 1 and 0 are both used for success, but maybe success as well.
Consider grep. Here's a file to search:
$ cat original.txt
This is the original
This is from append.txt
This is from append.txt
This is from append.txt
Search for something that exists:
$ grep original original.txt
This is the original
The exit code is 0 ($? on the shell, but Perl's $? is more complicated). That's what you expect by convention. You found something and there were no errors:
$ echo $?
0
But now search for something that doesn't exist. You don't find something but there were no errors:
$ grep foo original.txt
Now the exit code is 1, because that's what grep does:
$ echo $?
1
This is non-zero but not an error. You called grep correctly and it produced the correct and expected output.
It's not up to system to judge those exit values. It passes through that exit code and lets you interpret them.
Now, we get lazy and assume that 0 means that it worked. We do that because many people are used to seeing it and it works most of the time. But, "works most if the time" is different than "won't bite me in the ass". Compare the result to system to exactly what you will accept as success:
my $expected = 0;
system('...') == $expected or die ...
If the task's definition of success is that grep finds matching lines, that code works fine because the exit code in that case happens to be zero. But, that accidentally works because two different ideas happen to align at that one point.
But, what you accept as success is not the same thing as the exit code. Consider that grep example. So far you know that both 0 and 1 are normal operation. Both of those mean that grep did not encounter an error. That's different than calling grep incorrectly and getting a different exit value:
$ grep -X foo original.txt
grep: original.txt: Undefined error: 0
$ echo $?
2
That 2 is conventionally used to signal that you called the program incorrectly. For example, you are using switches that exist on one implementation of a tool that don't exist on another (there are other things than line and gnu, but less so).
You might end up with something like this:
my %acceptable = map { $_ => 1 } qw(0 1);
my $rc = system(...);
die ... unless exists $acceptable{$rc};
And, the die can pass through that exit code (see it's method for choosing a value):
$ perl -e 'system( "grep -X foo original.txt" ) == 0 or die'
grep: original.txt: Undefined error: 0
Died at -e line 1.
$ echo $?
2
Sure, that's not as pretty as the single statement, but pretty shouldn't trump correct.
But that's not even good enough. My linux version of grep says it will exit with 0 on error in certain conditions:
Normally the exit status is 0 if a line is selected, 1 if no lines were selected, and 2 if an error occurred. However, if the -q or --quiet or --silent is used and a line is selected, the exit status is 0 even if an error occurred.
Going a step further, there's a list of conventional exit codes in sysexits.h that signal very particular conditions. For example, that exit code of 2 gets the symbol EX_USAGE and indicates a problem with the way the command was formed.
When I'm writing system code, I'm hard core on proper exit codes so other programs can know what happened without parsing error output. The die called by the user without a failure in a system will probably return 255. That's not very useful for other things to figure out what went wrong.

A one-line solution:
system("if printf '' > tmp.txt; then exit 1; else exit 0; fi ;") or die("unable to clobber tmp.txt");

Related

how to return exit code from one program to another program in perl

I have perl Code 2, which is called from perl Code1. return value of code 2 determines, if to proceed further with code 1, or exit. How to write this logic.
Code 1:
my $exit = `perl /home/username/code2.pl -f #flag_value`;
if($exit == 1){
#Do something;
exit ;
}else{
#Do somethingelse;
exit;
Code 2:
subroutine(#flag_value);
sub subroutine{
my (#P)= #_;
#do req ops and assigns $exitcode
exit ($exitcode);
return $exitcode;
}
I am unable to build the logic on how to return the exit code from code 2 to 1. $exit has nothing if I return the $exitcode value. Please help me know my mistake.
Tried to reference available solutions, couldnot understand my mistake.
What a program wants to communicate to its caller as its exit is generally the number passed to exit.† But then that number is packaged into the high (most significant) 8 bits of a 16-bit word by the UNIX wait call, lower bits being a single bit indicating whether there was a core dump, and the number of the signal that terminated it (if any) in the right-hand (least-significant) seven bits. Then that is returned to the caller and in a Perl program that is available in the $? variable.
As docs for $? in perlvar put it
This is just the 16-bit status word returned by the traditional Unix wait() system call (or else is made up to look like it). Thus, the exit value of the subprocess is really ($? >> 8), and $? & 127 gives which signal, if any, the process died from, and $? & 128 reports whether there was a core dump.
So in the program that is called from another ("Code 2") you'd exit with: exit $exit_value;.‡
Then the 16-bit status word described above, that the calling program ("Code 1") receives, is in the $? variable. To get the actual exit value intended by the called program we need to extract the higher bits, by shifting to the right, $? >> 8. One way to organize this
my $output = qx( code2.pl ... );
# First investigate $? & 127 and $? & 128, see "system" docs for an example
my $exit_value = $? >> 8;
if ($exit_value == 1) ...
Here qx is the operator form of backticks used in the question, which I think is far nicer. See system for one way to look into possible errors.
Finally, a good option is to use libraries for calling and managing external programs, which make much of this, and way more, much easier. In the tentative order of simpler to more complex some are IPC::System::Simple, Capture::Tiny, IPC::Cmd (core), IPC::Run.
What is posted here is a bit simplified and perhaps loosely phrased at places; the provided links are all minimal required study, and more will come up as you follow through. Among the many SO pages on this, this page has clear ascii art and explanation(s) for that "16-bit word."
† However, normally the $? with the status is set prior to unwinding the call stack, and then END blocks run (if any are defined) and global destructors are called (if needed), which both can alter the exit value.
This goes after an explicit exit is issued as well; the program may not actually exit right away after exit. If you want or need to make sure no END blocks run and no destructors are called then use POSIX::_exit($status)
‡ This "Code 2" is a program, which could build the exit code in a subroutine, as indicated in the question. Then the subroutine would return that so that it can be used by exit
# code2.pl
use warnings;
use strict;
...
sub subroutine{
my (#P)= #_;
#do req ops and assigns $exitcode
...
return $exitcode;
}
...
exit subroutine(#flag_value);

Why to shift bits "$? >> 8" when using Perl system function to execute command

I have seen this in many places in Perl scripts:
system("binary");
$exit_code = ($? >> 8)
exit($exit_code)
Why should I use this? Is there an alternative way?
I am calling some binary in system(binary) which I created in C++, which does some stuff and if it fails it gives assert. When I reboot a Linux machine, the stuff which is going fails, and my binary generates assert as expected. But on the Perl side where I called it, it throws a 134 error code, which after 134 >> 8, becomes 0. Ultimately it is making my failure operation a success (which I don't want).
perldoc -f system snippet:
The return value is the exit status of the program
as returned by the wait call. To get the actual
exit value divide by 256.
You can check all the failure possibilities by
inspecting $? like this:
$exit_value = $? >> 8;
$signal_num = $? & 127;
$dumped_core = $? & 128;
The program executed under system may return a specific code when it exits. That is packaged into high bits of a number (exit status) which Perl gets and which is available in $?. This is why you want to test that number, for example (but see below)
system($cmd) == 0 or die "Error: $?";
Or, you can separately check $? after the call. If true (non-zero), that only means that there was something other than a clean exit by the application, or a problem with system itself. When you unpack that you are looking for what the application communicated on its exit. In order to merely see whether there was an error you only look at $?, and you are getting a value. So there is no conflict.
What code that program returned is its business (design decision). Programs ideally exit with codes when they fail (if they can detect and handle the problem), and have documentation explaining what code means what. For an example see this answer, and for another comment this one.
As seen in system, you readily get: exit code $? >> 8, signal number $? & 127, and whether core was dumped $? & 128. With your exit status of 134 the signal number is 6, which man 7 signal lists as SIGABRT. The core should be there as well. So that is what you got from your program, and no explicit exit code. Apparently, the program caused abort and dumped core.
In your case, you know where all this comes from -- assert is a macro which calls abort, whereby SIGABRT is raised (man 3 assert abort). Perl gets back that 6 packaged into 134.
Note that assert prints a message to STDERR, so you may want to run the program via qx (backticks), in which case you can capture that error. See the first link above, or search on SO.

What is the meaning of the built-in variable $? in Perl [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
perl “dollar sign and question mark” question
I am trying to understand the Perl script written by someone. I don't understand the use of $? in the script. Can anyone explain me the purpose of below line?
system( "perform_task.sh", "-param1");
if( ( $? >> 8 ) != 0 ) {
print( "perform_task.sh failed " );
}
To find the meaning of any variable, you can either type
$ perldoc -v '$?'
on the command line with relatively recent versions of Perl or scan perldoc perlvar installed on your computer. Usually, it is best to read the documentation for the specific version of perl you have, but in a pinch, bearing in mind any possible gotchas due to version differences, the online version will do as well: perldoc -v '$?':
The status returned by the last pipe close, backtick (``) command, successful call to wait() or waitpid(), or from the system() operator. This is just the 16-bit status word returned by the traditional Unix wait() system call (or else is made up to look like it). Thus, the exit value of the subprocess is really ($? >> 8), and $? & 127 gives which signal, if any, the process died from, and $? & 128 reports whether there was a core dump.
Further information can be gleaned from the documentation for perldoc -f system:
If you'd like to manually inspect "system"'s failure, you can check all possible failure modes by inspecting $? like this:
if ($? == -1) {
print "failed to execute: $!\n";
}
elsif ($? & 127) {
printf "child died with signal %d, %s coredump\n",
($? & 127), ($? & 128) ? 'with' : 'without';
}
else {
printf "child exited with value %d\n", $? >> 8;
}
While there is nothing wrong with asking even elementary questions on Stackoverflow, if you actually want to become a capable programmer, you'll need to get into the habit of reading the documentation yourself, and develop the capacity to understand it yourself.
Otherwise, you'll waste valuable time you could be using to solve problems instead on waiting for others to read the documentation for you.
It really doesn't affect the rest of us if you choose never to expend any effort in trying to understand documentation, but it will hurt you in the long run.
The use English; name for the $? is $CHILD_ERROR. It is child's exit status including possible signal number that caused it exit. See perldoc perlvar.
$? is the error code of the child process (perform_task.sh).
In the case of your script the return code is shifted eight bits to the right and the result compared with 0.
This means the run is considered a failure only if the returned code is > than 255.

Why does this perl script sometimes stop printing after the system function

After i changed:
system("recode ISO-8859-1..UTF-8 $csvPath");
if($? == -1) {
die(" failed\n");
}
to:
if(system("recode ISO-8859-1..UTF-8 $csvPath")) {
die(" failed!");
}
it works better but still not always.
Code snippet:
print("recoding file to utf-8...");
#`recode ISO-8859-1..UTF-8 $csvPath`;
system("recode ISO-8859-1..UTF-8 $csvPath");
if($? == -1) {
die(" failed\n");
}
print(" done!\n");
This doesn't happen all the times. Sometimes it works sometimes not.
If it doesn't works (script doesn't finish in that case) the first print works, but the the second one with "done!" is never printed even if it the external command worked.
Your test if($? == -1) doesn't make much sense. $? will be 0 if the command succeeded, non-0 if it failed. For example, if the command did an exit(1), $? should be 256 -- at least on Unix-like systems. And it's probably cleaner to test the value returned by system() rather than examining $# after the fact.
Try setting $| = 1; at the top of your program; this makes your output unbuffered, and might let you see some output your script is producing before it dies.
Print the exact value of the command string before executing it. If $cvsPath is an empty string (or if it's undef and you haven't enabled warnings), then you're invoking recode with no file argument, and it will read from standard input. That might explain the behavior you're seeing.
Though this isn't directly relevant to your problem, if system() is given a single argument, and if that argument contains shell metacharacters (including blanks), then the command is executed by the shell. With multiple arguments, it invokes the command directly:
system('recode', 'ISO-8859-1..UTF-8', $csvPath);
perldoc -f system for more information.
And in fact, if a bad value of $cvsPath is the cause of your problem, that could have revealed it, since it would pass an empty string as a separate argument to recode. The error message recode produces in that case is:
recode: fopen (): No such file or directory
(the empty file name is shown between the parentheses).
Note that a lot of this is guesswork based on incomplete information. Posting a small complete program that exhibits the problem would make this a lot easier.

How do I check the exit code in Test::More?

According to Test::More documentation, it will exit with certain exit codes depending on the out come of your tests.
My question is, how do I check these exit codes?
ps. I am trying to build a harness myself. Is there a simple harness I can use?
ps2. Test::Harness did the trick for me. In particular execute_tests function. This function returns all the statistics I want. Thank you everyone who gave useful links.
Any decent harness program (such as prove) will do that for you, there is absolutely no reason for you to do that yourself.
It's a process return code. If running under some Unix variant's shell, you can usually retrieve it as $?. (which probably makes your question more about bash than perl)
Common idioms:
perl your_file.t
if [[ $? -gt 10 ]]; then
echo "Wow, that's a lot of failures"
elif [[ $? -gt 0 ]]; then
echo "Tests failed"
else
echo "Success!"
fi
Alternately, if you're only interested in success/failure:
perl your_file.t || echo "Something bad happened."
If you're calling your test program from Perl, then the hard (and traditional) way involves doing dark and horrid bit-shifts to $?. You can read about that in the system function documentation if you really want to see how to do it.
The nice way involves using a module which gives you a system style function that processes return values for you:
use IPC::System::Simple qw(systemx EXIT_ANY);
my $exit_value = systemx( EXIT_ANY, 'mytest.t' );
The EXIT_ANY symbol allows your script to return any exit value, which we can then capture. If you just want to make sure that your scripts are passing (ie, returning a zero exit status), and halt as soon as any fail, that's IPC::System::Simple's default behaviour:
use IPC::System::Simple qw(systemx);
systemx( 'mytest.t' ); # Run this command successfully or die.
In all the above examples, you can ask for a replacement system command rather than systemx if you're happy for the possibility of the shell getting involved. See the IPC::System::Simple documentation for more details.
There are other modules that may allow you to easily run a command and capture its exit value. TIMTOWTDI.
Having said that, all the good harnesses should check return values for you, so it's only if you're writing our own testing testers that you should need to look at this yourself.
All the best,
Paul
Disclosure: I wrote IPC::System::Simple, and so may have some positive bias toward it.