I working on a project at the moment that uses a bit of Linux shell script, and I'm finding myself writing snippets of code that look like this:
#!/bin/sh
x=1
if [ $x -eq 1 ]; then
echo "Hello, Dave!"
fi
if command -v pip3 >/dev/null ; then
echo "PIP3 is already present."
fi
The above seems to work perfectly well, but it upsets my sense of inner harmony. Why should the first if statement only function properly with square brackets, while the second will only function properly without? That's (largely) a rhetorical question; I have a vague understanding that it's something to do with the fact that [ expression ] is actually an abbreviation for test expression. But it still looks ugly.
I seem to have come across an way of ameliorating the ugliness, by putting parentheses in the second if statement, i.e.
if ( command -v pip3 >/dev/null ); then
echo "PIP3 is already present."
fi
Obviously it's subjective, but I find the above a lot easier on the eyes, especially when the command which the if statement is assessing is quite long. But, putting aesthetics to one side, my concern now is: Could using parentheses as above have unintended consequences? My grasp of what exactly ( command ) does is even weaker than my understanding of square brackets, but I'm told that it runs said command in a subshell. That doesn't sound like it would cause problems, but I'd be grateful to be told if my use of parentheses is imprudent.
[ is indeed a synonym for test. The following are equivalent:
if [ $x -eq 1 ]; then
echo "Hello, Dave!"
fi
if test $x -eq 1; then
echo "Hello, Dave!"
fi
In your second example, the parentheses are somewhat redundant. They create a subshell command, in which the enclosed command is executed by a subshell, rather than by the current shell. Often, adding the parentheses makes no practical difference, but that's not always the case. Consider the following artificial example.
x=1
if ( x=3; true ); then
echo $x
fi
versus
x=1
if x=3; true; then
echo $x
fi
If you really need something to "enclose" the condition, use braces:
if { x=3; true; } then
echo $x
fi
but I would recommend adjusting your idea of what shell code should look like.
Related
I'm new to Perl, and am currently tasked with tidying and maintaining a large and pretty messy Perl project. I'm using perl-critic to help me detect issues in the code (and also to teach me best practices).
The existing code has places where the coder has created unreachable code. For instance, they added '&& 0' as a lazy way of commenting out some of the code branches:
if ($req->param('donut') && 0) {
unreachable code...
} else {
always branches to here...
}
I'd hoped that perl or Critic would warn me about unreachable code in such instances (where a conditional has a constant value evaluating to false), but it doesn't.
Is there a tool or a piece of script I could use which can reliably detect this kind of thing?
Obviously I could search for '&& 0' in the source but there are a number of ways that the coder could have created unreachable code besides appending '&& 0' to an if statement.
Using B::Deparse, you can detect unreachable code in some situations:
perl -MO=Deparse -e 'if (0 && $x) {print 1} else {print 2}'
do {
print 2
};
-e syntax OK
It's not so easy if the 0 is not the first condition, though:
perl -MO=Deparse -e 'if ($x && 0) {print 1} else {print 2}'
if ($x and 0) {
print 1;
}
else {
print 2;
}
-e syntax OK
Why is it different? Well, if 0 comes last, all the conditions before it must be checked. They can have side-effects which will still happen. Also, && forces a scalar context, so it can change the behaviour of the code called when evaluating the condition.
This doesn't explain why the block itself isn't compiled away, sorry. My guess would be it just seemed too complicated.
As per choroba's answer, B::Deparse will be able to show you cases where the code is so obviously unreachable that the Perl compiler optimizes it away. But, in the general case it's impossible to detect. The following code includes an effectively unreachable block.
use 5.006;
if ($] < 5) { ... }
Because $] is a variable which returns the currently running version of Perl, which is guaranteed to be at least 5.006 by the use line. But you'd need some pretty clever techniques to figure that out using static analysis of the source code. (As an aside, although an unusual thing to do, it is possible to alter the value of $] at run-time — see Acme::Futuristic::Perl — in which case the code will become reachable.)
If you have a decent test suite for your code, Devel::Cover may be useful. You set the environment variable PERL5OPT to -MDevel::Cover, then run your test suite (note it will run a little slower than usual), then run the command cover which will produce a pretty HTML report. This report will highlight which subs were not executed, which branches were never used, etc.
How do I specify the "before-the loop" code when using "perl -ne", without resorting to either BEGIN/END blocks or replacing "-n" with actually spelled-out while loop?
To explain in detail:
Say, I have the following Perl code:
use MyModule;
SETUP_CODE;
while (<>) {
LOOP_CODE;
}
FINAL_CODE;
How can I replace that with a one-liner using perl -ne?
Of course, the loop part is handled by the -n itself, while the FINAL_CODE can be done using a trick of adding "} { FINAL_CODE" at the end; whereas the use statement can be handled via "-M" parameter.
So, if we had no SETUP_CODE before the loop, I could write the following:
perl -MMyModule -ne 'LOOP_CODE } { FINAL_CODE'
But, how can we insert SETUP_CODE here?
The only idea I have is to try to add it after the loop via a BEGIN{} block, ala
perl -MMyModule -ne 'LOOP_CODE } BEGIN { SETUP_CODE } { FINAL_CODE'
But this seems at best hacky.
Any other solution?
Just to be clear - I already know I can do this by either spelling out the while loop instead of using "-n" or by using BEGIN/END blocks (and might even agree that from certain points of view, doing "while" is probably better).
What I'm interested in is whether there is a different solution.
Write BEGIN and END blocks without ceremony:
$ perl -lne 'BEGIN { print "hi" }
print if /gbacon/;
END { print "bye" }' /etc/passwd
hi
gbacon:x:700:700:Greg Bacon,,,:/home/gbacon:/bin/bash
bye
Sneak your extra code into the -M option
perl -M'Module;SETUP CODE' -ne 'LOOP CODE'
$ perl -MO=Deparse -M'MyModule;$SETUP=1' -ne '$LOOP=1}{$FINAL=1'
use MyModule;
$SETUP = 1;
LINE: while (defined($_ = <ARGV>)) {
$LOOP = 1;
}
{
$FINAL = 1;
}
-e syntax OK
Put your extra code in a module and use ‑M. That’ll run before the loop.
You might even be able to sneak something in via $ENV{PERL5OPT}, although the switches are pretty limited; no ‑e or ‑E, for example.
I suppose you could do something outrageous with $ENV{PERL_ENCODING} too, if you really wanted to.
This is all Acme:: territory. Please don’t. ☹
EDIT: The only solution I much like is the very uncreative and completely straightforward INIT{}.
Remove the -n and add while (<>) { ... }.
What? It's shorter and more straightforward than the BEGIN thing.
Maybe there is some obscure way to achieve this with -ne, but yeah, it is much easer just to use perl -e and code in the while(<>) yourself.
You can have multiple -e switches on the command line.
Assuming test contains
1
2
3
4
perl -e '$a=5;' -ne '$b+=$_ + $a}{print "$b\n"' test
will print 30.
I have a terrible nested script structure I am maintaining that takes user input from the keyboard periodically and I am trying to write a script that will automate this input. Essentially, the script structure looks like this:
cmd1.csh takes 3 lines of input and
then calls cmd2.csh, then exits
normally
cmd2.csh calls cmd3.pl twice
and then takes 1 line of input, then
exits normally
cmd3.pl takes 1 line
of input and exits normally.
Unfortunately, I can't permanently change any of these scripts, so anything I can do has to be done as a wrapper around the whole script chain.
I've tried several variations of the following perl script, but maybe I'm just not hitting the magic combination of quotes and escapes:
$cmd = "echo 'a\\
b\\
c\\
d\\
e\\
f' | ~/pltest/cmd1.csh";
system_tcsh
sub system_tcsh
{
#args = ("tcsh", "-c", shift);
return system(#args);
}
The problem seems to be that once cmd3.pl is called once and d is read, no more input is read by either the second call to cmd3.pl or by cmd2.csh. Putting multiple reads in cmd3.pl works fine and more items are read off of STDIN, but once that first call to cmd3.pl completes, STDIN is empty (or in some kind of unuseable state).
What happens to STDIN when the perl script returns and is there any way to simulate this kind of input using perl or something else?
Thanks in advance.
I created the conditions to reproduce the problem, but all went well:
caller.pl:
#! /usr/bin/perl
$cmd = "echo 'a\\
b\\
c\\
d\\
e\\
f' | ~/doc/Answers/src/pltest/cmd1.csh";
sub system_tcsh
{
#args = ("tcsh", "-c", $cmd);
return system(#args);
}
system_tcsh
cmd1.csh:
#! /bin/csh
echo "${0}(cmd1) $argv[*]"
set line1 = $<
set line2 = $<
set line3 = $<
setenv lineno 3
echo "cmd1: read: $line1 $line2 $line3"
./cmd2.csh
cmd2.csh:
#! /bin/csh
echo " ${0}(cmd2) $argv[*] now running..."
./cmd3.csh
./cmd3.csh
set line6 = $<
echo " ${0}: Read: $line6"
cmd3.csh:
#! /bin/csh
# cmd3
set line = $<
echo " ${0}: Read: '$line'"
Trial run:
frayser#gentoo ~/doc/Answers/src/pltest $ ./caller.pl
/export/home/frayser/doc/Answers/src/pltest/cmd1.csh(cmd1)
cmd1: read: a b c
./cmd2.csh(cmd2) now running...
./cmd3.csh: Read: 'd'
./cmd3.csh: Read: 'e'
./cmd2.csh: Read: f
Maybe you can modify this to recreate the trouble, or use it to implement your solution.
--
UPDATE
This is an update that reproduces the Perl STDIN problem, and provides a work-around for it.
Replacing cmd3 with a Perl version gives the reported, problematic result:
cmd3.pl that replaces cmd3.csh:
#! /usr/bin/perl
# cmd3
$_=<STDIN>;
chomp();
print " $0: Read: '$_'\n";
Result: using cmd3.pl. After the "d" is read, no more input is available.
./caller.pl
./cmd1.csh(cmd1)
cmd1: read: a b c
./cmd2.csh(cmd2) now running...
./cmd3.pl: Read: 'd'
./cmd3.pl: Read: ''
./cmd2.csh: Read:
In order to remedy this situation, cmd2 is changed to only send 1 line to cmd3. The line (1) command handily does this:
#! /bin/csh
echo " ${0}(cmd2) $argv[*] now running..."
line | ./cmd3.pl
line | ./cmd3.pl
set line6 = $<
echo " ${0}: Read: $line6"
This is cmd2.csh optimized to avoid overhead of executing the line command.
#! /bin/csh
echo " ${0}(cmd2) $argv[*] now running..."
set x = $<
echo $x | ./cmd3.pl
set y = $<
echo $y | ./cmd3.pl
set line6 = $<
echo " ${0}: Read: $line6"
Here is the output with the updated cmd2.csh. The functionality is now the same using Perl as it was with using csh as the final script: no lost of stdin.
./cmd1.csh(cmd1)
cmd1: read: a b c
./cmd2.csh(cmd2) now running...
./cmd3.pl: Read: 'd'
./cmd3.pl: Read: 'e'
./cmd2.csh: Read: f
To start with, I would use Expect to script input, it's probably more reliable, and available on most Unix systems.
Barring that, depending on your version of echo, many allow use of "\n" in quoted strings, although that should be functionally equivalent. Something like:
$cmd = '/bin/echo -e "a\nb\nc\nd\ne\nf" | ~/pltest/cmd1.csh'
(This works on my Linux box.. man echo to see what your local version is capable of.)
But, without actually looking at how the Perl script is reading input, it's hard to guess exactly where the output is going, and this is certainly a very convoluted situation.
And what happens if you do:
echo 'a\
b\
c\
d\
e\
f' > tmpfile
cat tmpfile | ~/pltest/cmd1.csh
If that does work you could organise your Perl script around pre-fetching input, saving it to a temp file and then doing the above cat /path/to/tempfile | /path/to/cmd1.csh trick.
#just jon: Unfortunately, expect is not available to me, and I was trying to find a way with the tools I have (because our request process isn't exactly expedient). Thanks for the response, though! The perl script is just performing a , or <>, neither of which seemed to work properly.
#user268396: That doesn't seem to work either, and I would imagine it is the functional equivalent since the pipe forces the input into STDIN regardless of where the data is coming from.
#Frayser: Have you tried the third script (cmd3.csh) as a perl script? That is what my problem involves.
This answer about greedy perl stdin from Greg Bacon seems to be a method to solve this. Of course, it means I have to change the innermost perl script, but it's the best solution so far if I can get permission to change it. It isn't the most robust solution - probably using the Expect pm is the best, but barring that, this should work.
According to Test::More documentation, it will exit with certain exit codes depending on the out come of your tests.
My question is, how do I check these exit codes?
ps. I am trying to build a harness myself. Is there a simple harness I can use?
ps2. Test::Harness did the trick for me. In particular execute_tests function. This function returns all the statistics I want. Thank you everyone who gave useful links.
Any decent harness program (such as prove) will do that for you, there is absolutely no reason for you to do that yourself.
It's a process return code. If running under some Unix variant's shell, you can usually retrieve it as $?. (which probably makes your question more about bash than perl)
Common idioms:
perl your_file.t
if [[ $? -gt 10 ]]; then
echo "Wow, that's a lot of failures"
elif [[ $? -gt 0 ]]; then
echo "Tests failed"
else
echo "Success!"
fi
Alternately, if you're only interested in success/failure:
perl your_file.t || echo "Something bad happened."
If you're calling your test program from Perl, then the hard (and traditional) way involves doing dark and horrid bit-shifts to $?. You can read about that in the system function documentation if you really want to see how to do it.
The nice way involves using a module which gives you a system style function that processes return values for you:
use IPC::System::Simple qw(systemx EXIT_ANY);
my $exit_value = systemx( EXIT_ANY, 'mytest.t' );
The EXIT_ANY symbol allows your script to return any exit value, which we can then capture. If you just want to make sure that your scripts are passing (ie, returning a zero exit status), and halt as soon as any fail, that's IPC::System::Simple's default behaviour:
use IPC::System::Simple qw(systemx);
systemx( 'mytest.t' ); # Run this command successfully or die.
In all the above examples, you can ask for a replacement system command rather than systemx if you're happy for the possibility of the shell getting involved. See the IPC::System::Simple documentation for more details.
There are other modules that may allow you to easily run a command and capture its exit value. TIMTOWTDI.
Having said that, all the good harnesses should check return values for you, so it's only if you're writing our own testing testers that you should need to look at this yourself.
All the best,
Paul
Disclosure: I wrote IPC::System::Simple, and so may have some positive bias toward it.
I would like to evaluate an environment variable and set the result to a variable:
$x=eval($ENV{EDITOR});
print $x;
outputs:
/bin/vi
works fine.
If I set an environment variable QUOTE to \' and try the same thing:
$x=eval($ENV{QUOTE});
print $x;
outputs:
(nothing)
$# set to: "Can't find a string terminator anywhere before ..."
I do not wish to simply set $x=$ENV{QUOTE}; as the eval is also used to call a script and return its last value (very handy), so I would like to stick with the eval(); Note that all of the Environment variables eval'ed in this manner are set by me in a different place so I am not concerned with malicious access to the environment variables eval-ed in this way.
Suggestions?
Well, of course it does nothing.
If your ENV varaible contains text which is half code, but isn't and you give the resulting string to something that evaluates that code as Perl, of course it's not going to work.
You only have 3 options:
Programmatically process the string so it doesn't have invalid syntax in it
Manually make sure your ENV variables are not rubbish
Find a solution not involving eval but gives the right result.
You may as well complain that
$x = '
Is not valid code, because that's essentially what's occurring.
Samples of Fixing the value of 'QUOTE' to work
# Bad.
QUOTE="'" perl -wWe 'print eval $ENV{QUOTE}; print "$#"'
# Can't find string terminator "'" anywhere before EOF at (eval 1) line 1.
# Bad.
QUOTE="\'" perl -wWe 'print eval $ENV{QUOTE}; print "$#"'
# Can't find string terminator "'" anywhere before EOF at (eval 1) line 1.
# Bad.
QUOTE="\\'" perl -wWe 'print eval $ENV{QUOTE}; print "$#"'
# Can't find string terminator "'" anywhere before EOF at (eval 1) line 1.
# Good
QUOTE="'\''" perl -wWe 'print eval $ENV{QUOTE}; print "$#"'
# '
Why are you eval'ing in the first place? Should you just say
my $x = $ENV{QUOTE};
print "$x\n";
The eval is executing the string in $ENV{QUOTE} as if it were Perl code, which I certainly hope it isn't. That is why \ disappears. If you were to check the $# variable you would find an error message like
syntax error at (eval 1) line 2, at EOF
If you environment variables are going to contain code that Perl should be executing then you should look into the Safe module. It allows you to control what sort of code can execute in an eval so you don't accidentally wind up executing something like "use File::Find; find sub{unlink $File::Find::file}, '.'"
Evaluating an environment value is very dangerous, and would generate errors if running under taint mode.
# purposely broken
QUOTE='`rm system`'
$x=eval($ENV{QUOTE});
print $x;
Now just imagine if this script was running with root access, and was changed to actually delete the file system.
Kent's answer, while technically correct, misses the point. The solution is not to use eval better, but to not use eval at all!
The crux of this problem seems to be in understanding what eval STRING does (there is eval BLOCK which is completely different despite having the same name). It takes a string and runs it as Perl code. 99.99% this is unnecessary and dangerous and results in spaghetti code and you absolutely should not be using it so early in your Perl programming career. You have found the gun in your dad's sock drawer. Discovering that it can blow holes in things you are now trying to use it to hang a poster. It's better to forget it exists, your code will be so much better for it.
$x = eval($ENV{EDITOR}); does not do what you think it does. I don't even have to know what you think it does, that you even used it there means you don't know. I also know that you're running with warnings off because Perl would have screamed at you for that. Why? Let's assume that EDITOR is set to /bin/vi. The above is equivalent to $x = /bin/vi which isn't even valid Perl code.
$ EDITOR=/bin/vi perl -we '$x=eval($ENV{EDITOR}); print $x'
Bareword found where operator expected at (eval 1) line 1, near "/bin/vi"
(Missing operator before vi?)
Unquoted string "vi" may clash with future reserved word at (eval 1) line 2.
Use of uninitialized value $x in print at -e line 1.
I'm not sure how you got it to work in the first place. I suspect you left something out of your example. Maybe tweaking EDITOR until it worked?
You don't have to do anything magical to read an environment variable. Just $x = $ENV{EDITOR}. Done. $x is now /bin/vi as you wanted. It's just the same as $x = $y. Same thing with QUOTE.
$ QUOTE=\' perl -wle '$x=$ENV{QUOTE}; print $x'
'
Done.
Now, I suspect what you really want to do is run that editor and use that quote in some shell command. Am I right?
Well, you could double-escape the QUOTE's value, I guess, since you know that it's going to be evaled.
Maybe what you want is not Perl's eval but to evaluate the environment variable as the shell would. For this, you want to use backticks.
$x = `$ENV{QUOTE}`