Related
I have a Perl script which runs fine under Perl 5.005 on HPUX 11i v3 and that causes a small problem under Perl 5.10 on my Ubuntu 11.04 box.
It boils down to a line like:
open (CMD, "do something | grep -E 'sometext$' |");
(it's actually capturing ps -ef output to see if a process is running but I don't think that's important (a)).
Now this runs fine on the HPUX environment but, when I try it out under Ubuntu, I get:
sh: Syntax error: Unterminated quoted string
By inserting copious debug statements, I tracked it down to that offending line then started removing characters one-by-one until it stopped complaining. Luckily, $ was the first one I tried and it stopped giving me the error, so I then changed the line to:
open (CMD, "do something | grep -E 'sometext\$' |");
and it worked fine (under Linux anyway - I haven't tested that on HPUX since I don't have access to that machine today - if it does work, I'll just use that approach but I'd still like to know why it's a problem).
So it seems obvious that the $ is "swallowing" the closing single quote under my Linux environment but not apparently on the HPUX one.
My question is simply, why? Surely there weren't any massive changes between 5.005 and 5.10. Or is there some sort of configuration item I'm missing?
(a) But, if you know a better way to do this without external CPAN modules (ie, with just the baseline Perl 5.005 installation), I'd be happy to know about it.
$' is a special variable (see perldoc perlvar). 5.005 was many versions ago, so it's possible that something has changed in the regexp engine to make this variable different (although it appears to be in 5.005 also)
As for the better way, you could at least only run the 'ps -ef' in a pipeline and do the 'grep' in perl.
Use the following!!!
use strict;
use warnings;
You would have gotten
Use of uninitialized value $' in concatenation (.) or string
A sigil followed by any punctuation symbol (on a standard keyboard) is a variable in Perl, regardless of if it is defined or not. So in a double quoted string [$#][symbol] will always be read as one token and interpolated unless the sigil is escaped.
I have a feeling that the difference you are seeing has to do with different system shells rather than different versions of perl.
Consider your line:
open (CMD, "do something | grep -E 'sometext$' |");
When perl sees that, it will interpolate the empty $' variable into the double quoted string, so the string becomes:
open (CMD, "do something | grep -E 'sometext |");
At that point, your shell gets to process a line that looks like:
do something | grep -E 'sometext
And if it succeeds or fails will depend on the shell's rules regarding unterminated strings (some shells will complain loudly, others will automatically terminate the string at eof).
If you were using the warnings pragma in your script, you probably would have gotten a warning about interpolating an undefined variable.
A shorter and cleaner way to read in the output of ps would be:
my #lines = grep /sometext\$/, `ps -ef`;
Or using an explicit open:
my #lines = grep /sometext\$/, do {
open my $fh, '|-', 'ps -ef' or die $!;
<$fh>
};
Because $' is a special variable in recent versions of Perl.
From the official documentation (perlvar):
$':
The string following whatever was matched by the last successful
pattern match (not counting any matches hidden within a BLOCK or
eval() enclosed by the current BLOCK).
If there were no successful pattern matches, $' is empty and your statement essentially interpolates to
open (CMD, "do something | grep -E 'sometext |");
Escaping the dollar sign (the solution that works for you on Linux) should work on HPUX too.
I'm not sure when was this variable added, but I can confirm that it exists in Perl 5.8.5. What's New for Perl 5.005 mentions $' (not as a new feature), so I think it was already there before then.
You should probably use single-quotes rather than double quotes around the string since there is nothing in the string that should be interpolated:
open (CMD, q{do something | grep -E 'sometext$' |});
Better would be to use the 3-argument form of open with a lexical file handle:
open my $cmd, '-|', q{do something | grep -E 'sometext$'} or die 'a horrible death';
I don't have a good explanation for why $' is being recognized as the special variable in 5.10 and yet it was not in 5.005. That is unexpected.
Is there a really good reason you can't upgrade to something like 5.14.1? Even if you don't change the system-provided Perl, there's no obvious reason you can't install a recent version in some other location and use that for all your scripting work.
$' is a special variable.
If you want to avoid variable expansion, just use q():
open (CMD, q(do something | grep -E 'sometext$' |));
We (the company I work for) need to run the find2perl script on over a thousand different Unix servers of different flavors (Linux, Solaris, HP-UX, AIX) and different versions.
The one thing that all the servers have in common, is that they all have at least one implementation of perl installed. However, not all systems have it configured the same way.
Finding the location of perl is easy enough using the which command. However, on 70% of the servers, the actual directory containing find2perl (the bin folder of perl) is not present in the $PATH variable and can't be located that way.
On some servers, perl is actually a symbolic link pointing another location, in which case I can use ls -l and sed to extract the target of the link to find where perl is actually installed.
On other servers however, it's more complicated, as it seems perl was compiled to a custom location and the binary of perl present in /bin or /usr/bin (or wherever perl is found) is not a symbolic link, but rather a full blown executable. In this case, I thought about using the #INC variable of perl to try to find find2perl but it seems rather excessive.
What would be the better/best/fullproof method (one-liner if possible) to always get the location of find2perl on a Unix system?
Ways to locate find2perl
Two ways, both of which rely on asking the perl install how it was configured:
Config.pm
Its probably scriptdirexp from Config.pm.
$ perl -MConfig -E 'say $Config{scriptdirexp}'
/usr/bin
And indeed, that's where find2perl is on my system. You can use Config; in your perl scripts, which is its major advantage over the next method.
perl -V:varname
As per Yanick Girouard's comment, you can also use perl -V:scriptdirexp to get this, in a format suitable to passing to eval in a shell script. There are actually several formats available (so, you don't need to use e.g., cut to parse it):
OPTION OUTPUT (\n = actual newline) NOTES
-V:scriptdirexp scriptdirexp='/usr/bin';\n full shell syntax, even if multiple -V options
-V:scriptdirexp: scriptdirexp='/usr/bin' trailing colon omits semicolon and newline
-V::scriptdirexp '/usr/bin'; \n extra leading colon omits var= part
-V::scriptdirexp: '/usr/bin' you can combine them.
Full documentation is in the perlrun manpage.
Ways to embed find2perl
If you decide to copy over find2perl, as per evil otto's comment, you can actually do that by embedding it in your shell script. There are many ways. If neither of the two below work, then you can certainly use shar (which has an extremely long history, and is likely compatible with everything).
Quoted here-document
The easiest way is if your shell supports quoted here-documents. They all should, as its a POSIX requirement:
#!/bin/sh
perl - -name 'foo' -mtime 2 -print <<'FIND2PERL'
#!/usr/bin/perl
eval 'exec /usr/bin/perl -S $0 ${1+"$#"}'
if $running_under_some_shell;
⋮
FIND2PERL
Hex dump in a non-quoted here-document
If some of your shells don't implement quoted here-documents (POSIX‽ what's that!), then you have to protect find2perl from shell expansion. An easy way is to hex dump it, as 0–9 and a–f are all safe from shell expansion. The dump is easily done with xxd -p /usr/bin/find2perl, which only requires xxd on one machine. To read back the dump, you can use plain perl:
#!/bin/sh
perl -n -e 'chomp; print pack("H*", $_)' <<HEX | perl - -name 'foo'
23212f7573722f62696e2f7065726c0a202020206576616c202765786563
202f7573722f62696e2f7065726c202d5320243020247b312b222440227d
⋮
HEX
Using find2perl several times
Naturally, with either approach, you could also write find2perl to a temporary file (if you need to invoke it multiple times, for example). You could also embed it in a shell function.
perl -lwe '$_ = $^X; s/perl$/find2perl/; -f or die qq($_ not -f); print'
Copy the interpreter executable path into dollar default argument. Patch the value, assuming that find2perl is in the same directory as perl itself. (This is specified as UNIX only, so you don't have to cater for perl.exe, which would be easy enough to deal with.) Then test the file exists, and die if it doesn't. (You might invent some better error handling.) Then print the path if we're still alive. That's it.
Okay, here's a version that works for Windows, too:
perl -lwe "$_ = $^X; s/perl(\.exe)?$/find2perl/;
-f or -f qq($_.bat) or die qq($_ not -f); print"
Note the double quotes, de rigueur on Windows for cmd.exe. And it has to go on one line, I just wrapped it for readability.
I'm new to Perl and was hoping someone could tell me what this means exactly
eval 'exec ${PERLHOME}/bin/perl -S $0 ${1+"$#"}' # -*- perl -*-
if 0;
This is explained in perldoc perlrun:
-S
makes Perl use the PATH environment variable to search for the program
unless the name of the program contains path separators.
...
Typically this is used to emulate #! startup on platforms that don't
support #! . It's also convenient when debugging a script that uses
#! , and is thus normally found by the shell's $PATH search
mechanism.
This example works on many platforms that have a shell compatible with
Bourne shell:
#!/usr/bin/perl
eval 'exec /usr/bin/perl -wS $0 ${1+"$#"}'
if $running_under_some_shell;
The system ignores the first line and feeds the program to /bin/sh,
which proceeds to try to execute the Perl program as a shell script.
The shell executes the second line as a normal shell command, and thus
starts up the Perl interpreter. On some systems $0 doesn't always
contain the full pathname, so the -S tells Perl to search for the
program if necessary. After Perl locates the program, it parses the
lines and ignores them because the variable
$running_under_some_shell is never true. If the program will be
interpreted by csh, you will need to replace ${1+"$#"} with $* ,
even though that doesn't understand embedded spaces (and such) in the
argument list. To start up sh rather than csh, some systems may
have to replace the #! line with a line containing just a colon,
which will be politely ignored by Perl.
In short, it mimics shebang behavior for platforms that have shells compatible with Bash.
It's valid both as shell script and as a Perl program. It is used to run the Perl interpreter after all on systems where the shebang doesn't work, for some reason. It's rarely seen these days but used to be common in the early 1990s.
The comment is just a comment, but it has special meaning in Emacs, which will open the file in perl mode.
I just read #Zaid's response, which is better and more correct than mine as long as this code is on the first line of the script being executed, and no shebang exists. I've never seen this kind of substitute. Quite interesting, really.
The second line, if 0; is a part of the first line. You can tell since the first line lacks a ;. It would be more obvious if this was one long single line with the comment being after the semicolon.
So it's equivalent to:
if(0) {
eval 'exec ${PERLHOME}/bin/perl -S $0 ${1+"$#"}
}
In perl, 0 will be evaluated to false, and so the eval-clause will never execute. Presumably this condition(the if) was a quick way to disable the line. Perhaps the evaluation was once something real instead of an always-false.
See perl --help, perldoc -f eval and perldoc -f exec for information on the evaluation block itself.
The remaining trickyness (${1+"$#"}) I have no idea about. This isn't perl anyway; it's interpreted by whichever shell exec is launching (Correct me if I'm wrong on this!). If it's bash, I don't think it does anything at all and can be substituted with $#, which is the environment variable holding all commandline arguments (ie #ARGV in perl).
I often write one-liners on the command line like so:
perl -Magic -wlnaF'\t' -i.orig -e 'abracadabra($_) for (#F)'
In order to scriptify this, I could pass the same flags to the shebang line:
#!/usr/bin/perl -Magic -wlnaF'\t' -i.orig
abracadabra($_) for (#F);
However, there's two problems with this. First, if someone invokes the script by passing it to perl directly (as 'perl script.pl', as opposed to './script.pl'), the flags are ignored. Also, I can't use "/usr/bin/env perl" for this because apparently I can't pass arguments to perl when calling it with env, so I can't use a different perl installation.
Is there anyway to tell a script "Hey, always run as though you were invoked with -wlnaF'\t' -i.orig"?
You're incorrect about the perl script.pl version; Perl specifically looks for and parses options out of a #! line, even on non-Unix and if run as a script instead of directly.
The #! line is always examined for switches as the line is being
parsed. Thus, if you're on a machine that allows only one argument
with the #! line, or worse, doesn't even recognize the #! line, you
still can get consistent switch behavior regardless of how Perl was
invoked, even if -x was used to find the beginning of the program.
(...)
Parsing of the #! switches starts wherever perl is mentioned in the
line. The sequences "-*" and "- " are specifically ignored so that you
could, if you were so inclined, say
#!/bin/sh
#! -*-perl-*-
eval 'exec perl -x -wS $0 ${1+"$#"}'
if 0;
to let Perl see the -p switch.
Now, the above quote expects perl -x, but it works just as well if you start the script with
#! /usr/bin/env perl -*-perl -p-*-
(with enough characters to get past the 32-character limit on systems with that limit; see perldoc perlrun for details on that and the rest of what I quoted above).
I had the same problem with #!env perl -..., and env ended up being helpful:
$ env 'perl -w'
env: ‘perl -w’: No such file or directory
env: use -[v]S to pass options in shebang lines
So, just modify the shebang to #!/usr/bin/env -S perl -...
How can I start an interactive console for Perl, similar to the irb command for Ruby or python for Python?
You can use the perl debugger on a trivial program, like so:
perl -de1
Alternatively there's Alexis Sukrieh's Perl Console application, but I haven't used it.
Not only did Matt Trout write an article about a REPL, he actually wrote one - Devel::REPL
I've used it a bit and it works fairly well, and it's under active development.
BTW, I have no idea why someone modded down the person who mentioned using "perl -e" from the console. This isn't really a REPL, true, but it's fantastically useful, and I use it all the time.
I wrote a script I call "psh":
#! /usr/bin/perl
while (<>) {
chomp;
my $result = eval;
print "$_ = $result\n";
}
Whatever you type in, it evaluates in Perl:
> gmtime(2**30)
gmtime(2**30) = Sat Jan 10 13:37:04 2004
> $x = 'foo'
$x = 'foo' = foo
> $x =~ s/o/a/g
$x =~ s/o/a/g = 2
> $x
$x = faa
If you want history, use rlwrap. This could be your ~/bin/ips for example:
#!/bin/sh
echo 'This is Interactive Perl shell'
rlwrap -A -pgreen -S"perl> " perl -wnE'say eval()//$#'
And this is how it looks like:
$ ips
This is Interactive Perl shell
perl> 2**128
3.40282366920938e+38
perl>
I think you're asking about a REPL (Read, Evaluate, Print, Loop) interface to perl. There are a few ways to do this:
Matt Trout has an article that describes how to write one
Adriano Ferreira has described some options
and finally, you can hop on IRC at irc.perl.org and try out one of the eval bots in many of the popular channels. They will evaluate chunks of perl that you pass to them.
I use the command line as a console:
$ perl -e 'print "JAPH\n"'
Then I can use my bash history to get back old commands. This does not preserve state, however.
This form is most useful when you want to test "one little thing" (like when answering Perl questions). Often, I find these commands get scraped verbatim into a shell script or makefile.
There isn't an interactive console for Perl built in like Python does. You can however use the Perl Debugger to do debugging related things. You turn it on with the -d option, but you might want to check out 'man perldebug' to learn about it.
After a bit of googling, there is a separate project that implements a Perl console which you can find at
Perl Console - Perl code interactive evaluator with completion.
Hope this helps!
There are two popular Perl REPLs.
Devel::REPL is great.
But IMO Reply is better.
For reply just run it as a command. The module install the reply script. If you had installed the module and you don't have the command, check your PATH variable.
$ reply --help
reply [-lb] [-I dir] [-M mod] [--version] [--help] [--cfg file]
You can always just drop into the built-in debugger and run commands from there.
perl -d -e 1
I've created perli, a Perl REPL that runs on Linux, macOS, and Windows.
Its focus is automatic result printing, convenient documentation lookups, and easy
inspection of regular-expression matches.
You can see screenshots here.
It works stand-alone (has no dependencies other than Perl itself), but installation of rlwrap is strongly recommended so as to support command-line editing, persistent command history, and tab-completion - read more here.
Installation
If you happen to have Node.js installed:
npm install -g perli
Otherwise:
Unix-like platforms: Download this script as perli to a folder in your system's path and make it executable with chmod +x.
Windows: Download the this script as perli.pl (note the .pl extension) to a folder in your system's path.
If you don't mind invoking Perli as perli.pl, you're all set.
Otherwise, create a batch file named perli.cmd in the same folder with the following content: #%~dpn.pl %*; this enables invocation as just perli.
Perl doesn't have a console but the debugger can be used as one. At a command prompt, type perl -de 1. (The value "1" doesn't matter, it's just a valid statement that does nothing.)
There are also a couple of options for a Perl shell:
Archived "perlfaq3" page which contain question "Is there Perl Shell?"
For more information read perlfaq3 (current version).
perl -d is your friend:
% perl -de 0
re.pl from Devel::REPL
I always did:
rlwrap perl -wlne'eval;print$#if$#'
With 5.10, I've switched to:
rlwrap perl -wnE'say eval()//$#'
(rlwrap is optional)
You could look into psh here: http://gnp.github.io/psh/
It's a full on shell (you can use it in replacement of bash for example), but uses perl syntax.. so you can create methods on the fly etc.
Read-eval-print loop:
$ perl -e'while(<>){print eval,"\n"}'
Update: I've since created a downloadable REPL - see my other answer.
With the benefit of hindsight:
The third-party solutions mentioned among the existing answers are either cumbersome to install and/or do not work without non-trivial, non-obvious additional steps - some solutions appear to be at least half-abandoned.
A usable REPL needs the readline library for command-line-editing keyboard support and history support - ensuring this is a trouble spot for many third-party solutions.
If you install CLI rlwrap, which provides readline support to any command, you can combine it with a simple Perl command to create a usable REPL, and thus make do without third-party REPL solutions.
On OSX, you can install rlwrap via Homebrew with brew install rlwrap.
Linux distros should offer rlwrap via their respective package managers; e.g., on Ubuntu, use sudo apt-get install rlwrap.
See Ján Sáreník's answer for said combination of rlwrap and a Perl command.
What you do NOT get with Ján's answer:
auto-completion
ability to enter multi-line statements
The only third-party solution that offers these (with non-trivial installation + additional, non-obvious steps), is psh, but:
it hasn't seen activity in around 2.5 years
its focus is different in that it aims to be a full-fledged shell replacement, and thus works like a traditional shell, which means that it doesn't automatically evaluate a command as a Perl statement, and requires an explicit output command such as print to print the result of an expression.
Ján Sáreník's answer can be improved in one way:
By default, it prints arrays/lists/hashtables as scalars, i.e., only prints their element count, whereas it would be handy to enumerate their elements instead.
If you install the Data::Printer module with [sudo] cpan Data::Printer as a one-time operation, you can load it into the REPL for use of the p() function, to which you can pass lists/arrays/hashtables for enumeration.
Here's an alias named iperl with readline and Data::Printer support, which can you put in your POSIX-like shell's initialization file (e.g., ~/.bashrc):
alias iperl='rlwrap -A -S "iperl> " perl -MData::Printer -wnE '\''BEGIN { say "# Use `p #<arrayOrList>` or `p %<hashTable>` to print arrays/lists/hashtables; e.g.: `p %ENV`"; } say eval()//$#'\'
E.g., you can then do the following to print all environment variables via hashtable %ENV:
$ iperl # start the REPL
iperl> p %ENV # print key-value pairs in hashtable %ENV
As with Ján's answer, the scalar result of an expression is automatically printed; e.g.:
iperl> 22 / 7 # automatically print scalar result of expression: 3.14285714285714
Under Debian/Ubuntu:
$ sudo apt-get install libdevel-repl-perl
$ re.pl
$ sudo apt-get install libapp-repl-perl
$ iperl
Matt Trout's overview lists five choices, from perl -de 0 onwards, and he recommends Reply, if extensibility via plugins is important, or tinyrepl from Eval::WithLexicals, for a minimal, pure-perl solution that includes readline support and lexical persistence.
See also Stylish REPL (for GNU Emacs)
Also look for ptkdb on CPAN:
http://search.cpan.org/search?query=ptkdb&mode=all
Sepia and PDE have also own REPLs (for GNU Emacs).
You can do it online (like many things in life) here:
https://www.tutorialspoint.com/execute_perl_online.php
You can use org-babel in emacs; Open an org-mode file, i.e., tmp.org, and then you can do:
#+begin_src perl :results output
#a = (1,5,9);
print ((join ", ", #a) . "\n");
$b = scalar #a;
print "$#a, $b\n";
print "$#a, " . #a . "\n";
print join ", ", 1..$#a; print "\n";
print join ", ", #a[0..$#a]
#+end_src
Pressing CTRL-c CTRL-c evals the block:
#+RESULTS:
#+begin_example
1, 5, 9
2, 3
2, 3
1, 2
1, 5, 9
#+end_example
I am not sure what emacs config this needs to work, but I think you can just install https://github.com/hlissner/doom-emacs and enable its perl and org-mode modules.