I have a dozen or so Perl files with this line:
my $version = "<span class=\"foottext\"><em>version 0.11</em></span>";
This is line 7 of each perl file. The element I need to batch-modify is "X.XX" -- the version number.
What is the most elegant script I can run from the shell (in Perl) to open each file, change the version number on line 7, and then write the file?
A quick Google search reveals that the following one-liner should help:
perl -pi -w -e 's/version 0\.11/version 0.12/g;' *.pl
On a more general note, you should avoid code duplication and move that line into one (single) library file that is called by your dozen other files.
Related
I found some command line with Perl that inserts headers into my files without going through the tedious process of inserting them one by one. Can someone walk me through the Perl aspect of this command line? I'm new to this and can't seem to find the right explanations for what I wrote.
cat header.txt | perl -0 -i -pe 'BEGIN{$h = <STDIN>}; print $h' 1*
-e
rather than provide a script in a xxxx.pl file, provide it on the command line
-p
makes it iterate over filename arguments somewhat like sed but also prints the contents of $_ at the end of the script.
the two above are combined in -pe
-i
indicate you want to edit the file in place and write the output to the same file. In practice, Perl renames the input file and reads from this renamed version while writing to a new file with the original name
-0
redefines the end of record character (\n by default) so that you can read the entire input file as a single line
1*
is the command line argument to your script, so I guess you are modifying any file with a name that starts with 1 (you could have used *.c, or whatever depending on the type of files you are trying to modify)
print $h
prints the variable $h that is the "main" of your script. if it was initialized with the content of the header file (the intent of this one-liner) then it will print the header file
BEGIN{ some code here }
this is stuff you execute before the script starts. this is where I'm stumped. this doesn't seem like valid perl code
so basically:
this will supposedly slurp the entire header file (because of -0) in the BEGIN block and store it in the variable $h
iterate over all the files specified by the wildcards at the end of the command line
for each file: print the header (print $h) then print hte file itself (because of -pe)
so it's equivalent to spelling the script out:
$h = gets content of the entire header file
while (<>){ #loop implied by -pe, iterates over all the 1* files
# the main contents of the "-e" script are inserted below as part of executing -pe
print h$; #print the header we saved
print $_; # implied by -pe, and since we are using -0, this prints the entire content in one shot
# end of the "-e" script. again it was a single print $h statement, the second print is implied by -pe
}
It's a bit hard to explain, take a look at the perlrun documentation for details (run man perlrun).
This is not 100% complete explanation because I don;t think the BEGIN block is right. I tried it on my ubuntu machine and it complained about its syntax too
Here's something similar, with an explanation. The program in the question doesn't run on my mac.
I needed to add the #nullable disable directive to the top of all my csharp files as part of migrating to nullable reference types.
perl -w -i -p -0777 -e 's/^/#nullable disable\n\n/' $(find . -iname '*.cs')
-w enable warnings
-i edit files in place
-p read each file block by block, printing each block after applying a perl expression. the default block size is one line
-0777 changes the default block size to the entire file
-e the perl expression to execute
The final argument uses shell command substitution to create a list of files. It passes that list of file paths to the perl command. The find command searches for files that end in .cs.
The perl program is a single substitution command. It matches the very beginning of the block and replaces (prepends, really) with "#nullable disable" and a couple new-lines.
I have observed that there are two ways of executing a perl program:
perl test.pl
and
./test.pl
What is the exact difference between these two and which one is recommendable?
I will rephrase slightly what other answers stated.
The first case will run the program called "perl" - presumably, a Perl language interpreter, and pass the value "test.pl" to it as the first parameter. Please note that this will do one of 3 things, depending on what "perl" is and what "test.pl" is:
If "perl" does not exist as an executable in your $PATH or a shell alias (check by running which perl), your shell will try to find a non-existing executable, and fail with perl: Command not found error.
If "perl" is an executable in your path (or a shell alias) that is NOT actually a Perl interpreter program, that will get executed instead. As example, try this in csh:
alias perl echo
which perl # Will print "perl: aliased to echo"
perl test.pl # Will print "test.pl". NOT what you intended!
unalias perl
This will execute your "perl" alias and simply echo the word "test.pl"
If "perl" is an executable in your path that IS a real perl interpreter, it will pass "test.pl" to it as a first parameter. In that case, Perl interpreter will treat this parameter (as it doesn't start with a "-") as the name of a file containing Perl code to execute and try to read the file in, compile it as Perl code and execute it.
Note that, since the program being run is actually "perl" and "test.pl" is just a text file being read in, "test.pl" does NOT need to have the "execute" Unix file permission.
The second case, shell will try to find a file called "test.pl" in your current directory, and - if it exists AND is executable - try to execute it as a program.
If the file does not exist OR if the execute bit on it is not set, the shell will fail with "command not found" error.
If the file has the execute bit set, shell (or actually process loader in Unix kernel) will try to execute it. The rules by which Unix executes a given executable file is governed by the first 2 bytes of the file, aka "magic number".
For a VERY good in-depth coverage of how magic numbers work, see "How does the #! work?" question on SO.
In a special case where the "magic number" is "#!" (aka "shebang"), the loader will read the first line of the file, and treat the contents of that line (sans the first 2 bytes) as a command to run instead of the given executable file; and append the path to the executable file as one more parameter to the command it read from shebang line. As examples:
if "test.pl" is a text file with a first line of #!/bin/sh -x, the kernel will execute /bin/sh -x ./test.pl.
if "test.pl" is a text file with a first line of #!/usr/bin/perl, the kernel will execute /usr/bin/perl ./test.pl.
if "test.pl" is a text file with a first line of #!perl, the kernel will execute perl ./test.pl.
if "test.pl" is a text file with a first line of my $var = 1; (or any other first 2 bytes it doesn't know what to do with), it will either error out or (at least on RedHat Linux) will pretend that there was an implied #!/bin/sh shebang and try to execute the file as Bourne Shell script. Which will of course fail since it was Perl code, not shell script
In the first case you are starting the perl interpreter and asking it to use your file and run it.
In the second case you are asking your shell to execute your file. This requires that the file starts with
#!/<path to perl>/perl
and that the file has the execute bit set.
The best method to use is the one that fits your usecase the best.
The first one will always run the script as the perl code.
The second one will do it only in case the perl is specified in she-bang. Otherwise it will run it as shell code or whatever is specified in she-bang (if there is no she-bang at all it will run as current shell code).
The first one will be executed even noexec mount option is enabled.
The second one will fail in that case.
The same stuff with execute bit. The first one will work if +x isn't setted, the second will fail.
The first executes the program using the perl that is found first in your $PATH. The second uses whatever shebang line in the program says.
If u set the executable permissions to the file , then you can run the file by ./ or else run using perl filename.pl
perl test.pl
Specify to the shell that you want the current Perl executable (as is found in your $PATH) to execute the test.pl file that is located in your $PATH.
Run which perl to quickly see what version of perl is the default
Run echo $PATH to see where the '.' (current directory) is. ALL directories BEFORE the '.' will be checked FIRST for the test.pl file! Use ./test.pl instead so the shell looks in the current directory only...Unless you want it to hunt in the $PATH for the test.pl file.
./test.pl
Specify to the shell that you want the test.pl file, in the current directory, to be run by the executable as specified inside the test.pl file at the line with the she-bang (line that starts with #!).
We (the company I work for) need to run the find2perl script on over a thousand different Unix servers of different flavors (Linux, Solaris, HP-UX, AIX) and different versions.
The one thing that all the servers have in common, is that they all have at least one implementation of perl installed. However, not all systems have it configured the same way.
Finding the location of perl is easy enough using the which command. However, on 70% of the servers, the actual directory containing find2perl (the bin folder of perl) is not present in the $PATH variable and can't be located that way.
On some servers, perl is actually a symbolic link pointing another location, in which case I can use ls -l and sed to extract the target of the link to find where perl is actually installed.
On other servers however, it's more complicated, as it seems perl was compiled to a custom location and the binary of perl present in /bin or /usr/bin (or wherever perl is found) is not a symbolic link, but rather a full blown executable. In this case, I thought about using the #INC variable of perl to try to find find2perl but it seems rather excessive.
What would be the better/best/fullproof method (one-liner if possible) to always get the location of find2perl on a Unix system?
Ways to locate find2perl
Two ways, both of which rely on asking the perl install how it was configured:
Config.pm
Its probably scriptdirexp from Config.pm.
$ perl -MConfig -E 'say $Config{scriptdirexp}'
/usr/bin
And indeed, that's where find2perl is on my system. You can use Config; in your perl scripts, which is its major advantage over the next method.
perl -V:varname
As per Yanick Girouard's comment, you can also use perl -V:scriptdirexp to get this, in a format suitable to passing to eval in a shell script. There are actually several formats available (so, you don't need to use e.g., cut to parse it):
OPTION OUTPUT (\n = actual newline) NOTES
-V:scriptdirexp scriptdirexp='/usr/bin';\n full shell syntax, even if multiple -V options
-V:scriptdirexp: scriptdirexp='/usr/bin' trailing colon omits semicolon and newline
-V::scriptdirexp '/usr/bin'; \n extra leading colon omits var= part
-V::scriptdirexp: '/usr/bin' you can combine them.
Full documentation is in the perlrun manpage.
Ways to embed find2perl
If you decide to copy over find2perl, as per evil otto's comment, you can actually do that by embedding it in your shell script. There are many ways. If neither of the two below work, then you can certainly use shar (which has an extremely long history, and is likely compatible with everything).
Quoted here-document
The easiest way is if your shell supports quoted here-documents. They all should, as its a POSIX requirement:
#!/bin/sh
perl - -name 'foo' -mtime 2 -print <<'FIND2PERL'
#!/usr/bin/perl
eval 'exec /usr/bin/perl -S $0 ${1+"$#"}'
if $running_under_some_shell;
⋮
FIND2PERL
Hex dump in a non-quoted here-document
If some of your shells don't implement quoted here-documents (POSIX‽ what's that!), then you have to protect find2perl from shell expansion. An easy way is to hex dump it, as 0–9 and a–f are all safe from shell expansion. The dump is easily done with xxd -p /usr/bin/find2perl, which only requires xxd on one machine. To read back the dump, you can use plain perl:
#!/bin/sh
perl -n -e 'chomp; print pack("H*", $_)' <<HEX | perl - -name 'foo'
23212f7573722f62696e2f7065726c0a202020206576616c202765786563
202f7573722f62696e2f7065726c202d5320243020247b312b222440227d
⋮
HEX
Using find2perl several times
Naturally, with either approach, you could also write find2perl to a temporary file (if you need to invoke it multiple times, for example). You could also embed it in a shell function.
perl -lwe '$_ = $^X; s/perl$/find2perl/; -f or die qq($_ not -f); print'
Copy the interpreter executable path into dollar default argument. Patch the value, assuming that find2perl is in the same directory as perl itself. (This is specified as UNIX only, so you don't have to cater for perl.exe, which would be easy enough to deal with.) Then test the file exists, and die if it doesn't. (You might invent some better error handling.) Then print the path if we're still alive. That's it.
Okay, here's a version that works for Windows, too:
perl -lwe "$_ = $^X; s/perl(\.exe)?$/find2perl/;
-f or -f qq($_.bat) or die qq($_ not -f); print"
Note the double quotes, de rigueur on Windows for cmd.exe. And it has to go on one line, I just wrapped it for readability.
I've got a batch script that does some processing and calls some perl scripts.
My question is if there was a way to put the perl code directly into the batch script and have it run both types of scripts.
Active Perl has been doing this for years!
Below is a skeleton. You can only call perl once though. Because passing it the -x switch says that you'll find the perl code embedded in this file, and perl reads down the file until it finds a perl shebang (#!...perl) and starts executing there. Perl will ignore everything past the __END__ and because you told DOS to goto endofperl it won't bother with anything until it gets to the label.
#rem = '--*-Perl-*--
#echo off
perl -x -S %0 %*
goto endofperl
#rem -- BEGIN PERL -- ';
#!d:/Perl/bin/perl.exe -w
#line 10
use strict;
__END__
:endofperl
Yes you can.
In fact this is exactly what the pl2bat tool does: it transforms a perl program into a batch file which embeds the perl program. Have a look to pl2bat.bat itself.
So you can take the .pl, convert it with pl2bat, and then tweak the batch part as you need. The biggest part of the batch code must be put at the end of the file (near the :end_of_perl label) because in the code at the top you are limited to not using single quotes.
However:
this simple approach will not work if you need to embed more that one perl file
this will be a maintenance nightmare.
So I suggest instead to write the whole process in one Perl program.
Update: if you have one script and some Perl modules that you want to combine in a single batch file, you can combine the Perl file using fatpack, and then apply pl2bat on the result.
There is a way to do this, but it wont be pretty. You can echo your perl code into a temp .pl file and then run that file from within your .bat.
Line directives (#line) are used to reconfigure perl's idea of the current filename and line number. When is this required to get right filename and line number (in an error message)?
Usually such markers are put into code that has been pre-processed or mechanically generated in order to refer back to the human-generated source.
For example, if there was a program that converted Python to Perl, it might insert a
# line 812 "foo.py"
so that error messages would refer to the original Python code which would make more sense to the programmer.
They're useful when wrapping a Perl script in another file, like pl2bat does. Perl doesn't see the batch commands at the beginning of the file which throws off its idea of line numbers. A #line directive at the beginning of the Perl source compensates for this.
I've seen several times that people incorrectly write the current line's number into the #line directive. It should contain the next line's number. Example code of linetest.pl (using a ksh wrapper to set an environment variable for the perl script):
1 #!/usr/bin/ksh
2 MY_ENV_VAR='something'
3 export MY_ENV_VAR
4 /usr/bin/perl -x $0 $# 2>&1
5 exit $?
6
7 #!/usr/bin/perl
8 #line 9
9 print "MY_ENV_VAR is $ENV{MY_ENV_VAR}\n";
10 die "This is line 10.";
Run the script and check the result:
$ ./linetest.pl
MY_ENV_VAR is something
This is line 10. at ./linetest.pl line 10.
You can see that line numbers are matching after writing #line 9 on line 8.
In addition to the already mentioned reasons perl has a (strongly discouraged) -P option that runs the Perl file through a C preprocessor before it is executed. Since most C preprocessor's will use line directives when they include or remove part of a file so any errors will be reported from where they were located in the original source instead of the processed source.
Line directives can also be very useful if you are generating code in strings that is then passed to eval. Normally if there is a warning or error in such code you get an error reported like "died at (eval 1) line 1." Using line directives you can supply a useful file name and line number.
The #line directive is also very helpful when doing perl -e in a shell script. I write
perl -e '#line X
more perl code here...
'
where X is the current shell script line +1 so that any Perl errors tell me the shell line where the failed Perl statement is.