I have a Perl script called replaceUp:
#!/usr/bin/perl
search=$1
replace=$2
find . -type f -exec perl -p -i -e "s/$search/$replace/g" {} \;
The script does not get loaded. This suggests me that my script is wrong.
How can you make a shell script using Perl?
The first line should be #!/bin/sh, not #!/usr/local/bin/perl. You are mistaken that that is a Perl script; it is a shell script that calls Perl.
It's also not going to actually work because $search and $replace are not going to get interpolated inside single quotes. Try single quotes inside double quotes.
Or better yet, try my mass search/replace Perl script. I keep a pure-Perl script for this around because, dangerous as mass search/replace is, you don't need multiple levels of shell metacharacter interpretation taking it from dangerous to absolutely lethal.
Related
I am trying to test the code snippet below for a bigger script that I am writing. However, I can't get the search working with parentheses and variables.
Appreciate any help someone can give me.
Code snippet:
#!/usr/bin/perl
$file="test4.html";
$Search="Help (Test)";
$Replace="Testing";
print "/usr/bin/sed -i cb 's/$Search/$Replace/g' $file\n";
`/usr/bin/sed -i cb 's/$Search/$Replace/g' $file`;
Thanks,
Ash
The syntax to run a command in a child process and wait for its termination in perl is system "cmd", "arg1", "arg2",...:
#!/usr/bin/perl
$file="test4.html";
$Search="Help (Test)";
$Replace="Testing";
print "/usr/bin/sed -icb -e 's/$Search/$Replace/g' -- $file\n";
system "/usr/bin/sed", "-icb", "-e", "s/$Search/$Replace/g", "--", $file;
(error checking left as an exercise, see perldoc -f system for details)
Note that -i is not a standard sed option. The few implementations that support it (yours must be the FreeBSD one as you've separated the cb backup extension from -i) have actually copied it from perl! It does feel a bit silly to be calling sed from perl here.
Looking at your approach:
The `...` operator itself is reminiscent of the equivalent `...` shell operator. In perl, what's inside is evaluated as if inside double quoted, in that $var, #var... perl variables are expanded, and a shell is started with -c and the resulting string as arguments and with its stdout redirected to a pipe.
The shell interprets that argument as code in the shell syntax. Perl reads the output of that inline shell script from the other end of the pipe and that makes up the expansion of `...`. Same as in shell command substitution except that there's is no stripping of zero bytes or of trailing newlines.
sed -i produces no output, so it's pointless to try and capture its output with `...` here.
Now in your case, the code that sh is asked to interpret is:
/usr/bin/sed -i cb 's/Help (Test)/Testing/g' test4.html
That should work fine on FreeBSD or macOS at least. If $file had been test$(reboot).html, that would have been worse though.
Here, because you have the contents of variables that end up interpreted as code in an interpreter (here sh), you have a potential arbitrary command injection vulnerability.
In the system approach, we remove sh, so that particular vulnerability is removed. However sed is also an interpreter of some language. That language is not as omnipotent as that of sh, but for instance sed can write to arbitrary files with its w command. The GNU implementation (which you don't seem to be using) can run arbitrary commands as well.
So you still potentially have a code injection vulnerability in the case of $Search or $Replace coming from an external source.
If that's the case, you'd need to make sure your properly sanitise those values before running sed. See for instance: How to ensure that string interpolated into `sed` substitution escapes all metachars
I need to write a perl script that calls a c-shell script that calls yet another perl script. I cannot change the c-shell script or the perl script it calls. One of the args that needs to be passed is a quotes string with spaces. If I use backticks to call the c-shell, and I run the c-shell with tcsh, the quoted string is respected as a single entity. However, if I run the c-shell with source, it is not.
I feel that I need to use 'source' because when the c-shell is called by users from the command line, it is called through an alias that sources the c-shell. E.g.
alias top "source top.csh"
Consider these...
topmost.pl
#!/usr/bin/env perl
use strict;
print "Try with tcsh...\n";
my $msg = `tcsh ./top.csh -arg1 "this line has spaces"`;
print "$msg\n";
print "Try with source...\n";
my $msg = `source ./top.csh -arg1 "this line has spaces"`;
print "$msg\n";
exit;
top.csh is simply....
perl ./subperl.pl $*:q
exit
And subperl.pl is...
#!/usr/bin/env perl
use strict;
print "In subperl.pl\n";
foreach $x (#ARGV) {
print "$x\n";
}
print "The End\n";
exit;
When I run the topmost.pl script, I get...
Try with tcsh...
In subperl.pl
-arg1
this line has spaces
The End
Try with source...
In subperl.pl
-arg1
this
line
has
spaces:q
The End
Why does the "sourced" call to the top.csh script fail to respect the quotes ?
#Kaz has the answer as to why your code isn't working. This answer is about how to avoid this class of problems entirely.
First, if you can, add a #!/bin/tcsh to top.csh and make it executable (ie. chmod +x). Now it can be executed as top.csh without needing to know what shell to use.
Then you'll want to avoid using `` for anything but very simple commands. This is because `` is interpreted by the shell and now you need to worry about shell special characters and escapes and spaces... it's a mess. What you need is a way to call external programs without invoking a shell.
You can do this by passing a list to system, but system cannot capture the output.
system "tcsh", "./top.csh", "-arg1", "this line has spaces";
While you can cobble something together with open and pipes, it's better to use a pre-existing library such as IPC::System::Simple.
use IPC::System::Simple qw(capturex);
# Or capturex("./top.csh", ...) if you add a #! to top.csh.
my $msg = capturex("tcsh", "./top.csh", "-arg1", "this line has spaces");
For more involved interactions with executables, look into System::Command or IPC::Run.
Needless to say, Perl scripts which call shell scripts which call Perl scripts is a bit of a nightmare to maintain. Rather than do that, it is better to scoop the guts of subperl.pl out into a Perl library and have both subperl.pl and your code use that library.
The command in backticks is being interpreted by your system interpreter (invoked via /bin/sh), which I'm guessing might be GNU Bash. Or, in any case, it seems to be some shell which understands the source command, and almost certainly a POSIX-like shell. That source command quite probably tells that shell to read a script written in that shell's own language. So, for instance, if that shell happens to be Bash, it will treat that as a Bash script1, not as a Tcsh script.
The only way both could work is if the script is a "polyglot": a program which can be interpreted by either tcsh or the system shell that is used by perl to implement backticks.
(An easy example of a C Shell + POSIX shell polyglot is a script that contains nothing but a sequence of trivial commands consisting of space separated words like cp from to.)
Your script isn't a polyglot; only Tcsh understands the :q syntax, not the other shell.
More precisely, if /bin/sh is Bash, the original source ... command as well as the contents of the sourced top.csh script will be treated as a POSIX-mode Bash script, since when Bash is invoked as /bin/sh, it turns off its POSIX-incompatible behaviors. So even if Bash's pathname expansion supported the Tcsh :q mechanism, it would almost certainly be turned off under POSIX mode because $*:q already has a firm meaning in POSIX.
I need to get an example file file from a find command in a Perl script to create another system call afterwards. For some reason, the find command gets stuck when I call it from the script. Here is what I need to do:
my $search_dir = "/something/like/this/??/??/??";
# the triple '??' are needed here
my $cmd = "find $search_dir -name \"\*.$var1.token1.$var2.ext\" | head -n 1";
my $first_example_file = `$cmd`; chomp $first_example_file;
This gets stuck when I run it through Perl, it never finishes executing the command, whereas the constructed $cmd runs in no time if I copy+paste it and run in in my bash terminal. Any ideas?
Try using the File::Find perl module for finding files. If you would like to use bash's find in your perl then you might have to use $(..) in your command.
I am not in to perl … just trying to help out.
Update:
As stated in the comments by Rohaq you can also use File::Find::Rule
I'd wager globbing (shell metacharacter expansion) is involved. But regardless, try and chop the command up. Does it work without the pipe? What about without the ?? in the pathname? What happens if you prepend 'echo' ("echo find ...")? Still hanging? Then you can try it under perl -d - the debugger; perldoc perldebug is your friend.
We (the company I work for) need to run the find2perl script on over a thousand different Unix servers of different flavors (Linux, Solaris, HP-UX, AIX) and different versions.
The one thing that all the servers have in common, is that they all have at least one implementation of perl installed. However, not all systems have it configured the same way.
Finding the location of perl is easy enough using the which command. However, on 70% of the servers, the actual directory containing find2perl (the bin folder of perl) is not present in the $PATH variable and can't be located that way.
On some servers, perl is actually a symbolic link pointing another location, in which case I can use ls -l and sed to extract the target of the link to find where perl is actually installed.
On other servers however, it's more complicated, as it seems perl was compiled to a custom location and the binary of perl present in /bin or /usr/bin (or wherever perl is found) is not a symbolic link, but rather a full blown executable. In this case, I thought about using the #INC variable of perl to try to find find2perl but it seems rather excessive.
What would be the better/best/fullproof method (one-liner if possible) to always get the location of find2perl on a Unix system?
Ways to locate find2perl
Two ways, both of which rely on asking the perl install how it was configured:
Config.pm
Its probably scriptdirexp from Config.pm.
$ perl -MConfig -E 'say $Config{scriptdirexp}'
/usr/bin
And indeed, that's where find2perl is on my system. You can use Config; in your perl scripts, which is its major advantage over the next method.
perl -V:varname
As per Yanick Girouard's comment, you can also use perl -V:scriptdirexp to get this, in a format suitable to passing to eval in a shell script. There are actually several formats available (so, you don't need to use e.g., cut to parse it):
OPTION OUTPUT (\n = actual newline) NOTES
-V:scriptdirexp scriptdirexp='/usr/bin';\n full shell syntax, even if multiple -V options
-V:scriptdirexp: scriptdirexp='/usr/bin' trailing colon omits semicolon and newline
-V::scriptdirexp '/usr/bin'; \n extra leading colon omits var= part
-V::scriptdirexp: '/usr/bin' you can combine them.
Full documentation is in the perlrun manpage.
Ways to embed find2perl
If you decide to copy over find2perl, as per evil otto's comment, you can actually do that by embedding it in your shell script. There are many ways. If neither of the two below work, then you can certainly use shar (which has an extremely long history, and is likely compatible with everything).
Quoted here-document
The easiest way is if your shell supports quoted here-documents. They all should, as its a POSIX requirement:
#!/bin/sh
perl - -name 'foo' -mtime 2 -print <<'FIND2PERL'
#!/usr/bin/perl
eval 'exec /usr/bin/perl -S $0 ${1+"$#"}'
if $running_under_some_shell;
⋮
FIND2PERL
Hex dump in a non-quoted here-document
If some of your shells don't implement quoted here-documents (POSIX‽ what's that!), then you have to protect find2perl from shell expansion. An easy way is to hex dump it, as 0–9 and a–f are all safe from shell expansion. The dump is easily done with xxd -p /usr/bin/find2perl, which only requires xxd on one machine. To read back the dump, you can use plain perl:
#!/bin/sh
perl -n -e 'chomp; print pack("H*", $_)' <<HEX | perl - -name 'foo'
23212f7573722f62696e2f7065726c0a202020206576616c202765786563
202f7573722f62696e2f7065726c202d5320243020247b312b222440227d
⋮
HEX
Using find2perl several times
Naturally, with either approach, you could also write find2perl to a temporary file (if you need to invoke it multiple times, for example). You could also embed it in a shell function.
perl -lwe '$_ = $^X; s/perl$/find2perl/; -f or die qq($_ not -f); print'
Copy the interpreter executable path into dollar default argument. Patch the value, assuming that find2perl is in the same directory as perl itself. (This is specified as UNIX only, so you don't have to cater for perl.exe, which would be easy enough to deal with.) Then test the file exists, and die if it doesn't. (You might invent some better error handling.) Then print the path if we're still alive. That's it.
Okay, here's a version that works for Windows, too:
perl -lwe "$_ = $^X; s/perl(\.exe)?$/find2perl/;
-f or -f qq($_.bat) or die qq($_ not -f); print"
Note the double quotes, de rigueur on Windows for cmd.exe. And it has to go on one line, I just wrapped it for readability.
I have written a library in Perl that contains a certain function, that returns information about a server as a character string. Can I call this function from a shell directly?
My boss asks "Can you call it from a shell directly for the time being?" Because he said that, I think I should be able to do it, but how do I do it?
perl -MServerlib=server_information -e 'print server_information()'
Is another way to do this, but only if Serverlib exports server_information sub. If it doesn't, you would need to do the below instead:
perl -MServerlib -e 'print MServerlib::server_information()'
As perl's command line arguments are a bit inscrutable, I'd wrap it in a simpler perl script that calls the function. For example, create a script serverinfo which contains:
#!/usr/bin/perl
require 'library.pl';
say library::getServerInformation();
then run:
chmod u+x serverinfo
The advantage of doing it this way is the output and arguments of the script can be corrected if the function itself changes. A command line script like this can be thought of as an API, which shouldn't change when the implementation changes.