how to execute multiline shell command in perl? - perl

I want to run shell command as follow in perl:
tar --exclude="*/node_modules" \
--exclude="*/vendor" \
--exclude='.git' \
-zvcf /tmp/robot.tgz .
But it seems perl can not excute this:
`tar --exclude="cv/node_modules" \
--exclude="*/vendor" \
--exclude='.git' \
-zvcf /tmp/robot.tgz .`;
Here is the error:
tar: Must specify one of -c, -r, -t, -u, -x
sh: line 1: --exclude=*/vendor: No such file or directory
sh: line 2: --exclude=.git: command not found
sh: line 3: -zvcf: command not found
it seems perl treat each line as one command.

Update
I apologise. My original diagnosis was wrong
It is hard to clearly express in a post like this the contents of strings that contain Perl escape characters. If anything below is unclear to you then please write a comment to say so. I hope I haven't made things unnecessarily complicated
My original solution below is still valid, and will give you better control over the contents of the command, but my reasons for why the OP's code doesn't work for them were wrong, and the truth offers other resolutions
The problem is that the contents of backticks (or qx/.../) are evaluated as a double-quoted string, which means that Perl variables and escape sequences like \t and \x20 are expanded before the string is executed. One of the consequences of this is that a backslash is deleted if it is followed by a literal newline, leaving just the newline
That means that a statement like this
my $output = `ls \
-l`;
will be preprocessed to "ls \n-l" and will no longer contain the backslash that is needed to signal to the shell that the newline should be removed (or indeed to get the command passed to the shell in the first place)
Apart from manipulating the command string directly as I described in my original post below, there are two solutions to this. The first is to escape the backslash itself by doubling it up, like this
my $output = `ls \\
-l`;
which will prevent it from being removed by Perl. That will pass the backslash-newline sequence to the shell, which will remove it as normal
The other is to use qx'...' instead of backticks together with single-quote delimiters, which will prevent the contents from being processed as a double-quoted string
my $output = qx'ls \
-l';
This will work fine unless you have used Perl variables in the string that you want to be interpolated
Original post
The problem is that the shell removes newlines preceded by backslashes from the command string before executing it. Without that step the command is invalid
So you must do the same thing yourself in Perl, and to do that you must put the command in a temporary variable
use strict;
use warnings 'all';
my $cmd = <<'END';
tar --exclude="*/node_modules" \
--exclude="*/vendor" \
--exclude='.git' \
-zvcf /tmp/robot.tgz .
END
$cmd =~ s/\\\n//g;
my $output = `$cmd`;
There is no need for the backslashes of course; you can simply use newlines and remove those before executing the command
Or you may prefer to wrap the operations in a subroutine, like this
use strict;
use warnings 'all';
my $output = do_command(<<'END');
tar --exclude="*/node_modules" \
--exclude="*/vendor" \
--exclude='.git' \
-zvcf /tmp/robot.tgz .
END
sub do_command {
my ($cmd) = #_;
$cmd =~ s/\\\n//g;
`$cmd`;
}

Related

How to execute gpg command with Perl

I am trying to execute the following gpg command from within Perl:
`gpg --yes -e -r me#mydomain.com "$backupPath\\$backupname"`;
However I get the following error:
Global symbol "#mydomain" requires explicit package name (did you forget to declare "my #mydomain"?)
Obviously I need to escape the '#' symbol but don't know how. How do I execute this command in Perl?
When you do:
`gpg --yes -e -r me#mydomain.com "$backupPath\\$backupname"`;
perl sees the #mydomain part and assumes you want to interpolate the #mydomain array right into the string.
But since there was no #domain array declared, it gives you the error.
The fix is simple: To tell perl that you want to treat #mydomain as a string and not as an array, simply put a backslash (\) before the #, like this:
`gpg --yes -e -r me\#mydomain.com "$backupPath\\$backupname"`;
Backticks run process in sub shell (slower, consumes more resources) and have some issues which you should investigate.
Following code demonstrates other approach which does not spawn sub shell.
use strict;
use warnings;
use feature 'say';
my $backupPath = '/some/path/dir';
my $backupname = 'backup_name';
my $command = "gpg --yes -e -r me\#mydomain.com $backupPath/$backupname";
my #to_run = split ' ', $command;
system(#to_run);
Handling backticks in Perl
Problem with backticks in multi-threaded Perl script on Windows

Pass command line parameters to perl via file?

Could command lines parameters been saved to a file and then pass the file to perl to parse out the options? Like response file (prefix the name with #) for some Microsoft tools.
I am trying to pass expression to perl via command line, like perl -e 'print "\n"', and Windows command prompt makes using double quotes a little hard.
There are several solutions, from most to least preferable.
Write your program to a file
If your one liner is too big or complicated, write it to a file and run it. This avoids messing with shell escapes. You can reuse it and debug it and work in a real editor.
perl path\to\some_program
Command line options to perl can be put on the otherwise useless on Windows #! line. Here's an example.
#!/usr/bin/perl -i.bak -p
# -i.bak Backs up the file.
# -p Puts each line into $_ and writes out the new value of $_.
# So this changes all instances in a file of " with '.
s{"}{'}g;
Use alternative quote delimiters
Perl has a slew of alternative ways to write quotes. Use them instead. This is good for both one liners as well as things like q[<tag key='value'>].
perl -e "print qq[\n]"
Escape the quote
^ is the cmd.exe escape character. So ^" is treated as a literal quote.
perl -e "print ^"\n^""
Pretty yucky. I'd prefer using qq[] and reserve ^" for when you need to print a literal quote.
perl -e "print qq[^"\n]"
Use the ASCII code
The ASCII and UTF-8 hex code for " is 22. You can supply this to Perl with qq[\x22].
perl -e "print qq[\x22\n]"
You can read the file into a string and then use
use Getopt::Long qw(GetOptionsFromString);
$ret = GetOptionsFromString($string, ...);
to parse the options from that.

What is a perl one liner to replace a substring in a string?

Suppose you've got this C shell script:
setenv MYVAR "-l os="\""redhat4.*"\"" -p -100"
setenv MYVAR `perl -pe "<perl>"`
Replace with code that will either replace "-p -100" with "-p -200" in MYVAR or add it if it doesn't exist, using a one liner if possible.
The topic does not correspond to content, but I think it may be usefull if someone posts an answer to topic-question. So here is the perl-oneliner:
echo "my_string" | perl -pe 's/my/your/g'
What you want will look something like
perl -e' \
use Getopt::Long qw( :config posix_default ); \
use String::ShellQuote; \
GetOptions(\my %opts, "l=s", "p=s") or die; \
my #opts; \
push #opts, "-l", $opts{l} if defined($opts{l}); \
push #opts, "-p", "-100"; \
print(shell_quote(#opts)); \
' -- $MYVAR
First, you need to parse the command line. That requires knowing the format of the arguments of the application for which they are destined.
For example, -n is an option in the following:
perl -n -e ...
Yet -n isn't an option in the following:
perl -e -n ...
Above, I used Getopt::Long in POSIX mode. You may need to adjust the settings or use an entirely different parser.
Second, you need to produce csh literals.
I've had bad experiences trying to work around csh's defective quoting, so I'll leave those details to you. Above, I used String::ShellQuote's shell_quote which produces sh (rather than csh) literals. You'll need to adjust.
Of course, once you got this far, you need to get the result back into the environment variable unmangled. I don't know if that's possible in csh. Again, I leave the csh quoting to you.

running a shell command that is quoted

In my Perl program I get to a point where I have a variable that has the following:
echo -e \"use\nseveral\nlines\"
I would like to run this command through the shell (using exec) as
echo -e "use\nseveral\nlines"
I tried eval on the variable before I passed it to exec, but that interpreted the \n and changed them to newlines.
EDIT:
Note that I am given the variable and do not have control over how it is input. Thus, given that the variable was input as such, is there a way to "unquote" it?
In Perl, you should use q{} or qq{} (or qx{} for execution) to avoid complicated quote escaping.
This should work for you (using q{} to avoid interpolating \n):
my $str = q{echo -e "use\nseveral\nlines"};
Now, you can execute it using qx:
qx{$str}
When you pass
echo -e \"use\nseveral\nlines\"
to the shell, it passes the following three args to the exec systems call:
echo
-e
use\nseveral\nlines
How does one create that last string? Here are a few ways:
"use\\nseveral\\nlines" # Escape \W chars in double-quoted strings.
'use\\nseveral\\nlines' # Escape \ and delimiters in single-quoted strings
'use\nseveral\nlines' # ...although it's optional if unambiguous.
The corresponding Perl command would be therefore be
exec('echo', '-e', 'use\nseveral\nlines');
system('echo', '-e', 'use\nseveral\nlines');
open(my $fh, '-|', 'echo', '-e', 'use\nseveral\nlines');
my #output = <$fh>;

Perl one-liner to remove trailing space from a file

I am calling a perl one-liner inside a Perl script.
The intention of the one-liner is to remove the trailing space from a file.
Inside the main perl script:
`perl -pi -e 's/\s+$//' tape.txt`;
Now it is throwing me an error Substitution replacement not terminated at -e line 2.
Where is it going wrong?
It's because of the $/ (special variable) inside your main perl script. Note that variables are interpolated inside `` strings just like inside "" strings, and the fact that there are some single quotes in there doesn't change that. You need to escape that $:
`perl -pi -e 's/\s+\$//' tape.txt;`
The backtick syntax invokes a shell and when invoked, the shell assumes it should interpolate the string passed.
A cleaner syntax might be:
system('perl -pli -e "s/\s*$//" tape.txt');
Since you aren't capturing the output of the command, using backticks or qx in lieu of system isn't an issue.
Too, adding the -l switch autochomps each line read and then adds a newline back --- probably what you want.
\s matches [ \t\n\r\f] and do not want to match \n.
Notice use of {} for subst delimiters:
$ echo -e 'hi \nbye'| perl -pe 's{[\t\040]+$}{};' | cat -A
hi$
bye$