How can I use bash syntax in Perl's system() command?
I have a command that is bash-specific, e.g. the following, which uses bash's process substitution:
diff <(ls -l) <(ls -al)
I would like to call it from Perl, using
system("diff <(ls -l) <(ls -al)")
but it gives me an error because it's using sh instead of bash to execute the command:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `sort <(ls)'
Tell Perl to invoke bash directly. Use the list variant of system() to reduce the complexity of your quoting:
my #args = ( "bash", "-c", "diff <(ls -l) <(ls -al)" );
system(#args);
You may even define a subroutine if you plan on doing this often enough:
sub system_bash {
my #args = ( "bash", "-c", shift );
system(#args);
}
system_bash('echo $SHELL');
system_bash('diff <(ls -l) <(ls -al)');
system("bash -c 'diff <(ls -l) <(ls -al)'")
should do it, in theory. Bash's -c option allows you to pass a shell command to execute, according to the man page.
The problem with vladr's answers is that system won't capture the output to STDOUT from the command (which you would usually want), and it also doesn't allow executing more than one command (given the use of shift rather than accessing the full contents of #_).
Something like the following might be more suited to the problem:
my #cmd = ( 'diff <(ls -l) <(ls -al)', 'grep fu' );
my #stdout = exec_cmd( #cmd );
print join( "\n", #stdout );
sub exec_cmd
{
my $cmd_str = join( ' | ', #_ );
my #result = qx( bash -c '$cmd_str' );
die "Failed to exec $cmd_str: $!" unless( $? == 0 && #result );
return #result;
}
Unfortunately this won't prevent you from invoking /bin/sh just to run bash, however I don't see a workaround for this issue.
I prefer to execute bash commands in perl with backticks "`". This way I get a return value, e.g.:
my $value = \`ls`;
Also, I don't have to use "bash -c" just to run a commmand. Works for me.
Inspired by the answer from #errant.info, I created something simpler and worked for me:
my $line="blastn -subject <(echo -e \"$seq1\") -query <(echo -e \"$seq2\") -outfmt 6";
my $result=qx(bash -c '$line');
print "$result\n";
The introduced $line variable allows modifying inputs ($seq1 and $seq2) each time.
Hope it helps!
Related
I have a Perl script progA.pl which needs to run another Perl script progB.pl using the system command. However, progB.pl has been aliased in ~/.bashrc so I need to ensure that it is run after ~/.bashrc has been loaded. I can achieve this by using bash with the -lc option.
For this question, I simplify the problem as much as I think is needed, by considering the following version of progB.pl
use feature qw(say);
use strict;
use warnings;
use Data::Dump qw(dd dump);
say "Received \#ARGV: " . dump #ARGV;
and here is progA.pl:
use feature qw(say);
use strict;
use warnings;
use Data::Dump qw(dd dump);
my $cmd = qq(progB.pl --opt='This option contains '"'"'single'"'"' quotes');
say "cmd = " . dump($cmd);
system( "$cmd" );
say "-----";
system( 'bash -c ' . "$cmd" );
say "-----";
system( 'bash -c ' . "'$cmd'" );
say "-----";
system( "bash -c \"$cmd\"" );
Running
$ progA.pl
gives output:
cmd = "progB.pl --opt='This option contains '\"'\"'single'\"'\"' quotes'"
Received #ARGV: "--opt=This option contains 'single' quotes"
-----
Received #ARGV: ()
-----
Received #ARGV: "--opt=This"
-----
Received #ARGV: "--opt=This option contains single quotes"
We see that this works fine, when progB.pl is run directly without using bash -c. When I use bash -c to run the command, none of the three alternatives are working correctly.
How can I run progB.pl with an argument containing single quotes and at the same time using using bash -c ?
You should avoid this quoting madness at first place but if you insist, you should avoid at least one level of quoting by using system ARRAY version.
my $cmd = q{progB.pl --opt='This option contains '"'"'single'"'"' quotes'};
system( qw(bash -c), $cmd );
It makes it only one level of quoting madness.
my $option = q{This option contains 'single' quotes} =~ s/'/'"'"'/gr; # '
my $cmd = qq{progB.pl --opt='$option'};
system( qw(bash -c), $cmd );
There you can make some simple helper
sub sq ($) { "'" . $_[0] =~ s/'/'"'"'/gr . "'" } # "
my $option = q{This option contains 'single' quotes};
my $cmd = qq{progB.pl --opt=#{[sq $option]}};
system( qw(bash -c), $cmd );
After some trial and error, I arrived at:
use feature qw(say);
use strict;
use warnings;
my $cmd = qq(print_first_arg.pl --opt='This option contains '"'"'single'"'"' quotes');
$cmd =~ s/'/'"'"'/g;
system( 'bash -c ' . "'$cmd'" );
It seems to work, for this test case at least..
This also follows the approach suggested by #ysth in this answer:
https://stackoverflow.com/a/24869016/2173773
My syntax is
my $pstree = `pstree -p $pid|wc`;
but i am getting an error.
sh: -c: line 1: syntax error near unexpected token `|'
any thoughts?
Your variable $pid isn't just a number; it probably has a trailing newline character.
See it with:
use Data::Dumper;
print Data::Dumper->new([$pid])->Terse(1)->Useqq(1)->Dump;
It's valid perl, your shell is what is complaining. Did you put the #!/bin/perl at the top of the script? It's probably being interpreted by bash, not perl.
host:/var/tmp root# ./try.pl
5992 zsched
6875 /usr/local/sbin/sshd -f /usr/local/etc/sshd_config
3691 /usr/local/sbin/sshd -f /usr/local/etc/sshd_config -R
3711 -tcsh
6084 top 60
===
5 16 175
host:/var/tmp root# cat try.pl
#!/bin/perl
my $pstree = `ptree 3691`;
my $wc = `ptree 3691 | wc`;
print STDOUT $pstree;
print STDOUT "===\n";
print STDOUT $wc;
Instead of using the shell to do your counting, you can use Perl, which saves you a process and some complexity in your shell command:
my $count = () = qx(pstree -p $pid);
qx() does the same thing as backticks. The empty parentheses puts the qx() in list context, which makes it return a list, which then in scalar context is the size. It is a shortcut for:
my #list = qx(pstree -p $pid);
my $count = #list;
How I can make the following external command within ticks work with variables instead?
Or something similar?
sed -i.bak -e '10,16d;17d' $docname; (this works)
I.e., sed -i.bak -e '$line_number,$line_end_number;$last_line' $docname;
my $result =
qx/sed -i.bak -e "$line_number,${line_end_number}d;${last_line}d" $docname/;
Where the line split avoid the horizontal scroll-bar on SO; otherwise, it would be on one line.
Or, since it is not clear that there's any output to capture:
system "sed -i.back '$line_number,${line_end_number}d;${last_line}d' $docname";
Or you could split that up into arguments yourself:
system "sed", "-i.back", "$line_number,${line_end_number}d;${last_line}d", "$docname";
This tends to be safer since the shell doesn't get a chance to interfere with the interpretation of the arguments.
#args = ("command", "arg1", "arg2");
system(#args) == 0 or die "system #args failed: $?"
Furthermore on the manual:
perldoc -f system
I think you should read up on using qq for strings.
You probably want something like this:
use strict;
use warnings;
my $line_number = qq|10|;
my $line_end_number = qq|16d|;
my $last_line = qq|17d|;
my $doc_name = qq|somefile.bak|;
my $sed_command = qq|sed -i.bak -e '$line_number,$line_end_number;$last_line' $doc_name;|;
print $sed_command;
qx|$sed_command|;
Is there a neater way of climbing up multiple directory levels from the location of a script.
This is what I currently have.
# get the full path of the script
D=$(cd ${0%/*} && echo $PWD/${0##*/})
D=$(dirname $D)
D=$(dirname $D)
D=$(dirname $D)
# second level parent directory of script
echo $D
I would like a neat way of finding the nth level. Any ideas other than putting in a for loop?
dir="/path/to/somewhere/interesting"
saveIFS=$IFS
IFS='/'
parts=($dir)
IFS=$saveIFS
level=${parts[3]}
echo "$level" # output: somewhere
#!/bin/sh
ancestor() {
local n=${1:-1}
(for ((; n != 0; n--)); do cd $(dirname ${PWD}); done; pwd)
}
Usage:
$ pwd
/home/nix/a/b/c/d/e/f/g
$ ancestor 3
/home/nix/a/b/c/d
A solution without loops would be to use recursion. I wanted to find a config file for a script by traversing backwards up from my current working directory.
rtrav() { test -e $2/$1 && echo $2 || { test $2 != / && rtrav $1 `dirname $2`;}; }
To check if the current directory is in a GIT repo: rtrav .git $PWD
rtrav will check the existence of a filename given by the first argument in each parent folder of the one given as the second argument. Printing the directory path where the file was found or exiting with an error code if the file was not found.
The predicate (test -e $2/$1) could be swapped for checking of a counter that indicates the traversal depth.
If you're OK with including a Perl command:
$ pwd
/u1/myuser/dir3/dir4/dir5/dir6/dir7
The first command lists the directory containing first N (in my case 5) directories
$ perl-e 'use File::Spec; \
my #dirs = File::Spec->splitdir( \
File::Spec->rel2abs( File::Spec->curdir() ) ); \
my #dirs2=#dirs[0..5]; print File::Spec->catdir(#dirs2) . "\n";'
/u1/myuser/dir3/dir4/dir5
The second command lists the directory N levels up (in my case 5) directories (I think you wanted the latter).
$ perl -e 'use File::Spec; \
my #dirs = File::Spec->splitdir( \
File::Spec->rel2abs( File::Spec->curdir() ) ); \
my #dirs2=#dirs[0..$#dir-5]; print File::Spec->catdir(#dirs2)."\n";'
/u1/myuser
To use it in your bash script, of course:
D=$(perl -e 'use File::Spec; \
my #dirs = File::Spec->splitdir( \
File::Spec->rel2abs( File::Spec->curdir() ) ); \
my #dirs2=#dirs[0..$#dir-5]; print File::Spec->catdir(#dirs2)."\n";')
Any ideas other than putting in a for loop?
In shells, you can't avoid the loop, because traditionally they do not support regexp, but glob matching instead. And glob patterns do not support the any sort of repeat counters.
And BTW, simplest way is to do it in shell is: echo $(cd $PWD/../.. && echo $PWD) where the /../.. makes it strip two levels.
With tiny bit of Perl that would be:
perl -e '$ENV{PWD} =~ m#^(.*)(/[^/]+){2}$# && print $1,"\n"'
The {2} in the Perl's regular expression is the number of directory entries to strip. Or making it configurable:
N=2
perl -e '$ENV{PWD} =~ m#^(.*)(/[^/]+){'$N'}$# && print $1,"\n"'
One can also use Perl's split(), join() and splice() for the purpose, e.g.:
perl -e '#a=split("/", $ENV{PWD}); print join("/", splice(#a, 0, -2)),"\n"'
where -2 says that from the path the last two entries has to be removed.
Two levels above the script directory:
echo "$(readlink -f -- "$(dirname -- "$0")/../..")"
All the quoting and -- are to avoid problems with tricky paths.
This method uses the actual full path to the perl script itself ... TIMTOWTDI
You could just easily replace the $RunDir with the path you would like to start with ...
#resolve the run dir where this scripts is placed
$0 =~ m/^(.*)(\\|\/)(.*)\.([a-z]*)/;
$RunDir = $1 ;
#change the \'s to /'s if we are on Windows
$RunDir =~s/\\/\//gi ;
my #DirParts = split ('/' , $RunDir) ;
for (my $count=0; $count < 4; $count++) { pop #DirParts ; }
$confHolder->{'ProductBaseDir'} = $ProductBaseDir ;
This allows you to work your way up until whatever condition is desired
WORKDIR=$PWD
until test -d "$WORKDIR/infra/codedeploy"; do
# get the full path of the script
WORKDIR=$(dirname $WORKDIR)
done
I noticed that when I use backticks in perl the commands are executed using sh, not bash, giving me some problems.
How can I change that behavior so perl will use bash?
PS. The command that I'm trying to run is:
paste filename <(cut -d \" \" -f 2 filename2 | grep -v mean) >> filename3
The "system shell" is not generally mutable. See perldoc -f exec:
If there is more than one argument in LIST, or if LIST is an array with more than one value, calls execvp(3) with the arguments in LIST. If
there is only one scalar argument or an array with one element in it, the argument is checked for shell metacharacters, and if there are any, the
entire argument is passed to the system's command shell for parsing (this is "/bin/sh -c" on Unix platforms, but varies on other platforms).
If you really need bash to perform a particular task, consider calling it explicitly:
my $result = `/usr/bin/bash command arguments`;
or even:
open my $bash_handle, '| /usr/bin/bash' or die "Cannot open bash: $!";
print $bash_handle 'command arguments';
You could also put your bash commands into a .sh file and invoke that directly:
my $result = `/usr/bin/bash script.pl`;
Try
`bash -c \"your command with args\"`
I am fairly sure the argument of -c is interpreted the way bash interprets its command line. The trick is to protect it from sh - that's what quotes are for.
This example works for me:
$ perl -e 'print `/bin/bash -c "echo <(pwd)"`'
/dev/fd/63
To deal with running bash and nested quotes, this article provides the best solution: How can I use bash syntax in Perl's system()?
my #args = ( "bash", "-c", "diff <(ls -l) <(ls -al)" );
system(#args);
I thought perl would honor the $SHELL variable, but then it occurred to me that its behavior might actually depend on your system's exec implementation. In mine, it seems that exec
will execute the shell
(/bin/sh) with the path of the
file as its first argument.
You can always do qw/bash your-command/, no?
Create a perl subroutine:
sub bash { return `cat << 'EOF' | /bin/bash\n$_[0]\nEOF\n`; }
And use it like below:
my $bash_cmd = 'paste filename <(cut -d " " -f 2 filename2 | grep -v mean) >> filename3';
print &bash($bash_cmd);
Or use perl here-doc for multi-line commands:
$bash_cmd = <<'EOF';
for (( i = 0; i < 10; i++ )); do
echo "${i}"
done
EOF
print &bash($bash_cmd);
I like to make some function btck (which integrates error checking) and bash_btck (which uses bash):
use Carp;
sub btck ($)
{
# Like backticks but the error check and chomp() are integrated
my $cmd = shift;
my $result = `$cmd`;
$? == 0 or confess "backtick command '$cmd' returned non-zero";
chomp($result);
return $result;
}
sub bash_btck ($)
{
# Like backticks but use bash and the error check and chomp() are
# integrated
my $cmd = shift;
my $sqpc = $cmd; # Single-Quote-Protected Command
$sqpc =~ s/'/'"'"'/g;
my $bc = "bash -c '$sqpc'";
return btck($bc);
}
One of the reasons I like to use bash is for safe pipe behavior:
sub safe_btck ($)
{
return bash_btck('set -o pipefail && '.shift);
}