Why is it that simply changing from enclosing my one-liner with ' instead of " affects the behavior of the code? The first line of code produces what is expected and the second line of code gives (to me!) an unexpected result, printing out an unexpected array reference.
$ echo "puke|1|2|3|puke2" | perl -lne 'chomp;#a=split(/\|/,$_);print $a[4];'
puke2
$ echo "puke|1|2|3|puke2" | perl -lne "chomp;#a=split(/\|/,$_);print $a[4];"
This is the Perl version:
$ perl -v
This is perl, v5.10.1 (*) built for x86_64-linux-thread-multi
ARRAY(0x1f79b98)
With double quotes you are letting the shell interpolate variables first.
As you can check, $_ and $a are unset in the subshell forked for pipe by the parent shell. See a comment on $_ below.
So the double-quoted version is effectively
echo "puke|1|2|3|puke2" | perl -lne 'chomp;#a=split(/\|/);print [4];'
what prints the arrayref [4].
A comment on the effects of having $_ exposed to Bash. Thanks to Borodin for bringing this up.
The $_ is one of a handful of special shell parameters in Bash. It contains the last argument of the previous command, or the pathname of what invoked the shell or commands (via _ environment variable). See the link for a full description.
However, here it is being interpreted in a subshell forked to run the perl command, its first. Apparently it is not even set, as seen with
echo hi; echo hi | echo $_
which prints an empty line (after first hi). The reason may be that the _ environment variable just isn't set for a subshell for a pipe, but I don't see why this would be the case. For example,
echo hi; (echo $_)
prints two lines with hi even though ( ) starts a subshell.
In any case, $_ in the given pipeline isn't set.
The split part is then split(/\|/), so via default split(/\|/, $_) -- with nothing to split. With -w added this indeed prints a warning for use of uninitialized $_.
Note that this behavior depends on the shell. The tcsh won't run this with double quotes at all. In ksh and zsh the last part of pipeline runs in the main shell, not a subshell, so $_ is there.
This is actual a shell topic, not a perl topic.
In shell:
Single quotes preserve the literal value of all of the characters they contain, including the $ and backslash. However, with double quotes, the $, backtick, and backslash characters have special meaning.
For example:
'\"' evaluates to \"
whereas
"\'" evaluates to just '
because with double quotes, the backslash gets a special meaning as the escape character.
Related
How can I run this command in my Jenkins file?
sh "perl -p -e 's/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg; s/\$\{([^}]+)\}//eg' .env"
I tried everything.
Like so:
sh """
perl -p -e 's/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg; s/\$\{([^}]+)\}//eg' .env
"""
Or escaping the backslahes.
But I keep getting the error:
WorkflowScript: 13: unexpected char: '\' # line 13, column 23.
Depending on how this command is run, the string interpolation issues can be awful to predict. Is the double quoted string interpolated by sh? Does the backslash in front of $ mean that it is escaped from sh, but not from Perl interpolation? When I ran a test string in pastebin, it simply removed the $ENV{$1}.
I'm sure there's a way to do it the hard way (this way), but an easy way is to just write the Perl code in a file instead, and run the file.
I would write your regexes like this, in a separate file, say foo.pl:
s|\${([^}]+)}|$ENV{$1} // $&|eg;
s/\${([^}]+)}//g;
Using the logical defined-or operator // is slightly prettier than using the ternary operator. We change delimiter on the substitution operator to facilitate that.
I removed unused e modifier on second substitution.
You should note that all strings that match the regex ${....} will be removed from the input by the second substitution. So the fact that you attempt to put them back with the first substitution with $& is quite meaningless. Moreover using $& carries a notable performance reduction. Assuming that is a mistake from your side, the code can be shortened to:
s/\${([^}]+)}/$ENV{$1}/g;
Note that now you can also skip the dangerous eval modifier /e.
If you run it without warnings, which you do in your original code, you will not notice the undefined values in the %ENV hash, it will just return the empty string -- i.e. remove undefined values.
This code can now be run by your other script without interpolation issues:
sh "perl -p foo.pl .env"
Just remove the -e switch since you are no longer providing command line code.
I want to rename files with 'sr' in their names, replacing 'sr' with 'SR'. This one succeeded：
ls | perl -e 'while(<>){chomp;if(/(.*)sr(.*)/){rename $_,$1."SR".$2}}'
But this one failed:
ls | perl -e "while(<>){chomp;if(/sr/){rename $_,$\`.'SR'.($')}}"
with this error message:
Not enough arguments for rename at -e line 1, near "rename ,"`
Execution of -e aborted due to compilation errors.
It seems that $_ has become an empty string, but I don't quite understand why. Thanks for any explanations.
Now quotes have been an interesting problem and this is my test:
ls | perl -e "while(<>){chomp;if(/sr/){print $_;print\"\n\";print $\`,$&,($');print \"\n\";print $_,$\`,$&,($');print\"\n\";print $_;print\"\n\"}}"
outputs this:
3sr
3sr
3sr
3sr
sr1
sr1
sr1
sr1
sr2
sr2
sr2
sr2
it seems that when using alone, $_ is not empty; but it become empty when using along with $`,$& and $'. According to the last line of each file, I guess $_ has temporarily changed when not using alone?
Besides, according to a1111exe's answer, I test this:
ls | perl -e "while(<>){chomp;if(/sr/){print \$_,$\`,$&,($');print \"\n\"}}"
and got this:
3sr3sr
sr1sr1
sr2sr2
First in linux we should use single quote instead of double quote.
And instead of ls command you can use perl inbuilt function glob
And to capture the pre and post match you can use the $POSTMATCH and $PREMATCH from English module
so your one liner should be
perl -MEnglish -e 'while(<*>){chomp;if(/sr/){rename $_,$PREMATCH."SR".$POSTMATCH}}'
EDITED
Single quote and double quote is not about Perl this is about shell.
Single quote
Enclosing characters in single quotes (') preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
Double quote
Enclosing characters in double quotes (‘"’) preserves the literal value of all characters within the quotes, with the exception of ‘$’, ‘`’, ‘\’, and, when history expansion is enabled, ‘!’.
In shell script we are accessing the shell variable prefix with $, so while using $ inside the double quote it is looking for the shell variable not a Perl variable. For example you can run the following line in your terminal,
m=4; perl -e "print $m;"
Here
m=4; perl -e "print $m;"
^ ^
| Accessing shell variable
Assigning shell variable
Output is 4. Because m is shell variable you are accessing the shell variable inside your Perl script.
And in windows, we need to use double-quote instead of single quote
It seems that double quotes mess between your shell environment and Perl. You can certainly do what #mkHun suggested. One other way:
ls | perl -e 'while(<>){chomp;($new=$_)=~s/sr/SR/g;rename $_,$new}'
Also, if you escape the '$' sigil in '$_', your oneliner will work too:
ls | perl -e "while(<>){chomp;if(/sr/){rename \$_,$\`.'SR'.$'}}"
I still don't get why though.. But it really seems like bash/perl interpolation issue.
I'm having an issue with some code and I'm wondering if anyone can assist.
Basically I'm trying to execute an isql query against a database and assign it to a scalar variable. The isql command makes use of the column seperator which is defined as the pipe symbol.
So I have it set-up as such:
my $command = "isql -S -U -s| -i";
my $isql_output = `$command`;
The isql command works in isolation but when issued as a backtick it stops at the pipe. I've tried concatenating the $command string using sub-strings, using single quotes and backslash escaping items such as -s\"\|\" to no avail. I've also tried using qx instead of backticks.
Unfortunately I'm currently using an older version of perl (v5.6.1) with limited scope for upgrade so I'm not sure if I can resolve this.
You have to quote the | in a way that the shell does not recognize it as a special character. Two ways:
Put the -s| into single quotes: '-s|'. Perl will leave single quotes inside double quoted strings alone and pass them to the shell unmodified.
Escape the | with two backslashes: -s\\|. Why two? The first one is seen by Perl telling it to pass the next character through unmodified. Therefore the shell sees -s\| and does something very similar: it sees the single \ and knows not to treat the next char, |, special.
The problem is that the command is being executed through a shell.
You can avoid this by passing the command and arguments in a list rather than a single string.
The backtick construct does not support that, so you would need to use the open() function instead.
I haven't tested the following code but it gives the idea:
my #command = (qw(isql -Sserver -Uuser -Ppassword -s| -w4096), '–i' . $file);
print join(' ', #command), "\n";
open(my $fh, '-|', #command)
or die "failed to run isql command: $#\n";
my #isql_output = <$fh>;
close($fh);
my $isql_output = $isql_output[0]; chomp($isql_output);
If you're working with a 15 year old version of Perl (which Oracle users tend to do) I'm not sure this will all be supported. For instance, you may need to write chop instead of chomp.
UPDATE: the problem is not the Perl version, but this construct not being supported on Windows, according to the documentation. This must be qualified: I use Perl on Cygwin and it works fine there, but I don't know whether you can use Cygwin.
Single quotes should work. Try to run test perl script:
my $cmd = "./test.sh -S -U -s '|' -i";
print `$cmd`;
With test.sh:
#!/bin/sh
echo $#
Output should be -S -U -s | -i
I have some output data from some Bash Shell commands. The output is delimited line by line with "\n" or "\0". I would like to know that is there any way to pipe the output into Perl and process the data line by line within Perl (just like piping the output to awk, but in my case it is in the Perl context.). I suppose the command may be something like this :
Bash Shell command | perl -e 'some perl commands' | another Bash Shell command
Suppose I want to substitute all ":" character to "#" character in a "line by line" basis (not a global substitution, I may use a condition, e.g. odd or even line, to determine whether the current line should have the substitution or not.), then how could I achieve this.
See perlrun.
perl -lpe's/:/#/g' # assumes \n as input record separator
perl -0 -lpe's/:/#/g' # assumes \0 as input record separator
perl -lne'if (0 == $. % 2) { s/:/#/g; print; }' # modify and print even lines
Yes, Perl may appear at any place in a pipeline, just like awk.
The command line switch -p (if you want automatic printing) or -n (if you don't want it) will do what you want. The line contents are in $_ so:
perl -pe's/\./\#/g'
would be a solution. Generally, you want to read up on the '<>' (diamond) operator which is the way to go for non-oneliners.
I have a variable in a shell script,
var=1234_number
I want to replace all other than integer of $var .. how can I do it using a perl onliner?
You might be looking for something to edit the shell script, in which case, this might be sufficient:
perl -i.bak -e 's/\b(var=\d+).*/$1/' shellscript.sh
The '-i' overwrites the original file, saving a copy in shellscript.sh.bak; the substitute command finds assignments to 'var' (and not any longer name ending 'var') followed by an equals sign, some digits, and any non-digits, and leaves behind just the assignment of digits.
In the example, it gives:
var=1234
Note that the Perl regex is not foolproof - it will mangle this (dropping the closing brace).
: ${var=1234_number}
Dealing with all such possible variants is extremely fairly tricky:
echo $var=$other
OTOH, you might be looking to eliminate digits from a variable within a shell script, in which case:
var=$(echo $var | perl -e 's/\D//g')
You could also use 'sed' for the job:
var=$(echo $var | sed 's/[^0-9]//g')
No need to use anything but the shell for this
var=1234_abcd
var=${var%_*}
echo $var # => 1234
See 'Parameter Expansion' in the bash manual.