I have a file that contains a lot of these
"/watch?v=VhsnHIUMQGM"
and I would like to output the letter code using a perl one-liner. So I try
perl -nle 'm/\"\/watch\?v=(.*?)\"/g' filename.txt
but it doesn't print anything.
What am I doing wrong?
The -n option processes each line but doesn't print anything out. So you need to add an explicit print if you successfully match.
perl -ne 'while ( m/\"\/watch\?v=(.+?)\"/g ) { print "$1\n" }' filename.txt
Another approach, if you're sure every line will match, is to use the -p option which prints out the value of $_ after processing, e.g.:
perl -pe 's/\"\/watch\?v=(.+?)\"/$1//' filename.txt
Your regex is fine. You're getting no output because the -n option won't print anything. It simply wraps a while (<>) { ... } loop around your program (run perl --help for brief explanations of the Perl options).
The following uses your regex, but add some printing. In list context, regexes with the /g option return all captures. Effectively, we print each capture.
perl -nle 'print for m/\"\/watch\?v=(.*?)\"/g' data.dat
You can split the string on "=" instead of matching:
perl -paF= -e '$_= #F[1]' filename.txt
Related
I have a text file (input.txt) like this:
NP_414685.4: 15-26, 131-138, 441-465
NP_418580.2: 493-500
NP_418780.2: 36-48, 44-66
NP_418345.2:
NP_418473.3: 1-19, 567-1093
NP_418398.2:
I want a perl one-liner that keeps only those lines in file where ":" is followed by number range (that means, here, the lines containing "NP_418345.2:" and "NP_418398.2:" get deleted). For this I have tried:
perl -ni -e "print unless /: \d/" -pi.bak input.txt del input.txt.bak
But it shows exactly same output as the input file.
What will be the exact pattern that I can match here?
Thanks
First, print unless means print if not -- opposite to what you want.
More to the point, it doesn't make sense using both -n and -p, and when you do -p overrides the other. While both of them open the input file(s) and set up the loop over lines, -p also prints $_ for every iteration. So with it you are reprinting every line. See perlrun.
Finally, you seem to be deleting the .bak file ... ? Then don't make it. Use just -i
Altogether
perl -i -ne 'print if /:\s*\d+\s*-\s*\d+/' input.txt
If you do want to keep the backup file use -i.bak instead of -i
You can see the code equivalent to a one-liner with particular options with B::Deparse (via O module)
Try: perl -MO=Deparse -ne 1 and perl -MO=Deparse -pe 1
This way:
perl -i.bak -ne 'print if /:\s+\d+-\d/' input.txt
This:
perl -ne 'print if /:\s*(\d+\s*-\s*\d+\s*,?\s*)+\s*$/' input.txt
Prints:
NP_414685.4: 15-26, 131-138, 441-465
NP_418580.2: 493-500
NP_418780.2: 36-48, 44-66
NP_418473.3: 1-19, 567-1093
I'm not sure if you want to match lines that are possibly like this:
NP_418580.2: 493-500, asdf
or this:
NP_418580.2: asdf
This answer will not print these lines, if given to it.
I am sure I have run this before but for the life of me cant find any reference in perlrun or through Google.
Hopefully some of the perl boffins here will be able to answer it.
When running a perl one liner with the -ne switch. Is there an option to have the code, that perl will compile, to be outputted to the console?
So if I run:
crontab -l | perl -ne 'print if /^00/'
Then Perl will compile this to:
while(<>){
print if /^00/;
}
I am sure there is a way to have perl spit out the code its going to use including any begin or end blocks. hopefully someone knows how.
You may be thinking of the B::Deparse function:
perl -MO=Deparse -ne 'print if /^00/'
Try
perl -MO=Deparse -ne 'print if /^00/'
It will give you the output like this:
LINE: while (defined($_ = <ARGV>)) {
print $_ if /^00/;
}
-e syntax OK
Is possible somewhat write a perl script in a bash script as heredoc?
This is not working (example only)
#/bin/bash
perl <<EOF
while(<>) {
chomp;
print "xxx: $_\n";
}
EOF
Is here some nice way how to embed a perl script into a bash script? Want run perl script from an bash script and don't want put it into external file.
The problem here is that the script is being passed to perl on stdin, so trying to process stdin from the script doesn't work.
1. String literal
perl -e '
while(<>) {
chomp;
print "xxx: $_\n";
}
'
Using a string literal is the most direct way to write this, though it's not ideal if the Perl script contains single quotes itself.
2. Use perl -e
#/bin/bash
script=$(cat <<'EOF'
while(<>) {
chomp;
print "xxx: $_\n";
}
EOF
)
perl -e "$script"
If you pass the script to perl using perl -e then you won't have the stdin problem and you can use any characters you like in the script. It's a bit roundabout to do this, though. Heredocs yield input on stdin and we need strings. What to do? Oh, I know! This calls for $(cat <<HEREDOC).
Make sure to use <<'EOF' rather than just <<EOF to keep bash from doing variable interpolation inside the heredoc.
You could also write this without the $script variable, although it's getting awfully hairy now!
perl -e "$(cat <<'EOF'
while(<>) {
chomp;
print "xxx: $_\n";
}
EOF
)"
3. Process substitution
perl <(cat <<'EOF'
while(<>) {
chomp;
print "xxx: $_\n";
}
EOF
)
Along the lines of #2, you can use a bash feature called process substitution which lets you write <(cmd) in place of a file name. If you use this you don't need the -e since you're now passing perl a file name rather than a string.
You know I never thought of this.
The answer is "YES!" it does work. As others have mentioned, <STDIN> can't be used, but this worked fine:
$ perl <<'EOF'
print "This is a test\n";
for $i ( (1..3) ) {
print "The count is $i\n";
}
print "End of my program\n";
EOF
This is a test
The count is 1
The count is 2
The count is 3
End of my program
In Kornshell and in BASH, if you surround your end of here document string with single quotes, the here document isn't interpolated by the shell.
Only small corection of #John Kugelman's answer. You can eliminate the useless cat and use:
read -r -d '' perlscript <<'EOF'
while(<>) {
chomp;
print "xxx: $_\n";
}
EOF
perl -e "$perlscript"
Here's another way to use a PERL HEREDOC script within bash, and take full advantage it.
#!/bin/sh
#If you are not passing bash var's and single quote the HEREDOC tag
perl -le "$(cat <<'MYPL'
# Best to build your out vars rather than writing directly
# to the pipe until the end.
my $STDERRdata="", $STDOUTdata="";
while ($i=<STDIN>){ chomp $i;
$STDOUTdata .= "To stdout\n";
$STDERRdata .= "Write from within the heredoc\n";
MYPL
print $STDOUTdata; #Doing the pipe write at the end
warn $STDERRdata; #will save you a lot of frustration.
)" [optional args] <myInputFile 1>prints.txt 2>warns.txt
or
#!/bin/sh
set WRITEWHAT="bash vars"
#If you want to include your bash var's
#Escape the $'s that are not bash vars, and double quote the HEREDOC tag
perl -le "$(cat <<"MYPL"
my $STDERRdata="", $STDOUTdata="";
while (\$i=<STDIN>){ chomp \$i;
\$STDOUTdata .= "To stdout\n";
\$STDERRdata .= "Write $WRITEWHAT from within the heredoc\n";
MYPL
print \$STDOUTdata; #Doing the pipe write at the end
warn \$STDERRdata; #will save you a lot of frustration.
)" [optional args] <myInputFile 1>prints.txt 2>warns.txt
This question already has answers here:
How can I process options using Perl in -n or -p mode?
(2 answers)
Closed last year.
I often need to run some Perl one-liners for fast data manipulations, like
some_command | perl -lne 'print if /abc/'
Reading from a pipe, I don't need a loop around the command arg filenames. How can I achieve the next?
some_command | perl -lne 'print if /$ARGV[0]/' abc
This gives the error:
Can't open abc: No such file or directory.
I understand that the '-n' does the
while(<>) {.... }
around my program, and the <> takes args as filenames, but doing the next every time is a bit impractical
#/bin/sh
while read line
do
some_command | perl -lne 'BEGIN{$val=shift #ARGV} print if /$val/' "$line"
done
Is there some better way to get "inside" the Perl ONE-LINER command line arguments without getting them interpreted as filenames?
Some solutions:
perl -e'while (<STDIN>) { print if /$ARGV[0]/ }' pat
perl -e'$p = shift; while (<>) { print if /$p/ }' pat
perl -e'$p = shift; print grep /$p/, <>' pat
perl -ne'BEGIN { $p = shift } print if /$p/' pat
perl -sne'print if /$p/' -- -p=pat
PAT=pat perl -ne'print if /$ENV{PAT}/'
Of course, it might make more sense to create a pattern that's an ORing or all patterns rather than executing the same command for each pattern.
Also reasonably short:
... | expr=abc perl -lne 'print if /$ENV{expr}/'
Works in bash shell but maybe not other shells.
It depends on what you think will be in the lines you read, but you could play with:
#/bin/sh
while read line
do
some_command | perl -lne "print if /$line/"
done
Clearly, if $line might contain slashes, this is not going to fly. Then, AFAIK, you're stuck with the BEGIN block formulation.
I'm trying to extract all ip addresses from a file. So far, I'm just using
cat foo.txt | perl -pe 's/.*?((\d{1,3}\.){3}\d{1,3}).*/\1/'
but this also prints lines that don't contain a match. I can fix this by piping through grep, but this seems like it ought to be unnecessary, and could lead to errors if the regexes don't match up perfectly.
Is there a simpler way to accomplish this?
Try this:
cat foo.txt | perl -ne 'print if s/.*?((\d{1,3}\.){3}\d{1,3}).*/\1/'
or:
<foo.txt perl -ne 'print if s/.*?((\d{1,3}\.){3}\d{1,3}).*/\1/'
It's the shortest alternative I can think of while still using Perl.
However this way might be more correct:
<foo.txt perl -ne 'if (/((\d{1,3}\.){3}\d{1,3})/) { print $1 . "\n" }'
If you've got grep, then just call grep directly:
grep -Po "(\d{1,3}\.){3}\d{1,3}" foo.txt
You've already got a suitable answer of using grep to extract the IP addresses, but just to explain why you were seeing non-matches being printed:
perldoc perlrun will tell you about all the options you can pass Perl on the command line.
Quoting from it:
-p causes Perl to assume the following loop around your program, which makes it
iterate over filename arguments somewhat like sed:
LINE:
while (<>) {
... # your program goes here
} continue {
print or die "-p destination: $!\n";
}
You could have used the -n switch instead, which does similar, but does not automatically print, for example:
cat foo.txt | perl -ne '/((?:\d{1,3}\.){3}\d{1,3})/ and print $1'
Also, there's no need to use cat; Perl will open and read the filenames you give it, so you could say e.g.:
perl -ne '/((?:\d{1,3}\.){3}\d{1,3})/ and print $1' foo.txt
ruby -0777 -ne 'puts $_.scan(/((?:\d{1,3}\.){3}\d{1,3})/)' file