perl one liner to replace part of file with other file - perl

I'm having problems with this one liner:
perl -pe 's/FINDME/`cat rep.txt`/ge' in.txt
If i use it exactly like this, it works, but I also want to add some text before and after the replaced content:
perl -pe 's/FINDME/SOMETHING1`cat rep.txt`SOMETHING2/ge' in.txt
I get the error:
syntax error at -e line 1, near "SOMETHING1`cat rep.txt`"
Shouldn't the output of the command be treated like a string?
Adicionally, I'm also confused by the fact I can't replace to something with the character <
perl -pe 's/SOMETHING/<SOMETHINGELSE/ge' in.txt
Unterminated <> operator at -e line 1.
Escaping the < (\<) gives me the same error.

The problem is that you are using the e modifier to the regexp, which means to eval{} (or in other words, execute) the replacement string as a code snippet, but you are treating it like a shell replacement. The e modifier expects CODE, not TEXT.
So, a normal (global) replace would use:
s/FINDME/REPLACE/g
... and this is fine. However, when you use an e flag, the replacement is run as code. Thus:
s/FINDME/`cat foo.txt`/ge;
... is equivalent to ...
$replace = `cat foo.txt`;
s/FINDME/$replace/g;
So, you can see how this:
s/FINDME/SOMETHING`cat foo.txt`/ge;
... is equivalent to...
$replace = SOMETHING`cat foo.txt`;
s/FINDME/$replace/g;
... and this is clearly a syntax error. Try this way instead:
s/FINDME/"SOMETHING".`cat foo.txt`/ge;
and you will find that it works, because this is valid code:
$replace = "SOMETHING".`cat foo.txt`;
( You can of course put even more complex things in there; since what is going on behind the scenes is an eval{}, your code is actually doing this:
eval { "SOMETHING".`cat foo.txt`; }
s/FINDME/$_/g;
however I'm simplifying for ease of comprehension :-)

Related

Perl: Bareword found where operator expected at [duplicate]

This question already has answers here:
Why is the escape of quotes lost in this regex substitution?
(2 answers)
Closed 4 years ago.
I am trying to use perl to replace a string like so:
perl -pe "s/FlDigVal/$DIGN/" Header.xml > $DDIRP/$FNAME.xml
If DIGN=+q4T/h/B8Saf0im3LtBevNvMPsd1PRG5Tz+Iq/uwjXA= i get the following syntax error:
Having no space between pattern and following word is deprecated at -e line 1.
Bareword found where operator expected at -e line 1, near "s/FlDigVal/+q4T/h"
Having no space between pattern and following word is deprecated at -e line 1.
syntax error at -e line 1, near "s/FlDigVal/+q4T/h"
Execution of -e aborted due to compilation errors.
I guess this is related to /hbeing in variable DIGN. Is there a way to escape those reserved words?
Don't use shell variables, which are just non-hygienic macros. Export the variable to Perl's environment:
DIGN=$DIGN perl -pe 's/FlDigVal/$ENV{DIGN}/'
Note the single quotes: we don't want the shell to change the Perl commands.
or pass the value as an argument:
perl -pe 'BEGIN { $replace = shift } s/FlDigVal/$replace/' "$DIGN" Header.xml
Nevertheless, you seem to be editing an XML document with regular expressions. It's a painful way, there are libraries like XML::LibXML that handle XML correctly. E.g. what would happen if DIGN contained & or <?
The problem is not with reserved words but /.
If DIGN contains +q4T/h/B8Saf0im3LtBevNvMPsd1PRG5Tz+Iq/uwjXA=, your command passes the following code to perl:
s/FlDigVal/+q4T/h/B8Saf0im3LtBevNvMPsd1PRG5Tz+Iq/uwjXA=/
Here s/FlDigVal/+q4T/ parses as a substitution command, but the rest is garbage.
The solution is to not let the shell interpolate variables into code. Instead you can pass strings via the environment:
DIGN="$DIGN" perl -pe 's/FlDigVal/$ENV{DIGN}/' Header.xml
(If DIGN is already exported, you don't need the DIGN="$DIGN" part.)
Here we use single quotes (no shell interpolation) and let perl grab a value from the environment.

How to modify the matched pattern

Just wondering if there is a handy way to modify matched pattern variable in Perl one liner. For instance in the string abcdef I'd like to replace def with e (output abce) using a command looking like this :
echo "abcdef" | perl -pne 's/(def)/{command that trims first and last character of $1 and returns it as a string for perl to use it as a replacement}/'
It would be easy to use such functionality to perform various formating tasks. Can we do this in sed ?
This is easy in Perl with the /e flag:
echo 'abcdef' | perl -pe 's/(def)/substr $1, 1, -1/e'
e tells perl to parse the replacement part as a block of code, not a string. You can put arbitrary code in there.
But your concrete task (trimming the first and last character) can also be done like this:
echo 'abcdef' | perl -pe 's/d(e)f/$1/'
(Also, perl -p already implies -n. No need to specify both.)

The perl -pe command

So I've done a research about the perl -pe command and I know that it takes records from a file and creates an output out of it in a form of another file. Now I'm a bit confused as to how this line of command works since it's a little modified so I can't really figure out what exactly is the role of perl pe in it. Here's the command:
cd /usr/kplushome/entities/Standalone/config/webaccess/WebaccessServer/etc
(PATH=/usr/ucb:$PATH; ./checkall.sh;) | perl -pe "s,^, ,g;"
Any idea how it works here?
What's even more confusing in the above statement is this part : "s,^, ,g;"
Any help would be much appreciated. Let me know if you guys need more info. Thank you!
It simply takes an expression given by the -e flag (in this case, s,^, ,g) and performs it on every line of the input, printing the modified line (i.e. the result of the expression) to the output.
The expression itself is something called a regular expression (or "regexp" or "regex") and is a field of learning in and of itself. Quick googles for "regular expression tutorial" and "getting started with regular expressions" turn up tons of results, so that might be a good place to start.
This expression, s,^, ,g, adds ten spaces to the start of the line, and as I said earlier, perl -p applies it to every line.
"s,^, ,g;"
s is use for substitution. syntax is s/somestring/replacement/.
In your command , is the delimiter instead of /.
g is for work globally, means replace all occurrence.
For example:
perl -p -i -e "s/oldstring/newstring/g" file.txt;
In file.txt all oldstring will replace with newstring.
i is for inplace file editing.
See these doc for information:
perlre
perlretut
perlop

Removing text with command line?

I have a huge list of locations in this form in a text file:
ar,casa de piedra,Casa de Piedra,20,,-49.985133,-68.914673
gr,riziani,Ríziani,18,,39.5286111,20.35
mx,tenextepec,Tenextepec,30,,19.466667,-97.266667
Is there any way with command line to remove everything that isn't between the first and second commas? For example, I want my list to look like this:
casa de piedra
riziani
tenextepec
with Perl
perl -F/,/ -ane 'print $F[1]."\n"' file
Use cut(1):
cut -d, -f2 inputfile
With perl:
perl -pe 's/^.*?,(.*?),.*/$1/' filename
Breakdown of the above code
perl - the command to use the perl programming language.
-pe - flags.
e means "run this as perl code".
p means:
Set $_ to the first line of the file (given by filename)
Run the -e code
Print $_
Repeat from step 1 with the next line of the file
what -p actually does behind the scenes is best explained here.
s/.*?,(.*?),.*/$1/ is a regular expression:
s/pattern/replacement/ looks for pattern in $_ and replaces it with replacement
.*? basically means "anything" (it's more complicated than that but outside the scope of this answer)
, is a comma (nothing special)
() capture whatever is in them and save it in $1
.* is another (slightly different) "anything" (this time it's more like "everything")
$1 is what we captured with ()
so the whole thing basically says to search in $_ for:
anything
a comma
anything (save this bit)
another comma
everything
and replace it with the bit it saved. This effectively saves the stuff between the first and second commas, deletes everything, and then puts what it saved into $_.
filename is the name of your text file
To review, the code goes through your file line by line, applies the regular expression to extract your needed bit, and then prints it out.
If you want the result in a file, use this:
perl -pe 's/^.*?,(.*?),.*/$1/' filename > out.txt
and the result goes into a file named out.txt (that will be placed wherever your terminal is pointed to at the moment.) What this pretty much does is tell the terminal to print the command's result to a file instead of on the screen.
Also, if it isn't crucial to use the command line, you can just import into Excel (it's in CSV format) and work with it graphically.
With awk:
$ awk -F ',' '{ print $2 }' file

sed in perl script

I am using following kind of script in my perl script and expecting entry at line 1. I am getting some error as below; any help?
plz ignore perl variable....
Error Messaage -
sed: -e expression #1, char 22: extra characters after command
# Empty file will not work for Sed line number 1
`"Security Concerns Report" > $outputFile`;
`sed '1i\
\
DATE :- $CDate \
Utility accounts with super access:- $LNumOfSupUserUtil \
Users not found in LDAP with super access: - $LNumOfSupUserNonLdap\
' $outputFile > $$`;
`mv $$ $outputFile`;
}
Your immediate problem is that the backslash character is interpreted by Perl inside the backtick operator, as is the dollar character. So your backslash-newline sequence turns into a newline in the shell command that is executed. If you replace these backslashes by \\, you'll go over this hurdle, but you'll still have a very brittle program.
Perl is calling a shell which calls sed. This requires an extra level of quoting for the shell which you are not performing. If your file names and data contain no special characters, you may be able to get away with this, until someone uses a date format containing a ' (among many things that would break your code).
Rather than fix this, it is a lot simpler to do everything in Perl. Everything sed and shells can do, Perl can do almost as easily or easier. It's not very clear from your question what you're trying to do. I'll focus on the sed call, but this may not be the best way to write your program.
If you really need to prepend some text to an existing file, there's a widely-used module on CPAN that already does this well. Use existing libraries in preference to reinventing the wheel. File::Slurp has a prepend_file method just for that. In the code below I use a here-document operator for the multiline string.
use File::Slurp; # at the top of the script with the other use directives
File::Slurp->prepend_file($outputFile, <<EOF);
DATE :- $CDate
Utility accounts with super access:- $LNumOfSupUserUtil
Users not found in LDAP with super access: - $LNumOfSupUserNonLdap
EOF