This question already has answers here:
Why is the escape of quotes lost in this regex substitution?
(2 answers)
Closed 4 years ago.
I am trying to use perl to replace a string like so:
perl -pe "s/FlDigVal/$DIGN/" Header.xml > $DDIRP/$FNAME.xml
If DIGN=+q4T/h/B8Saf0im3LtBevNvMPsd1PRG5Tz+Iq/uwjXA= i get the following syntax error:
Having no space between pattern and following word is deprecated at -e line 1.
Bareword found where operator expected at -e line 1, near "s/FlDigVal/+q4T/h"
Having no space between pattern and following word is deprecated at -e line 1.
syntax error at -e line 1, near "s/FlDigVal/+q4T/h"
Execution of -e aborted due to compilation errors.
I guess this is related to /hbeing in variable DIGN. Is there a way to escape those reserved words?
Don't use shell variables, which are just non-hygienic macros. Export the variable to Perl's environment:
DIGN=$DIGN perl -pe 's/FlDigVal/$ENV{DIGN}/'
Note the single quotes: we don't want the shell to change the Perl commands.
or pass the value as an argument:
perl -pe 'BEGIN { $replace = shift } s/FlDigVal/$replace/' "$DIGN" Header.xml
Nevertheless, you seem to be editing an XML document with regular expressions. It's a painful way, there are libraries like XML::LibXML that handle XML correctly. E.g. what would happen if DIGN contained & or <?
The problem is not with reserved words but /.
If DIGN contains +q4T/h/B8Saf0im3LtBevNvMPsd1PRG5Tz+Iq/uwjXA=, your command passes the following code to perl:
s/FlDigVal/+q4T/h/B8Saf0im3LtBevNvMPsd1PRG5Tz+Iq/uwjXA=/
Here s/FlDigVal/+q4T/ parses as a substitution command, but the rest is garbage.
The solution is to not let the shell interpolate variables into code. Instead you can pass strings via the environment:
DIGN="$DIGN" perl -pe 's/FlDigVal/$ENV{DIGN}/' Header.xml
(If DIGN is already exported, you don't need the DIGN="$DIGN" part.)
Here we use single quotes (no shell interpolation) and let perl grab a value from the environment.
Related
I'm having problems with this one liner:
perl -pe 's/FINDME/`cat rep.txt`/ge' in.txt
If i use it exactly like this, it works, but I also want to add some text before and after the replaced content:
perl -pe 's/FINDME/SOMETHING1`cat rep.txt`SOMETHING2/ge' in.txt
I get the error:
syntax error at -e line 1, near "SOMETHING1`cat rep.txt`"
Shouldn't the output of the command be treated like a string?
Adicionally, I'm also confused by the fact I can't replace to something with the character <
perl -pe 's/SOMETHING/<SOMETHINGELSE/ge' in.txt
Unterminated <> operator at -e line 1.
Escaping the < (\<) gives me the same error.
The problem is that you are using the e modifier to the regexp, which means to eval{} (or in other words, execute) the replacement string as a code snippet, but you are treating it like a shell replacement. The e modifier expects CODE, not TEXT.
So, a normal (global) replace would use:
s/FINDME/REPLACE/g
... and this is fine. However, when you use an e flag, the replacement is run as code. Thus:
s/FINDME/`cat foo.txt`/ge;
... is equivalent to ...
$replace = `cat foo.txt`;
s/FINDME/$replace/g;
So, you can see how this:
s/FINDME/SOMETHING`cat foo.txt`/ge;
... is equivalent to...
$replace = SOMETHING`cat foo.txt`;
s/FINDME/$replace/g;
... and this is clearly a syntax error. Try this way instead:
s/FINDME/"SOMETHING".`cat foo.txt`/ge;
and you will find that it works, because this is valid code:
$replace = "SOMETHING".`cat foo.txt`;
( You can of course put even more complex things in there; since what is going on behind the scenes is an eval{}, your code is actually doing this:
eval { "SOMETHING".`cat foo.txt`; }
s/FINDME/$_/g;
however I'm simplifying for ease of comprehension :-)
I have such line from https://camlistore.googlesource.com/camlistore/+/master/third_party/rewrite-imports.sh
find . -type f -name '*.go' -exec perl -pi -e 's!"code.google.com/!"camlistore.org/third_party/code.google.com/!' {} \;
I would like help understanding what exactly this does:
perl -pi -e 's!"code.google.com/!"camlistore.org/third_party/code.google.com/!'
Especialy exclamation marks and ". Thanks!
From perldoc perlrun:
-p means "run the expression for each line, and print the result"
-i means "edit the input file in place"
-e means "the next parameter is the Perl expression to evaluate"
For the expression itself:
The ! marks are the separators for the s (substitution) operator. Any non-alphanumeric character can be used for that - whatever follows the s.
The " characters don't mean anything special, they're just part of the text to be replaced, and the replacement.
So we have:
s: substitute
!: (separator)
"code.google.com/: text to find
!: (separator)
"camlistore.org/third_party/code.google.com/: replacement text
!: (separator)
Which all means:
For each line in the file
Find the text "code.google.com/
And (if found) replace it with "camlistore.org/third_party/code.google.com/
The bangs ! are just an alternative delimiter for the search and replace regex s///.
Because the content of the search and replace includes forward slashes, it makes sense to use a different delimiter to avoid having to escape them all. Exclamation points are sometimes used for this purpose s!!!, but my preferred alternate are braces: s{}{}.
As for what that code is done, it's replacing all references to "code.google.com/ with "camlistore.org/third_party/code.google.com/ in the found files.
This is a pretty straightforward search-and-replace. The s/PATTERN/REPLACEMENT/ operator sees if a string matches the regular expression pattern and replaces the part that matches with the value of the replacement string.
Since sometimes / characters are an inconvenient delimiter (such as dealing with web URIs), Perl allows you to swap them out for other characters, in this case they chose to use !.
The -p switch causes Perl to assume a loop around the code in question for processing lines. The -i switch allows input lines to be edited in-place as they are processed, optionally preserving the original in another file. (See perldoc perlrun for the gory details.)
So all this code is doing is replacing lines that contain "code.google.com/ with "camlistore.org/third_party/code.google.com/.
Suppose I have a text file with content like below:
'Jack', is a boy
'Jenny', is a girl
...
...
...
I'd like to use perl in Cli to only capture the names between pairs of single quotes
cat text| perl -ne 'print $1."\n" if/\'(\w+?)\'/'
Above command was what I ran but it didn't work. It seems like "'" messed up with Shell.
I know we have other options like writing a perl script. But given my circumstances, I'd like to find a way to fulfill this in Shell command line.
Please advise.
The shell has the interesting property of concatenating quoted strings. Or rather, '...' or "..." should not be considered strings, but modifiers for available escapes. The '...'-surrounded parts of a command have no escapes available. Outside of '...', a single quote can be passed as \'. Together with the concatenating property, we can embed a single quote like
$ perl -E'say "'\''";'
'
into the -e code. The first ' exits the no-escape zone, \' is our single quote, and ' re-enters the escapeless zone. What perl saw was
perl // argv[0]
-Esay "'"; // argv[1]
This would make your command
cat text| perl -ne 'print $1."\n" if/'\''(\w+?)'\''/'
(quotes don't need escaping in regexes), or
cat text| perl -ne "print \$1.qq(\n) if/'(\w+?)'/"
(using double quotes to surround the command, but using qq// for double quoted strings and escaping the $ sigil to avoid shell variable interpolation).
Here are some methods that do not require manually escaping the perl statement:
(Disclaimer: I'm not sure how robust these are – they haven't been tested extensively)
Cat-in-the-bag technique
perl -ne "$(cat)" text
You will be prompted for input. To terminate cat, press Ctrl-D.
One shortcoming of this: The perl statement is not reusable. This is addressed by the variation:
$pline=$(cat)
perl -ne "$pline" text
The bash builtin, read
Multiple lines:
read -rd'^[' pline
Single line:
read -r pline
Reads user input into the variable pline.
The meaning of the switches:
-r: stop read from interpreting backslashes (e.g. by default read interprets \w as w)
-d: determines what character ends the read command.
^[ is the character corresponding to Esc, you insert ^[ by pressing Ctrl-V then Esc.
Heredoc and script.
(You said no scripts, but this is quick and dirty, so might as well...)
cat << 'EOF' > scriptonite
print $1 . "\n" if /'(\w+)'/
EOF
then you simply
perl -n scriptonite text
I am using following kind of script in my perl script and expecting entry at line 1. I am getting some error as below; any help?
plz ignore perl variable....
Error Messaage -
sed: -e expression #1, char 22: extra characters after command
# Empty file will not work for Sed line number 1
`"Security Concerns Report" > $outputFile`;
`sed '1i\
\
DATE :- $CDate \
Utility accounts with super access:- $LNumOfSupUserUtil \
Users not found in LDAP with super access: - $LNumOfSupUserNonLdap\
' $outputFile > $$`;
`mv $$ $outputFile`;
}
Your immediate problem is that the backslash character is interpreted by Perl inside the backtick operator, as is the dollar character. So your backslash-newline sequence turns into a newline in the shell command that is executed. If you replace these backslashes by \\, you'll go over this hurdle, but you'll still have a very brittle program.
Perl is calling a shell which calls sed. This requires an extra level of quoting for the shell which you are not performing. If your file names and data contain no special characters, you may be able to get away with this, until someone uses a date format containing a ' (among many things that would break your code).
Rather than fix this, it is a lot simpler to do everything in Perl. Everything sed and shells can do, Perl can do almost as easily or easier. It's not very clear from your question what you're trying to do. I'll focus on the sed call, but this may not be the best way to write your program.
If you really need to prepend some text to an existing file, there's a widely-used module on CPAN that already does this well. Use existing libraries in preference to reinventing the wheel. File::Slurp has a prepend_file method just for that. In the code below I use a here-document operator for the multiline string.
use File::Slurp; # at the top of the script with the other use directives
File::Slurp->prepend_file($outputFile, <<EOF);
DATE :- $CDate
Utility accounts with super access:- $LNumOfSupUserUtil
Users not found in LDAP with super access: - $LNumOfSupUserNonLdap
EOF
What I need is:
I need to delete the entire line but need to keep the matching strings.
matching pattern starting with Unhandled and ending with a :
I tried the below code which prints the matching pattern, but I need to delete the extra lines from the file.
perl -0777 -ne 'print "Unhandled error at$1\n" while /Unhandled\ error\ at(.*?):/gs' filename
Below is the sample input:
2012-04-09 01:52:13,717 - uhrerror - ERROR - 22866 - /home/shabbir/web/middleware.py process_exception - 217 - Unhandled error at /user/resetpassword/: : {'mod_wsgi.listener_port': '8080', 'HTTP_COOKIE': "__utma=1.627673239.1309689718.1333823126.1333916263.156; __utmz=1.1333636950.152.101.utmgclid=CMmkz934na8CFY4c6wod_R8JbA|utmccn=(not%20set)|utmcmd=(not%20set)|utmctr=non-stick%20kadai%20online; subpopdd=yes; _msuuid_1690zlm11992=FCC09820-3004-413A-97A3-1088EE128CE9; _we_wk_ls_=%7Btime%3A'1322900804422'%7D; _msuuid_lf2uu38ua0=08D1CEFE-3C19-4B9E-8096-240B92BA0ADD; nevermissadeal=True; sessionid=c1e850e2e7db09e98a02415fc1ef490; __utmc=1; __utmb=1.7.10.1333916263; 'wsgi.file_wrapper': , 'HTTP_ACCEPT_ENCODING': 'gzip, deflate'}
The code you gave already provides the requested behaviour.
That said, there's a huge redundant string in your program you can eliminate.
perl -0777nE'say $1 while /(Unhandled error at .*?):/gs' filename
Finally, slurping the entire file seems entirely superfluous.
perl -nE'say $1 if /(Unhandled error at .*?):/g' filename
perl -0777 -i -pe 's/.*?(Unhandled error .*?):.*/$1/g' filename
This will replace error block with matched string in the file.
-0777 : will force Perl to read the whole file in one shot.
-i : means in-place editing of files.
-p : means loop line-by-line through contents of file,execute code in single quotes i.e.'s/.*?(Unhandled error .*?):.*/$1/g',and print the result(matched string),which is written back to file using -i option.
-e : for command-line
If one match is all you want to keep from the whole string, you could replace the string value with the match afterwards. (i.e. Simply assign the new value)
If you have several matches within the string, the least complicated method may be to store the matches temporarily in an array. Then just discard the original variable if you don't need it anymore.
I would use -l option to handle line endings (less version dependent, prints a new line for each match), and a for loop to print all the matches, not just the first one $1. No need to slurp the file with -0777.
perl -nwle 'print for /Unhandled error at .*?:/g'
Note that with the /g modifier, a capturing parenthesis is not required.
If only one (the first) match is to be printed, /g is redundant and you can just use $1:
perl -nlwe 'print $1 if /(Unhandled error at .*?):/'