I'm basically trying to modify tomcat server.xml connector tag and add a address attribute to it.
I want to find the below string in server.xml
I'm doing the below with sed,
export currlistener=\<Connector\ port\=\"18443\"
export newlistener=\<Connector\ port\=\"18443\"\ address\=\"127.0.0.1\"\
echo $currlistener
echo $newlistener
sed -i -e 's/'$currlistener'/'$newlistener'/g' server.xml
But I get the error
sed: -e expression #1, char 12: unterminated `s' command
I guess sed is interpreting the special characters and erroring out.
How would I do the same using awk?
Regards,
Anand.
Using sed
The problem was that the shell variables were unquoted. Try:
sed -i -e "s/$currlistener/$newlistener/g" server.xml
Using awk
The sed solution requires that you trust the source of your shell variables. For a case like this, awk is safer. Using a modern GNU awk:
awk -i inplace -v a="$currlistener" -v b="$newlistener" '{gsub(a, b)} 1' server.xml
Or, using other awk:
awk -v a="$currlistener" -v b="$newlistener" '{gsub(a, b)} 1' server.xml >tmp && mv tmp server.sml
Simplifying the variable assignments
Separately, the shell variables can be defined without requiring so many escapes:
currlistener='<Connector port="18443"'
newlistener='<Connector port="18443" address="127.0.0.1"'
It is only necessary to export them if they are to be used in a child process.
Related
I want to replace a placeholder on a file <<string>> in the example to the contents of a var that has several special characters.
file.txt
My string: <<string>>
script.sh
STRING="something-else;device=name.of.device;key=abcd1234/wtp="
sed -i "s/<<string>>/${STRING}/g" file.txt
I get this error:
sed: -e expression #1, char 165: unknown option to `s'
I already use this sed command for other vars that do not have special characters. Any way to escape the var ${STRING} entirely?
You can't do this job easily and robustly with sed, see Is it possible to escape regex metacharacters reliably with sed. Instead just use a tool like awk that understands literal strings:
$ string='~`!##$%^&*()-_+={[}]|\:;"'\''<,>.?/\1'
$ echo "$string"
~`!##$%^&*()-_+={[}]|\:;"'<,>.?/\1
$ string="$string" awk -i inplace 'match($0,/(.*)(<<string>>)(.*)/,a){ $0=a[1] ENVIRON["string"] a[3] } 1' file.txt
$ cat file.txt
My string: ~`!##$%^&*()-_+={[}]|\:;"'<,>.?/\1
That above will work for any characters (or backreference substrings like \1) that string might contain because it's simply using a literal string operation (concatenation) for the replacement.
It's using GNU awk for -i inplace just the same as your original script used GNU sed for -i.
Don't use all upper case for non-exported variable names by the way to avoid clashes with exported and built-in variables and not obfuscate your code by making it look like you're using exported variables, see Correct Bash and shell script variable capitalization.
Note that if you have multiple <<whatever>> placeholders you can easily parameterize the above, e.g.:
$ foo='Now is the Winter'
$ bar='Of our discontent'
$ cat file.txt
My foo string: <<foo>>
My bar string: <<bar>>
$ foo="$foo" bar="$bar" awk -i inplace 'match($0,/(.*)<<(\w+)>>(.*)/,a) && (a[2] in ENVIRON){ $0=a[1] ENVIRON[a[2]] a[3] } 1' file.txt
$ cat file.txt
My foo string: Now is the Winter
My bar string: Of our discontent
If you don't want to set foo and bar on the awk command line you can export them before it, or read them from a config file or a here-doc or ... - lots of options.
Since STRING contains a /, you should use a other delimiter, for example, you can use ^ like so:
sed 's^<<string>>^'"$STRING"'^g' file.txt
The quoting logic (''""'') is explained nicely on this SO answer.
Example on my locale machine:
$
$ cat file.txt
My string: <<string>>
$
$
$ STRING="something-else;device=name.of.device;key=abcd1234/wtp="
$
$
$ sed -i 's^<<string>>^'"$STRING"'^g' file.txt
$
$ cat file.txt
My string: something-else;device=name.of.device;key=abcd1234/wtp=
$
$
I'm looking for an equivalent of perl -pe. Ideally, it would be replace with sed if it's possible. Any help is highly appreciated.
The code is:
perl -pe 's/^\[([^\]]+)\].*$/$1/g'
$ echo '[foo] 123' | perl -pe 's/^\[([^\]]+)\].*$/$1/g'
foo
$ echo '[foo] 123' | sed -E 's/^\[([^]]+)\].*$/\1/'
foo
sed by default accepts code from command line, so -e isn't needed (though it can be used)
printing the pattern space is default, so -p isn't needed and sed -n is similar to perl -n
-E is used here to be as close as possible to Perl regex. sed supports BRE and ERE (not as feature rich as Perl) and even that differs from implementation to implementation.
with BRE, the command for this example would be: sed 's/^\[\([^]]*\)\].*$/\1/'
\ isn't special inside character class unless it is an escape sequence like \t, \x27 etc
backreferences use \N format (and limited to maximum 9)
Also note that g flag isn't needed in either case, as you are using line anchors
sed inplace change on a file is not working inside Make object.
I want to replace a line in a file with sed called in a make object. But it does not seem to be working. How can I fix this?
change_generics:
ifeq ($(run_TESTNAME), diagnostics)
ifeq ($(run_TESTCASE), 1)
sed -i -e "s/SIM_MULTI\==[a-z,A-Z]*/SIM_MULTI=TRUE/" ./generics.f
else ifeq ($(TESTCASE), 2)
sed -i -e "s/SIM_MISSED\==[a-z,A-Z]*/SIM_MISSED=TRUE/" ./generics.f
endif
endif
I would like the generics.f file changed with that one line change. But it remains the same as the original. The sed command works outside make.
I can't reproduce this using GNU sed 4.2.2 and GNU make 3.82, or at least, I can't reproduce any scenario where the same sed command works from the command line but not in a Makefile.
Simpler Makefile:
all:
# Contrived just so I can test your 2 sed commands.
sed -i -e "s/SIM_MULTI\==[a-z,A-Z]*/SIM_MULTI=TRUE/" ./generics.f
sed -i -e "s/SIM_MISSED\==[a-z,A-Z]*/SIM_MISSED=TRUE/" ./generics.f
Sample file content in generics.f:
SIM_MULTI=foo
SIM_MISSED=bar
Testing:
$ make all
sed -i -e "s/SIM_MULTI\==[a-z,A-Z]*/SIM_MULTI=TRUE/" ./generics.f
sed -i -e "s/SIM_MISSED\==[a-z,A-Z]*/SIM_MISSED=TRUE/" ./generics.f
Confirmed that both sed commands fail to edit a file with this content.
To fix:
Probably, you need to simply remove the \= from your regular expression. The backslash there has no effect, and causes your regex to simply match two equals signs ==. Thus this works:
all:
sed -i 's/SIM_MULTI=[a-zA-Z]*/SIM_MULTI=TRUE/' ./generics.f
sed -i 's/SIM_MISSED=[a-zA-Z]*/SIM_MISSED=TRUE/' ./generics.f
Testing:
$ make all
sed -i 's/SIM_MULTI=[a-zA-Z]*/SIM_MULTI=TRUE/' ./generics.f
sed -i 's/SIM_MISSED=[a-zA-Z]*/SIM_MISSED=TRUE/' ./generics.f
$ cat generics.f
SIM_MULTI=TRUE
SIM_MISSED=TRUE
Further explanation:
There is no need to specify -e there.
There is no need to enclose the script in double quotes, which is riskier because it allows the contents to be modified by the shell.
The bug appears to be \= and I deleted those characters, as mentioned above.
Note that I removed the comma , as well in [a-z,A-Z]. I think that probably isn't what you meant, and it would cause a class of characters including a-z, A-Z and a comma , to be matched by the regex. (And if it is what you mean, you might consider writing it as [a-zA-Z,] as that would be less confusing.)
If this has not resolved your issue, I would need to know things like:
What is the version of your sed.
What is the contents in generics.f.
POSIX/GNU sed have c for "change":
sed -i '/SIM_MULTI=/c\SIM_MULTI=TRUE'
sed -i '/SIM_MISSED=/c\SIM_MISSED=TRUE'
I have a file: foo
The file has a line:
JVMDATA="$(${TIMEOUT} sudo /usr/lib/jvm/jdk1.8.0_92/bin/java -jar ${JVMINSPECTOR} ${PID} 2>&1)"
I would like to replace /usr/lib/jvm/jdk1.8.0_92/bin/java with:
JAVA_HOME/bin/java
How do I do this?
One way to do it could be this:
sed -e 's|/usr/lib/jvm/jdk1.8.0_92/bin/java|JAVA_HOME/bin/java|' original.txt > new.txt
cp original.txt original.txt.sav
mv new.txt original.txt
This uses the sed pattern substitution s/old/new/.
Normally it is a pain to do with slashes because you need to escape them.
On my system sed tolerates vertical-bar pattern delimiters so that can make it more readable.
I suspect you're going to want to change it to $JAVA_HOME... (note the leading dollar sign).
Just for completeness, here's a version using the more common slash pattern delimiter:
sed -e 's/\/usr\/lib\/jvm\/jdk1.8.0_92\/bin\/java/JAVA_HOME\/bin\/java/' original.txt > new.txt
I have the following sed command:
sed -i -E "s/\{\{(.*)\}\}/$(echo "$\1")/g" test.conf
In test.conf, I have this:
this is a {{TEST}}
and this is an {{ANSWER}} here.
And I have the follow environment variables set:
export TEST=1234
export ANSWER=5678
When I run the sed command, I end up with this result:
this is a $TEST
and this is an $ANSWER here.
I want 1234 and 5678 there respectively. Is there a reason the echo command is interpreting things literally?
Backreferences are used internally by a single sed command. The echo has no idea about sed backreferences and would have been invoked by the shell before the sed command has even run so the $(echo "$\1") is outputing $\1 so
sed -i -E "s/\{\{(.*)\}\}/$(echo "$\1")/g" test.conf
is really:
sed -i -E "s/\{\{(.*)\}\}/$\1/g" test.conf
hence the output you are seeing.
Anyway, sed is for simple subsitutions on individual lines, for anything else you should be using awk:
$ export TEST=1234 ANSWER=5678
$ awk 'match($0,/(.*)\{\{(.*)\}\}(.*)/,a){$0=a[1] ENVIRON[a[2]] a[3]} 1' file
this is a 1234
and this is an 5678 here.
The above uses GNU awk for the 3rd arg to match(), with other awks it'd be:
$ awk 'match($0,/\{\{(.*)\}\}/){$0=substr($0,1,RSTART-1) ENVIRON[substr($0,RSTART+2,RLENGTH-4)] substr($0,RSTART+RLENGTH)} 1' file
this is a 1234
and this is an 5678 here.
If anyone suggests running eval or similar on the sed output - don't do it (google eval is evil and friends), just use the awk command above for simple string operations.
You can use perl which conveniently has a %ENV hash variable, see perldoc for more info
perl -pe 's/\{\{(.*)\}\}/$ENV{$1}/' test.conf