sed comand is not replacing value after first match - sed

I am trying to add a line in a file using sed after first match. I looked my threads online, addressing the same issue and tried below versions of the command:
sudo sed -e '/cfg_file/cfg_file='$NEW_FILE -e '/cfg_file/cfg_file=/q' MAIN_CONF_FILE
sudo sed -i '0,/cfg_file/ a\cfg_file='$NEW_FILE $MAIN_CONF_FILE
sudo sed -i '1s/cfg_file/cfg_file='$NEW_FILE $MAIN_CONF_FILE
sudo sed -i '1/cfg_file/s/cfg_file='$NEW_FILE $MAIN_CONF_FILE
Unfortunately, nothing worked for me. Either they show error, in case of point 3, or show similar behavior of adding lines after each match.
SAMPLE FILE
cfg_file=some_line1
cfg_file=some_line2
Now I want to add a line after first match of cg_file, like below.
EXPECTED RESULT
cfg_file=some_line1
cfg_file=my_added_line_after_first_match_only.
cfg_file=some_line2
Help me in adding line after first match and correcting my command.

Since you're on Ubuntu, you are using GNU sed. GNU sed has some weird features and some useful ones; you should ignore the weird ones and use the useful ones.
In context, the useful one is ranges starting at line 0. Weird ones are the way it messes with a, i and c commands.
MAIN_CONF_FILE=/tmp/copy.of.main.config.file
NEWFILE="my_added_line_after_first_match_only."
sed -e '0,/^cfg_file=/ { /^cfg_file/ a\' \
-e "cfg_file=$NEWFILE" \
-e '}' \
"$MAIN_CONF_FILE"
In classic sed, the a command is followed by backslash-newline, and each subsequent line of the script is appended up to and including the first line without a backslash at the end (and the backslash is removed). Each -e argument functions as a line in the script. Distinguish between the shell lines escaped with backslash at the end and the sed script lines with backslash at the end.
Example at work
$ cat /tmp/copy.of.main.config.file | so
cfg_file=some_line1
cfg_file=some_line2
$ cat script.sh
MAIN_CONF_FILE=/tmp/copy.of.main.config.file
NEWFILE="my_added_line_after_first_match_only."
SED=/opt/gnu/bin/sed
${SED} \
-e '0,/^cfg_file=/ { /^cfg_file/ a\' \
-e "cfg_file=$NEWFILE" \
-e '}' \
"$MAIN_CONF_FILE"
$ bash script.sh
cfg_file=some_line1
cfg_file=my_added_line_after_first_match_only.
cfg_file=some_line2
$
This is based on your attempt 2, but avoids some of the weird stuff.
Basic sanity
As I noted, it is not sensible to experiment with sudo and the -i option to sed. You don't use those until you know that the script will do the job correctly. It is dangerous to do anything as root via sudo. It is doubly dangerous when you don't know whether what you're trying to use will work. Don't risk wrecking your system.

Related

SED inplace file change inside make - How?

sed inplace change on a file is not working inside Make object.
I want to replace a line in a file with sed called in a make object. But it does not seem to be working. How can I fix this?
change_generics:
ifeq ($(run_TESTNAME), diagnostics)
ifeq ($(run_TESTCASE), 1)
sed -i -e "s/SIM_MULTI\==[a-z,A-Z]*/SIM_MULTI=TRUE/" ./generics.f
else ifeq ($(TESTCASE), 2)
sed -i -e "s/SIM_MISSED\==[a-z,A-Z]*/SIM_MISSED=TRUE/" ./generics.f
endif
endif
I would like the generics.f file changed with that one line change. But it remains the same as the original. The sed command works outside make.
I can't reproduce this using GNU sed 4.2.2 and GNU make 3.82, or at least, I can't reproduce any scenario where the same sed command works from the command line but not in a Makefile.
Simpler Makefile:
all:
# Contrived just so I can test your 2 sed commands.
sed -i -e "s/SIM_MULTI\==[a-z,A-Z]*/SIM_MULTI=TRUE/" ./generics.f
sed -i -e "s/SIM_MISSED\==[a-z,A-Z]*/SIM_MISSED=TRUE/" ./generics.f
Sample file content in generics.f:
SIM_MULTI=foo
SIM_MISSED=bar
Testing:
$ make all
sed -i -e "s/SIM_MULTI\==[a-z,A-Z]*/SIM_MULTI=TRUE/" ./generics.f
sed -i -e "s/SIM_MISSED\==[a-z,A-Z]*/SIM_MISSED=TRUE/" ./generics.f
Confirmed that both sed commands fail to edit a file with this content.
To fix:
Probably, you need to simply remove the \= from your regular expression. The backslash there has no effect, and causes your regex to simply match two equals signs ==. Thus this works:
all:
sed -i 's/SIM_MULTI=[a-zA-Z]*/SIM_MULTI=TRUE/' ./generics.f
sed -i 's/SIM_MISSED=[a-zA-Z]*/SIM_MISSED=TRUE/' ./generics.f
Testing:
$ make all
sed -i 's/SIM_MULTI=[a-zA-Z]*/SIM_MULTI=TRUE/' ./generics.f
sed -i 's/SIM_MISSED=[a-zA-Z]*/SIM_MISSED=TRUE/' ./generics.f
$ cat generics.f
SIM_MULTI=TRUE
SIM_MISSED=TRUE
Further explanation:
There is no need to specify -e there.
There is no need to enclose the script in double quotes, which is riskier because it allows the contents to be modified by the shell.
The bug appears to be \= and I deleted those characters, as mentioned above.
Note that I removed the comma , as well in [a-z,A-Z]. I think that probably isn't what you meant, and it would cause a class of characters including a-z, A-Z and a comma , to be matched by the regex. (And if it is what you mean, you might consider writing it as [a-zA-Z,] as that would be less confusing.)
If this has not resolved your issue, I would need to know things like:
What is the version of your sed.
What is the contents in generics.f.
POSIX/GNU sed have c for "change":
sed -i '/SIM_MULTI=/c\SIM_MULTI=TRUE'
sed -i '/SIM_MISSED=/c\SIM_MISSED=TRUE'

Sed. Save lines that are ending with specific file-type

I have a textfile and would like to save all lines that are ending with .m2 or .M2. I tried several ways including this here
D:\filetype\core\sed.exe -n -e "s/^\(.*\)\.M2//" D:\filetype\listfile\test.txt > D:\filetype\listfile\test2.txt
But as result i only get a emtpy textfile, so i guess something is wrong with my code.
The other way was
D:\filetype\core\sed.exe -n -e "^/\(.*)\/.M2\/" D:\filetype\listfile\test.txt > D:\filetype\listfile\test2.txt
But in this case i wasn't able to locate the source of the error
unknown command: `^'
Thanks if someone can see my fault.
You can use below sed command:
sed -n -e '/\.[mM]2$/p' <file_name>
This will print all the lines which have .m2 or .M2 at the end
Now comming to issues with your commands. Your first command does:
sed -n -e "s/^\(.*\)\.M2//"
which is a search and replace command indiacated by s in the command. Syntax for this command is s/search_text/replace_text/. Now if you are look at your command carefully, you are searching for something and replacing it with nothing - as indicated by last // in your command - where replace_text is missing.
Your second command does
sed -n -e "^/\(.*)\/.M2\/"
which is incorrect syntax. General syntax of a sed command is :
sed -e 'line_range command'
where line range - which is optional - can be line numbers like1, 5 , 2-5, or a regular expression like /[gG]/, /[gG][iIuU]rl/.
If line_range is missing, the first letter in sed command should be a command. In your case: line_range is missing - which is fine syntax wise -, however the first letter is ^ - which is not a sed command - because of which you are getting syntax error.
The command that I suggested is
sed -n -e '/\.[mM]2$/p'
Here, line_range is the regular expression /\.[mM]2$/ - which says "any line which has .m2 or .M2 at the end", and command is p, which is the letter for print command.
Sed is mostly used to transform text. Why not use grep instead?
grep -i "\.m2$"
This will match case insensitively (-i) any line ending with .m2.

Strange sed command

I was trying to install docker in Ubuntu, and following the instructions I have come across a very strange sed command which I do not understand (this seems to be used to set-up bash autocompletion for docker):
sudo sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io
What is that command doing? The -i command means in-place editing, but what does the $acomplete -F _docker docker mean? The $ is matching the last line, but what is it doing? I do not even recognize any sed command there! For example, a substitution command would look like:
$s/in/out/
Could somebody explain that expression for me?
It appends complete -F _docker docker to the file.
From sed's manual:
a \
text Append text, which has each embedded newline preceded by a backslash.

Replace a word with another set of strings in a UNIX file

When I try to replace a string using sed command it works perfectly fine.
For eg :
When i used the below sed command:
sed 's/DB_ALTER/DB_REPRISE/g' /product/dwhrec1/abc.ksh > /product/dwhrec1/abc1.ksh
This command works perfectly fine and replace all the "DB_ALTER" with "DB_REPRISE" and writes the result to abc1.ksh script.
But when I place all such values in a file. for eg:
cat Repla.txt
DB_ALTER
DB_CMD
DB_GEST_COMM
for i in `cat Repla.txt`
do
sed 's/$i/DB_REPRISE/g' /product/dwhrec1/abc.ksh > /product/dwhrec1/abc1.ksh
done
But this does not work. In my file Repla.txt is just an example. In actual it has many values.
Can anyone please help me on this command or suggest some alternative.
Thanks
There are two problems with your script. The first is that the $i variable appears within single quotes. That means that bash will not substitute for the value of i. It needs to be in double-quotes.
Secondly, every time that you run sed, it overwrites the previous abc1.ksh file. You should copy abc.ksh to abc1.ksh and then modify in place abc1.ksh as many times as needed:
cp abc.ksh abc1.ksh
for i in `cat Repla.txt`; do
sed -i'' "s/$i/DB_REPRISE/g" abc1.ksh
done
The -i flag to sed causes it to modify the file in place.
Also, bash will apply word splitting to cat Repla.txt. This can surprise people who were expecting it to work line-by-line, not word-by-word.
Workaround in case your sed does not support -i
The sed on both linux (GNU) and Mac OSX (BSD) support -i. If your sed does not, try:
cmd=
for i in `cat Repla.txt`; do
[ "$cmd" ] && cmd="$cmd;"
cmd="$cmd s/$i/DB_REPRISE/g"
done
sed "$cmd" abc.ksh >abc1.ksh
The above puts all the substitution commands that you need in a single shell variable. This way, sed only needs to be run once and -i is not used.
Another option
If it is acceptable to overwrite the source file, then:
for i in $(cat Repla.txt)
do
sed 's/'$i'/DB_REPRISE/g' abc.ksh >abc1.ksh
mv -f abc1.ksh abc.ksh
done
The above puts in single quotes all of the sed command except for the part that we want the shell to expand. This is not needed in this example but could be useful if your replacement text had shell-active characters. The above also uses the more modern $(...) in place of backquotes for command substitution.
If $i were to contain spaces (it doesn't here), we would need to enclose it in double-quotes to protect it against shell word splitting as in:
for i in $(cat Repla.txt)
do
sed 's/'"$i"'/DB_REPRISE/g' abc.ksh >abc1.ksh
mv -f abc1.ksh abc.ksh
done

perl find and replace from command line. special characters?

So, I have looked around for an answer to this, and indeed I have found some, but none seem to work...
I have a folder full of bash scripts that I need to modify. specifically, I need to replace the line
INPUT=/data/scratch02/mpgussert/HAWC-30/${RUN}_reco
with
INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/${RUN}
I have tried this
perl -w -i -p -e "s'INPUT=/data/scratch02/mpgussert/HAWC-30/${RUN}_reco'INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/${RUN}'g" *.sh
which executes without error, but does not find and replace the desired text. From my understanding, using ' to deliminate the regex should search without special character replacement. Is that correct? If so, any ideas why it fails?
I have also tried
perl -w -i -p -e "s/INPUT=\/data\/scratch02\/mpgussert\/HAWC-30\/\$\{RUN\}_reco/INPUT=\/data\/hawc01\/hawcroot\/data\/hawc\/reconstructed\/quesadilla\/\$\{RUN\}/g" *.sh
the backslash should ignore special character replacement, but this returns the following error.
Backslash found where operator expected at -e line 1, near "RUN\"
syntax error at -e line 1, near "RUN\"
Execution of -e aborted due to compilation errors.
So it's searching for RUN\, which is not what I want... Any thoughts? I would appreciate any help you can give.
Thanks!
You want the pattern to be ...\$\{RUN\}..., but that's not what you're passing:
$ echo "...\$\{RUN\}..."
...$\{RUN\}...
You either need do more escaping, or switch to single quotes.
$ echo '...\$\{RUN\}...'
...\$\{RUN\}...
All together:
perl -i -wpe'
s{INPUT=/data/scratch02/mpgussert/HAWC-30/\$\{RUN\}_reco}
{INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/\${RUN}}g
' *.sh