I already know that sed uses own approach to deal with single quote but I think it still possible to use it in my automation script.
I had to replace value of fingerprint in Saltstack config file.
Current value:
#master_finger: ''
Target value
master_finger: 'some:value'
My current command which doesn't work:
$ sed -i 's/#master_finger: ''/master_finger: 'some:value'/g' /etc/salt/minion
returns:
master_finger: some:value''
How can I solve this?
just use the double quotes to enclose the script.
$ echo "#master_finger: ''" | sed "s/#master_finger: ''/master_finger: 'some:value'/"
master_finger: 'some:value'
It's not sed that's making handling of 's difficult, it's the shell because the shell does not allow 's within any '-quoted string, including scripts.
You could save the sed script in a file and run it with -f or use a here document:
$ sed -f- file <<'EOF'
s/#master_finger: ''/master_finger: 'some:value'/g
EOF
master_finger: 'some:value'
To see the difference between the above and #karakfas suggestion:
$ sed -f- file <<'EOF'
s/#master_finger: ''/master_finger: '$(date)'/g
EOF
master_finger: '$(date)'
$ sed "s/#master_finger: ''/master_finger: '$(date)'/" file
master_finger: 'Sun Feb 14 06:50:43 CST 2021'
and imagine if date was replace by rm -rf * or something worse.
Also consider:
$ sed 's/#master_finger: '\'\''/master_finger: '\''$(date)'\''/' file
master_finger: '$(date)'
I've successfully used the following sed command to search/replace text in Linux:
sed -i 's/old_link/new_link/g' *
However, when I try it on my Mac OS X, I get:
"command c expects \ followed by text"
I thought my Mac runs a normal BASH shell. What's up?
EDIT:
According to #High Performance, this is due to Mac sed being of a different (BSD) flavor, so my question would therefore be how do I replicate this command in BSD sed?
EDIT:
Here is an actual example that causes this:
sed -i 's/hello/gbye/g' *
If you use the -i option you need to provide an extension for your backups.
If you have:
File1.txt
File2.cfg
The command (note the lack of space between -i and '' and the -e to make it work on new versions of Mac and on GNU):
sed -i'.original' -e 's/old_link/new_link/g' *
Create 2 backup files like:
File1.txt.original
File2.cfg.original
There is no portable way to avoid making backup files because it is impossible to find a mix of sed commands that works on all cases:
sed -i -e ... - does not work on OS X as it creates -e backups
sed -i'' -e ... - does not work on OS X 10.6 but works on 10.9+
sed -i '' -e ... - not working on GNU
Note Given that there isn't a sed command working on all platforms, you can try to use another command to achieve the same result.
E.g., perl -i -pe's/old_link/new_link/g' *
I believe on OS X when you use -i an extension for the backup files is required. Try:
sed -i .bak 's/hello/gbye/g' *
Using GNU sed the extension is optional.
This works with both GNU and BSD versions of sed:
sed -i'' -e 's/old_link/new_link/g' *
or with backup:
sed -i'.bak' -e 's/old_link/new_link/g' *
Note missing space after -i option! (Necessary for GNU sed)
Had the same problem in Mac and solved it with brew:
brew install gnu-sed
and use as
gsed SED_COMMAND
you can set as well set sed as alias to gsed (if you want):
alias sed=gsed
Or, you can install the GNU version of sed in your Mac, called gsed, and use it using the standard Linux syntax.
For that, install gsed using ports (if you don't have it, get it at http://www.macports.org/) by running sudo port install gsed. Then, you can run sed -i 's/old_link/new_link/g' *
Your Mac does indeed run a BASH shell, but this is more a question of which implementation of sed you are dealing with. On a Mac sed comes from BSD and is subtly different from the sed you might find on a typical Linux box. I suggest you man sed.
Insead of calling sed with sed, I do ./bin/sed
And this is the wrapper script in my ~/project/bin/sed
#!/bin/bash
if [[ "$OSTYPE" == "darwin"* ]]; then
exec "gsed" "$#"
else
exec "sed" "$#"
fi
Don't forget to chmod 755 the wrapper script.
Sinetris' answer is right, but I use this with find command to be more specific about what files I want to change. In general this should work (tested on osx /bin/bash):
find . -name "*.smth" -exec sed -i '' 's/text1/text2/g' {} \;
In general when using sed without find in complex projects is less efficient.
I've created a function to handle sed difference between MacOS (tested on MacOS 10.12) and other OS:
OS=`uname`
# $(replace_in_file pattern file)
function replace_in_file() {
if [ "$OS" = 'Darwin' ]; then
# for MacOS
sed -i '' -e "$1" "$2"
else
# for Linux and Windows
sed -i'' -e "$1" "$2"
fi
}
Usage:
$(replace_in_file 's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' "./mysql/.env")
Where:
, is a delimeter
's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' is pattern
"./mysql/.env" is path to file
As the other answers indicate, there is not a way to use sed portably across OS X and Linux without making backup files. So, I instead used this Ruby one-liner to do so:
ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml
In my case, I needed to call it from a rake task (i.e., inside a Ruby script), so I used this additional level of quoting:
sh %q{ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml}
Here's how to apply environment variables to template file (no backup need).
1. Create template with {{FOO}} for later replace.
echo "Hello {{FOO}}" > foo.conf.tmpl
2. Replace {{FOO}} with FOO variable and output to new foo.conf file
FOO="world" && sed -e "s/{{FOO}}/$FOO/g" foo.conf.tmpl > foo.conf
Working both macOS 10.12.4 and Ubuntu 14.04.5
Here is an option in bash scripts:
#!/bin/bash
GO_OS=${GO_OS:-"linux"}
function detect_os {
# Detect the OS name
case "$(uname -s)" in
Darwin)
host_os=darwin
;;
Linux)
host_os=linux
;;
*)
echo "Unsupported host OS. Must be Linux or Mac OS X." >&2
exit 1
;;
esac
GO_OS="${host_os}"
}
detect_os
if [ "${GO_OS}" == "darwin" ]; then
sed -i '' -e ...
else
sed -i -e ...
fi
sed -ie 's/old_link/new_link/g' *
Works on both BSD & Linux with gnu sed
I am trying to add a line in a file using sed after first match. I looked my threads online, addressing the same issue and tried below versions of the command:
sudo sed -e '/cfg_file/cfg_file='$NEW_FILE -e '/cfg_file/cfg_file=/q' MAIN_CONF_FILE
sudo sed -i '0,/cfg_file/ a\cfg_file='$NEW_FILE $MAIN_CONF_FILE
sudo sed -i '1s/cfg_file/cfg_file='$NEW_FILE $MAIN_CONF_FILE
sudo sed -i '1/cfg_file/s/cfg_file='$NEW_FILE $MAIN_CONF_FILE
Unfortunately, nothing worked for me. Either they show error, in case of point 3, or show similar behavior of adding lines after each match.
SAMPLE FILE
cfg_file=some_line1
cfg_file=some_line2
Now I want to add a line after first match of cg_file, like below.
EXPECTED RESULT
cfg_file=some_line1
cfg_file=my_added_line_after_first_match_only.
cfg_file=some_line2
Help me in adding line after first match and correcting my command.
Since you're on Ubuntu, you are using GNU sed. GNU sed has some weird features and some useful ones; you should ignore the weird ones and use the useful ones.
In context, the useful one is ranges starting at line 0. Weird ones are the way it messes with a, i and c commands.
MAIN_CONF_FILE=/tmp/copy.of.main.config.file
NEWFILE="my_added_line_after_first_match_only."
sed -e '0,/^cfg_file=/ { /^cfg_file/ a\' \
-e "cfg_file=$NEWFILE" \
-e '}' \
"$MAIN_CONF_FILE"
In classic sed, the a command is followed by backslash-newline, and each subsequent line of the script is appended up to and including the first line without a backslash at the end (and the backslash is removed). Each -e argument functions as a line in the script. Distinguish between the shell lines escaped with backslash at the end and the sed script lines with backslash at the end.
Example at work
$ cat /tmp/copy.of.main.config.file | so
cfg_file=some_line1
cfg_file=some_line2
$ cat script.sh
MAIN_CONF_FILE=/tmp/copy.of.main.config.file
NEWFILE="my_added_line_after_first_match_only."
SED=/opt/gnu/bin/sed
${SED} \
-e '0,/^cfg_file=/ { /^cfg_file/ a\' \
-e "cfg_file=$NEWFILE" \
-e '}' \
"$MAIN_CONF_FILE"
$ bash script.sh
cfg_file=some_line1
cfg_file=my_added_line_after_first_match_only.
cfg_file=some_line2
$
This is based on your attempt 2, but avoids some of the weird stuff.
Basic sanity
As I noted, it is not sensible to experiment with sudo and the -i option to sed. You don't use those until you know that the script will do the job correctly. It is dangerous to do anything as root via sudo. It is doubly dangerous when you don't know whether what you're trying to use will work. Don't risk wrecking your system.
I was trying to install docker in Ubuntu, and following the instructions I have come across a very strange sed command which I do not understand (this seems to be used to set-up bash autocompletion for docker):
sudo sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io
What is that command doing? The -i command means in-place editing, but what does the $acomplete -F _docker docker mean? The $ is matching the last line, but what is it doing? I do not even recognize any sed command there! For example, a substitution command would look like:
$s/in/out/
Could somebody explain that expression for me?
It appends complete -F _docker docker to the file.
From sed's manual:
a \
text Append text, which has each embedded newline preceded by a backslash.
I'm using Sed on a Mac. I am trying to do a simple string replace on a file that is not in the directory. I do:
sed -i 's/old/new/' /Users/A/file
and it says invalid command code A.
What do I need to do?
The -i option in OSX/BSD sed is a little different than the GNU/Linux version in that it requires a backup extension to be given, even if it's an empty string (which means that no backup will be made). The "invalid command code" error message occurs because s/old/new is taken as the backup extension and /Users/A/file is taken as the script (where A is seen as an invalid command name). So it needs to be something like:
sed -i '' 's/old/new/' /Users/A/file
if you have perl:
perl -p -i -e 's/old/new/' /Users/A/file
The -i in sed is not a standard across OS's (not a POSIX standard). This should work every time:
cp /Users/A/file /Users/A/file.sed
cat /Users/A/file.sed | sed 's/old/new' > /Users/A/file