I've successfully used the following sed command to search/replace text in Linux:
sed -i 's/old_link/new_link/g' *
However, when I try it on my Mac OS X, I get:
"command c expects \ followed by text"
I thought my Mac runs a normal BASH shell. What's up?
EDIT:
According to #High Performance, this is due to Mac sed being of a different (BSD) flavor, so my question would therefore be how do I replicate this command in BSD sed?
EDIT:
Here is an actual example that causes this:
sed -i 's/hello/gbye/g' *
If you use the -i option you need to provide an extension for your backups.
If you have:
File1.txt
File2.cfg
The command (note the lack of space between -i and '' and the -e to make it work on new versions of Mac and on GNU):
sed -i'.original' -e 's/old_link/new_link/g' *
Create 2 backup files like:
File1.txt.original
File2.cfg.original
There is no portable way to avoid making backup files because it is impossible to find a mix of sed commands that works on all cases:
sed -i -e ... - does not work on OS X as it creates -e backups
sed -i'' -e ... - does not work on OS X 10.6 but works on 10.9+
sed -i '' -e ... - not working on GNU
Note Given that there isn't a sed command working on all platforms, you can try to use another command to achieve the same result.
E.g., perl -i -pe's/old_link/new_link/g' *
I believe on OS X when you use -i an extension for the backup files is required. Try:
sed -i .bak 's/hello/gbye/g' *
Using GNU sed the extension is optional.
This works with both GNU and BSD versions of sed:
sed -i'' -e 's/old_link/new_link/g' *
or with backup:
sed -i'.bak' -e 's/old_link/new_link/g' *
Note missing space after -i option! (Necessary for GNU sed)
Had the same problem in Mac and solved it with brew:
brew install gnu-sed
and use as
gsed SED_COMMAND
you can set as well set sed as alias to gsed (if you want):
alias sed=gsed
Or, you can install the GNU version of sed in your Mac, called gsed, and use it using the standard Linux syntax.
For that, install gsed using ports (if you don't have it, get it at http://www.macports.org/) by running sudo port install gsed. Then, you can run sed -i 's/old_link/new_link/g' *
Your Mac does indeed run a BASH shell, but this is more a question of which implementation of sed you are dealing with. On a Mac sed comes from BSD and is subtly different from the sed you might find on a typical Linux box. I suggest you man sed.
Insead of calling sed with sed, I do ./bin/sed
And this is the wrapper script in my ~/project/bin/sed
#!/bin/bash
if [[ "$OSTYPE" == "darwin"* ]]; then
exec "gsed" "$#"
else
exec "sed" "$#"
fi
Don't forget to chmod 755 the wrapper script.
Sinetris' answer is right, but I use this with find command to be more specific about what files I want to change. In general this should work (tested on osx /bin/bash):
find . -name "*.smth" -exec sed -i '' 's/text1/text2/g' {} \;
In general when using sed without find in complex projects is less efficient.
I've created a function to handle sed difference between MacOS (tested on MacOS 10.12) and other OS:
OS=`uname`
# $(replace_in_file pattern file)
function replace_in_file() {
if [ "$OS" = 'Darwin' ]; then
# for MacOS
sed -i '' -e "$1" "$2"
else
# for Linux and Windows
sed -i'' -e "$1" "$2"
fi
}
Usage:
$(replace_in_file 's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' "./mysql/.env")
Where:
, is a delimeter
's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' is pattern
"./mysql/.env" is path to file
As the other answers indicate, there is not a way to use sed portably across OS X and Linux without making backup files. So, I instead used this Ruby one-liner to do so:
ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml
In my case, I needed to call it from a rake task (i.e., inside a Ruby script), so I used this additional level of quoting:
sh %q{ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml}
Here's how to apply environment variables to template file (no backup need).
1. Create template with {{FOO}} for later replace.
echo "Hello {{FOO}}" > foo.conf.tmpl
2. Replace {{FOO}} with FOO variable and output to new foo.conf file
FOO="world" && sed -e "s/{{FOO}}/$FOO/g" foo.conf.tmpl > foo.conf
Working both macOS 10.12.4 and Ubuntu 14.04.5
Here is an option in bash scripts:
#!/bin/bash
GO_OS=${GO_OS:-"linux"}
function detect_os {
# Detect the OS name
case "$(uname -s)" in
Darwin)
host_os=darwin
;;
Linux)
host_os=linux
;;
*)
echo "Unsupported host OS. Must be Linux or Mac OS X." >&2
exit 1
;;
esac
GO_OS="${host_os}"
}
detect_os
if [ "${GO_OS}" == "darwin" ]; then
sed -i '' -e ...
else
sed -i -e ...
fi
sed -ie 's/old_link/new_link/g' *
Works on both BSD & Linux with gnu sed
Related
I am reviewing the source code for gitflow-avh (A VirtualHome edition), version 1.12.3, which ships with Git for Windows, version 2.31.1. I'm looking at lines 67-73 of the script git-flow.
*MINGW*)
export GITFLOW_DIR=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
pwd () {
builtin pwd -W
}
;;
*)
# The sed expression here replaces all backslashes by forward slashes.
# This helps our Windows users, while not bothering our Unix users.)
export GITFLOW_DIR=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
;;
esac
What is the -W option? I'm only aware of -L and -P in bash. I'm fairly confident this is a "MINGW-ism", but I'm having trouble finding documentation online.
Does anyone know what the -W option does?
Ok, so I found this specifically has to do with MSYS2, as opposed to MINGW.
Per MSYS2's website, "How does MSYS2 differ from Cygwin?":
I had to run MSYS2 itself to actually see what the -W option provided:
In my "root" directory in Git Bash on Windows, pwd just gives me /.
But with -W, I get the real location:
$ pwd -W
C:/Program Files/Git
There's nothing on the man page for pwd, of course. Based on #A. Hendry's answer, I think I'll just alias pwd so that it always uses the -W option.
I'm trying to use sed, but I want to run properly both under Linux and Mac. Currently, I have something like this:
if test -f ${GENESISFILE};
then
echo "Replacing ..."
sed -i '' "s/ADDRESS/${ADDRESS}/g" ${GENESISFILE}
else
echo "No such file"
fi
Now, the point is that using -i '' part it runs properly under Mac, but doesn't under Linux, and if I remove it then it doesn't work under Mac. What's proper way to make it cross-platform compatible?
Instead of sed one-liner:
sed -i '' "s/ADDRESS/${ADDRESS}/g" ${GENESISFILE}
use this cross-platform Perl one-liner, which runs OK on both Linux and macOS:
perl -i.bak -pe 's/ADDRESS/$ENV{ADDRESS}/g' ${GENESISFILE}
The Perl one-liner uses these command line flags:
-e : Tells Perl to look for code in-line, instead of in a file.
-p : Loop over the input one line at a time, assigning it to $_ by default. Add print $_ after each loop iteration.
-i.bak : Edit input files in-place (overwrite the input file). Before overwriting, save a backup copy of the original file by appending to its name the extension .bak. Use -i alone, without .bak, to skip making the backup.
SEE ALSO:
perldoc perlrun: how to execute the Perl interpreter: command line switches
In a previous project I used the sed utility to replace newlines with NUL characters for compatibility with the API for adding a release note on the CI server.
sed -zE 's/\r\n|\n/\\n/g' < CHANGELOG.md
However, in a different project I am using an alpine based docker image (mcr.microsoft.com/dotnet/core/sdk:3.1-alpine) and the -z or --null-data image is unrecognised on alpine linux when using the sed utility.
How do I achieve the equivalent of the -z or --null-data with Alpine Linux sed utility?
What Is The Equivalent Of -z or --null-data Switch When Using Alpine sed Utility?
There is no equivalent.
How do I achieve the equivalent of the -z or --null-data with Alpine Linux sed utility?
Instead of using busybox sed, install GNU sed.
/ # sed --help
BusyBox v1.31.1 () multi-call binary.
/ # apk add sed
...
/ # sed --version
sed (GNU sed) 4.8
I used the sed utility to replace newlines with NUL
Use tr, it's a tool created to "translate or delete characters".
tr '\n' '\0'
sed -zE 's/\r\n|\n/\\n/g'
If you want to remote \r, no need to use zero separated streams. Just remove \r character from the actual end of line, why match the newline, if sed already works on lines.
sed 's/\r$//'
but really, again, to remove characters use tr:
tr -d '\r'
I'm using Sed on a Mac. I am trying to do a simple string replace on a file that is not in the directory. I do:
sed -i 's/old/new/' /Users/A/file
and it says invalid command code A.
What do I need to do?
The -i option in OSX/BSD sed is a little different than the GNU/Linux version in that it requires a backup extension to be given, even if it's an empty string (which means that no backup will be made). The "invalid command code" error message occurs because s/old/new is taken as the backup extension and /Users/A/file is taken as the script (where A is seen as an invalid command name). So it needs to be something like:
sed -i '' 's/old/new/' /Users/A/file
if you have perl:
perl -p -i -e 's/old/new/' /Users/A/file
The -i in sed is not a standard across OS's (not a POSIX standard). This should work every time:
cp /Users/A/file /Users/A/file.sed
cat /Users/A/file.sed | sed 's/old/new' > /Users/A/file
I have this line inside a file:
ULNET-PA,client_sgcib,broker_keplersecurities
,KEPLER
I try to get rid of that ^M (carriage return) character so I used:
sed 's/^M//g'
However this does remove everything after ^M:
[root#localhost tmp]# vi test
ULNET-PA,client_sgcib,broker_keplersecurities^M,KEPLER
[root#localhost tmp]# sed 's/^M//g' test
ULNET-PA,client_sgcib,broker_keplersecurities
What I want to obtain is:
[root#localhost tmp]# vi test
ULNET-PA,client_sgcib,broker_keplersecurities,KEPLER
Use tr:
tr -d '^M' < inputfile
(Note that the ^M character can be input using Ctrl+VCtrl+M)
EDIT: As suggested by Glenn Jackman, if you're using bash, you could also say:
tr -d $'\r' < inputfile
still the same line:
sed -i 's/^M//g' file
when you type the command, for ^M you type Ctrl+VCtrl+M
actually if you have already opened the file in vim, you can just in vim do:
:%s/^M//g
same, ^M you type Ctrl-V Ctrl-M
You can simply use dos2unix which is available in most Unix/Linux systems. However I found the following sed command to be better as it removed ^M where dos2unix couldn't:
sed 's/\r//g' < input.txt > output.txt
Hope that helps.
Note: ^M is actually carriage return character which is represented in code as \r
What dos2unix does is most likely equivalent to:
sed 's/\r\n/\n/g' < input.txt > output.txt
It doesn't remove \r when it is not immediately followed by \n and replaces both with just \n. This fails with certain types of files like one I just tested with.
alias dos2unix="sed -i -e 's/'\"\$(printf '\015')\"'//g' "
Usage:
dos2unix file
If Perl is an option:
perl -i -pe 's/\r\n$/\n/g' file
-i makes a .bak version of the input file
\r = carriage return
\n = linefeed
$ = end of line
s/foo/bar/g = globally substitute "foo" with "bar"
In awk:
sub(/\r/,"")
If it is in the end of record, sub(/\r/,"",$NF) should suffice. No need to scan the whole record.
This is the better way to achieve
tr -d '\015' < inputfile_name > outputfile_name
Later rename the file to original file name.
I agree with #twalberg (see accepted answer comments, above), dos2unix on Mac OSX covers this, quoting man dos2unix:
To run in Mac mode use the command-line option "-c mac" or use the
commands "mac2unix" or "unix2mac"
I settled on 'mac2unix', which got rid of my less-cmd-visible '^M' entries, introduced by an Apple 'Messages' transfer of a bash script between 2 Yosemite (OSX 10.10) Macs!
I installed 'dos2unix', trivially, on Mac OSX using the popular Homebrew package installer, I highly recommend it and it's companion command, Cask.
This is clean and simple and it works:
sed -i 's/\r//g' file
where \r of course is the equivalent for ^M.
Simply run the following command:
sed -i -e 's/\r$//' input.file
I verified this as valid in Mac OSX Monterey.
remove any \r :
nawk 'NF+=OFS=_' FS='\r'
gawk 3 ORS= RS='\r'
remove end of line \r :
mawk2 8 RS='\r?\n'
mawk -F'\r$' NF=1