I have this list of files:
$ more files
one_this_2017_1_abc.txt
two_that_2018_1_abc.txt
three_another_2017_10.abc.txt
four_again_2018_10.abc.txt
five_back_2018_1a.abc.txt
I would like to get this output:
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_YY.abc.txt
four_again_XXXX_YY.abc.txt
five_back_XXXX_YY.abc.txt
I am trying to remove the year and the bit after the year and replace them with another string--this is to generate test cases.
I can get the year just fine, but it's that one or two character piece after it I can't seem to match.
This should work, right?
~/test_cases
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\{1,2\}_/_YY_/'
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Except it doesn't for the 2 character cases.
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\
{2\}_/_YY_/'
one_this_XXXX_1_abc.txt
two_that_XXXX_1_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Doesn't work for the two character cases either, and this works not at all (but according to the docs it should):
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\+_/_YY_/'
one_YY_XXXX_1_abc.txt
two_YY_XXXX_1_abc.txt
three_YY_XXXX_10.abc.txt
four_YY_XXXX_10.abc.txt
five_YY_XXXX_1a.abc.txt
Other random experiments that don't work:
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\+_/_YY_/'
one_YY_XXXX_1_abc.txt
two_YY_XXXX_1_abc.txt
three_YY_XXXX_10.abc.txt
four_YY_XXXX_10.abc.txt
five_YY_XXXX_1a.abc.txt
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\{1\}_/_YY_/'
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\{2\}_/_YY_/'
one_this_XXXX_1_abc.txt
two_that_XXXX_1_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Tried with both GNU sed version 4.2.1 under Linux and sed (GNU sed) 4.4 under Cygwin.
And yes, I realize I can pipe this through multiple sed calls to get it to work, but that regex SHOULD work, right?
if your Input_file is same as shown sample then following may help you in same.
sed 's/\([^_]*\)_\([^_]*\)_\(.*_\)\(.*\)/\1_\2_XXXX_YY_\4/g' Input_file
Output will be as follows.
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_YY_10.abc.txt
four_again_XXXX_YY_10.abc.txt
five_back_XXXX_YY_1a.abc.txt
I've successfully used the following sed command to search/replace text in Linux:
sed -i 's/old_link/new_link/g' *
However, when I try it on my Mac OS X, I get:
"command c expects \ followed by text"
I thought my Mac runs a normal BASH shell. What's up?
EDIT:
According to #High Performance, this is due to Mac sed being of a different (BSD) flavor, so my question would therefore be how do I replicate this command in BSD sed?
EDIT:
Here is an actual example that causes this:
sed -i 's/hello/gbye/g' *
If you use the -i option you need to provide an extension for your backups.
If you have:
File1.txt
File2.cfg
The command (note the lack of space between -i and '' and the -e to make it work on new versions of Mac and on GNU):
sed -i'.original' -e 's/old_link/new_link/g' *
Create 2 backup files like:
File1.txt.original
File2.cfg.original
There is no portable way to avoid making backup files because it is impossible to find a mix of sed commands that works on all cases:
sed -i -e ... - does not work on OS X as it creates -e backups
sed -i'' -e ... - does not work on OS X 10.6 but works on 10.9+
sed -i '' -e ... - not working on GNU
Note Given that there isn't a sed command working on all platforms, you can try to use another command to achieve the same result.
E.g., perl -i -pe's/old_link/new_link/g' *
I believe on OS X when you use -i an extension for the backup files is required. Try:
sed -i .bak 's/hello/gbye/g' *
Using GNU sed the extension is optional.
This works with both GNU and BSD versions of sed:
sed -i'' -e 's/old_link/new_link/g' *
or with backup:
sed -i'.bak' -e 's/old_link/new_link/g' *
Note missing space after -i option! (Necessary for GNU sed)
Had the same problem in Mac and solved it with brew:
brew install gnu-sed
and use as
gsed SED_COMMAND
you can set as well set sed as alias to gsed (if you want):
alias sed=gsed
Or, you can install the GNU version of sed in your Mac, called gsed, and use it using the standard Linux syntax.
For that, install gsed using ports (if you don't have it, get it at http://www.macports.org/) by running sudo port install gsed. Then, you can run sed -i 's/old_link/new_link/g' *
Your Mac does indeed run a BASH shell, but this is more a question of which implementation of sed you are dealing with. On a Mac sed comes from BSD and is subtly different from the sed you might find on a typical Linux box. I suggest you man sed.
Insead of calling sed with sed, I do ./bin/sed
And this is the wrapper script in my ~/project/bin/sed
#!/bin/bash
if [[ "$OSTYPE" == "darwin"* ]]; then
exec "gsed" "$#"
else
exec "sed" "$#"
fi
Don't forget to chmod 755 the wrapper script.
Sinetris' answer is right, but I use this with find command to be more specific about what files I want to change. In general this should work (tested on osx /bin/bash):
find . -name "*.smth" -exec sed -i '' 's/text1/text2/g' {} \;
In general when using sed without find in complex projects is less efficient.
I've created a function to handle sed difference between MacOS (tested on MacOS 10.12) and other OS:
OS=`uname`
# $(replace_in_file pattern file)
function replace_in_file() {
if [ "$OS" = 'Darwin' ]; then
# for MacOS
sed -i '' -e "$1" "$2"
else
# for Linux and Windows
sed -i'' -e "$1" "$2"
fi
}
Usage:
$(replace_in_file 's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' "./mysql/.env")
Where:
, is a delimeter
's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' is pattern
"./mysql/.env" is path to file
As the other answers indicate, there is not a way to use sed portably across OS X and Linux without making backup files. So, I instead used this Ruby one-liner to do so:
ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml
In my case, I needed to call it from a rake task (i.e., inside a Ruby script), so I used this additional level of quoting:
sh %q{ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml}
Here's how to apply environment variables to template file (no backup need).
1. Create template with {{FOO}} for later replace.
echo "Hello {{FOO}}" > foo.conf.tmpl
2. Replace {{FOO}} with FOO variable and output to new foo.conf file
FOO="world" && sed -e "s/{{FOO}}/$FOO/g" foo.conf.tmpl > foo.conf
Working both macOS 10.12.4 and Ubuntu 14.04.5
Here is an option in bash scripts:
#!/bin/bash
GO_OS=${GO_OS:-"linux"}
function detect_os {
# Detect the OS name
case "$(uname -s)" in
Darwin)
host_os=darwin
;;
Linux)
host_os=linux
;;
*)
echo "Unsupported host OS. Must be Linux or Mac OS X." >&2
exit 1
;;
esac
GO_OS="${host_os}"
}
detect_os
if [ "${GO_OS}" == "darwin" ]; then
sed -i '' -e ...
else
sed -i -e ...
fi
sed -ie 's/old_link/new_link/g' *
Works on both BSD & Linux with gnu sed
This question already has answers here:
How to use sed to replace only the first occurrence in a file?
(25 answers)
Closed 8 years ago.
I am using sed command make substitution in the file.
Assume my file1 is:
10
11
10
11
Then I want to substitute the 1st "10" in file1 to "12", and dump to file2. The file2 should be:
12
11
10
11
I tried this command:
sed 's/10/12/' file1 >file2
But it changed the 2nd "10" also. So, how can I write the sed command to do that?
If you can use awk instead of sed, you can have more control like this:
awk '!changed && /10/ { sub(/10/, "12"); changed = 1}1' file1
12
11
10
11
try:
sed '0,/10/s/10/12/' file1 >file2
Like GriffinG said, you can do something like this:
sed '0,/10/ s/10/12/' file1 > file2
The 0,/10/ at the beginning sets the bounds on the following substitution command. It says start from line 0 (the first line), and go until a line it matches the pattern /10/. So After that first match, sed will stop processing that substitution.
If you do not have GNU sed, try:
echo | cat - file | sed '1,/10/s/10/12/; 1d'
or
sed '1,/10/s/10/12/; 1d' << EOF
$(cat file)
EOF
or in bash / ksh93
sed '1,/10/s/10/12/; 1d' <(echo; cat file)
GNU sed knows 0,/10/ so the extra empty line is not required
This question already has answers here:
How to remove every other line with sed?
(5 answers)
Closed 8 years ago.
I am somewhat new to posix, and i can't use: sed '1~2p'
My goal is to skip every one line from line 1:
1
2
3
4
would become
1
3
I was wondering what is the posix equivalent of ~.
Code for sed:
sed -e n -e d file
or:
sed -e 'n;d' file
The simpler, portable solution would be:
awk 'NR%2' file
bash solution:
while read -r line; do
[ $((i++ % 2)) -eq 0 ] && echo "$line";
done < file
Consider the file test.txt:
#include "foo.h"
#include "bar.h"
#include "baz.h"
using GNU sed version 4.2.1 (on Ubuntu 10.04.4 LTS), I can extract foo.h, bar.h and baz.h with:
SHELL$) sed -n -e 's:^\s*\#include\s*"\(.*\)".*:\1:p' test.txt
foo.h
bar.h
baz.h
using BSD sed (on Mac OS X lion), and modifying the above command, I can extract foo.h, bar.h and baz.h, but with double quotes:
SHELL) sed -n -e 's:^\s*\#include\s*\(.*\).*:\1:p' test.txt
"foo.h"
"bar.h"
"bar.h"
How can to extract names without the quotes with BSD sed? The output of theses commands are empty:
SHELL) sed -n -e 's:^\s*\#include\s*"\(.*\)".*:\1:p' test.txt
SHELL) sed -n -e 's:^\s*\#include\s*\"\(.*\)\".*:\1:p' test.txt
BSD sed (unsurprisingly, really) doesn't support the \s Perlism -- it is interpreted as just a literal s. Try this instead;
sed -n -e 's!^[[:space:]]*\#include[[:space:]]*"\(.*\)".*!\1!p' test.txt
The character class [[:space:]] should work in all POSIX regex implementations. (Other seds may or may not want backslashes before grouping parentheses.)