I have a bunch of textfiles that contain the token -filename-. I need to replace it by the real path and filename of the file.
Thats what I have so far:
grep -lr -e '-filename-' *.txt | xargs sed -i
's/-filename-/therealname/g'
Is there a way to replace therealname with the name of the file?
Just do a bit more bash-fu
for x in *.txt; do
sed -i "s/-filename-/$x/g" $x;
done
Of course, the newlines are just for clarity. Feel free to cram that into one line.
like
for f in $(grep...) ; do sed -i "s,-filename-,$f,g" $f ; done
you mean?
With xargs it will be something like this.
grep ... | xargs -I% -n 1 sed -i "s,-filename-,%,g" %
for f in YOUR_FILE_LIST_MASK ; do
sed -i "s:-filename-:${f}" ${f}
done
can do it.
Related
I am trying to use sed to replace the following but not working
replace datetime.now(pytz.utc) with datetime.utcnow() recursively
i have tried the following
grep -rl "datetime.now(pytz.utc)" . | xargs sed -i 's/datetime.now\(pytz.utc\)/datetime.utcnow\(\)/g'
mac command equivalent
LC_ALL=C
grep -e "datetime.now(pytz.utc)" -rl . | xargs sed -i '' 's/datetime.now\(pytz.utc\)/datetime.utcnow\(\)/g'
as you can see i tried to escape all the parentheses but does not work
anyone know how to properly use sed to replace datetime.now(pytz.utc) with datetime.utcnow()?
I tried to explain in the comments, but obviously I wasn't clear. Here are two potential solutions to your problem:
Using your 'grep/xargs' method:
grep -rl "datetime.now(pytz.utc)" . | xargs sed -i 's/datetime.now(pytz.utc)/datetime.utcnow()/g'
Using the 'find/exec' method:
find . -type f -exec sed -i 's/datetime.now(pytz.utc)/datetime.utcnow()/g' {} \;
Both options will replace "datetime.now(pytz.utc)" with "datetime.utcnow()" in the files found. Both answers are platform independent provided you have GNU sed, not BSD sed.
This is mostly by curiosity, I am trying to have the same behavior as:
echo -e "test1:test2:test3"| sed 's/:/\n/g' | grep 1
in a single sed command.
I already tried
echo -e "test1:test2:test3"| sed -e "s/:/\n/g" -n "/1/p"
But I get the following error:
sed: can't read /1/p: No such file or directory
Any idea on how to fix this and combine different types of commands into a single sed call?
Of course this is overly simplified compared to the real usecase, and I know I can get around by using multiple calls, again this is just out of curiosity.
EDIT: I am mostly interested in the sed tool, I already know how to do it using other tools, or even combinations of those.
EDIT2: Here is a more realistic script, closer to what I am trying to achieve:
arch=linux64
base=https://chromedriver.storage.googleapis.com
split="<Contents>"
curl $base \
| sed -e 's/<Contents>/<Contents>\n/g' \
| grep $arch \
| sed -e 's/^<Key>\(.*\)\/chromedriver.*/\1/' \
| sort -V > out
What I would like to simplify is the curl line, turning it into something like:
curl $base \
| sed 's/<Contents>/<Contents>\n/g' -n '/1/p' -e 's/^<Key>\(.*\)\/chromedriver.*/\1/' \
| sort -V > out
Here are some alternatives, awk and sed based:
sed -E "s/(.*:)?([^:]*1[^:]*).*/\2/" <<< "test1:test2:test3"
awk -v RS=":" '/1/' <<< "test1:test2:test3"
# or also
awk 'BEGIN{RS=":"} /1/' <<< "test1:test2:test3"
Or, using your logic, you would need to pipe a second sed command:
sed "s/:/\n/g" <<< "test1:test2:test3" | sed -n "/1/p"
See this online demo. The awk solution looks cleanest.
Details
In sed solution, (.*:)?([^:]*1[^:]*).* pattern matches an optional sequence of any 0+ chars and a :, then captures into Group 2 any 0 or more chars other than :, 1, again 0 or more chars other than :, and then just matches the rest of the line. The replacement just keeps Group 2 contents.
In awk solution, the record separator is set to : and then /1/ regex is used to only return the record having 1 in it.
This might work for you (GNU sed):
sed 's/:/\n/;/^[^\n]*1/P;D' file
Replace each : and if the first line in the pattern space contains 1 print it.
Repeat.
An alternative:
sed -Ez 's/:/\n/g;s/^[^1]*$//mg;s/\n+/\n/;s/^\n//' file
This slurps the whole file into memory and replaces all colons by newlines. All lines that do not contain 1 are removed and surplus newlines deleted.
An alternative to the really ugly sed is: grep -o '\w*2\w*'
$ printf "test1:test2:test3\nbob3:bob2:fred2\n" | grep -o '\w*2\w*'
test2
bob2
fred2
grep -o: only matching
Or: grep -o '[^:]*2[^:]*'
echo -e "test1:test2:test3" | sed -En 's/:/\n/g;/^[^\n]*2[^\n]*(\n|$)/P;//!D'
sed -n doesn't print unless told to
sed -E allows using parens to match (\n|$) which is newline or the end of the pattern space
P prints the pattern buffer up to the first newline.
D trims the pattern buffer up to the first newline
[^\n] is a character class that matches anything except a newline
// is sed shorthand for repeating a match
//! is then matching everything that didn't match previously
So, after you split into newlines, you want to make sure the 2 character is between the start of the pattern buffer ^ and the first newline.
And, if there is not the character you are looking for, you want to D delete up to the first newline.
At that point, it works for one line of input, with one string containing the character you're looking for.
To expand to several matches within a line, you have to ta, conditionally branch back to label :a:
$ printf "test1:test2:test3\nbob3:bob2:fred2\n" | \
sed -En ':a s/:/\n/g;/^[^\n]*2[^\n]*(\n|$)/P;D;ta'
test2
bob2
fred2
This is simply NOT a job for sed. With GNU awk for multi-char RS:
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' '/1/'
test1
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' 'NR%2'
test1
test3
test5
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' '!(NR%2)'
test2
test4
test6
$ echo "foo1:bar1:foo2:bar2:foo3:bar3" | awk -v RS='[:\n]' '/foo/ || /2/'
foo1
foo2
bar2
foo3
With any awk you'd just have to strip the \n from the final record before operating on it:
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS=':' '{sub(/\n$/,"")} /1/'
test1
Suppose I have a string like this
<start><a></a><a></a><a></a></start>
I want to replace values inside <start></start> like this
<start><ab></ab><ab></ab><ab></ab><more></more><vale></value></start>
How do I do this using Sed?
Try this :
sed 's#<start>.*</start>#<start><ab></ab><ab></ab><ab></ab></start>#' file
I get this line with gnu sed :
sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:a>:ab>:g")"\3"#ge'
see example:
kent$ echo "<start><a></a><a></a><a></a><foo></foo><bar></bar></start>"|sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:a>:ab>:g")"\3"#ge'
<start><ab></ab><ab></ab><ab></ab><foo></foo><bar></bar></start>
note
this will replace the tags between <start>s which ending with a . which worked for your example. but if you have <aaa></aaa>:
you could do: (I break it into lines for better reading)
sed -r 's#(<start>)(.*)(</start>)
#echo "\1"$(echo "\2"\|sed "s:<a>:<ab>:g;s:</a>:</ab>:g")"\3"
#ge'
e.g.
kent$ echo "<start><a></a><a></a><a></a><aaa></aaa><aba></aba></start>" \
|sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:<a>:<ab>:g;s:</a>:</ab>:g")"\3"#ge'
<start><ab></ab><ab></ab><ab></ab><aaa></aaa><aba></aba></start>
sed 's/(\<\/?)a\>/\1ab\>/g' yourfile, though that would get <a></a> that was outside <start> as well...
grep -rl 'abc' a.txt | xargs sed -i 's/abc/def/g'
This question already has answers here:
How can I do a recursive find/replace of a string with awk or sed?
(37 answers)
Closed 1 year ago.
Using just grep and sed, how do I replace all occurrences of:
a.example.com
with
b.example.com
within a text file under the /home/user/ directory tree recursively finding and replacing all occurrences in all files in sub-directories as well.
Try this:
find /home/user/ -type f | xargs sed -i 's/a\.example\.com/b.example.com/g'
In case you want to ignore dot directories
find . \( ! -regex '.*/\..*' \) -type f | xargs sed -i 's/a\.example\.com/b.example.com/g'
Edit: escaped dots in search expression
Try this:
grep -rl 'SearchString' ./ | xargs sed -i 's/REPLACESTRING/WITHTHIS/g'
grep -rl will recursively search for the SEARCHSTRING in the directories ./ and will replace the strings using sed.
Ex:
Replacing a name TOM with JERRY using search string as SWATKATS in directory CARTOONNETWORK
grep -rl 'SWATKATS' CARTOONNETWORK/ | xargs sed -i 's/TOM/JERRY/g'
This will replace TOM with JERRY in all the files and subdirectories under CARTOONNETWORK wherever it finds the string SWATKATS.
On macOS, none of the answers worked for me. I discovered that was due to differences in how sed works on macOS and other BSD systems compared to GNU.
In particular BSD sed takes the -i option but requires a suffix for the backup (but an empty suffix is permitted)
grep version from this answer.
grep -rl 'foo' ./ | LC_ALL=C xargs sed -i '' 's/foo/bar/g'
find version from this answer.
find . \( ! -regex '.*/\..*' \) -type f | LC_ALL=C xargs sed -i '' 's/foo/bar/g'
Don't omit the Regex to ignore . folders if you're in a Git repo. I realized that the hard way!
That LC_ALL=C option is to avoid getting sed: RE error: illegal byte sequence if sed finds a byte sequence that is not a valid UTF-8 character. That's another difference between BSD and GNU. Depending on the kind of files you are dealing with, you may not need it.
For some reason that is not clear to me, the grep version found more occurrences than the find one, which is why I recommend to use grep.
I know this is a really old question, but...
#vehomzzz's answer uses find and xargs when the questions says explicitly grep and sed only.
#EmployedRussian and #BrooksMoses tried to say it was a dup of awk and sed, but it's not - again, the question explicitly says grep and sed only.
So here is my solution, assuming you are using Bash as your shell:
OLDIFS=$IFS
IFS=$'\n'
for f in `grep -rl a.example.com .` # Use -irl instead of -rl for case insensitive search
do
sed -i 's/a\.example\.com/b.example.com/g' $f # Use /gi instead of /g for case insensitive search
done
IFS=$OLDIFS
If you are using a different shell, such as Unix SHell, let me know and I will try to find a syntax adjustment.
P.S.: Here's a one-liner:
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl a.example.com .`;do sed -i 's/a\.example\.com/b.example.com/g' $f;done;IFS=$OLDIFS
Sources:
Bash: Iterating over lines in a variable
grep(1) - Linux man page
Official Grep Manual
sed(1) - Linux man page
Official sed Manual
For me works the next command:
find /path/to/dir -name "file.txt" | xargs sed -i 's/string_to_replace/new_string/g'
if string contains slash 'path/to/dir' it can be replace with another character to separate, like '#' instead '/'.
For example: 's#string/to/replace#new/string#g'
We can try using the more powerful ripgrep as
rg "BYE_BYE_TEXT" ./ --files-with-matches | xargs sed -i "s/BYE_BYE_TEXT/WELCOME_TEXT/g"
Because ripgrep is good at finding and sed is great at replacing.
it is much simpler than that.
for i in `find *` ; do sed -i -- 's/search string/target string/g' $i; done
find i => will iterate over all the files in the folder and in subfolders.
sed -i => will replace in the files the relevant string if exists.
Try this command:
/home/user/ directory - find ./ -type f \
-exec sed -i -e 's/a.example.com/b.example.com/g' {} \;
The command below will search all the files recursively whose name matches the search pattern and will replace the string:
find /path/to/searchdir/ -name "serachpatter" -type f | xargs sed -i 's/stringone/StrIngTwo/g'
Also if you want to limit the depth of recursion you can put the limits as well:
find /path/to/searchdir/ -name "serachpatter" -type f -maxdepth 4 -mindepth 2 | xargs sed -i 's/stringone/StrIngTwo/g'
I need to do the following for hundreds of files:
Append the name of the file (which may contain spaces) to the end of each line in the file.
It seems to me there should be some way to do this:
sed -e 's/$/FILENAME/' *
where FILENAME represents the name of the current file. Is there a sed variable representing the current filename? Or does anyone have a different solution using bash, awk, etc.?
I'm sure there are other ways to do it, I'd use perl:
perl -p -i -e 's/$/$ARGV/;' *
Some versions of sed support the "--in-place" argument so you can condense Tyler's solution to
for i in * ; do
sed -e "s/\$/$i/" --in-place "$i"
done
You could do it with a bash script
for i in *
do
sed -e "s/\$/$i/" "$i"
done
One-liner version:
for i in * ; do sed -e "s/\$/$i/" "$i" ; done
Edit: If you want to replace the contents of the file with the new, name-appended lines, do this:
TFILE=`mktemp`
for i in *
do
sed -e "s/\$/$i/" "$i" > $TFILE
cp -f $TFILE "$i"
done
rm -f $TFILE
awk '{print $0,FILENAME}' > tmpfile
In BASH, I'd do something to the effect of:
for f in *; do echo $f >> $f; done
More or less how Tyler suggested, just with some modifications to allow for spaces in the name. I was hoping for a one-liner though...
(
OLDIFS=$IFS
IFS=$'\n'
for f in *
do
IFS=OLDIFS
sed -e "s/\$/$f/" $f > tmpfile
mv tmpfile $f
IFS=$'\n'
done
)
This might work for you:
printf "%s\n" * | sed 's/.*/sed -i "s|$| &|" &/' | bash