Replace a given string in folders throughout a directory - perl

I want to do the equivalent of this, but [maybe recursively] for all, say, .md files within a directory tree.
perl -pi -e 's/FOO/BAR/g' *.md

Use find:
find /path -name "*.md" -exec perl -pi -e 's/FOO/BAR/g' {} \;

A simple & pure bash one line solution by using bash parameter expansion:
$ cd ~/
$ mkdir test
$ cd test/
$ touch foo{1..10}.md
$ ls
foo1.md foo10.md foo2.md foo3.md foo4.md foo5.md foo6.md foo7.md foo8.md foo9.md
$ for file in ./*.md; do mv "$file" "${file/foo/bar}"; done
$ ls
bar1.md bar10.md bar2.md bar3.md bar4.md bar5.md bar6.md bar7.md bar8.md bar9.md
Of course it can be combined with find as suggested by devnull:
$ files=($(find ./test -name "*.md"))
$ for file in "${files[#]}"; do mv "$file" "${file/foo/bar}"; done
Or pipe the ouptut of find to the loop:
$ find ./test -name "*.md" | for file in $(xargs -0); do mv "$file" "${file/foo/bar}"; done

Related

how to create folder recursively

there are two file path .
now i am in eee folder
/Volumes/aaa/bbb/ccc/ddd/eee/text.txt
1) i am going to mv text.txt file to below path
/Volumes/aaa/bbb/ccc/ddd_1/eee/text.txt
or just create folder structure only
/Volumes/aaa/bbb/ccc/ddd_1/eee/
with below command with recursively , but not works all
there are some solution related with this .
but not works .
mkdir -p $(`pwd | sed 's/ddd/ddd_l/'`)
or
rsync -av -f"+ */" -f"- *" "$pwd" "$(`pwd | sed 's/ddd/ddd_l/'`)"
or
mv test.MTS `pwd | sed 's/ddd/ddd_l/'`
or
cp -R test.MTS `pwd | sed 's/ddd/ddd_l/'`
who can do this ?
Try this:
new_file="/Volumes/aaa/bbb/ccc/ddd/eee/text.txt"
mkdir -p "$(dirname "$new_file")"
touch "$new_file"

duplicate weblogic domain from solaris 9 command line

I need to search for a string of characters in a weblogic domain and then replace that string with a new one. I tried doing a grep -r but solaris 9 doesn't support it. I tried a find command:
find <mydir>-type f -exec sed -i 's/string1/string2/g' {}+ .
This did not work.
I have multiple directory names, file names and file content to change.
I am essentially coping a weblogic domain, tar and zipping it, moving it, unzip, untar, rename and reconfig and then launch. all from command prompt.
ANY help would be very much appreciated.
-i is a GNU sed extension.
Here is one way to achieve what you want with Solaris 9 commands :
find <mydir> -type f -exec ksh -c '
for i do sed "s/string1/string2/g" $i > /tmp/foo && cat /tmp/foo > $i; done' ksh {} +
Thanks guys but i found my answer:
find . -type f -exec grep -l <search string> {} \; | while read i; do perl -pi -e 's/<string1>/<string2>/g' $i;done
Thanks for the replies.

sed not working properly with multiple input files

sed -i is creating a backup of all files in subdirectories before editing in place (as expected) but it's not actually editing files in subdirectories.
$ mkdir -p a/b
$ echo "A" > a/a.txt
$ echo "B" > a/b/b.txt
Now I have two text files, one in a one in a subdirectory of a
$ sed -i.bac "1s/^/PREPENDED /" a/**/*.txt
Backups are created for both:
$ find a
a
a/a.txt
a/a.txt.bac
a/b
a/b/b.txt
a/b/b.txt.bac
Only a.txt is edited:
$ cat a/a.txt
PREPENDED A
$ cat a/b/b.txt
B
I'm using ZSH (so I have globstar support) and I'm on Mac.
Why is this happening and how can I fix it?
It's happening because your sed invocation only has a single line 1, which happens to be in a.txt. If you want it to do it for each file then you need to invoke sed multiple times.
for f in a/**/*.txt
do
sed ... "$f"
done
Since you are needing to descend through several levels of directories, a single invocation of sed alone is not sufficient. However, using find you can accomplish what you want in a single line. If you are not familiar with find ... -exec '{}' \; it is worth taking a few minutes with startpage.com and do a quick search. In your case, the following invocation works well:
find a -type f -name "*.txt" -exec sed -i.bac 's/^/PREPENDED /' '{}' \;
Here find searches directory a and all below for any file (-type f) matching *.txt, then for each file (indicated by '{}') -exec executes sed -i.bac 's/^/PREPENDED /' and lastly an escaped \; is given to indicate the end of the -exec command.
results:
$ ls -1 a
b
a.txt
a.txt.bac
$ ls -1 a/b
b.txt
b.txt.bac
$ cat a/a.txt
PREPENDED A
$ cat a/b/b.txt
PREPENDED B
As was correctly pointed out, with globstar set shopt -s globstar it is unnecessary to use find as the following invocation of sed is sufficient:
sed -i.bac 's/^/PREPENDED /' a/**/*.txt

Change multiple files

The following command is correctly changing the contents of 2 files.
sed -i 's/abc/xyz/g' xaa1 xab1
But what I need to do is to change several such files dynamically and I do not know the file names. I want to write a command that will read all the files from current directory starting with xa* and sed should change the file contents.
I'm surprised nobody has mentioned the -exec argument to find, which is intended for this type of use-case, although it will start a process for each matching file name:
find . -type f -name 'xa*' -exec sed -i 's/asd/dsg/g' {} \;
Alternatively, one could use xargs, which will invoke fewer processes:
find . -type f -name 'xa*' | xargs sed -i 's/asd/dsg/g'
Or more simply use the + exec variant instead of ; in find to allow find to provide more than one file per subprocess call:
find . -type f -name 'xa*' -exec sed -i 's/asd/dsg/g' {} +
Better yet:
for i in xa*; do
sed -i 's/asd/dfg/g' $i
done
because nobody knows how many files are there, and it's easy to break command line limits.
Here's what happens when there are too many files:
# grep -c aaa *
-bash: /bin/grep: Argument list too long
# for i in *; do grep -c aaa $i; done
0
... (output skipped)
#
You could use grep and sed together. This allows you to search subdirectories recursively.
Linux: grep -r -l <old> * | xargs sed -i 's/<old>/<new>/g'
OS X: grep -r -l <old> * | xargs sed -i '' 's/<old>/<new>/g'
For grep:
-r recursively searches subdirectories
-l prints file names that contain matches
For sed:
-i extension (Note: An argument needs to be provided on OS X)
Those commands won't work in the default sed that comes with Mac OS X.
From man 1 sed:
-i extension
Edit files in-place, saving backups with the specified
extension. If a zero-length extension is given, no backup
will be saved. It is not recommended to give a zero-length
extension when in-place editing files, as you risk corruption
or partial content in situations where disk space is exhausted, etc.
Tried
sed -i '.bak' 's/old/new/g' logfile*
and
for i in logfile*; do sed -i '.bak' 's/old/new/g' $i; done
Both work fine.
#PaulR posted this as a comment, but people should view it as an answer (and this answer works best for my needs):
sed -i 's/abc/xyz/g' xa*
This will work for a moderate amount of files, probably on the order of tens, but probably not on the order of millions.
Another more versatile way is to use find:
sed -i 's/asd/dsg/g' $(find . -type f -name 'xa*')
I'm using find for similar task. It is quite simple: you have to pass it as an argument for sed like this:
sed -i 's/EXPRESSION/REPLACEMENT/g' `find -name "FILE.REGEX"`
This way you don't have to write complex loops, and it is simple to see, which files you are going to change, just run find before you run sed.
u can make
'xxxx' text u search and will replace it with 'yyyy'
grep -Rn '**xxxx**' /path | awk -F: '{print $1}' | xargs sed -i 's/**xxxx**/**yyyy**/'
There's some good answers above. I thought I'd throw in one more that is succinct and parallelizable, using GNU parallel, which I often prefer to xargs:
parallel sed -i 's/abc/xyz/g' {} ::: xa*
Combine this with the -j N option to run N jobs in parallel.
If you are able to run a script, here is what I did for a similar situation:
Using a dictionary/hashMap (associative array) and variables for the sed command, we can loop through the array to replace several strings. Including a wildcard in the name_pattern will allow to replace in-place in files with a pattern (this could be something like name_pattern='File*.txt' ) in a specific directory (source_dir).
All the changes are written in the logfile in the destin_dir
#!/bin/bash
source_dir=source_path
destin_dir=destin_path
logfile='sedOutput.txt'
name_pattern='File.txt'
echo "--Begin $(date)--" | tee -a $destin_dir/$logfile
echo "Source_DIR=$source_dir destin_DIR=$destin_dir "
declare -A pairs=(
['WHAT1']='FOR1'
['OTHER_string_to replace']='string replaced'
)
for i in "${!pairs[#]}"; do
j=${pairs[$i]}
echo "[$i]=$j"
replace_what=$i
replace_for=$j
echo " "
echo "Replace: $replace_what for: $replace_for"
find $source_dir -name $name_pattern | xargs sed -i "s/$replace_what/$replace_for/g"
find $source_dir -name $name_pattern | xargs -I{} grep -n "$replace_for" {} /dev/null | tee -a $destin_dir/$logfile
done
echo " "
echo "----End $(date)---" | tee -a $destin_dir/$logfile
First, the pairs array is declared, each pair is a replacement string, then WHAT1 will be replaced for FOR1 and OTHER_string_to replace will be replaced for string replaced in the file File.txt. In the loop the array is read, the first member of the pair is retrieved as replace_what=$i and the second as replace_for=$j. The find command searches in the directory the filename (that may contain a wildcard) and the sed -i command replaces in the same file(s) what was previously defined. Finally I added a grep redirected to the logfile to log the changes made in the file(s).
This worked for me in GNU Bash 4.3 sed 4.2.2 and based upon VasyaNovikov's answer for Loop over tuples in bash.
The Silver Searcher Solution
I'm adding another option for those people who don't know about the amazing tool called The Silver Searcher (command line tool is ag).
Note: You can use grep and other tools to do the same thing here, but The Silver Searcher is fantastic :)
TLDR
ag -l 'abc' | xargs sed -i 's/abc/xyz/g'
Install The Silver Searcher
sudo apt install silversearcher-ag # Debian / Ubuntu
sudo pacman -S the_silver_searcher # Arch / EndeavourOS
sudo yum install epel-release the_silver_searcher # RHEL / CentOS
Demo Files
Paste the following into your terminal to create some demonstration files:
mkdir /tmp/food
cd /tmp/food
content="Everybody loves to abc this food!"
echo "$content" > ./milk
echo "$content" > ./bread
mkdir ./fastfood
echo "$content" > ./fastfood/pizza
echo "$content" > ./fastfood/burger
mkdir ./fruit
echo "$content" > ./fruit/apple
echo "$content" > ./fruit/apricot
Using 'ag'
The following ag command will recursively find all the files that contain the string 'abc'. It ignores the .git directory, .gitignore files, and other ignore files:
$ ag 'abc'
milk
1:Everybody loves to abc this food!
bread
1:Everybody loves to abc this food!
fastfood/burger
1:Everybody loves to abc this food!
fastfood/pizza
1:Everybody loves to abc this food!
fruit/apple
1:Everybody loves to abc this food!
fruit/apricot
1:Everybody loves to abc this food!
To just list the files that contain the string 'abc', use the -l switch:
$ ag -l 'abc'
bread
fastfood/burger
fastfood/pizza
fruit/apricot
milk
fruit/apple
Changing Multiple Files
Finally, using xargs and sed, we can replace the 'abc' string with another string:
ag -l 'abc' | xargs sed -i 's/abc/eat/g'
In the above command, ag is listing all the files that contain the string 'abc'. The xargs command is splitting the file names and piping them individually into the sed command.

Use grep / sed for filename search & replace?

I have a bunch of image files that were incorrectly named 'something#x2.png' and they need to be 'something#2x.png'. They're spread across multiple directories like so:
/images
something#x2.png
/icons
icon#x2.png
/backgrounds
background#x2.png
How can I use grep + sed to find/replace as needed?
Ruby(1.9+)
$ ruby -e 'Dir["**/*#x2.png"].each{|x| File.rename( x, x.sub(/#x2/,"#2x") ) }'
Look at qmv and rename
find -iname '*.png' -print0 | xargs -0 qmv -d
will launch your default editor and allow you to interactively edit the names
rename s/#x2/#2x/ *.png
Slashes look linuxy/unixoid to me. Do you have find and rename?
find -name "*#x2*" -execdir rename 's/#x2/#2x/' {} +
rename is worth installing, comes in some perl-package.
With bash 2.x/3.x
#!/bin/bash
while IFS= read -r -d $'\0' file; do
echo mv "$file" "${file/#x2/#2x}"
done < <(find images/ -type f -name "something*#x2*.png" -print0)
With bash 4.x
#!/bin/bash
shopt -s globstar
for file in images/**; do
[[ "$file" == something*#x2*.png ]] && echo mv "$file" "${file/#x2/#2x}"
done
Note:
In each case I left in an echo so you can do a dry-run, remove the echo if the output is sufficient