I'm very new to some of the command line utilities and have been looking for a while for a command that would accomplish my goal.
The goal is to find files that contain a string of text, replace it with a new string, and then write the results to a file that is named the same as the original, but in a different directory.
Obviously this is not working, so I am asking how you who know about this stuff would go about it.
grep -rl 'stringToFind' *.* | sed 's|oldString|newString|g' < fileNameFromGrep > ./new/fileNameFromGrep
Thanks for your input!
John
for f in "`find /YOUR/SEARCH/DIR/ROOT -type f -exec fgrep -l 'stirngToFind' \{\} \;`" ; do
sed 's|oldString|newString|g' < "${f} > ./new/"${f}
done
Will do it for you.
If you have spaces in filenames:
OLDIFS=$IFS
IFS=''
find /PATH -print0 -type f | while read -r -d $'' file
do
fgrep -l 'stirngToFind' "$file" && \
sed 's|oldString|newString|g' < "${file} > ./new/"${file}
done
IFS=$OLDIFS
#!/bin/bash
for file in *; do
if grep -qF 'stringToFind' "$file"; then
sed 's/oldString/newString/g' "$file" > "./new/$file"
fi
done
for file in path/to/dir/*
do
grep -q 'pattern' "$file" > /dev/null
if [ $? == 0 ]; then
sed 's/oldString/newString/g' "$file" > /path/to/newdir/"$file"
fi
done
You try:
sed -ie "s/oldString/newString/g" \
$(grep -Rsi 'pattern' path/to/dir/ | cut -d: -f1)
sed:
i in_place
e exec other command or script
grep:
R recursive
s Suppress error messages
i ignore case sensitive
Related
I have a set of filenames in a directory, some of which are likely to have identical substrings but not known in advance. This is a sorting exercise. I want to move the files with the maximum substring ordered letter match together in a subdirectory named with that number of letters and progress to the minimum match until no matches of 2 or more letters remain. Ignore extensions. Case insensitive. Ignore special characters.
Example.
AfricanElephant.jpg
elephant.jpg
grant.png
ant.png
el_gordo.tif
snowbell.png
Starting from maximum length matches to minimum length matches will result in:
./8/AfricanElephant.jpg and ./8/elephant.jpg
./3/grant.png and ./3/ant.png
./2/snowbell.png and ./2/el_gordo.tif
Completely lost on an efficient bash or python way to do what seems a complex sort.
I found some awk code which is almost there:
{
count=0
while ( match($0,/elephant/) ) {
count++
$0=substr($0,RSTART+1)
}
print count
}
where temp.txt contains a list of the files and is invoked as eg
awk -f test_match.awk temp.txt
Drawback is that a) this is hardwired to look for "elephant" as a string (I don't know how to make it take an input string (rather than file) and an input test string to count against, and
b) I really just want to call a bash function to do the sort as specified
If I had this I could wrap some bash script around this core awk to make it work.
function longest_common_substrings () {
shopt -s nocasematch
for file1 in * ; do for file in * ; do \
if [[ -f "$file1" ]]; then
if [[ -f "$file" ]]; then
base1=$(basename "$file" | cut -d. -f1)
base2=$(basename "$file1" | cut -d. -f1)
if [[ "$file" == "$file1" ]]; then
echo -n ""
else
echo -n "$file $file1 " ; $HOME/Scripts/longest_common_substring.sh "$base1" "$base2" | tr -d '\n' | wc -c | awk '{$1=$1;print}' ;
fi
fi
fi
done ;
done | sort -r -k3 | awk '{ print $1, $3 }' > /tmp/filesort_substring.txt
while IFS= read -r line; do \
file_to_move=$(echo "$line" | awk '{ print $1 }') ;
directory_to_move_to=$(echo "$line" | awk '{ print $2 }') ;
if [[ -f "$file_to_move" ]]; then
mkdir -p "$directory_to_move_to"
\gmv -b "$file_to_move" "$directory_to_move_to"
fi
done < /tmp/filesort_substring.txt
shopt -u nocasematch
where $HOME/Scripts/longest_common_substring.sh is
#!/bin/bash
shopt -s nocasematch
if ((${#1}>${#2})); then
long=$1 short=$2
else
long=$2 short=$1
fi
lshort=${#short}
score=0
for ((i=0;i<lshort-score;++i)); do
for ((l=score+1;l<=lshort-i;++l)); do
sub=${short:i:l}
[[ $long != *$sub* ]] && break
subfound=$sub score=$l
done
done
if ((score)); then
echo "$subfound"
fi
shopt -u nocasematch
Kudos to the original solution for computing the match in the script which I found elsewhere in this site
I have this:
for file in **/*.ogg;
if ffprobe "$file" 2>&1 | sed -E -n 's/^ *GENRE *: (.*)/\1/p' | grep -q "$argv";
echo "$file"
else
end
end
but I would like to turn it into a function which will take a list of filenames as standard-input:
$ find . -maxdepth 1 -not -type d -exec du -h {} + | cut -f2 | filterByGenre Classical
You could do
function filterByGenre
while read line
do stuff with $line
end
end
or
function filterByGenre
set listOfLines (cat)
for line in $listOfLines
do stuff with $line
end
end
Can anybody help me please?
grep " 287 " file.txt | grep "HI" | sed -i 's/HIS/HID/g'
sed: no input files
Tried also xargs
grep " 287 " file.txt | grep HI | xargs sed -i 's/HIS/HID/g'
sed: invalid option -- '6'
This works fine
grep " 287 " file.txt | grep HI
If you want to keep your pipeline:
f=file.txt
tmp=$(mktemp)
grep " 287 " "$f" | grep "HI" | sed 's/HIS/HID/g' > "$tmp" && mv "$tmp" "$f"
Or, simplify:
sed -i -n '/ 287 / {/HI/ s/HIS/HID/p}' file.txt
That will filter out any line that does not contain " 287 " and "HI" -- is that what you want? I suspect you really want this:
sed -i '/ 287 / {/HI/ s/HIS/HID/}' file.txt
For lines that match / 287 /, execute the commands in braces. In there, for lines that match /HI/, search for the first "HIS" and replace with "HID". sed implicitly prints all lines if -n is not specified.
Other commands that do the same thing:
awk '/ 287 / && /HI/ {sub(/HIS/, "HID")} {print}' file.txt > new.txt
perl -i -pe '/ 287 / and /HI/ and s/HIS/HID/' file.txt
awk does not have an "in-place" option (except gawk -i inplace for recent gawk versions)
I have a example cut down from a log file.
112 172.172.172.1#50912 (ssl.bing.com):
I would like some how to remove the # and numbers after and (): from the url.
Would like the result.
112 172.172.172.1 ssl.bing.com
Here is the sed oneliner I have been working on.
cat newdns.log | sed -e 's/.*query: //' | cut -f 1 -d' ' | sort | uniq -c | sort -k2 > old.log
Thanks
Using sed, you could say:
sed 's/#[0-9]*//;s/(\(.*\)):$/\1/' filename
or, in a single substitution:
sed 's/#[0-9]* *(\(.*\)):$/ \1/' filename
Another sed:
sed -r 's/#[^ ]+|[():]//g'
$ echo '112 172.172.172.1#50912 (ssl.bing.com):' | sed -r 's/#[^ ]+|[():]//g'
112 172.172.172.1 ssl.bing.com
Before:
eng-vshakya:scripts vshakya$ ls
American Samoa.png Faroe Islands.png Saint Barthelemy.png
After:
eng-vshakya:scripts vshakya$ ls
AmericanSamoa.png FaroeIslands.png SaintBarthelemy.png
Tried below prototype, but it does not work :( Sorry, not very good when it comes to awk/sed :(
ls *.png | sed 's/\ /\\\ /g' | awk '{print("mv "$1" "$1)}'
[ Above is prototype, real command, I guess, would be:
ls *.png | sed 's/\ /\\\ /g' | awk '{print("mv "$1" "$1)}' | sed 's/\ //g'
]
No need to use awk or sed when you can do this in pure bash.
[ghoti#pc ~/tmp1]$ ls -l
total 2
-rw-r--r-- 1 ghoti wheel 0 Aug 1 01:19 American Samoa.png
-rw-r--r-- 1 ghoti wheel 0 Aug 1 01:19 Faroe Islands.png
-rw-r--r-- 1 ghoti wheel 0 Aug 1 01:19 Saint Barthelemy.png
[ghoti#pc ~/tmp1]$ for name in *\ *; do mv -v "$name" "${name// /}"; done
American Samoa.png -> AmericanSamoa.png
Faroe Islands.png -> FaroeIslands.png
Saint Barthelemy.png -> SaintBarthelemy.png
[ghoti#pc ~/tmp1]$
Note that the ${foo/ /} notation is bash, and does not work in classic Bourne shell.
ghoti's solution is the right thing to do. Since you ask how to do it in sed, here's one way:
for file in *; do newfile=$( echo "$file" | tr -d \\n | sed 's/ //g' );
test "$file" != "$newfile" && mv "$file" "$newfile"; done
The tr is there to remove newlines in the filename, and is necessary to ensure that sed sees the entire filename in one line.