problems while reading log file with tail -n0 -F - sed

i am monitoring the asterisk log file for peers that get offline.
the if part is working correct, but the sed command is not executed in the else part, although the echo command works. What do i need to change
tail -n0 -F /var/log/asterisk/messages | \
while read LINE
do
if echo "$LINE" | /bin/grep -q "is now UNREACHABLE!"
then
EXTEN=$(echo $LINE | /bin/grep -o -P "(?<=\').*(?=\')")
echo "$EXTEN is now UNREACHABLE!"
CALLERID=$(/bin/sed -n '/^\['"$EXTEN"'\]/,/^\[.*\]/{/^callerid*/p}' "$SIP" | /usr/bin/awk -F'=' '{ print $2 }')
if .......
then
.......
fi
elif echo "$LINE" | /bin/grep -q "is now REACHABLE!"
then
EXTEN=$(echo $LINE | /bin/grep -o -P "(?<=\').*(?=\')")
echo "$EXTEN is now REACHABLE!"
if /bin/grep -qi "^$EXTEN;" $OFFLINE; then
/bin/sed -i '/^$EXTEN;/d' $OFFLINE
fi
fi
done

You have a quoting problem - you've used single quotes when the string includes a shell variable:
if /bin/grep -qi "^$EXTEN;" $OFFLINE; then
/bin/sed -i '/^$EXTEN;/d' $OFFLINE
fi
Try using double quotes instead:
if /bin/grep -qi "^$EXTEN;" $OFFLINE; then
/bin/sed -i "/^$EXTEN;/d" $OFFLINE
fi

Related

Bash or Python efficient substring matching and filtering

I have a set of filenames in a directory, some of which are likely to have identical substrings but not known in advance. This is a sorting exercise. I want to move the files with the maximum substring ordered letter match together in a subdirectory named with that number of letters and progress to the minimum match until no matches of 2 or more letters remain. Ignore extensions. Case insensitive. Ignore special characters.
Example.
AfricanElephant.jpg
elephant.jpg
grant.png
ant.png
el_gordo.tif
snowbell.png
Starting from maximum length matches to minimum length matches will result in:
./8/AfricanElephant.jpg and ./8/elephant.jpg
./3/grant.png and ./3/ant.png
./2/snowbell.png and ./2/el_gordo.tif
Completely lost on an efficient bash or python way to do what seems a complex sort.
I found some awk code which is almost there:
{
count=0
while ( match($0,/elephant/) ) {
count++
$0=substr($0,RSTART+1)
}
print count
}
where temp.txt contains a list of the files and is invoked as eg
awk -f test_match.awk temp.txt
Drawback is that a) this is hardwired to look for "elephant" as a string (I don't know how to make it take an input string (rather than file) and an input test string to count against, and
b) I really just want to call a bash function to do the sort as specified
If I had this I could wrap some bash script around this core awk to make it work.
function longest_common_substrings () {
shopt -s nocasematch
for file1 in * ; do for file in * ; do \
if [[ -f "$file1" ]]; then
if [[ -f "$file" ]]; then
base1=$(basename "$file" | cut -d. -f1)
base2=$(basename "$file1" | cut -d. -f1)
if [[ "$file" == "$file1" ]]; then
echo -n ""
else
echo -n "$file $file1 " ; $HOME/Scripts/longest_common_substring.sh "$base1" "$base2" | tr -d '\n' | wc -c | awk '{$1=$1;print}' ;
fi
fi
fi
done ;
done | sort -r -k3 | awk '{ print $1, $3 }' > /tmp/filesort_substring.txt
while IFS= read -r line; do \
file_to_move=$(echo "$line" | awk '{ print $1 }') ;
directory_to_move_to=$(echo "$line" | awk '{ print $2 }') ;
if [[ -f "$file_to_move" ]]; then
mkdir -p "$directory_to_move_to"
\gmv -b "$file_to_move" "$directory_to_move_to"
fi
done < /tmp/filesort_substring.txt
shopt -u nocasematch
where $HOME/Scripts/longest_common_substring.sh is
#!/bin/bash
shopt -s nocasematch
if ((${#1}>${#2})); then
long=$1 short=$2
else
long=$2 short=$1
fi
lshort=${#short}
score=0
for ((i=0;i<lshort-score;++i)); do
for ((l=score+1;l<=lshort-i;++l)); do
sub=${short:i:l}
[[ $long != *$sub* ]] && break
subfound=$sub score=$l
done
done
if ((score)); then
echo "$subfound"
fi
shopt -u nocasematch
Kudos to the original solution for computing the match in the script which I found elsewhere in this site

jboss-fuse-6.3.0.redhat-187 Fabric ssh script Error - Command not found: function

Created a Fabric Root
started ./fuse and then executed bellow command
fabric:create --wait-for-provisioning --verbose --clean --new-user admin --new-user-role admin --new-user-password admin --zookeeper-password zoopassword --resolver manualip --manual-ip 127.0.0.1
Started another fuse server ./fuse with ssh listening on 8102
Executed bellow command in the root console(fuse running at step 1 as root)
fabric:container-create-ssh --host 127.0.0.1 --user admin --password admin --port 8102 --new-user admin --new-user-password admin --resolver manualip --manual-ip 127.0.0.1 mqgateway
Then getting this Error :
--- command ---
#!/bin/bash function run { echo "Running: $*" ; $* ; rc=$? ; if [ "${rc}" -ne 0 ]; then echo "Command Failed:Error running installation script: $*" ; exit ${rc} ; fi ; }
function sudo_n { SUDO_NON_INTERACTIVE=`sudo -h | grep "\-n"` if [
-z "$SUDO_NON_INTERACTIVE" ]; then
sudo $* else
sudo -n $* fi }
function download { echo "Downloading: $1"; ret=`curl -C - --retry 10 --write-out %{http_code} --silent --output $2 $1`; if [ "${ret}"
-ne 200 ]; then
echo "Download failed with code: ${ret}";
rm $2; fi; }
function maven_download { echo "Downloading Maven Artifact with groupId: $2 artifactId: $3 and version: $4 from repository: $1"; export REPO=$1 export GROUP_ID=$2 export ARTIFACT_ID=$3 export VERSION=$4 export TYPE=$5 export TARGET_FILE=$ARTIFACT_ID-$VERSION.$TYPE
export GROUP_ID_PATH=`echo $GROUP_ID | sed 's/\./\//g'`
export ARTIFACT_BASE_URL=`echo $REPO$GROUP_ID_PATH/$ARTIFACT_ID/$VERSION/`
if [[ "$VERSION" == *SNAPSHOT* ]]; then
export ARTIFACT_URL=`curl --location -C - --retry 10 --silent $ARTIFACT_BASE_URL | grep href | grep zip\" | sed 's/^.*<a href="//' | sed 's/".*$//' | tail -1` else
export ARTIFACT_URL=`echo $REPO$GROUP_ID_PATH/$ARTIFACT_ID/$VERSION/$ARTIFACT_ID-$VERSION.$TYPE` fi
if [ -z "$ARTIFACT_URL" ]; then
export ARTIFACT_URL=`echo $REPO$GROUP_ID_PATH/$ARTIFACT_ID/$VERSION/$ARTIFACT_ID-$VERSION.$TYPE` fi
echo "Using URL: $ARTIFACT_URL" ret=`curl --location --write-out %{http_code} --silent --output $TARGET_FILE $ARTIFACT_URL` if [ "${ret}" -ne 200 ]; then
echo "Download failed with code: ${ret}"
rm $TARGET_FILE fi }
function update_pkgs() { if which dpkg &> /dev/null; then
sudo_n apt-get update elif which rpm &> /dev/null; then
sudo_n yum check-update fi }
function install_curl() { echo "Checking if curl is present." if which curl &> /dev/null; then
echo "Curl is already installed." else
echo "Installing curl."
if which dpkg &> /dev/null; then
sudo_n apt-get -y install curl
elif which rpm &> /dev/null; then
sudo_n yum -y install curl
fi fi }
function install_unzip() { echo "Checking if unzip is present." if which unzip &> /dev/null; then
echo "Unzip is already installed." else
echo "Installing unzip."
if which dpkg &> /dev/null; then
sudo_n apt-get -y install unzip
elif which rpm &> /dev/null; then
sudo_n yum -y install unzip
fi fi }
function install_openjdk_deb() { sudo_n apt-get -y install openjdk-7-jdk
# Try to set JAVA_HOME in a number of commonly used locations # Lifting JAVA_HOME detection from jclouds
for CANDIDATE in `ls -d /usr/lib/jvm/java-1.7.0-openjdk-* /usr/lib/jvm/java-7-openjdk-* /usr/lib/jvm/java-7-openjdk 2>&-`; do
if [ -n "$CANDIDATE" -a -x "$CANDIDATE/bin/java" ]; then
export JAVA_HOME=$CANDIDATE
break
fi
done
if [ -f /etc/profile ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> /etc/profile" fi if [ -f /etc/bashrc ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> /etc/bashrc" fi if [ -f ~root/.bashrc ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> ~root/.bashrc" fi if [ -f /etc/skel/.bashrc ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> /etc/skel/.bashrc" fi if [ -f "$DEFAULT_HOME/$NEW_USER" ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> $DEFAULT_HOME/$NEW_USER" fi
sudo_n update-alternatives --install /usr/bin/java java $JAVA_HOME/bin/java 17000 sudo_n update-alternatives --set java $JAVA_HOME/bin/java java -version }
function install_openjdk_rpm() { sudo_n yum -y install java-1.7.0-openjdk-devel
# Try to set JAVA_HOME in a number of commonly used locations # Lifting JAVA_HOME detection from jclouds
for CANDIDATE in `ls -d /usr/lib/jvm/java-1.7.0-openjdk-* /usr/lib/jvm/java-7-openjdk-* /usr/lib/jvm/java-7-openjdk 2>&-`; do
if [ -n "$CANDIDATE" -a -x "$CANDIDATE/bin/java" ]; then
export JAVA_HOME=$CANDIDATE
break
fi
done
if [ -f /etc/profile ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> /etc/profile" fi if [ -f /etc/bashrc ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> /etc/bashrc" fi if [ -f ~root/.bashrc ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> ~root/.bashrc" fi if [ -f /etc/skel/.bashrc ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> /etc/skel/.bashrc" fi if [ -f "$DEFAULT_HOME/$NEW_USER" ]; then
sudo_n "echo 'export JAVA_HOME=$JAVA_HOME' >> $DEFAULT_HOME/$NEW_USER" fi
sudo_n alternatives --install /usr/bin/java java $JAVA_HOME/bin/java 17000 sudo_n alternatives --set java $JAVA_HOME/bin/java java
-version }
function install_openjdk() {
echo "Checking if java is present."
ARCH=`uname -m`
JAVA_VERSION=`java -version 2>&1`
if [[ $JAVA_VERSION == *1.7* ]]; then
echo "Java is already installed."
else
echo "Installing java."
if which dpkg &> /dev/null; then
install_openjdk_deb
elif which rpm &> /dev/null; then
install_openjdk_rpm
fi
fi }
function validate_requirements() { if ! which curl &> /dev/null; then
echo "Command Failed:Curl is not installed."; fi if ! which java &> /dev/null; then
echo "Command Failed:Java is not installed.";
exit -1; else
check_java_version fi }
function check_java_version() { JAVA_VERSION=`java -version 2>&1 | grep "[java|openjdk] version" | awk '{print $3}' | tr -d \" | awk '{split($0, array, ".")} END{print array[2]}'` if [ $JAVA_VERSION
-ge 6 ]; then
echo "Java version is greater than 1.6." else
echo "Command Failed:Unsupported java version: 1.$JAVA_VERSION.x found."
exit -1; fi }
function exit_if_not_exists() { if [ ! -f $1 ]; then
echo "Command Failed:Could not find file $1";
exit -1; fi local zipFile="$1" local size="$(du $zipFile | awk '{ print $1}')" if [ $size -lt 100 ]; then
echo "Command Failed: Zip archive is empty. Check $1";
exit -1; fi
}
function copy_node_metadata() { echo "Copying metadata for container: $1"; TARGET_PATH="./fabric/import/fabric/registry/containers/config/$1/" mkdir -p $TARGET_PATH ENCODED_METADATA=$2 echo $ENCODED_METADATA > ./fabric/import/fabric/registry/containers/config/$1/metadata.cfg }
function karaf_check() { KARAF_HOME=$1 INSTANCES_FILE=$KARAF_HOME/instances/instance.properties for i in {1..5};
do
if [ ! -f $INSTANCES_FILE ]; then
sleep 1
else
break
fi
done if [ -f $INSTANCES_FILE ]; then
for j in {1..5};
do
PID=`cat $INSTANCES_FILE | grep "item.0.pid" | awk -F "=" '{print $2}'`
if [ "$PID" = "" ]; then
sleep 1
else
break
fi
done
if ps -p $PID > /dev/null; then
echo "Fabric is started successfully"
else
echo "Command Failed: Karaf process ($PID) is not running"
fi else
echo "Command Failed:Could not find Karaf instance.properties" fi }
function replace_in_file { sed "s/$1/$2/g" $3 > $3.tmp rm $3 mv $3.tmp $3 }
function replace_property_value { echo "Setting value $2 for key $1 in $3" sed "s/$1[ \t]*=.*/$1 = $2/g" $3 > $3.tmp rm $3 mv $3.tmp $3 }
function configure_hostnames() { CLOUD_PROVIDER=$1 case $CLOUD_PROVIDER in
openstack-nova | ec2 | aws-ec2 )
echo "Resolving public hostname for ec2 node"
export PUBLIC_HOSTNAME=`curl http://169.254.169.254/latest/meta-data/public-hostname | sed 's/ /_/g'`
echo PUBLIC_HOSTNAME
;;
cloudservers | cloudservers-uk | cloudservers-us )
echo "Resovling public hostname for rackspace node"
PRIVATE_IP=`/sbin/ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}'`
export PUBLIC_HOSTNAME=`echo $PRIVATE_IP | tr . -`.static.cloud-ips.com
;; esac if [ ! -z ${PUBLIC_HOSTNAME} ]; then
LOOKUP_ADDRESS=`nslookup $PUBLIC_HOSTNAME > /dev/null | grep Address | tail -n 1 | cut -d " " -f 3 | sed 's/ /_/g'`
echo "Found hostname: $PUBLIC_HOSTNAME matching with address: $LOOKUP_ADDRESS"
echo "publichostname=$PUBLIC_HOSTNAME" >> etc/system.properties
cat etc/system.properties | grep -v 'local.resolver=' | grep -v 'global.resolver=' > etc/system.properties.tmp
mv etc/system.properties.tmp etc/system.properties
echo "local.resolver=publichostname" >> etc/system.properties
echo "global.resolver=publichostname" >> etc/system.properties
echo $PUBLIC_HOSTNAME > hostname
sudo_n cp hostname /etc/
export JAVA_OPTS="-Djava.rmi.server.hostname=$PUBLIC_HOSTNAME $JAVA_OPTS"
echo "RESOLVER OVERRIDE:publichostname" fi }
function find_free_port() { START_PORT=$1 END_PORT=$2 for port in `eval echo {$START_PORT..$END_PORT}`;do
if [[ $OSTYPE == darwin* ]]; then
# macosx has a different syntax for netstat
netstat -atp tcp | tr -s ' ' ' '| cut -d ' ' -f 4 | grep ":$port" > /dev/null 2>&1 && continue || echo $port && break;
else
netstat -utan | tr -s ' ' ' '| cut -d ' ' -f 4 | grep ":$port" > /dev/null 2>&1 && continue || echo $port && break;
fi done }
function wait_for_port() {
PORT=$1
for i in {1..5};
do
if [[ $OSTYPE == darwin* ]]; then
# macosx has a different syntax for netstat
netstat -an -ptcp | grep LISTEN | tr -s ' ' ' '| cut -d ' ' -f 4 | grep ":$PORT" > /dev/null 2>&1 && break;
else
netstat -lnt | tr -s ' ' ' '| cut -d ' ' -f 4 | grep ":$PORT" > /dev/null 2>&1 && break;
fi
sleep 5;
done
return 0 }
function extract_zip { if ! which unzip &> /dev/null; then
jar xf $1 else
unzip -o $1 fi }
function generate_ssh_keys { if [ ! -f ~/.ssh/id_rsa ]; then
mkdir -p ~/.ssh
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa fi }
run mkdir -p ~/containers/ run cd ~/containers/ run mkdir -p mqgateway run cd mqgateway validate_requirements cp /tmp/fabric8-karaf-1.2.0.redhat-630187.zip fabric8-karaf-1.2.0.redhat-630187.zip if ! jar xf fabric8-karaf-1.2.0.redhat-630187.zip &> /dev/null; then rm fabric8-karaf-1.2.0.redhat-630187.zip ; fi if [ ! -f fabric8-karaf-1.2.0.redhat-630187.zip ] && [ ! -s fabric8-karaf-1.2.0.redhat-630187.zip ] ; then maven_download http://127.0.0.1:8181/maven/download/ io.fabric8 fabric8-karaf
1.2.0.redhat-630187 zip exit_if_not_exists fabric8-karaf-1.2.0.redhat-630187.zip run extract_zip fabric8-karaf-1.2.0.redhat-630187.zip run cd `ls -l | grep fabric8-karaf | grep ^d | awk '{ print $NF }' | sort -n | head -1` run mkdir -p system/io/fabric8/fabric8-karaf/1.2.0.redhat-630187 run cp ../fabric8-karaf-1.2.0.redhat-630187.zip system/io/fabric8/fabric8-karaf/1.2.0.redhat-630187/ run rm ../fabric8-karaf-1.2.0.redhat-630187.zip run chmod +x bin/* cat >> etc/system.properties <<'END_OF_FILE' global.resolver=localhostname END_OF_FILE replace_property_value "karaf.name" "mqgateway" etc/system.properties replace_property_value "importDir" "fabric" etc/io.fabric8.datastore.cfg replace_property_value "felix.fileinstall.filename" "file:\/C:\/Fabric\/root\/etc\/io.fabric8.datastore.cfg" etc/io.fabric8.datastore.cfg replace_property_value "component.name" "io.fabric8.datastore" etc/io.fabric8.datastore.cfg replace_property_value "gitRemotePollInterval" "60000" etc/io.fabric8.datastore.cfg replace_property_value "service.pid" "io.fabric8.datastore" etc/io.fabric8.datastore.cfg BIND_ADDRESS=0.0.0.0 SSH_PORT="`find_free_port 8101 65535`" RMI_REGISTRY_PORT="`find_free_port 1099 65535`" RMI_SERVER_PORT="`find_free_port 44444 65535`" JMX_SERVER_URL="service:jmx:rmi:\/\/${BIND_ADDRESS}:${RMI_SERVER_PORT}\/jndi\/rmi:\/\/${BIND_ADDRESS}:${RMI_REGISTRY_PORT}\/karaf-mqgateway" HTTP_PORT="`find_free_port 8181 65535`" replace_property_value "sshPort" "$SSH_PORT" etc/org.apache.karaf.shell.cfg replace_property_value "sshHost" "$BIND_ADDRESS" etc/org.apache.karaf.shell.cfg replace_property_value "rmiRegistryPort" "$RMI_REGISTRY_PORT" etc/org.apache.karaf.management.cfg replace_property_value "rmiServerPort" "$RMI_SERVER_PORT" etc/org.apache.karaf.management.cfg replace_property_value "rmiServerHost" "$BIND_ADDRESS" etc/org.apache.karaf.management.cfg replace_property_value "rmiRegistryHost" "$BIND_ADDRESS" etc/org.apache.karaf.management.cfg replace_property_value "org.osgi.service.http.port" "$HTTP_PORT" etc/org.ops4j.pax.web.cfg replace_in_file "8181" "$HTTP_PORT" etc/jetty.xml cat >> etc/system.properties <<'END_OF_FILE' minimum.port=0 END_OF_FILE cat >> etc/system.properties <<'END_OF_FILE' maximum.port=65535 END_OF_FILE cat >> etc/system.properties <<'END_OF_FILE'
END_OF_FILE cat >> etc/system.properties <<'END_OF_FILE' preferred.network.address=127.0.0.1 END_OF_FILE cat >> etc/system.properties <<'END_OF_FILE' zookeeper.url = 127.0.0.1:2181 END_OF_FILE cat >> etc/system.properties <<'END_OF_FILE' zookeeper.password = ZKENC=em9vcGFzc3dvcmQ= END_OF_FILE cat >> etc/system.properties <<'END_OF_FILE' zookeeper.password.encode = true END_OF_FILE cat >> etc/system.properties <<'END_OF_FILE' agent.auto.start=true END_OF_FILE sed 's/featuresBoot=/&fabric-agent,fabric-git,/' etc/org.apache.karaf.features.cfg > etc/org.apache.karaf.features.cfg.tmp mv etc/org.apache.karaf.features.cfg.tmp etc/org.apache.karaf.features.cfg sed 's/repositories=/&http:\/\/127.0.0.1:8181\/maven\/download\/,/' etc/org.ops4j.pax.url.mvn.cfg > etc/org.ops4j.pax.url.mvn.cfg.tmp mv etc/org.ops4j.pax.url.mvn.cfg.tmp etc/org.ops4j.pax.url.mvn.cfg generate_ssh_keys configure_hostnames none cat > bin/setenv <<'END_OF_FILE' export JAVA_OPTS=" -XX:+UnlockDiagnosticVMOptions
-XX:+UnsyncloadClass -server"
END_OF_FILE nohup bin/start & karaf_check `pwd` wait_for_port $SSH_PORT wait_for_port $RMI_REGISTRY_PORT
--- output --- Command not found: function
--- error ---
------
SSH containers should be created on Remote machines.An SSH container is just a Fabric container that is running on a remote host on your local network,
where that host is accessible through the SSH protocol. This section describes some basic administration tasks for these SSH containers.
Please refer https://access.redhat.com/documentation/en-us/red_hat_jboss_fuse/6.3/html/fabric_guide/chapter-fabric_container#ContSSH
If you want to create containers on the same machine (local host) create child containers
https://access.redhat.com/documentation/en-us/red_hat_jboss_fuse/6.3/html/fabric_guide/chapter-fabric_container#ContChild
Creating ssh container
fabric:container-create-ssh --host remotehost --user user1 --password pass1 --jvm-opts="-Djava.rmi.server.hostname=remotehost" --profile fabric fabric-eu03

Is there a way to achieve the same in "gnu parallel" with less nested quoting?

Although this script works, I am not happy about the excess of nested quoting, which makes the script very fragile and difficult to modify.
The script consists basically of parallel followed by a filter test argument followed by command arguments.
#!/bin/sh
uhrzeit_0=$(date --rfc-3339=ns) ; uhrzeit="$uhrzeit_0 $(echo "$uhrzeit_0" | sha512sum)" ;
find ./* -maxdepth 30 -type f -print0 | parallel --null \
"mimetype --output-format %m {} | grep video > /dev/null \
&& echo -en '\nSeparator: ' && echo -e $uhrzeit \
&& echo -n 'md5sum: ' && md5sum {} && echo -n 'du -h: ' \
&& du -h {} && ffmpeg -loglevel quiet -i {} -f ffmetadata - " > Film_metadata.txt
#!/bin/bash
uhrzeit_0=$(date --rfc-3339=ns)
uhrzeit="$uhrzeit_0 $(echo "$uhrzeit_0" | sha512sum)"
doit() {
F="$1"
mimetype --output-format %m "$F" | grep video > /dev/null &&
echo -en '\nSeparator: ' &&
echo -e $uhrzeit &&
echo -n 'md5sum: ' &&
md5sum "$F" &&
echo -n 'du -h: ' &&
du -h "$F" &&
ffmpeg -loglevel quiet -i "$F" -f ffmetadata -
}
export -f doit
find ./* -maxdepth 30 -type f -print0 |
parallel --null doit > Film_metadata.txt

Using grep with sed and writing a new file based on the results

I'm very new to some of the command line utilities and have been looking for a while for a command that would accomplish my goal.
The goal is to find files that contain a string of text, replace it with a new string, and then write the results to a file that is named the same as the original, but in a different directory.
Obviously this is not working, so I am asking how you who know about this stuff would go about it.
grep -rl 'stringToFind' *.* | sed 's|oldString|newString|g' < fileNameFromGrep > ./new/fileNameFromGrep
Thanks for your input!
John
for f in "`find /YOUR/SEARCH/DIR/ROOT -type f -exec fgrep -l 'stirngToFind' \{\} \;`" ; do
sed 's|oldString|newString|g' < "${f} > ./new/"${f}
done
Will do it for you.
If you have spaces in filenames:
OLDIFS=$IFS
IFS=''
find /PATH -print0 -type f | while read -r -d $'' file
do
fgrep -l 'stirngToFind' "$file" && \
sed 's|oldString|newString|g' < "${file} > ./new/"${file}
done
IFS=$OLDIFS
#!/bin/bash
for file in *; do
if grep -qF 'stringToFind' "$file"; then
sed 's/oldString/newString/g' "$file" > "./new/$file"
fi
done
for file in path/to/dir/*
do
grep -q 'pattern' "$file" > /dev/null
if [ $? == 0 ]; then
sed 's/oldString/newString/g' "$file" > /path/to/newdir/"$file"
fi
done
You try:
sed -ie "s/oldString/newString/g" \
$(grep -Rsi 'pattern' path/to/dir/ | cut -d: -f1)
sed:
i in_place
e exec other command or script
grep:
R recursive
s Suppress error messages
i ignore case sensitive

extract number from string

I have a string ABCD20110420.txt and I want to extract the date out of it. Expected 2011-04-20
I can use replace to remove the text part, but how do I insert the "-" ?
# echo "ABCD20110420.txt" | replace 'ABCD' '' | replace '.txt' ''
20110420
echo "ABCD20110420.txt" | sed -e 's/ABCD//' -e 's/.txt//' -e 's/\(....\)\(..\)\(..\)/\1-\2-\3/'
Read: sed FAQ
Just use the shell (bash)
$> file=ABCD20110420.txt
$> echo "${file//[^0-9]/}"
20110420
$> file="${file//[^0-9]/}"
$> echo $file
20110420
$> echo ${file:0:4}-${file:4:2}-${file:6:2}
2011-04-20
The above is applicable to files like your sample. If you have files like A1BCD20110420.txt, then will not work.
For that case,
$> file=A1BCD20110420.txt
$> echo ${file%.*} #get rid of .txt
A1BCD20110420
$> file=${file%.*}
$> echo "2011${file#*2011}"
20110420
Or you can use regular expression (Bash 3.2+)
$> file=ABCD20110420.txt
$> [[ $file =~ ^.*(2011)([0-9][0-9])([0-9][0-9])\.*$ ]]
$> echo ${BASH_REMATCH[1]}
2011
$> echo ${BASH_REMATCH[2]}
04
$> echo ${BASH_REMATCH[3]}
20
echo "ABCD20110420.txt" | sed -r 's/.+([0-9]{4})([0-9]{2})([0-9]{2}).+/\1-\2-\3/'
$ file=ABCD20110420.txt
$ echo "$file" | sed -e 's/^[A-Za-z]*\([0-9][0-9][0-9][0-9]\)\([0-9][0-9]\)\([0-9][0-9]\)\.txt$/\1-\2-\3/'
This only requires a single call to sed.
echo "ABCD20110420.txt" | sed -r 's/.{4}(.{4})(.{2})(.{2}).txt/\1-\2-\3/'