Want to change everything after security.server.ip=* with the result ip from the second grep.
First Grep:
cat admin.conf|grep security.server.ip|grep -v ^#
Result:
security.server.ip=10.10.1.2
Second Grep:
cat /etc/hosts|grep -i admin-server|head -1|awk '{ print $1}
Result:
10.10.1.2
Sometimes security.server.ip will be different on admin.conf and I'm wondering how to replace it with one command which will catch IP address form second grep and replace it in the first one.
You can use a script:
#!/bin/sh
IP=$(exec grep -i admin-server /etc/hosts | awk '{ print $1; exit }')
sed -i "/^security\.server\.ip=/s|=.*|=$IP|" admin.conf
You could save it in a variable:
NEWIP=`grep -i admin-server /etc/hosts|head -1|awk '{ print $1}'` \
sed -i "s/^security\.server\.ip=[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/security\.server\.ip=$NEWIP/" admin.conf
With GNU awk for inplace editing, nextfile, and gensub():
gawk -i inplace '
NR==FNR{ if (tolower($0) ~ /admin-server/) { ip=$1; nextfile } next }
{ $0=gensub(/(security\.server\.ip=).*/,"\\1"ip,""); print }
' /etc/hosts admin.conf
Related
I have a set of filenames in a directory, some of which are likely to have identical substrings but not known in advance. This is a sorting exercise. I want to move the files with the maximum substring ordered letter match together in a subdirectory named with that number of letters and progress to the minimum match until no matches of 2 or more letters remain. Ignore extensions. Case insensitive. Ignore special characters.
Example.
AfricanElephant.jpg
elephant.jpg
grant.png
ant.png
el_gordo.tif
snowbell.png
Starting from maximum length matches to minimum length matches will result in:
./8/AfricanElephant.jpg and ./8/elephant.jpg
./3/grant.png and ./3/ant.png
./2/snowbell.png and ./2/el_gordo.tif
Completely lost on an efficient bash or python way to do what seems a complex sort.
I found some awk code which is almost there:
{
count=0
while ( match($0,/elephant/) ) {
count++
$0=substr($0,RSTART+1)
}
print count
}
where temp.txt contains a list of the files and is invoked as eg
awk -f test_match.awk temp.txt
Drawback is that a) this is hardwired to look for "elephant" as a string (I don't know how to make it take an input string (rather than file) and an input test string to count against, and
b) I really just want to call a bash function to do the sort as specified
If I had this I could wrap some bash script around this core awk to make it work.
function longest_common_substrings () {
shopt -s nocasematch
for file1 in * ; do for file in * ; do \
if [[ -f "$file1" ]]; then
if [[ -f "$file" ]]; then
base1=$(basename "$file" | cut -d. -f1)
base2=$(basename "$file1" | cut -d. -f1)
if [[ "$file" == "$file1" ]]; then
echo -n ""
else
echo -n "$file $file1 " ; $HOME/Scripts/longest_common_substring.sh "$base1" "$base2" | tr -d '\n' | wc -c | awk '{$1=$1;print}' ;
fi
fi
fi
done ;
done | sort -r -k3 | awk '{ print $1, $3 }' > /tmp/filesort_substring.txt
while IFS= read -r line; do \
file_to_move=$(echo "$line" | awk '{ print $1 }') ;
directory_to_move_to=$(echo "$line" | awk '{ print $2 }') ;
if [[ -f "$file_to_move" ]]; then
mkdir -p "$directory_to_move_to"
\gmv -b "$file_to_move" "$directory_to_move_to"
fi
done < /tmp/filesort_substring.txt
shopt -u nocasematch
where $HOME/Scripts/longest_common_substring.sh is
#!/bin/bash
shopt -s nocasematch
if ((${#1}>${#2})); then
long=$1 short=$2
else
long=$2 short=$1
fi
lshort=${#short}
score=0
for ((i=0;i<lshort-score;++i)); do
for ((l=score+1;l<=lshort-i;++l)); do
sub=${short:i:l}
[[ $long != *$sub* ]] && break
subfound=$sub score=$l
done
done
if ((score)); then
echo "$subfound"
fi
shopt -u nocasematch
Kudos to the original solution for computing the match in the script which I found elsewhere in this site
I'm trying to turn a big list of data into a CSV. Its basically a giant list with no spaces, and the rows are separated by newlines. I have made a bash script that basically loops through the document, awks out the line, cuts the byte range, and then adds a comma and appends it to the end of the line. It looks like this:
awk -v n=$x 'NR==n { print;exit}' PROP.txt | cut -c 1-12 | tr -d '\n' >> $x.tmp
awk -v n=$x 'NR==n { print;exit}' PROP.txt | cut -c 13-17 | tr -d '\n' | xargs -I {} sed -i '' -e 's~$~,{}~' $x.tmp
awk -v n=$x 'NR==n { print;exit}' PROP.txt | cut -c 18-22 | tr -d '\n' | xargs -I {} sed -i '' -e 's~$~,{}~' $x.tmp
awk -v n=$x 'NR==n { print;exit}' PROP.txt | cut -c 23-34 | tr -d '\n' | xargs -I {} sed -i '' -e 's~$~,{}~' $x.tmp
The problem is this is EXTREMELY slow, and the data has about 400k rows. I know there must be a better way to accomplish this. Essentially I just need to add a comma after every 12/17/22/34 etc character of a line.
Any help is appreciated, thank you!
There are many many ways to do this with Perl. Here is one way:
perl -pe 's/(.{12})(.{5})(.{5})(.{12})/$1,$2,$3,$4,/' < input-file > output-file
The matching pattern in the substitution captures four groups of text from the beginning of each line with 12, 5, 5, and 12 arbitrary characters. The replacement pattern places a comma after each group.
With GNU awk, you could write
gawk 'BEGIN {FIELDWIDTHS="12 5 5 12"; OFS=","} {$1=$1; print}'
The $1=$1 part is to force awk to rewrite the like, incorporating the output field separator, without changing anything.
This is very much a job for substr.
use strict;
use warnings;
my #widths = (12, 5, 5, 12);
my $offset;
while (my $line = <DATA>) {
for my $width (#widths) {
$offset += $width;
substr $line, $offset, 0, ',';
++$offset;
}
print $line;
}
__DATA__
1234567890123456789012345678901234567890
output
123456789012,34567,89012,345678901234,567890
I'm very new to some of the command line utilities and have been looking for a while for a command that would accomplish my goal.
The goal is to find files that contain a string of text, replace it with a new string, and then write the results to a file that is named the same as the original, but in a different directory.
Obviously this is not working, so I am asking how you who know about this stuff would go about it.
grep -rl 'stringToFind' *.* | sed 's|oldString|newString|g' < fileNameFromGrep > ./new/fileNameFromGrep
Thanks for your input!
John
for f in "`find /YOUR/SEARCH/DIR/ROOT -type f -exec fgrep -l 'stirngToFind' \{\} \;`" ; do
sed 's|oldString|newString|g' < "${f} > ./new/"${f}
done
Will do it for you.
If you have spaces in filenames:
OLDIFS=$IFS
IFS=''
find /PATH -print0 -type f | while read -r -d $'' file
do
fgrep -l 'stirngToFind' "$file" && \
sed 's|oldString|newString|g' < "${file} > ./new/"${file}
done
IFS=$OLDIFS
#!/bin/bash
for file in *; do
if grep -qF 'stringToFind' "$file"; then
sed 's/oldString/newString/g' "$file" > "./new/$file"
fi
done
for file in path/to/dir/*
do
grep -q 'pattern' "$file" > /dev/null
if [ $? == 0 ]; then
sed 's/oldString/newString/g' "$file" > /path/to/newdir/"$file"
fi
done
You try:
sed -ie "s/oldString/newString/g" \
$(grep -Rsi 'pattern' path/to/dir/ | cut -d: -f1)
sed:
i in_place
e exec other command or script
grep:
R recursive
s Suppress error messages
i ignore case sensitive
I have a string ABCD20110420.txt and I want to extract the date out of it. Expected 2011-04-20
I can use replace to remove the text part, but how do I insert the "-" ?
# echo "ABCD20110420.txt" | replace 'ABCD' '' | replace '.txt' ''
20110420
echo "ABCD20110420.txt" | sed -e 's/ABCD//' -e 's/.txt//' -e 's/\(....\)\(..\)\(..\)/\1-\2-\3/'
Read: sed FAQ
Just use the shell (bash)
$> file=ABCD20110420.txt
$> echo "${file//[^0-9]/}"
20110420
$> file="${file//[^0-9]/}"
$> echo $file
20110420
$> echo ${file:0:4}-${file:4:2}-${file:6:2}
2011-04-20
The above is applicable to files like your sample. If you have files like A1BCD20110420.txt, then will not work.
For that case,
$> file=A1BCD20110420.txt
$> echo ${file%.*} #get rid of .txt
A1BCD20110420
$> file=${file%.*}
$> echo "2011${file#*2011}"
20110420
Or you can use regular expression (Bash 3.2+)
$> file=ABCD20110420.txt
$> [[ $file =~ ^.*(2011)([0-9][0-9])([0-9][0-9])\.*$ ]]
$> echo ${BASH_REMATCH[1]}
2011
$> echo ${BASH_REMATCH[2]}
04
$> echo ${BASH_REMATCH[3]}
20
echo "ABCD20110420.txt" | sed -r 's/.+([0-9]{4})([0-9]{2})([0-9]{2}).+/\1-\2-\3/'
$ file=ABCD20110420.txt
$ echo "$file" | sed -e 's/^[A-Za-z]*\([0-9][0-9][0-9][0-9]\)\([0-9][0-9]\)\([0-9][0-9]\)\.txt$/\1-\2-\3/'
This only requires a single call to sed.
echo "ABCD20110420.txt" | sed -r 's/.{4}(.{4})(.{2})(.{2}).txt/\1-\2-\3/'
I want to create a var from the section names of an ini file like:
[foo]
; ...
[bar]
; ...
[baz:bar]
;...
now I need a var like
SECTIONS="foo bar baz"
thanks in advance
One line solution could be:
export SECTIONS=`grep "^\[" test.ini |sort -u | xargs | tr '\[' ' ' | tr '\]' ' ' `
SECTIONS=$(crudini --get your.ini | sed 's/:.*//')
I'm now using this construct, don't need to know if a section exists. just read it, if it's empty it does not exist.
INI_FILE=test.ini
function ini_get
{
eval `sed -e 's/[[:space:]]*\=[[:space:]]*/=/g' \
-e 's/;.*$//' \
-e 's/[[:space:]]*$//' \
-e 's/^[[:space:]]*//' \
-e "s/^\(.*\)=\([^\"']*\)$/\1=\"\2\"/" \
< $INI_FILE \
| sed -n -e "/^\[$1\]/,/^\s*\[/{/^[^;].*\=.*/p;}"
echo ${!2}
}
IP=$(ini_get 50001 ip)
PORT=$(ini_get 50001 port)
echo $IP:$PORT