I want to use sed to do this. I have 2 files:
keys.txt:
host1
host2
test.txt
host1 abc
host2 cdf
host3 abaasdf
I want to use sed to remove any lines in test.txt that contains the keyword in keys.txt. So the result of test.txt should be
host3 abaasdf
Can somebody show me how to do that with sed?
Thanks,
I'd recommend using grep for this (especially fgrep since there are no regexps involved), so
fgrep -v -f keys.txt test.txt
does it fine. With sed quickly this works:
sed -i.ORIGKEYS.txt ^-e 's_^_/_' -e 's_$_/d_' keys.txt
sed -f keys.txt test.txt
(This modifies the original keys.txt in place - with backup - to a sourceable sed script.)
fgrep -v -f is the best solution. Here are a couple of alternatives:
A combination of comm and join
comm -13 <(join keys.txt test.txt) test.txt
or awk
awk 'NR==FNR {key[$1]; next} $1 in key {next} 1' keys.txt test.txt
This might work for you (GNU sed):
sed 's|.*|/^&\\>/d|' keys.txt | sed -f - test.txt
Related
can please somebody help me with this?
I have this line
test.txt
siemplog1.nw.lan / 172.31.180.22
I tried this command sed -Ei "s/^[a-z A-Z].*([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/" test.txt
result should be 172.31.180.22 but I got this 2.31.180.22
thank you
The .* matches as many chars as it can (it is "greedy") and since [0-9]{1,3} can match just 1 digit, the 17 is matched by the .* and 2 is matched by [0-9]{1,3}.
You may stop the .* before any non-digit:
sed -Ei 's~.*[^0-9]([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*~\1~' test.txt
Or, before /:
sed -Ei 's~.*/ *([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*~\1~' test.txt
See online sed demo:
s='siemplog1.nw.lan / 172.31.180.22'
sed -E 's~.*/ *([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*~\1~' <<< "$s"
# => 172.31.180.22
If you string is always in this format, you might simplify the sed command to
sed -E 's~.*/ *([0-9.]+)~\1~p'
sed -E 's~.*/ *([0-9.]+).*~\1~p'
If you have space before ip
$echo siemplog1.nw.lan / 172.31.180.22 | sed -E "s/.* ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/"
172.31.180.22
$
I have this list of files:
$ more files
one_this_2017_1_abc.txt
two_that_2018_1_abc.txt
three_another_2017_10.abc.txt
four_again_2018_10.abc.txt
five_back_2018_1a.abc.txt
I would like to get this output:
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_YY.abc.txt
four_again_XXXX_YY.abc.txt
five_back_XXXX_YY.abc.txt
I am trying to remove the year and the bit after the year and replace them with another string--this is to generate test cases.
I can get the year just fine, but it's that one or two character piece after it I can't seem to match.
This should work, right?
~/test_cases
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\{1,2\}_/_YY_/'
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Except it doesn't for the 2 character cases.
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\
{2\}_/_YY_/'
one_this_XXXX_1_abc.txt
two_that_XXXX_1_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Doesn't work for the two character cases either, and this works not at all (but according to the docs it should):
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\+_/_YY_/'
one_YY_XXXX_1_abc.txt
two_YY_XXXX_1_abc.txt
three_YY_XXXX_10.abc.txt
four_YY_XXXX_10.abc.txt
five_YY_XXXX_1a.abc.txt
Other random experiments that don't work:
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\+_/_YY_/'
one_YY_XXXX_1_abc.txt
two_YY_XXXX_1_abc.txt
three_YY_XXXX_10.abc.txt
four_YY_XXXX_10.abc.txt
five_YY_XXXX_1a.abc.txt
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\{1\}_/_YY_/'
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\{2\}_/_YY_/'
one_this_XXXX_1_abc.txt
two_that_XXXX_1_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Tried with both GNU sed version 4.2.1 under Linux and sed (GNU sed) 4.4 under Cygwin.
And yes, I realize I can pipe this through multiple sed calls to get it to work, but that regex SHOULD work, right?
if your Input_file is same as shown sample then following may help you in same.
sed 's/\([^_]*\)_\([^_]*\)_\(.*_\)\(.*\)/\1_\2_XXXX_YY_\4/g' Input_file
Output will be as follows.
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_YY_10.abc.txt
four_again_XXXX_YY_10.abc.txt
five_back_XXXX_YY_1a.abc.txt
thanks in advance for the help.
I have the following line that does work on linux.
myfile (extract)
active_instance_count=
aq_tm_processes=1
archive_lag_target=0
audit_file_dest=?/rdbms/audit
audit_sys_operations=FALSE
audit_trail=NONE
background_core_dump=partial
background_dump_dest=/home1/oracle/app/oracle/admin/iopecom/bdump
...
cat myfile |sed -r 's/ {1,}//g'|sed -r 's/\t*//g' |grep -v "^#"|sed -s "/^$/d" |sed =|sed 'N;s/\n/\t/'|sed -r "s/#.*//g" | sed "s/\t/;/g"|sed "s/\t/;/g"|sed -e "s,',\o042,g"
The result will be:
1;O7_DICTIONARY_ACCESSIBILITY=TRUE
2;active_instance_count=
3;aq_tm_processes=1
4;archive_lag_target=0
5;audit_file_dest=?/rdbms/audit
6;audit_sys_operations=FALSE
7;audit_trail=NONE
8;background_core_dump=partial
9;background_dump_dest=/home1/oracle/app/oracle/admin/iopecom/bdump
But, I can't figure out, how to perform the same command on AIX server.
Help is very welcome.
Regards.
Antonio.
Unless you have a compelling reason to use sed, you could use alternate tools:
awk -v OFS=';' '{print NR,$0}' filename
would produce the desired output.
You could also use perl:
perl -ne 'print "$.;$_"' filename
It appears that your sed expression would skip lines beginning with a #. As such, you could say:
perl -ne '$,=";"; !/^#/ && print ++$i,$_' filename
or something like:
grep -v '^#' filename | awk ...
reformatting your pipeline:
cat myfile |
sed -r 's/ {1,}//g' | # strip all spaces (1)
sed -r 's/\t*//g' | # strip all tabs (2)
grep -v "^#" | # delete all lines beginning `#` (3)
sed -s "/^$/d" | # delete all empty lines (4)
sed = | # interleave with line numbers (5)
sed 'N;s/\n/\t/' | # join line number and line with `\t` (6)
sed -r "s/#.*//g" | # strip all `#` comments (7)
sed "s/\t/;/g" | # replace all tabs with `;` (8)
sed "s/\t/;/g" | # do it again (9)
sed -e "s,',\o042,g" # replace all ' with " (10)
Boiling that down and using cat -n to provide the line numbers up front gets:
cat -n myfile |
sed "$(print 's/\t/;/')
$(print 's/[ \t]*//g')
s/#.*//g
/^$/d
s/'/\"/g"
which behaves identically unless I'm misreading the aix docs. The $(...) construction is command substitution, it runs that command and substitutes its output. print would be printf on linux.
This is probably a trivial one:
I have a file (my.file) with these lines:
>h1_c1
>h1_c2
>h1_c3
>h2_c1
>h2_c2
>h2_c3
and I want to change it in place to be:
>c1_h1
>c2_h1
>c3_h1
>c1_h2
>c2_h2
>c3_h3
I thought this ought to do it:
sed -i 's/\(\>\)\(h1\)\(\_\)\(.*\)/\1 \4 \3 \2/g' my.file
sed -i 's/\(\>\)\(h2\)\(\_\)\(.*\)/\1 \4 \3 \2/g' my.file
but it doesn't seem to work. How do I do it?
The obvious sed for your example is:
$ sed -i~ -e 's/^>\(h[0-9]\)_\(c[0-9]\)/>\2_\1/' *.foo
I tested this and it works for your example file.
Try this awk
awk -F">|_" '{print ">"$3"_"$2}' my.file > tmp && mv tmp my.file
awk -F">|_" '{print ">"$3"_"$2}' my.file
>c1_h1
>c2_h1
>c3_h1
>c1_h2
>c2_h2
>c3_h2
You can try this sed,
sed 's/>\(h[1-2]\)_\(.*\)/>\2_\1/' yourfile
(OR)
sed -r 's/>(h[1-2])_(.*)/>\2_\1/' yourfile
kent$ sed -r 's/>([^_]*)_(.*)/>\2_\1/' f
>c1_h1
>c2_h1
>c3_h1
>c1_h2
>c2_h2
>c3_h2
you add -i if you want it to happen "in-place"
Suppose I have a string like this
<start><a></a><a></a><a></a></start>
I want to replace values inside <start></start> like this
<start><ab></ab><ab></ab><ab></ab><more></more><vale></value></start>
How do I do this using Sed?
Try this :
sed 's#<start>.*</start>#<start><ab></ab><ab></ab><ab></ab></start>#' file
I get this line with gnu sed :
sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:a>:ab>:g")"\3"#ge'
see example:
kent$ echo "<start><a></a><a></a><a></a><foo></foo><bar></bar></start>"|sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:a>:ab>:g")"\3"#ge'
<start><ab></ab><ab></ab><ab></ab><foo></foo><bar></bar></start>
note
this will replace the tags between <start>s which ending with a . which worked for your example. but if you have <aaa></aaa>:
you could do: (I break it into lines for better reading)
sed -r 's#(<start>)(.*)(</start>)
#echo "\1"$(echo "\2"\|sed "s:<a>:<ab>:g;s:</a>:</ab>:g")"\3"
#ge'
e.g.
kent$ echo "<start><a></a><a></a><a></a><aaa></aaa><aba></aba></start>" \
|sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:<a>:<ab>:g;s:</a>:</ab>:g")"\3"#ge'
<start><ab></ab><ab></ab><ab></ab><aaa></aaa><aba></aba></start>
sed 's/(\<\/?)a\>/\1ab\>/g' yourfile, though that would get <a></a> that was outside <start> as well...
grep -rl 'abc' a.txt | xargs sed -i 's/abc/def/g'