I am using grep to pull the contents in a file and writing it in output. I wish to concatenate a text for all those pulled entries. Currently, I am doing it in two steps: 1. grep, 2. Modify the text output in excel.
grep -E -w "entry1" file1 >output.csv
In output.csv I get four lines matching
A, entry1
B, entry1
C, entry1
D, entry1
I am opening this file in excel and modifying the row header in column1 to
A_type1, entry1
B_type1, entry1
C_type1, entry1
D_type1, entry1
Is it possible to do the same thing in single line?
according to the grep output you given, I think this line should help:
awk -F, '$2~/entry1/{print $1"_type1" FS $2}' file
You can do something like
$ grep -E "A|B" sample.txt | perl -pe 's/A/A_type1/g; s/B/B_type1/g'
This will filter the input file and apply the transformation on the output. The input file remains unchanged.
If you would rather perform an inplace repalcement in the file without affecting the other lines, you could use "sed". In this case the input file does get updated.
sed -i 's/A/A_type1/g; s/B/B_typ2/g' sample.txt
admin#DESKTOP-J2E0MU7 /cygdrive/c/workspace/temp
$ cat sample.txt
A line
B file
C test1
D sample
E something else
F what
G else
A other line
B second occurence
admin#DESKTOP-J2E0MU7 /cygdrive/c/workspace/temp
$ grep -E "A|B" sample.txt | perl -pe 's/A/A_type1/g; s/B/B_type1/g'
A_type1 line
B_type1 file
A_type1 other line
B_type1 second occurence
A different solution using awk to append "_type1" to column1 of whatever comes out of grep.
$ grep -E "A|B" sample.txt | awk '{$1=$1"_type";print}''\
awk -F, '{print $1"_type1",$2}' OFS=, file
A_type1, entry1
B_type1, entry1
C_type1, entry1
D_type1, entry1
Related
I am trying to insert the contents of a file1.txt into file2.txt using sed. The content of file1.txt is just a single line, which is a path.
I want it to be added as a prefix to each line in file2.txt as well as add another / character.
$ cat file1.txt
/psot/rot8888/orce/db/tier/data/tine
$ cat file2.txt
o1_mf_users_abchwfg_.dbf
o1_mf_toptbs2_abchrq0_.dbf
o1_mf_toptbs1_abchrl2_.dbf
o1_mf_toptbs1_abchtlf_.dbf
Desired output should be like:
/psot/rot8888/orce/db/tier/data/tine/o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchtlf_.dbf
Tried command:
$ sed '/o1/ r file1.txt' file2.txt >> test.txt
$ cat test.txt
o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs1_abchtlf_.dbf
/psot/rot8888/orce/db/tier/data/tine
You can use pr for this without having to worry about sed metacharacters, delimiters, etc.
$ cat ip.txt
abcd.xyz
123.txt
foo_baz.txt
$ cat f1
/a/b/c/d/
$ pr -mts"$(< f1)" /dev/null ip.txt
/a/b/c/d/abcd.xyz
/a/b/c/d/123.txt
/a/b/c/d/foo_baz.txt
Where -m allows pasting files parallely and -s is the separator between the files to be merged. Here, /dev/null is used as a dummy for one of the files as only the separator has to be prefixed.
If you need to add some more characters after the contents of file containing the prefix:
$ cat ip.txt
abcd.xyz
123.txt
foo_baz.txt
$ cat f1
/a/b/c/d
$ pr -mts"$(< f1)"'/' /dev/null ip.txt
/a/b/c/d/abcd.xyz
/a/b/c/d/123.txt
/a/b/c/d/foo_baz.txt
This will work using any awk in any shell on every UNIX box:
$ awk 'NR==FNR{p=$0; next} {print p "/" $0}' file1 file2
/psot/rot8888/orce/db/tier/data/tine/o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchtlf_.dbf
This might work for you (GNU sed):
sed '1h;1d;G;s/\(.*\)\n\(.*\)/\2\1/' file1 file2
Copy file1 into the hold space and append it to each line in file2. Using regexp and back references, manipulate the two lines into one in the correct order.
Alternative:
sed 'x;s/.*/cat file1/e;G;s/\n//' file2
Insert file1 into the hold space, append the current line of file2, and remove the newline connecting them.
A third way:
sed 'r file1' file2 | sed -E 'N;s/(.*)\n(.*)/\2\/\1/'
My example string is in txt file /www/meteo/last.txt:
a:3:{i:0;s:4:"6.13";i:1;s:5:"19.94";i:2;s:5:"22.13";}
I would like to get line by line 3 numbers from that file to a new file.
(those values is temperature so they are changing in time - every 10 minutes)
New file /www/meteo/new.txt: (line by line)
6.13
19.94
22.13
Try this awk method
awk -F'"' 'BEGIN{OFS="\n"} {print $2,$4,$6}' last.txt > new.txt
OutPut:
cat new.txt
6.13
19.94
22.13
Or, if you wanted to use sed or grep:
sed -r 's/([^"]*)("[^"]*")([^"]*)/\2\n/g;s/"//g' /www/meteo/last.txt
grep -Eo '"[^"]*"' /www/meteo/last.txt | sed 's/"//g'
If you want a specific value, lets say the second temperature in quotes you can use sed:
grep -Eo '"[^"]*"' /www/meteo/last.txt | sed -n '2p'
I have the file test.txt similar to that:
aa:bbbbbb:22.3
a:bb:33.2
a:bbbb:22.3
aaaa:bb:39.9
I know how to count and sort them like:
awk -F ':' '{print $2}' test.txt | awk '{print length($0),$0}' | sort -nr
Now I want to remove the 1st and 3rd lines from the file because the length of the second field (containing "b") in those lines is larger than 3. How can I do that using awk/sed? Thanks.
With awk:
This will output the lines whose 2nd field is >3:
$ awk -F: 'length($2)>3' file
aa:bbbbbb:22.3
a:bbbb:22.3
To do the opposite:
$ awk -F: 'length($2)<=3' file
a:bb:33.2
aaaa:bb:39.9
Code for sed:
sed '/.*:..:.*/!d' file
or more general:
sed '/.*:.\{2\}:.*/!d' file
I have 'file1' with (say) 100 lines. I want to use sed or awk to print lines 23, 71 and 84 (for example) to 'file2'. Those 3 line numbers are in a separate file, 'list', with each number on a separate line.
When I use either of these commands, only line 84 gets printed:
for i in $(cat list); do sed -n "${i}p" file1 > file2; done
for i in $(cat list); do awk 'NR==x {print}' x=$i file1 > file2; done
Can a for loop be used in this way to supply line addresses to sed or awk?
This might work for you (GNU sed):
sed 's/.*/&p/' list | sed -nf - file1 >file2
Use list to build a sed script.
You need to do > after the loop in order to capture everything. Since you are using it inside the loop, the file gets overwritten. Inside the loop you need to do >>.
Good practice is to or use > outside the loop so the file is not open for writing during every loop iteration.
However, you can do everything in awk without for loop.
awk 'NR==FNR{a[$1]++;next}FNR in a' list file1 > file2
You have to >>(append to the file) . But you are overwriting the file. That is why, You are always getting 84 line only in the file2.
Try use,
for i in $(cat list); do sed -n "${i}p" file1 >> file2; done
With sed:
sed -n $(sed -e 's/^/-e /' -e 's/$/p/' list) input
given the example input, the inner command create a string like this: `
-e 23p
-e 71p
-e 84p
so the outer sed then prints out given lines
You can avoid running sed/awk in a for/while loop altgether:
# store all lines numbers in a variable using pipe
lines=$(echo $(<list) | sed 's/ /|/g')
# print lines of specified line numbers and store output
awk -v lineS="^($lines)$" 'NR ~ lineS' file1 > out
I need to replace the pattern ### with the current line number.
I managed to Print in the next line with both AWK and SED.
sed -n "/###/{p;=;}" file prints to the next line, without the p;, it replaces the whole line.
sed -e "s/###/{=;}/g" file used to make sense in my head, since the =; returns the line number of the matched pattern, but it will return me the the text {=;}
What am i Missing? I know this is a silly question. I couldn't find the answer to this question in the sed manual, it's not quite clear.
If possible, point me what was i missing, and what to make it work. Thank you
Simple awk oneliner:
awk '{gsub("###",NR,$0);print}'
Given the limitations of the = command, I think it's easier to divide the job in two (actually, three) parts. With GNU sed you can do:
$ sed -n '/###/=' test > lineno
and then something like
$ sed -e '/###/R lineno' test | sed '/###/{:r;N;s/###\([^\n]*\n\)\([^\n]*\)/\2\1/;tr;:c;s/\n\n/\n/;tc}'
I'm afraid there's no simple way with sed because, as well as the = command, the r and GNU extension R commands don't read files into the pattern space, but rather directly append the lines to the output, so the contents of the file cannot be modified in any way. Hence piping to another sed command.
If the contents of test are
fooo
bar ### aa
test
zz ### bar
the above will produce
fooo
bar 2 aa
test
zz 4 bar
This might work for you (GNU sed):
sed = file | sed 'N;:a;s/\(\(.*\)\n.*\)###/\1\2/;ta;s/.*\n//'
An alternative using cat:
cat -n file | sed -E ':a;s/^(\s*(\S*)\t.*)###/\1\2/;ta;s/.*\t//'
As noted by Lev Levitsky this isn't possible with one invocation of sed, because the line number is sent directly to standard out.
You could have sed write a sed-script for you, and do the replacement in two passes:
infile
a
b
c
d
e
###
###
###
a
b
###
c
d
e
###
Find the lines that contain the pattern:
sed -n '/###/=' infile
Output:
6
7
8
11
15
Pipe that into a sed-script writing a new sed-script:
sed 's:.*:&s/###/&/:'
Output:
6s/###/6/
7s/###/7/
8s/###/8/
11s/###/11/
15s/###/15/
Execute:
sed -n '/###/=' infile | sed 's:.*:&s/^/& \&/:' | sed -f - infile
Output:
a
b
c
d
e
6
7
8
a
b
11
c
d
e
15
is this ok ?
kent$ echo "a
b
c
d
e"|awk '/d/{$0=$0" "NR}1'
a
b
c
d 4
e
if match pattern "d", append line number at the end of the line.
edit
oh, you want to replace the pattern not append the line number... take a look the new cmd:
kent$ echo "a
b
c
d
e"|awk '/d/{gsub(/d/,NR)}1'
a
b
c
4
e
and the line could be written like this as well: awk '1+gsub(/d/,NR)' file
one-liner to modify the FILE in place, replacing LINE with the corresponding line number:
seq 1 `wc -l FILE | awk '{print $1}'` | xargs -IX sed -i 'X s/LINE/X/' FILE
Following on from https://stackoverflow.com/a/53519367/29924
If you try this on osx the version of sed is different and you need to do:
seq 1 `wc -l FILE | awk '{print $1}'` | xargs --verbose -IX sed -i bak "X s/__line__/X/" FILE
see https://markhneedham.com/blog/2011/01/14/sed-sed-1-invalid-command-code-r-on-mac-os-x/