is it possible to use the pipe to redirect the output of the previous command, to sed, and let sed use this as input(pattern or string) to access a file?
I know if you only use sed, you can use something like
sed -i '1 i\anything' file
But can I do something like
head -1 file1 | sed -i '1 i\OutputFromPreviousCmd' file2
This way, I don't need to manually copy the output and change the sed command everytime
Update:
Added the files I meant
head -3 file1.txt
Side A,Age(us),mm:ss.ms_us_ns_ps
84 Vendor Specific, 0000000009096, 0349588242
84 Vendor Specific, 0000000011691, 0349591828
head -3 file2.txt
84 Vendor Specific, 0000000000418, 0349575322
83 Vendor Specific, 0000000002099, 0349575343
83 Vendor Specific, 0000000001628, 0349576662
I'd like to grab the first line of file1 and insert it to file2, so the result should be :
head -3 file2.txt
Side A,Age(us),mm:ss.ms_us_ns_ps
84 Vendor Specific, 0000000000418, 0349575322
83 Vendor Specific, 0000000002099, 0349575343
83 Vendor Specific, 0000000001628, 0349576662
head -1 file1 | sed '1s/^/1i /' | sed -i -f- file2
This takes your one line of output, prepends the sed 1i command, the pipes that sed command stream to sed using -f- to take sed commands from stdin.
For example:
$ echo bob > bob.txt
$ echo alice | sed '1s/^/1i /' | sed -i -f- bob.txt
$ more bob.txt
alice
bob
This looks like pipes and not commands ending in > temp ; mv temp file2, but sed is doing that nonetheless when -i is used.
This might work for you (GNU sed):
head -1 file1 | sed -i '1e cat /dev/stdin' file2
Insert the first line of file1 into the start of file2.
But why not use cat?:
cat <(head -1 file1) file2
Related
file xz.txt
123
456
789
I want to merge
sed -i '1d' xz.txt
sed -i '1a abc' xz.txt
I tried
sed -i -e '1d' -e '1a abc' xz.txt
expect to get
456
abc
789
but I got
456
789
sed (GNU sed) 4.7
but it doesn't work, any help?
Sed goes line by line, first command 1d - deleted 1st line, 1st line is gone, there is no more 1st line, that is why second command 1a abc didn't work. Here is how it should be:
$ sed '1d; 2a abc' f
456
abc
789
What is going on is that the delete statement automatically ends the processing sequence:
[1addr]a\
text Write text to standard output as described previously (yes there is a new-line here)
[2addr]d: Delete the pattern space and start the next cycle
Source: Posix)
As the a command does not modify the pattern space but just writes to stdout, you can simply do
[POSIX]$ sed -e '1a\
abc' -e '1d'
[GNU]$ sed -e '1a abc' -e '1d'
However, the easiest is just to use the replace command c:
[POSIX]$ sed -e '1c\
abc'
[GNU]$ sed -e '1c abc`
Note: The reason the commands a and c write directly to the output and not to the pattern space is most likely that it would mess up the address ranging using line-numbers.
I am trying to insert the contents of a file1.txt into file2.txt using sed. The content of file1.txt is just a single line, which is a path.
I want it to be added as a prefix to each line in file2.txt as well as add another / character.
$ cat file1.txt
/psot/rot8888/orce/db/tier/data/tine
$ cat file2.txt
o1_mf_users_abchwfg_.dbf
o1_mf_toptbs2_abchrq0_.dbf
o1_mf_toptbs1_abchrl2_.dbf
o1_mf_toptbs1_abchtlf_.dbf
Desired output should be like:
/psot/rot8888/orce/db/tier/data/tine/o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchtlf_.dbf
Tried command:
$ sed '/o1/ r file1.txt' file2.txt >> test.txt
$ cat test.txt
o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs1_abchtlf_.dbf
/psot/rot8888/orce/db/tier/data/tine
You can use pr for this without having to worry about sed metacharacters, delimiters, etc.
$ cat ip.txt
abcd.xyz
123.txt
foo_baz.txt
$ cat f1
/a/b/c/d/
$ pr -mts"$(< f1)" /dev/null ip.txt
/a/b/c/d/abcd.xyz
/a/b/c/d/123.txt
/a/b/c/d/foo_baz.txt
Where -m allows pasting files parallely and -s is the separator between the files to be merged. Here, /dev/null is used as a dummy for one of the files as only the separator has to be prefixed.
If you need to add some more characters after the contents of file containing the prefix:
$ cat ip.txt
abcd.xyz
123.txt
foo_baz.txt
$ cat f1
/a/b/c/d
$ pr -mts"$(< f1)"'/' /dev/null ip.txt
/a/b/c/d/abcd.xyz
/a/b/c/d/123.txt
/a/b/c/d/foo_baz.txt
This will work using any awk in any shell on every UNIX box:
$ awk 'NR==FNR{p=$0; next} {print p "/" $0}' file1 file2
/psot/rot8888/orce/db/tier/data/tine/o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchtlf_.dbf
This might work for you (GNU sed):
sed '1h;1d;G;s/\(.*\)\n\(.*\)/\2\1/' file1 file2
Copy file1 into the hold space and append it to each line in file2. Using regexp and back references, manipulate the two lines into one in the correct order.
Alternative:
sed 'x;s/.*/cat file1/e;G;s/\n//' file2
Insert file1 into the hold space, append the current line of file2, and remove the newline connecting them.
A third way:
sed 'r file1' file2 | sed -E 'N;s/(.*)\n(.*)/\2\/\1/'
I have file1 containing some text, like:
abcdef 123456 abcdef
ghijkl 789123 abcdef
mnopqr 123456 abcdef
and I have file2 containing single line of text which I want to use as pattern:
ghijkl 789123
How can I use second file as a pattern to print lines containing it to third file using sed? like file3:
ghijkl 789123 abcdef
I've tried to use
sed -ne "s/r file2//p" file1 > file3
But the content of file3 is blank for some reason
P.S. using Windows
If you have sed, do have access to grep?
grep -f file2 file1 > file3
This is the simplest sed solution on linux: sed -n /`<file2`/p file1 > file3, but windows does not provides backticks. So the windows work-around would be:
set /p PATERN=<file2
sed -n /%PATERN%/p file1 > file3
The sed solution is:
cat f2.txt | xargs -I {} sed -n "/{}/p" f1.txt > f3.txt
but, as #Cyrus correctly notes, grep is the proper tool for this solution and it's much nicer:
grep -f f2.txt f1.txt > f3.txt
Note: using these incredibly powerful *nix tools like sed, grep, cat, xargs, bash, etc. on Microsoft Windows can be frustrating. Consider spinning up a Linux environment, instead -- you'll save yourself many hours of grief dealing with subtle path and environment issues from emulators like Cygwin, etc.
I have 'file1' with (say) 100 lines. I want to use sed or awk to print lines 23, 71 and 84 (for example) to 'file2'. Those 3 line numbers are in a separate file, 'list', with each number on a separate line.
When I use either of these commands, only line 84 gets printed:
for i in $(cat list); do sed -n "${i}p" file1 > file2; done
for i in $(cat list); do awk 'NR==x {print}' x=$i file1 > file2; done
Can a for loop be used in this way to supply line addresses to sed or awk?
This might work for you (GNU sed):
sed 's/.*/&p/' list | sed -nf - file1 >file2
Use list to build a sed script.
You need to do > after the loop in order to capture everything. Since you are using it inside the loop, the file gets overwritten. Inside the loop you need to do >>.
Good practice is to or use > outside the loop so the file is not open for writing during every loop iteration.
However, you can do everything in awk without for loop.
awk 'NR==FNR{a[$1]++;next}FNR in a' list file1 > file2
You have to >>(append to the file) . But you are overwriting the file. That is why, You are always getting 84 line only in the file2.
Try use,
for i in $(cat list); do sed -n "${i}p" file1 >> file2; done
With sed:
sed -n $(sed -e 's/^/-e /' -e 's/$/p/' list) input
given the example input, the inner command create a string like this: `
-e 23p
-e 71p
-e 84p
so the outer sed then prints out given lines
You can avoid running sed/awk in a for/while loop altgether:
# store all lines numbers in a variable using pipe
lines=$(echo $(<list) | sed 's/ /|/g')
# print lines of specified line numbers and store output
awk -v lineS="^($lines)$" 'NR ~ lineS' file1 > out
I need to replace the pattern ### with the current line number.
I managed to Print in the next line with both AWK and SED.
sed -n "/###/{p;=;}" file prints to the next line, without the p;, it replaces the whole line.
sed -e "s/###/{=;}/g" file used to make sense in my head, since the =; returns the line number of the matched pattern, but it will return me the the text {=;}
What am i Missing? I know this is a silly question. I couldn't find the answer to this question in the sed manual, it's not quite clear.
If possible, point me what was i missing, and what to make it work. Thank you
Simple awk oneliner:
awk '{gsub("###",NR,$0);print}'
Given the limitations of the = command, I think it's easier to divide the job in two (actually, three) parts. With GNU sed you can do:
$ sed -n '/###/=' test > lineno
and then something like
$ sed -e '/###/R lineno' test | sed '/###/{:r;N;s/###\([^\n]*\n\)\([^\n]*\)/\2\1/;tr;:c;s/\n\n/\n/;tc}'
I'm afraid there's no simple way with sed because, as well as the = command, the r and GNU extension R commands don't read files into the pattern space, but rather directly append the lines to the output, so the contents of the file cannot be modified in any way. Hence piping to another sed command.
If the contents of test are
fooo
bar ### aa
test
zz ### bar
the above will produce
fooo
bar 2 aa
test
zz 4 bar
This might work for you (GNU sed):
sed = file | sed 'N;:a;s/\(\(.*\)\n.*\)###/\1\2/;ta;s/.*\n//'
An alternative using cat:
cat -n file | sed -E ':a;s/^(\s*(\S*)\t.*)###/\1\2/;ta;s/.*\t//'
As noted by Lev Levitsky this isn't possible with one invocation of sed, because the line number is sent directly to standard out.
You could have sed write a sed-script for you, and do the replacement in two passes:
infile
a
b
c
d
e
###
###
###
a
b
###
c
d
e
###
Find the lines that contain the pattern:
sed -n '/###/=' infile
Output:
6
7
8
11
15
Pipe that into a sed-script writing a new sed-script:
sed 's:.*:&s/###/&/:'
Output:
6s/###/6/
7s/###/7/
8s/###/8/
11s/###/11/
15s/###/15/
Execute:
sed -n '/###/=' infile | sed 's:.*:&s/^/& \&/:' | sed -f - infile
Output:
a
b
c
d
e
6
7
8
a
b
11
c
d
e
15
is this ok ?
kent$ echo "a
b
c
d
e"|awk '/d/{$0=$0" "NR}1'
a
b
c
d 4
e
if match pattern "d", append line number at the end of the line.
edit
oh, you want to replace the pattern not append the line number... take a look the new cmd:
kent$ echo "a
b
c
d
e"|awk '/d/{gsub(/d/,NR)}1'
a
b
c
4
e
and the line could be written like this as well: awk '1+gsub(/d/,NR)' file
one-liner to modify the FILE in place, replacing LINE with the corresponding line number:
seq 1 `wc -l FILE | awk '{print $1}'` | xargs -IX sed -i 'X s/LINE/X/' FILE
Following on from https://stackoverflow.com/a/53519367/29924
If you try this on osx the version of sed is different and you need to do:
seq 1 `wc -l FILE | awk '{print $1}'` | xargs --verbose -IX sed -i bak "X s/__line__/X/" FILE
see https://markhneedham.com/blog/2011/01/14/sed-sed-1-invalid-command-code-r-on-mac-os-x/