I want to move multiple lines using SED to the end of a file. For example:
ABCDEF
GHIJKL
MNOPQR
STUVWX
YZ1234
should become:
STUVWX
YZ1234
ABCDEF
GHIJKL
MNOPQR
The question asks a method for using sed to move multiple lines. Although the question, as currently edited, does not show multiple lines, I will assume that they are actually meant to be separate lines.
To use sed to move the first three lines to the end:
$ cat file
ABCDEF
GHIJKL
MNOPQR
STUVWX
YZ1234
$ sed '1,3{H;d}; ${p;x;s/^\n//}' file
STUVWX
YZ1234
ABCDEF
GHIJKL
MNOPQR
Explanation:
1,3{H;d}
The 1,3 restricts these commands to operation only on lines 1 through 3. H tells sed to save the current line to the hold buffer. d tells sed not to print the current line at this time.
${p;x;s/^\n//}
The $ restricts this command to the last line. The p tells sed to print the last line. x exchanges the pattern buffer and hold buffer. The lines that we saved from the beginning of the file are now in the ready to be printed. Before printing, though, we remove the extraneous leading newline character.
Before continuing to the next line, sed will print anything left in the pattern buffer.
If you are on Mac OSX or other BSD platform, try:
sed -e '1,3{H;d;}' -e '${p;x;s/^\n//;}' file
Using head and tail
If you are already familiar with head and tail, this may be a simpler way of moving the first three lines to the end:
$ tail -n+4 file; head -n3 file
STUVWX
YZ1234
ABCDEF
GHIJKL
MNOPQR
Alternative sed command
In the comments, potong suggests:
$ sed '1,3{1h;1!H;d};$G' file
STUVWX
YZ1234
ABCDEF
GHIJKL
MNOPQR
The combination of h, H, and G commands eliminate the need for a substitution command to remove the extra newline.
Try this using awk
awk '{print $4,$5,$1,$2,$3}' file
STUVWX YZ1234 ABCDEF GHIJKL MNOPQR
PS, this is multiple fields, not lines.
To write it back to the original file
awk '{print $4,$5,$1,$2,$3}' file > tmp && mv tmp file
If data is in this format:
cat file
ABCDEF
GHIJKL
MNOPQR
STUVWX
Then this may do:
awk '{a[NR]=$0} END {print a[4],a[5],a[1],a[2],a[3]}' OFS="\n" file
STUVWX
YZ1234
ABCDEF
GHIJKL
MNOPQR
Simple ex should do:
ex file <<< $'1,3m$\nw'
Related
I have a file with the following content.
test1
test2
test3
test4
test5
If I want to concatenate all lines into one line separated by commas, I can use vi and run the following command:
:%s/\n/,/g
I then get this, which is what I want
test1,test2,test3,test4,test5,
I'm trying to use sed to do the same thing but I'm missing some unknown command/option to make it work. When I look at the file in vi and search for "\n" or "$", it finds the newline or end of line. However, when I tell sed to look for a newline, it pretends it didn't find one.
$ cat test | sed --expression='s/\n/,/g'
test1
test2
test3
test4
test5
$
If I tell sed to look for end of line, it finds it and inserts the comma but it doesn't concatenate everything into one line.
$ cat test | sed --expression='s/$/,/g'
test1,
test2,
test3,
test4,
test5,
$
What command/option do I use with sed to make it concatenate everything into one line and replace the end of line/newline with a comma?
sed reads one line at a time, so, unless you're doing tricky things, there's never a newline to replace.
Here's the trickiness:
$ sed -n '1{h; n}; H; ${g; s/\n/,/gp}' test.file
test1,test2,test3,test4,test5
h, H, g documented at https://www.gnu.org/software/sed/manual/html_node/Other-Commands.html
When using a non-GNU sed, as found on MacOS, semi-colons before the closing braces are needed.
However, paste is really the tool for this job
$ paste -s -d, test.file
test1,test2,test3,test4,test5
If you really want the trailing comma:
printf '%s,\n' "$(paste -sd, file)"
tr instead of sed for this one:
$ tr '\n' ',' < input.txt
test1,test2,test3,test4,test5,
Just straight up translate newlines to commas.
Based on how can i replace each newline n with a space using sed:
sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g' <file>
testing:
$ cat file.txt
test1
test2
test3
test4
test5
$ sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g' file.txt
test1,test2,test3,test4,test5
Of course, if the question would have been more generic: How do I replace \n with any character using sed then one should only replace the , with ones desired char:
export CHAR_TO_REPLACE=','
export FILE_TO_PROCESS=<filename>
sed -e ':a' -e 'N' -e '$!ba' -e "s/\n/${CHAR_TO_REPLACE}/g" $FILE_TO_PROCESS
This answer is to satisfy the requirement of using sed. Otherwise, you can use alternatives like tr, awk etc.
This might work for you (GNU sed):
sed 'H;1h;$!d;x;y/\n/,/' file
Append all lines but the first to the hold space (the first replaces the hold space).
If it is not the last line of the file, delete it.
Otherwise, swap to the hold space and translate all newlines to commas.
I am trying to insert the contents of a file1.txt into file2.txt using sed. The content of file1.txt is just a single line, which is a path.
I want it to be added as a prefix to each line in file2.txt as well as add another / character.
$ cat file1.txt
/psot/rot8888/orce/db/tier/data/tine
$ cat file2.txt
o1_mf_users_abchwfg_.dbf
o1_mf_toptbs2_abchrq0_.dbf
o1_mf_toptbs1_abchrl2_.dbf
o1_mf_toptbs1_abchtlf_.dbf
Desired output should be like:
/psot/rot8888/orce/db/tier/data/tine/o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchtlf_.dbf
Tried command:
$ sed '/o1/ r file1.txt' file2.txt >> test.txt
$ cat test.txt
o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs1_abchtlf_.dbf
/psot/rot8888/orce/db/tier/data/tine
You can use pr for this without having to worry about sed metacharacters, delimiters, etc.
$ cat ip.txt
abcd.xyz
123.txt
foo_baz.txt
$ cat f1
/a/b/c/d/
$ pr -mts"$(< f1)" /dev/null ip.txt
/a/b/c/d/abcd.xyz
/a/b/c/d/123.txt
/a/b/c/d/foo_baz.txt
Where -m allows pasting files parallely and -s is the separator between the files to be merged. Here, /dev/null is used as a dummy for one of the files as only the separator has to be prefixed.
If you need to add some more characters after the contents of file containing the prefix:
$ cat ip.txt
abcd.xyz
123.txt
foo_baz.txt
$ cat f1
/a/b/c/d
$ pr -mts"$(< f1)"'/' /dev/null ip.txt
/a/b/c/d/abcd.xyz
/a/b/c/d/123.txt
/a/b/c/d/foo_baz.txt
This will work using any awk in any shell on every UNIX box:
$ awk 'NR==FNR{p=$0; next} {print p "/" $0}' file1 file2
/psot/rot8888/orce/db/tier/data/tine/o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchtlf_.dbf
This might work for you (GNU sed):
sed '1h;1d;G;s/\(.*\)\n\(.*\)/\2\1/' file1 file2
Copy file1 into the hold space and append it to each line in file2. Using regexp and back references, manipulate the two lines into one in the correct order.
Alternative:
sed 'x;s/.*/cat file1/e;G;s/\n//' file2
Insert file1 into the hold space, append the current line of file2, and remove the newline connecting them.
A third way:
sed 'r file1' file2 | sed -E 'N;s/(.*)\n(.*)/\2\/\1/'
On a Unix system I am trying to add a new line in a file using sed or perl but it seems I am missing something.
Supposing my file has multiple lines of texts, always ending like this {TNG:}}${1:F01.
I am trying to find a to way to add a new line after the }$, in this way {1 should always start on a new line.
I tried it by escaping $ sign using this:
perl -e '$/ = "\${"; while (<>) { s/\$}\{$/}\n{/; print; }' but it does not work.
Any ideas will be appreciated.
give this a try:
sed 's/{TNG:}}\$/&\n/' file > newfile
The sed will by default use BRE, that is, the {}s are literal characters. But we must escape the $.
kent$ cat f
{TNG:}}${1:F01.
kent$ sed 's/{TNG:}}\$/&\n/' f
{TNG:}}$
{1:F01.
With perl:
$ cat input.txt
line 1 {TNG:}}${1:F01
line 2 {TNG:}}${1:F01
$ perl -pe 's/TNG:\}\}\$\K/\n/' input.txt
line 1 {TNG:}}$
{1:F01
line 2 {TNG:}}$
{1:F01
(Read up on the -p and -n options in perlrun and use them instead of trying to do what they do in a one-liner yourself)
I have file1 containing some text, like:
abcdef 123456 abcdef
ghijkl 789123 abcdef
mnopqr 123456 abcdef
and I have file2 containing single line of text which I want to use as pattern:
ghijkl 789123
How can I use second file as a pattern to print lines containing it to third file using sed? like file3:
ghijkl 789123 abcdef
I've tried to use
sed -ne "s/r file2//p" file1 > file3
But the content of file3 is blank for some reason
P.S. using Windows
If you have sed, do have access to grep?
grep -f file2 file1 > file3
This is the simplest sed solution on linux: sed -n /`<file2`/p file1 > file3, but windows does not provides backticks. So the windows work-around would be:
set /p PATERN=<file2
sed -n /%PATERN%/p file1 > file3
The sed solution is:
cat f2.txt | xargs -I {} sed -n "/{}/p" f1.txt > f3.txt
but, as #Cyrus correctly notes, grep is the proper tool for this solution and it's much nicer:
grep -f f2.txt f1.txt > f3.txt
Note: using these incredibly powerful *nix tools like sed, grep, cat, xargs, bash, etc. on Microsoft Windows can be frustrating. Consider spinning up a Linux environment, instead -- you'll save yourself many hours of grief dealing with subtle path and environment issues from emulators like Cygwin, etc.
I have 'file1' with (say) 100 lines. I want to use sed or awk to print lines 23, 71 and 84 (for example) to 'file2'. Those 3 line numbers are in a separate file, 'list', with each number on a separate line.
When I use either of these commands, only line 84 gets printed:
for i in $(cat list); do sed -n "${i}p" file1 > file2; done
for i in $(cat list); do awk 'NR==x {print}' x=$i file1 > file2; done
Can a for loop be used in this way to supply line addresses to sed or awk?
This might work for you (GNU sed):
sed 's/.*/&p/' list | sed -nf - file1 >file2
Use list to build a sed script.
You need to do > after the loop in order to capture everything. Since you are using it inside the loop, the file gets overwritten. Inside the loop you need to do >>.
Good practice is to or use > outside the loop so the file is not open for writing during every loop iteration.
However, you can do everything in awk without for loop.
awk 'NR==FNR{a[$1]++;next}FNR in a' list file1 > file2
You have to >>(append to the file) . But you are overwriting the file. That is why, You are always getting 84 line only in the file2.
Try use,
for i in $(cat list); do sed -n "${i}p" file1 >> file2; done
With sed:
sed -n $(sed -e 's/^/-e /' -e 's/$/p/' list) input
given the example input, the inner command create a string like this: `
-e 23p
-e 71p
-e 84p
so the outer sed then prints out given lines
You can avoid running sed/awk in a for/while loop altgether:
# store all lines numbers in a variable using pipe
lines=$(echo $(<list) | sed 's/ /|/g')
# print lines of specified line numbers and store output
awk -v lineS="^($lines)$" 'NR ~ lineS' file1 > out