I am using sed to combine lines of text files in a directory.
The command cd dir && sed -e 'N;s/\n//' *.txt works fine to do that but is there any way it can be tweaked to only combine the line sentences that start with ** with following sentence ending in **. So
This is Line1
**This is Line2
This is Line3**
This is Line4
This is Line5
Becomes
This is Line1
** This is Line2 This is Line3**
This is Line4
This is Line5
etc
sed is for simple subsitutions on individual lines, that is all. For anything else you should be using awk. This will do what you show with your sample input/output:
$ awk '{ORS=(/^\*\*/?FS:RS)}1' file
This is Line1
**This is Line2 This is Line3**
This is Line4
This is Line5
but of course it doesn't address any of the requirements you haven't shared with us yet (e.g. what to do when a line starts with ** but the next line doesn't end with ** or vice-versa or a line starts and ends with ** or a line starting with ** is at the end of the input file or....).
Sed is your friend
$ sed '/^\*\*/{:l1;/\*\*$/!{N;bl1};s/\n/ /g;}' file
This is Line1
**This is Line2 This is Line3**
This is Line4
This is Line5
You can use this sed:
sed '/^\*\*/{:loop; N; /\*\*$/{s/\n/ /g;p;d;}; b loop}' file
Test:
$ cat file
This is Line1
**This is Line2
in between
This is Line3**
This is Line4
**This is Line5
This is Line6**
$ sed '/^\*\*/{:loop; N; /\*\*$/{s/\n/ /g;p;d;}; b loop;}' file
This is Line1
**This is Line2 in between This is Line3**
This is Line4
**This is Line5 This is Line6**
$ cat ip.txt
This is Line1
**This is Line2
This is Line3**
This is Line4
This is Line5
$ # this slurps entire file
$ perl -0777 -pe 's/^(\*\*.*)\n(.*\*\*)$/$1 $2/mg' ip.txt
This is Line1
**This is Line2 This is Line3**
This is Line4
This is Line5
$ # can use this if testing start of line for ** is enough
$ perl -pe 's/\n/ / if /^\Q**/' ip.txt
Reference: How do I search and replace across multiple lines with Perl?
Related
I need to extract text between the first occurrence of a word called "BEGIN" and the last occurrence of a word called "END" using sed.
Input:
line1
BEGIN
line2
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
line9
line10
Expected Output:
BEGIN
line2
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
My approach:
It extracts text between BEGIN and END. Here there are two BEGIN & END statement, and my solution extracts text between these words.
My solution fails to extract text between first occurence of word1 (BEGIN) and last occurence of word2 (END).
dsonachalam$ sed -n -e '/^BEGIN$/,/^END$/p' logs.txt
BEGIN
line2
line3
END
BEGIN
line6
line7
ENDED
END
start=$(grep -n "BEGIN" $FILE_NAME |cut -f1 -d:|head -n 1)
end=$(grep -n "END" $FILE_NAME |cut -f1 -d:|tail -n 1)
sed -n $start,"$end"p $FILE_NAME
If the file is small enough to fit memory:
$ perl -0777 -ne 'print /(^BEGIN\n.*^END\n)/ms' ip.txt
BEGIN
line2
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
With a 2-pass approach to avoid having to store any text in memory so it'll work for any size input file and with 1 call to 1 standard UNIX tool to avoid spawning multiple subshells, the following will work using any awk in any shell on every UNIX box:
$ awk '
NR==FNR{ if (!beg && /BEGIN/) beg=NR; if (/END/) end=NR; next}
(beg <= FNR) && (FNR <= end)
' file file
BEGIN
line2
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
A one-liner sed command would suffice (using GNU sed):
sed -E '/^BEGIN$/,$!d; :a; /(^|\n).*END$/{p;d}; $d; N; ba'
/^BEGIN$/,$!d; deletes lines above the first BEGIN. :a; /(^|\n).*END$/{p;d}; $d; N; ba accumulates ("slurps") lines into pattern space. Whenever an END line is read then the accumulated lines are printed out and pattern space is deleted starting a new cycle. Note that this "slurping" approach may be slow, or even may crash the sed process if the input is too large.
Content of input file:
line1
BEGIN
line2
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
line9
line10
and using GNU sed 4.8
sed -E '/^BEGIN$/,$!d; :a; /(^|\n).*END$/{p;d}; $d; N; ba' inputfile
prints
BEGIN
line2
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
Another approach would be:
lastend=$(sed -n '/^END$/=' inputfile | tail -1)
[[ -n $lastend ]] && sed -n "/^BEGIN\$/,${lastend}p" inputfile
This two-pass approach doesn't suffer from "slurping" lines.
This might work for you (GNU sed):
sed -n '/\<BEGIN\>/{x;:a;n;/\<END\>/{x;p;ba};H;$!ba;x;//P}' file
Set automatic printing off by using the -n option and then focus on lines following one that contains the word BEGIN.
Swap to the hold space (HS) and initiate a loop that fetches the next line and if that line contains the word END swap to the HS, print its contents and repeat.
If the current line does not contain the word END, append the current line to the HS and unless it is the end of file repeat.
At the end of file, print the first line of the HS if it begins END and whatever the condition allow the file processing to terminate.
Thus processing of lines only occurs once the word BEGIN has been seen and printing of those lines every time the word END occurs.
I have a Verilog file that looks like this:
Line1
Line2
Line3
module1
Line4
Line5
Line6
endmodule
Line7
Line8
module2
Line9
Line11
Line12
Line13
endmodule
Line15
Line16
Here I want to delete whole modules and the module names will be specified by me. Ex: I want to delete module1 so I want lines from module1 to endmodule to be deleted(module1, Line4, Line5, Line6, endmodule). And keep the other remaining modules intact.
My expected output when I delete module1:
Line1
Line2
Line3
Line7
Line8
module2
Line9
Line11
Line12
Line13
endmodule
Line15
Line16
How do I go about it?
I'd use sed for this, not perl:
sed -e '/module1/,/endmodule/d' input.txt
X,Y specifies a range of lines to do something on, starting with the one matching X and ending with the one matching Y, and the d command basically says to delete the current line instead of printing it like normal.
If you're set on perl, the scalar form of the range operator (..) allows for the same sort of thing:
perl -ne 'print unless /module1/ .. /endmodule/' data.txt
How to delete the records that has '?' in the file ?
Input file data
12345 Line1
?
34567 Line2
?
89101 Line3
Expected Output
12345 Line1
34567 Line2
89101 Line3
sed '/?/d' yourfile
or
grep -v '?' yourfile
if you only wanted just a '?' and nothing else, do '^?$' instead of just the ?.
Seems like you want to replace a blankline followed by a ? symbol again followed blank line with a single blank line. If so then try the below perl command.
perl -0777 -pe 's/\n\?\n//g' file
Example:
$ perl -0777 -pe 's/\n\?\n//g' file
12345 Line1
34567 Line2
89101 Line3
I saw some answers here, but can't make them work for me.
I have text like this:
line1
line2 text=^M
line3
line4
basically what i need is to replace =^M\n with empty character something like s/=^M\n//, so the output is (^M is special character ctrl+v ctrl+m)
line1
line2 textline3
line4
I know it's some sed branches but I have problem with making them work.
One way:
$ sed '/^M/{N;s/=^M\n//;}' file
line1
line2 textline3
line4
Where ^M has to be typed as: Ctrl-V + Ctrl-M
awk solution for this
#awk -f myawk.sh temp.txt
BEGIN { print "Start Records"}
{
if ($2 ~ /=\^M/){
a=$1;
gsub("=\\^M","",$2);
b=$2; f=1
}
else {
if(f==1){
print a""b""$0;
a="";
b="";
}else{
print $0
}
}
}
END {print "Process Complete"}
So there are other similar questions, but here's in particular what I want to do -
I have one really long file. long.txt that looks like
line1
line2
line3
line4
line1
line1
line2
line8
line1
line2
now, I have another file, pattern.txt that looks like
line1
line2
Finally, replace.txt that looks like
newline1
newline2
Is there a way to call sed such that after running it on the above, I end up with
newline1
newline2
line3
line4
line1
newline1
newline2
line8
newline1
newline2
This might work for you (GNU sed):
cat <<\! >cat.sed
> :a;$!{N;ba};s/\n/\\n/
> !
sed ':a;$!'"{N;ba};s/$(sed -f cat.sed pattern.txt)/$(sed -f cat.sed replace.txt)/g" long.txt
newline1
newline2
line3
line4
line1
newline1
newline2
line8
newline1
newline2
Explanation:
Build the LHS (pattern) and RHS (replace) of a sed substitution using a generic sed script - cat.sed
Plug the above substitution into another sed script that processes the long.txt file.
$ paste -d'/' pattern.txt replace.txt | sed 's#.*#s/&/#' >script.sed
$ sed -f script.sed long.txt