sed - multi-line replace and combine - sed

I have the following sed code below that takes a number of lines and concatenates that them into one, while removing un-wanted data.
Here sed captures everything between the lines starting with To: and ending with X-RSMF-Generator, in addition, it also catutres everthign between the line starting with X-RSMF-Participants: and ending with MIME-Version
#!/bin/sh
#Sed turotial
shopt -s nullglob
FILES=/mnt/c/Temp/rsmf/*.rsmf
for f in $FILES
do
FileName=$f
#echo "$FileName"
#echo "Processing $f
sed -rn \
-e '/^To/,/X-RSMF-Generator:/{
#s/To: //
s/X-RSMF-Generator:.*/_/
s/\r\n//g
H
}' \
-e '/^X-RSMF-Participants:/,/^MIME-Version/{
s/X-RSMF-Participants: /_/
s/MIME-Version.*//
H
G
s/\r\n//gp
}' \
$f
done
Sample Data
To: Person One <>, "Person two" <person.two#gmail.com>, ,
"Person Three ***" <>
X-RSMF-Generator: Relativity RSMF Generator Sample Library
X-RSMF-Version: 1.0.0
X-RSMF-EventCount: 53
X-RSMF-BeginDate: 2022-09-20T04:33:11-04:00
X-RSMF-EndDate: 2022-09-20T16:47:56-04:00
X-RSMF-GroupID: GRP000000118
X-RSMF-SecondaryGroupID: GRP000000118_D_20220920
X-RSMF-ContainsDeleted: False
X-RSMF-Application: Native Messages
X-RSMF-Participants: Person eight <5148842954> next person available.
<Email.one#gmail.com> <342453136512> Next Person *** <9876582694>
<first.man#gmail.com> <888888889> Second Man *** <9999999999>
<fourth.man#gmail.com> <444444444> Thrid Man *** <3333333333>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-D+MPqNoS/tGpx2q22uID4A=="
Results
_Person eight <5148842954> next person available.
To: Person One <>, "Person two" <person.two#gmail.com>, , "Person Three ***" <>
_Person eight <5148842954> next person available.
<Email.one#gmail.com> <342453136512> Next Person *** <9876582694>
To: Person One <>, "Person two" <person.two#gmail.com>, , "Person Three ***" <>
_Person eight <5148842954> next person available. <Email.one#gmail.com> <342453136512> Next Person *** <9876582694>
<first.man#gmail.com> <888888889> Second Man *** <9999999999>
To: Person One <>, "Person two" <person.two#gmail.com>, , "Person Three ***" <>
_Person eight <5148842954> next person available. <Email.one#gmail.com> <342453136512> Next Person *** <9876582694> <first.man#gmail.com> <888888889> Second Man *** <9999999999>
<fourth.man#gmail.com> <444444444> Thrid Man *** <3333333333>
To: Person One <>, "Person two" <person.two#gmail.com>, , "Person Three ***" <>
_Person eight <5148842954> next person available. <Email.one#gmail.com> <342453136512> Next Person *** <9876582694> <first.man#gmail.com> <888888889> Second Man *** <9999999999> <fourth.man#gmail.com> <444444444> Thrid Man *** <3333333333>
To: Person One <>, "Person two" <person.two#gmail.com>, , "Person Three ***" <>
_Person eight <5148842954> next person available. <Email.one#gmail.com> <342453136512> Next Person *** <9876582694> <first.man#gmail.com> <888888889> Second Man *** <9999999999> <fourth.man#gmail.com> <444444444> Thrid Man *** <3333333333>
There are a few issues I am encountering is
The first lines does not concatenate with the second.
in the second capture, each time a new line is read, it adds it to the buffer and the line is printed each time. although the print command is only list once. I also not sure why the last line is printed twice. I am sure am missing something simple.
i am also looking for a method of printing the filename of the file being read at at the beginning of the line two know where the date came from as i have thousands of files.
assistance is greatly appreciated, thank you in advance.

Related

Retrieve results between two patterns plus one line in sed

I would like to extract all lines between INFO:root:id is and one line after the INFO:root:newId.
Can anyone advise how I can achieve this?
Currently I'm using
sed -n '/INFO:root:id is/,/INFO:root:newId/p' 1/python.log
and I'm trying to figure out how to print one line after the second pattern match.
INFO:root:id is
INFO:root:16836211
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): abc.hh.com
DEBUG:urllib3.connectionpool:https://abc.hh.com:443 "POST /api/v2/import/row.json HTTP/1.1" 201 4310
INFO:root:newId
INFO:root:35047536
INFO:root:id is
INFO:root:46836211
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): abc.hh.com
DEBUG:urllib3.connectionpool:https://abc.hh.com:443 "POST /api/v2/import/row.json HTTP/1.1" 201 4310
INFO:root:newId
INFO:root:55547536
If I am understanding question correctly
$ seq 10 | sed -n '/3/,/5/{/5/N;p;}'
3
4
5
6
/3/ is starting regex and /5/ is ending regex
/5/N get additional line for ending regex
tested on GNU sed, syntax might differ for other versions
With awk
$ seq 10 | awk '/3/{f=1} f; /5/{f=0; if((getline a)>0) print a}'
3
4
5
6
Unclear whether you want only the first set of lines after a match or all matches.
If you want the first set between the matching patterns, it is easy if you use /INFO:root:id/ for your end match as well and then use head -n -1 to print everything but the last line.
$ sed -n '/INFO:root:id is/,/INFO:root:id/p' test.txt | head -n -1
INFO:root:id is
INFO:root:16836211
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): abc.hh.com
DEBUG:urllib3.connectionpool:https://abc.hh.com:443 "POST /api/v2/import/row.json HTTP/1.1" 201 4310
INFO:root:newId
INFO:root:35047536
Just use flags to indicate when you've found the beginning and ending regexps and print accordingly:
$ seq 10 | awk 'e{print buf $0; buf=""; b=e=0} /3/{b=1} b{buf = buf $0 ORS; if (/5/) e=1}'
3
4
5
6
Note that this does not have the potential issue of printing lines when there's only the beginning or ending regexp present but not both. The other answers, including your currently accepted answer, have that problem:
$ seq 10 | sed -n '/3/,/27/{/27/N;p;}'
3
4
5
6
7
8
9
10
$ seq 10 | awk '/3/{f=1} f; /27/{f=0; if((getline a)>0) print a}'
3
4
5
6
7
8
9
10
$ seq 10 | awk 'e{print buf $0; buf=""; b=e=0} /3/{b=1} b{buf = buf $0 ORS; if (/27/) e=1}'
$
Note that the script I posted correctly didn't print anything because a block of text beginning with 3 and ending with 27 was not present in the input.

Add a label indicating duplicate names [duplicate]

This question already has answers here:
Add double quotation on duplicated name
(4 answers)
Closed 5 years ago.
I tried to use
sed 's/ */:/' file | awk -F: '{ if (arr[$1":"$2]) print "\""$1"\":"$2; else { arr[$1":"$2]++; print $0 }}'
but cannot get ideal output. Thanks.
The following is the file information and the desired output that I want.
Text File:
Jon DeLoach:408-253-3122:123 Park St., San Jose, CA 04086:7/25/53:85100
Karen Evich:284-758-2857:23 Edgecliff Place, Lincoln, NB 92086:7/25/53:85100
Karen Evich:284-758-2867:23 Edgecliff Place, Lincoln, NB 92743:11/3/35:58200
Karen Evich:284-758-2867:23 Edgecliff Place, Lincoln, NB 92743:11/3/35:58200
Fred Fardbarkle:674-843-1385:20 Parak Lane, DeLuth, MN 23850:4/12/23:780900
Fred Fardbarkle:674-843-1385:20 Parak Lane, DeLuth, MN 23850:4/12/23:780900
Lori Gortz:327-832-5728:3465 Mirlo Street, Peabody, MA 34756:10/2/65:35200
Paco Gutierrez:835-365-1284:454 Easy Street, Decatur, IL 75732:2/28/53:123500
Paco Gutierrez:835-365-1284:454 Easy Street, Decatur, IL 75732:2/28/53:123500
Jesse Neal:408-233-8971:45 Rose Terrace, San Francisco, CA 92303:2/3/36:25000
Jesse Neal:408-233-8971:45 Rose Terrace, San Francisco, CA 92303:2/3/36:25000
Zippy Pinhead:834-823-8319:2356 Bizarro Ave., Farmount, IL 84357:1/1/67:89500
Required output: Add stars indicating duplicated names
Jon DeLoach:408-253-3122:123 Park St., San Jose, CA 04086:7/25/53:85100
*Karen Evich*:284-758-2857:23 Edgecliff Place, Lincoln, NB 92086:7/25/53:85100
*Karen Evich*:284-758-2867:23 Edgecliff Place, Lincoln, NB 92743:11/3/35:58200
*Karen Evich*:284-758-2867:23 Edgecliff Place, Lincoln, NB 92743:11/3/35:58200
*Fred Fardbarkle*:674-843-1385:20 Parak Lane, DeLuth, MN 23850:4/12/23:780900
*Fred Fardbarkle*:674-843-1385:20 Parak Lane, DeLuth, MN 23850:4/12/23:780900
Lori Gortz:327-832-5728:3465 Mirlo Street, Peabody, MA 34756:10/2/65:35200
*Paco Gutierrez*:835-365-1284:454 Easy Street, Decatur, IL 75732:2/28/53:123500
*Paco Gutierrez*:835-365-1284:454 Easy Street, Decatur, IL 75732:2/28/53:123500
*Jesse Neal*:408-233-8971:45 Rose Terrace, San Francisco, CA 92303:2/3/36:25000
*Jesse Neal*:408-233-8971:45 Rose Terrace, San Francisco, CA 92303:2/3/36:25000
Zippy Pinhead:834-823-8319:2356 Bizarro Ave., Farmount, IL 84357:1/1/67:89500
Give a test to this. Seems to work ok.
$ awk -F":" 'NR==FNR{a[$1]++;next}(a[$1]>1){sub($1,"*" $1 "*")}1' file1 file1
Explanation:
This code reads the same file twice. This maybe has a performance penalty depending on the filesize.
-F":" : Global Input Fields Delimiter is defined as :
NR==FNR{a[$1]++;next} : The code in { } is executed when NR==FNR = the first file is read by awk
a[$1]++ : Creates an array a with index $1 and value ++ => +1 for each $1 found. So for record 1 we have a[Jon DeLoach]=1. For Record2 a[Karen Evich]=1, for record 3 a[Karen Evich]++ => 2,etc
next : instructs awk to go to the next record and skip the rest script.
(a[$1]>1){sub($1,"*" $1 "*")}1 : This condition & action is performed on the second file. For each a[$1] found in second file with a value >1 (as has been finalized when the first file finished), we insert * around $1 using awk sub function. sub function applies substitution directly to $0 = Whole record.
1 : prints the whole record of the second file.

If matched then print all using awk

I have a file which contains many sub-sections each starting with [begin] and ending with [end]:
[begin li1_1378184738754_91]
header=7075|lime|0|0|109582|0|1|2700073||0|0|0|[355]|1|0|ssb-li1-1378184738754-90||0||LIME |0|saved=true|0.002406508312038836|0|[ser=zu1:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=uzu6:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=xzs5:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=sv-stda-zu3:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=hzu8:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=lzu3:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=yzu2:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=xzu7:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer]|0|null|false|40||false|
attrs=0|0|0||0|
ptitle=690751404|1|1|1|Rest:1998636||||||2700401|175619|900.5636134725806|0.985486|39.166666666666664|$9.99|100.0|1|||
seller=1998636|1|9.99|1|-1||0|||||true||4.7937584|10412|false|
ptitle=5543369186|2|1|1|Rest:1533891||||||2700211|19615|886.8211044369053|0.776121|34.0|$119.99|100.0|1|||
seller=1533891|1|119.99|3|-1|1.0:text,In+size+6.0%2C7.0%2C8.0%2C8.5%2C9.0%2C9.5%2C10.0%2C...,0.0,,,,0,0,|2|||||true||2.95|20|true|
ptitle=622529158|3|1|1|||||||2700408|67402|796.5289827432475|0.893899|63.0|$5.27|100.0|1|||
seller=4281413|1|5.27|1|-1||0|||||true||4.695052|1769|true|
ptitle=5507199621|4|1|1|||||||2700220|56412|706.9031281251306|0.791171|45.0|$99.99|100.0|1|||
seller=4806107|1|-1.0|1|-1|1.0:sale,$,30.000000000000014,0.0,,,,0,0,:text,In+size+6.0%2C6.5%2C7.0%2C7.5%2C8.0%2C8.5%2C9.0%2C9...,0.0,,,,0,0,|2||||$130 $30.00 off|false||5.0|1|false|
ptitle=5502728013|5|1|1|||||||900000|0|698.7772340643119|0.836740|75.0|$40.95|100.0|1|||
seller=955448|1|40.95|1|-1||0|||||false||4.142857|7|false|
ptitle=840662011|6|1|1|Rest:265238||||||300233|62718|683.2927820751431|0.995513|52.0|$22.95|100.0|1|||
seller=265238|1|22.95|1|-1||0|||||false||4.478261|23|false|
ptitle=848084980|8|1|1|||||||2700073|145653|670.4809846773688|0.880587|60.0|$24.99|100.0|1|||
seller=5267046|1|24.99|1|-1||0|||||true||0.0|0|false|
ptitle=891200492|9|1|1|Rest:1030132||||||2701003|17215|668.8437575254773|0.825491|32.0|$519.99|100.0|1|||
seller=1030132|1|519.99|1|-1||0|||||false||4.7391305|23|false|
ptitle=641974054|10|1|1|||||||900000|69433|667.6678790058678|0.752129|57.0|$4.19|100.0|1|||
seller=3365158|1|4.19|1|-1||0|||||true||4.70907|4410|true|
ptitle=517591869|12|1|1|Rest:4802895||||||2700408|127644|643.0972570735605|0.893899|17.25|$23.95|100.0|1|||
seller=4318776|1|-1.0|3|-1||0|||||false||0.0|0|false|
ptitle=541549480|13|1|1|Rest:1180414||||||2702000|105832|597.4904572011968|0.752129|24.666666666666664|$8.27|100.0|1|||
seller=4636561|1|8.27|1|-1||0|||||false||4.8283377|734|true|
ptitle=1020561900|14|1|1|||||||2700063|159813|594.4717491579845|0.934869|75.0|$5.39|100.0|1|||
seller=4722645|1|5.39|1|-1|1.0:sale,$,0.6000000000000005,0.0,,,,0,0,:text,Free+Shipping+on+All+Orders%21,0.0,201301010000/,,,0,0,|2||||$5.99 $0.60 off|true||4.3942246|1593|true|
ptitle=507792308|15|1|1|Rest:4683455||||||2702000|105832|591.7739184402442|0.768311|22.5|$9.48|100.0|1|||
seller=4910651|1|-1.0|2|-1||0|||||false||5.0|1|false|
ptitle=1090571346|16|1|1|Rest:4452919||||||2700211|20824|776.4814913363535|0.776121|35.0|$59.99|100.0|1|||
seller=1533891|1|59.99|1|-1|1.0:sale,$,49.99999999999999,0.0,,,,0,0,:text,In+size+7.5%2C8.0%2C8.5%2C9.0%2C9.5%2C10.0%2C10.5...,0.0,,,,0,0,|2||||$110 $50.00 off|true||2.95|20|true|
ptitle=573017390|17|1|1|||||||2700073|91937|679.695660577044|0.880587|33.5|$14.85|100.0|1|||
seller=4281413|1|14.85|1|-1||0|||||true||4.695052|1769|true|
ptitle=5502723300|18|1|1|||||||900000|0|639.3095640940136|0.836740|75.0|$50.95|100.0|1|||
seller=955448|1|50.95|1|-1||0|||||false||4.142857|7|false|
ptitle=940022974|20|1|1|||||||2700600|58701|569.9503499778303|0.875839|59.0|$14.40|100.0|1|||
seller=4825227|1|14.4|1|12||0|||||true||4.0289855|276|true|
ptitle=5513277553|21|1|1|||||||2700220|56412|565.2712749001105|0.776121|44.33333333333333|$129.95|100.0|1|||
seller=4825252|1|129.95|1|23||0|||||true||4.0289855|276|true|
ptitle=890329961|22|1|1|||||||2700408|133796|564.7642425785796|0.837916|34.75|$61.95|100.0|1|||
seller=4825235|1|61.95|4|19||0|||||true||4.0289855|276|true|
ptitle=753852910|24|1|1|||||||2700073|146738|557.7419123688652|0.934869|47.69230769230769|$26.99|100.0|1|||
seller=4722645|1|26.99|10|-1|1.0:sale,$,3.0,0.0,,,,0,0,:text,Free+Shipping+on+All+Orders%21,0.0,201301010000/,,,0,0,|2||||$29.99 $3.00 off|true||4.3942246|1593|true|
ptitle=654738989|26|1|1|||||||900000|84012|554.7756559595525|0.752129|57.0|$3.19|100.0|1|||
seller=3365158|1|3.19|1|-1||0|||||true||4.70907|4410|true|
ptitle=707747307|27|1|1|Rest:4736009||||||2700063|76249|552.234395428327|0.889614|19.857142857142854|$6.39|100.0|1|||
seller=4736009|1|6.39|1|-1||0|||||false||4.8071113|15356|true|
ptitle=63531001|28|1|1|||||||2700408|82712|625.0421885589608|0.893899|47.166666666666664|$7.69|100.0|1|||
seller=4281413|1|7.69|3|-1||0|||||true||4.695052|1769|true|
ptitle=5502728016|29|1|1|||||||900000|0|605.9895507237038|0.836740|75.0|$503.00|100.0|1|||
seller=955448|1|503.0|1|-1||0|||||false||4.142857|7|false|
ptitle=507792308|31|1|1|Rest:4683455||||||2702000|105832|559.6902659046442|0.752129|22.5|$8.99|100.0|1|||
seller=5105812|1|-1.0|1|-1||0|||||false||0.0|0|false|
ptitle=753852910|32|1|1|||||||2700073|146738|545.9987095658629|0.870929|47.69230769230769|$22.49|100.0|1|||
seller=4143386|1|22.49|6|-1|1.0:sale,$,7.5,0.0,,,,0,0,:text,Free+Shipping+on+Orders+Over+%24100,0.0,201109010000/201409302359,,,0,0,|2||||$29.99 $7.50 off|false||4.7316346|2355|true|
ptitle=5513277553|33|1|1|Rest:1533891||||||2700220|56412|653.3133907916089|0.825491|44.33333333333333|$149.99|100.0|1|||
seller=1533891|1|149.99|3|-1|1.0:text,In+size+5.0%2C5.5%2C6.0%2C6.5%2C7.0%2C7.5%2C8.0%2C8...,0.0,,,,0,0,|2|||||true||2.95|20|true|
ptitle=63531001|34|1|1|||||||2700408|82712|541.8233547780552|0.893899|47.166666666666664|$7.72|100.0|1|||
seller=2370155|1|7.72|4|-1||0|||||false||4.85|40|false|
ptitle=1018957017|35|1|1|||||||2700073|145653|540.6093714604533|0.860614|56.0|$25.95|100.0|1|||
seller=5036683|1|25.95|1|-1||0|||||false||4.8405056|366|false|
ptitle=743682867|36|1|1|||||||2700073|63437|539.5985846455641|0.870929|58.0|$46.99|100.0|1|||
seller=193176|1|46.99|1|-1||0|||||true||4.8511987|1418|true|
ptitle=679858288|37|1|1|||||||2700063|188669|535.1360632897284|0.902031|30.0|$12.41|100.0|1|||
seller=4143386|1|12.41|2|-1|1.0:sale,$,1.379999999999999,0.0,,,,0,0,:text,Free+Shipping+on+Orders+Over+%24100,0.0,201109010000/201409302359,,,0,0,|2||||$13.79 $1.38 off|false||4.7316346|2355|true|
ptitle=994328713|38|1|1|||||||2700073|71463|534.7715925279717|0.870929|58.0|$1.29|100.0|1|||
seller=1787388|1|1.29|1|-1||0|||||false||4.680464|3624|false|
ptitle=886915818|40|1|1|||||||2700444|201835|529.7519801432289|0.934869|65.5|$44.99|100.0|1|||
seller=4561883|1|44.99|2|-1||0|||||true||4.7913384|508|false|
seller_hidden=227502|990765963|1147436601|-1
seller_hidden=5310958|622529158|5645627277|-1
seller_hidden=4825254|5543369186|5651114316|23
seller_hidden=5289138|5548930281|5653769481|-1
[end li1_1378184738754_91]
I am trying to run the command cat /home/nextag/logs/OutpdirImpressions.log.2013-09-02 | awk -F "$begin" '{print $0}' | awk '$0 ~ "header=7075" {print $0}'
As per this command i want to split the entire file into sub-sections beginning with the word 'begin'. Now in that i want those sub-sections which contains 'header=7075'
Expected output is that it will print the entire sub-section(those which contain that string), but i am getting only this portion as output:
header=7075|lime|0|0|109582|0|1|2700073||0|0|0|[355]|1|0|ssb-li1-1378184738754-90||0||LIME
|0|saved=true|0.002406508312038836|0|[ser=zu1:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=uzu6:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=xzs5:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=sv-stda-zu3:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=hzu8:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=lzu3:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=yzu2:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer][ser=xzu7:mtu=model_other_20120806calibex.csv:mu=model_other_20120806calibex.csv:scorerClassUsed=LinearPersonalizedProductSearchScorer]|0|null|false|40||false|
I have tried using if in various ways, but it doesn't works. Can somebody please help me.
I tried awk -F "$begin" '{if($0 ~ "header=7075") {print $0}}' /home/nextag/logs/OutpdirImpressions.log.2013-09-02. It gave the same result
Can somebody please suggest that how do i get the complete sub-section in the result
Try this awk one-liner:
awk '$1=="[end"{p=0}/^header=7075/{p=1}p' file
In parts:
$1=="[end"{p=0} if you reach a line, with the first word "[end", then set the flag to zero
/^header=7075/{p=1} If you reach a line, which begins with "header=7075", set set the flag to one
p if the flag is non-zero, print the current line (equivalent to p{print} or p{print $0} or p!=0{print $0}

search for a string between repetitive tags in a txt file

Need some help with respect to search for a string between repetitive tags
I have a text file with following format repeating many times within a file
========== File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_1 START ==========
Block 1
TC ID OK
Block 2
Input section OK
data section OK
Block 3
Input section OK
data section OK
Block 4
Input section OK
data section OK
========== File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_2 START ========
Block 1
TC ID OK
Block 2
Input section OK
line mismatch:
output line: "Buffer allocated from pool: MIF_CTRL_POOL, buffer_id: 1, size (words): 4"
reference line: "pdcp_pdu_delete_count = 0, reset_cip_rdy = 1"
line mismatch:
output line: "pdcp_pdu_delete_count = 0, reset_cip_rdy = 1"
reference line: "-- MIF CTRL output: --------------------------------------------------------------------------------"
line mismatch:
output line: "-- MIF CTRL output: --------------------------------------------------------------------------------"
reference line: "mif_ctrl_rlc_am_um_reset_reestablish_ind_t.rlc_reset_reestablish_ind = 3 (0x0003)"
line mismatch:
output line: "mif_ctrl_rlc_am_um_reset_reestablish_ind_t.rlc_reset_reestablish_ind = 3 (0x0003)"
reference line: "Buffer released from pool: MIF_CTRL_POOL, buffer_id 0, size (words) 6 (used 5)"
line count mismatch:
last output line: "mif_ctrl_rlc_am_um_reset_reestablish_ind_t.rlc_reset_reestablish_ind = 3 (0x0003)"
last reference line: "Buffer released from pool: MIF_CTRL_POOL, buffer_id 0, size (words) 6 (used 5)"
data section DIFFERS
Block 3
Input section OK
data section OK
Block 4
Input section OK
data section OK
========== File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_3 START ========
Block 1
TC ID OK
Block 2
Input section OK
data section OK
Block 3
Input section OK
data section OK
Block 4
Input section OK
data section OK
**I need to find if anything other than 'OK' exists between the Start tags and if yes i have to mark the particular block as failed
for example if i find any other than OK between Test Case: Test_Case_1 START and Test Case: Test_Case_2 START i have to mark Test Case: Test_Case_1 as failed
UPDATED
Expected Output in somewhat this format
File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_1 Status: PASS
(if there is no string as 'DIFFERS' between tags (==)
File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_2 Status: FAILED
(if there is string 'DIFFERS' between tags (==)
UPDATE -2
If in case Test Case fails
File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_2 Status: FAILED
Section of block differing is:
Block 2
Input section OK
line mismatch:
output line: "Buffer allocated from pool: MIF_CTRL_POOL, buffer_id: 1, size (words): 4"
reference line: "pdcp_pdu_delete_count = 0, reset_cip_rdy = 1"
line mismatch:
output line: "pdcp_pdu_delete_count = 0, reset_cip_rdy = 1"
reference line: "-- MIF CTRL output: --------------------------------------------------------------------------------"
line mismatch:
output line: "-- MIF CTRL output: --------------------------------------------------------------------------------"
reference line: "mif_ctrl_rlc_am_um_reset_reestablish_ind_t.rlc_reset_reestablish_ind = 3 (0x0003)"
line mismatch:
output line: "mif_ctrl_rlc_am_um_reset_reestablish_ind_t.rlc_reset_reestablish_ind = 3 (0x0003)"
reference line: "Buffer released from pool: MIF_CTRL_POOL, buffer_id 0, size (words) 6 (used 5)"
line count mismatch:
last output line: "mif_ctrl_rlc_am_um_reset_reestablish_ind_t.rlc_reset_reestablish_ind = 3 (0x0003)"
last reference line: "Buffer released from pool: MIF_CTRL_POOL, buffer_id 0, size (words) 6 (used 5)"
data section DIFFERS
Wild guess:
while(<>) {print '**FAIL**' unless /TC ID OK/; print $_; }
But really, you should specify your requirements.
Maybe something quirky along the lines of the following:
#!/usr/bin/env perl
use strict;
use warnings;
my ($failed, $file, $test);
while (<>)
{
chomp;
next if /^$/;
if (/^=/)
{
print "$file $test Status: $failed\n" if $failed;
($file, $test) = ($_ =~ /(File Name:\s+\S+).+\b(Test Case: Test_Case_\d+)\b/);
$failed = 'PASSED';
next;
};
$failed = 'FAILED' if /\bDIFFERS\b/g;
}
print "$file $test Status: $failed\n";
Data
$ cat testdata
========== File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_1 START ==========
Block 1 TC ID OK
Block 2 Input section OK data section OK
Block 3 Input section OK data section OK
Block 4 Input section OK data section OK
========== File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_2 START ========
Block 1 TC ID OK
Block 2 Input section OK data section DIFFERS
Block 3 Input section OK data section OK
Block 4 Input section OK data section OK
Invocation
$ ./t.pl < testdata
File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_1 Status: PASSED
File Name: fixed_am_7bitLI_HE10.txt Test Case: Test_Case_2 Status: FAILED

sed remove line containing a string and nothing but; automation using for loop

Q1: Sed specify the whole line and if the line is nothing but the string then delete
I have a file that contains several of the following numbers:
1 1
3 1
12 1
1 12
25 24
23 24
I want to delete numbers that are the same in each line. For that I have either been using:
sed '/1 1/d' < old.file > new.file
OR
sed -n '/1 1/!p' < old.file > new.file
Here is the main problem. If I search for pattern '1 1' that means I get rid of '1 12' as well. So for I want the pattern to specify the whole line and if it does, to delete it.
Q2: Automation of question 1
I am also trying to automate this problem. The range of numbers in the first column and the second column could be from 1 to 25.
So far this is what I got:
for ((i=1;i<26;i++)); do
sed "/'$i' '$i'/d" < oldfile > newfile; mv newfile oldfile;
done
This does nothing to the oldfile in the end. :(
This would be more readable with awk:
awk '$1 == $2 {next} {print}' oldfile > newfile
Update based on comment:
If the requirement is to remove lines where the two values are within 1 of each other:
awk '{d = $1-$2; if (-1 <= d && d <= 1) next; else print}' oldfile
Unfortunately, awk does not have abs() (at least nawk and gawk don't)
Just put the first number in a group (\([0-9]*\)) and then look for it with a backreference (\1). Since the line to delete should contain only the group, repeated, use the ^ to mark the beginning of line and the $ to mark the end of line. For example, for the following file:
$ cat input
1 1
3 1
12 1
1 12
12 12
12 13
13 13
25 24
23 24
...the result is:
$ sed '/^\([0-9]*\) \1$/d' input
3 1
12 1
1 12
12 13
25 24
23 24
You can also do it with grep:
grep -E -v "([0-9])*\s\1" testfile
Look for multiple digits in a row and remember them, followed by a single whitespace, followed by whatever digits you remembered.