I have a graph QL schema file that has comments between triple speech marks;
"""
some text
"""
I need to remove the blocks and was wondering if anyone knows how to do this with regular expressions as I can then use sed or grep to remove the blocks
I don't own the API so cannot change the output.
Perhaps sed:
sed '/"""/,/"""/d' <<END
a
"""
comment 1
"""
b
"""
comment 2
"""
c
END
a
b
c
Related
I need to append a few lines to a configuration file. The format is something like follows:
[Topic1]
param=foo
param=bar
param=foobar
[Topic2]
param=one
param=two
etc...
I am trying to write a script using sed to append parameters to a specific topic. Since all the topics have param=, I can't just insert a line after the last occurrence of that string. Also, I can't count on the value of the last parameter being consistent so for example I can't just insert a line after the string param=two
Any help would be appreciated. I'm not too familiar with mutliline sed-fu.
Thanks!
sed -i -r ':a; N; $!ba; s/\[Topic1\]\n(param=[a-zA-Z]*\n)*/¶m=VALUE\n/g' FILE_NAME
Basically what :a; N; $!ba; doing is append all line when not the last line (N) to the tag created by :a so that we can use \n in our expression.
Then match [Topic1] followed by arbitrary number of param=xxx, and append param=VALUE to the end of the matching result (&).
I want to insert a range of lines from a file, say something like 210,221r before the first occurrence of a pattern in a bunch of other files.
As I am clearly not a GNU sed expert, I cannot figure how to do this.
I tried
sed '0,/pattern/{210,221r file
}' bunch_of_files
But apparently file is read from line 210 to EOF.
Try this:
sed -r 's/(FIND_ME)/PUT_BEFORE\1/' test.text
-r enables extendend regular expressions
the string you are looking for ("FIND_ME") is inside parentheses, which creates a capture group
\1 puts the captured text into the replacement.
About your second question: You can read the replacement from a file like this*:
sed -r 's/(FIND_ME)/`cat REPLACEMENT.TXT`\1/' test.text
If replace special characters inside REPLACEMENT.TXT beforehand with sed you are golden.
*= this depends on your terminal emulator. It works in bash.
In https://stackoverflow.com/a/11246712/4328188 CodeGnome gave some "sed black magic" :
In order to insert text before a pattern, you need to swap the pattern space into the hold space before reading in the file. For example:
sed '/pattern/ {
h
r file
g
N
}' in
However, to read specific lines from file, one may have to use a two-calls solution similar to dummy's answer. I'd enjoy knowing of a one-call solution if it is possible though.
How does the begin and end keywords in sed actually works ? Do we have to specifically mention these keywords in the data file ?
For example, if I'm trying to delete empty lines using sed using the below code:
sed -n '/begin/,/end/ {
s/^$/ d
p
}
'
Now, should the data file should have begin and end keywork in it ? I'm sorry I've tried using these two keywords without actually entering them in the data and it doesn't give me the expected o/p.
If you want to remove all empty lines from your file, you use:
sed '/^$/d` file
or (remove also lines only contains tabs or spaces):
sed '/^\s*$/d' file
if you want to remove empty lines only between BBB line and AAA line:
sed '/BBB/,/AAA/{/^$/d}` file
and yes, BBB and AAA must be in your file.
I have a file like as below and I want to search for the pattern "Unix" and insert the content of another file two lines after the line where the pattern is matched. I want to do it in sed.
$ cat text1
Unix
Windows
Database
Wintel
Sql
Java
$
Output should be
Unix
Windows
Database
CONTENT OF ANOTHER FILE
Wintel
Sql
Java
It looks a bit funny, but this works with both GNU sed and BSD sed (on Mac OS X), and should work with most versions of sed:
sed -e '/Unix/{N;N;p;r content' -e 'd;}' data
Or:
sed -e '/Unix/{
N
N
p
r content
d
}' data
The N commands add extra lines to the pattern space (so the pattern space holds three lines containing Unix, Windows and Database); the p command prints the pattern space; the r content reads the file content and adds it to the output; the d deletes the pattern space; the {} group these operations so that they only occur when the input line matches Unix.
The r content must be at the end of a line of the script, or at the end of a -e argument, as shown. Trying to add a semicolon after it does not work (after all, the file name might contain a semicolon).
This might work for you (GNU sed):
sed '/Unix/!b;n;n;r another_file' text1
If the line doesn't contain unix bail out. Otherwise print it and get the next line, repeat and then read in the second file.
N.B. The second line following unix is printed first as this is now part of the current cycle, another_file is inserted into the pattern space following the end of the current cycle.
Let's say we want to do some substitutions only between some patterns, let them be <a> and </a> for clarity... (all right, all right, they're start and end!.. Jeez!)
So I know what to do if start and end always occur on the same line: just design a proper regex.
I also know what to do if they're guaranteed to be on different lines and I don't care about anything in the line containing end and I'm also OK with applying all the commands in the line containing start before start: just specify the address range as /start/,/end/.
This, however, doesn't sound very useful. What if I need to do a smarter job, for instance, introduce changes inside a {...} block?
One thing I can think of is breaking the input on { and } before processing and putting it back together afterwards:
sed 's/{\|}/\n/g' input | sed 'main stuff' | sed ':a $!{N;ba}; s/\n\(}\|{\)\n/\1/g'
Another option is the opposite:
cat input | tr '\n' '#' | sed 'whatever; s/#/\n/g'
Both of these are ugly, mainly because the operations are not confined within a single command. The second one is even worse because one has to use some character or substring as a "newline holder" assuming it isn't present in the original text.
So the question is: are there better ways or can the above-mentioned ones be optimized? This is quite a regular task from what I read in recent SO questions, so I'd like to choose the best practice once and for all.
P.S. I'm mostly interested in pure sed solutions: can the job be do with one invocation of sed and nothing else? Please no awk, Perl, etc.: this is more of a theoretical question, not a "need the job done asap" one.
This might work for you:
# create multiline test data
cat <<\! >/tmp/a
> this
> this { this needs
> changing to
> that } that
> that
> !
sed '/{/!b;:a;/}/!{$q;N;ba};h;s/[^{]*{//;s/}.*//;s/this\|that/\U&/g;x;G;s/{[^}]*}\([^\n]*\)\n\(.*\)/{\2}\1/' /tmp/a
this
this { THIS needs
changing to
THAT } that
that
# convert multiline test data to a single line
tr '\n' ' ' </tmp/a >/tmp/b
sed '/{/!b;:a;/}/!{$q;N;ba};h;s/[^{]*{//;s/}.*//;s/this\|that/\U&/g;x;G;s/{[^}]*}\([^\n]*\)\n\(.*\)/{\2}\1/' /tmp/b
this this { THIS needs changing to THAT } that that
Explanation:
Read the data into the pattern space (PS). /{/!b;:a;/}/!{$q;N;ba}
Copy the data into the hold space (HS). h
Strip non-data from front and back of string. s/[^{]*{//;s/}.*//
Convert data e.g. s/this\|that/\U&/g
Swap to HS and append converted data. x;G
Replace old data with converted data.s/{[^}]*}\([^\n]*\)\n\(.*\)/{\2}\1/
EDIT:
A more complicated answer which I think caters for more than one block per line.
# slurp file into pattern space (PS)
:a
$! {
N
ba
}
# check for presence of \v if so quit with exit value 1
/\v/q1
# replace original newlines with \v's
y/\n/\v/
# append a newline to PS as a delimiter
G
# copy PS to hold space (HS)
h
# starting from right to left delete everything but blocks
:b
s/\(.*\)\({.*}\).*\n/\1\n\2/
tb
# delete any non-block details form the start of the file
s/.*\n//
# PS contains only block details
# do any block processing here e.g. uppercase this and that
s/th\(is\|at\)/\U&/g
# append ps to hs
H
# swap to HS
x
# replace each original block with its processed one from right to left
:c
s/\(.*\){.*}\(.*\)\n\n\(.*\)\({.*}\)/\1\n\n\4\2\3/
tc
# delete newlines
s/\n//g
# restore original newlines
y/\v/\n/
# done!
N.B. This uses GNU specific options but could be tweaked to work with generic sed's.