Can I use sed to check the first line of some command's output (to stdout) and delete this very first line if it matches a certain pattern?
Say, the command's output is something like this:
"AB"
"CD"
"E"
"F"
I want it to become:
"CD"
"E"
"F"
But when the first line is "GH", I don't want to delete the line.
I tried this, but it doesn't work:
<some_command> |sed '1/<pattern>/d'
The shell told me:
sed: 0602-403 1/<pattern>/d is not a recognized function.
I only want to use sed to process the first line, leaving the other lines untouched.
What is the correct syntax here?
This might work for you:
sed -e '1!b' -e '/GH/!d' file
You want to reference the 1st line, then say delete:
$ sed '1 d' file
No need for any pattern if you know which line you want to delete.
With a pattern, use this syntax:
$ sed '0,/pattern/ d' file
This is what you want:
$ sed '1{/"GH"/!d}' file
sed '1{/<pattern>/{/GH/!d}}' input
The error in your expression can be fixed like this:
sed '1{/<pattern>/d}' input
Related
Can I avoid duplicate strings with the sed "a" command?
I added the word "apple" under "true" in my file.txt.
The problem is that every time I run the command "apple" is appended.
$ sed -i '/true/a\apple' file.txt ...execute 3 time
$ cat file.txt
true
apple
apple
apple
If the word "apple" already exists, repeating the sed command does not want to add any more.
I have no idea, please help me
...
I want to do this,
...execute sed command anytime
$ cat file.txt
true
apple
It seems you don't want to append the line apple if the line following the true already contains apple. Then this sed command should do the trick.
sed -i.backup '
/true/!b
$!{N;/\napple$/!s/\n/&apple&/;p;d;}
a\
apple
' file.txt
Explanation of sed commands:
If the line doesn't contain true then jump to the end of the script, which will print out the line read (/true/!b).
Otherwise the line contains true:
If it isn't the last line ($!) then• read the next line (N).• If the next line doesn't consist of apple (/\napple$/!) then insert the apple between two lines (s/\n/&apple&/).• Print out the pattern space (p) and start a new cycle (d)
Otherwise it is the last line (and contains true)
Append apple (a\ apple)
Edit:
The above sed script won't work properly if two consecutive true line occurs in the file, as pointed out by #potong. The version below should fix this, if I haven't overlooked something.
sed -i.backup ':a
/true/!b
a\
apple
n
/^apple$/d
ba
' file.txt
Explanation:
/true/!b: If the line doesn't contain true, no further processing is required. Jump to the end of the script. This will print the current pattern space.
a\ apple: Otherwise, the line contains true. Append apple.
n: Print the current pattern space and appended line (apple) and replace the pattern space with the next line. This will end the script if no next line available.
/^apple$/d: If the line read consists of string apple then delete it and start a new cycle (because it is already appended before)
ba: Jump to the start of the script (label a) without reading an input line.
There is no general solution for sed unless the file is sorted. If sorted, the following deletes the duplicate lines:
sed '$!N; /^\(.*\)\n\1$/!P; D'
This was taken from this link: https://www.unix.com/shell-programming-and-scripting/146404-command-remove-duplicate-lines-perl-sed-awk.html
Great answer by M. Nejat Aydin but to make things simpler just add grep:
grep -q apple file.txt || sed -i '/true/a\apple' file.txt
This might work for you (GNU sed):
sed -e ':a;/true/!b;$a apple' -e 'n;/apple/b;i apple' -e 'ba' file
If a line does not contain true just print it.
Otherwise, if it is the last line, append the line apple.
Otherwise, print that line and fetch the next.
If that line contains apple just print it.
Otherwise, insert a line apple and jump to the first sed instruction since the fetched line might be one containing true.
N.B. This uses both the a command (for end of file condition) and the i command for when there is a following line.
I couldn't make it work the way I want.
echo 'https://aaa/xx_one/bbb/xx_two/ccc' | sed 's|^\(https://.*\)xx_|\1OK_|g'
https://aaa/xx_one/bbb/OK_two/ccc
The output I wanted is: https://aaa/OK_one/bbb/OK_two/ccc
That is, I just want to replace every occurrences of xx_ with OK_ and with that specific constraint.
Need some help. I want it in sed. Thanks.
If the line should start with https://, which you originally didn't say, you can use an "address" in sed:
sed '\=^https://= s/xx_/OK_/g'
Original answer: (will replace xx_ by OK_ after https://, but the line doesn't have to start with it).
Perl to the rescue:
perl -pe '1 while s=https://.*\Kxx_=OK_=g'
-p reads the input line by line, prints each line after processing;
\K in a regular expression forgets what's been matched so far. In this case, it will only replace the xx_ by OK_, but it still has to match https://.* before it;
The substitution runs in a while loop until there's nothing to replace.
I am attempting to delete a line from a text file if it matches a regular expression. To accomplish this I was using sed in an Ubuntu environment combined with regular expressions. I have tried/referenced the following solutions: Sol1, Sol2, Sol3.
My current command is: sed '/[^"]+},/d' test.json with this command I am attempting to match and remove lines like:
{"hello},
{"go penguins!},
{"someone help1),
I am NOT trying to match or remove lines like: "should not match regex"}, Any line that ends with "}, should not be deleted.
I am not tied to using sed so any acceptable answer would work so long as my text file would look something like:
...
{"omg this is amazing"},
{"thanks for your help"},
{"no problem"},
...
How about sed '/\"},/!d' test.json?
It should by sed '/\"},/d' test.json (without !)
I want to substring the File name in unix using sed command.
File name : Test_Test1_Test2_10082019_030013.csv.20191008-075740
I want the characters after the 3rd underscore or (all the characters after Test2 ) i need to be printed .
Can this be done using sed command?
I have tried this command
sed 's/^.*_\([^_]*\)$/\1/' <<< 'Test_Test1_Test2_10082019_030013.csv.20191008-075740'
but this is giving result as 030013.csv.20191008-075740
I need it from 10082019_030013.csv.20191008-075740
Thanks
Neha
To remove from the beginning up to including the 3rd underscore you can use
sed 's/^\([^_]*_\)\{3\}//' <<< 'Test_Test1_Test2_10082019_030013.csv.20191008-075740'
This removes the initial part that consists of 3 groups of (any number of non-underscore characters followed by an underscore). The result is
10082019_030013.csv.20191008-075740
If you use GNU sed you can switch it to extended regular expressions and omit the backslashes.
sed -r 's/^([^_]*_){3}//' <<< 'Test_Test1_Test2_10082019_030013.csv.20191008-075740'
Could you please try following.
sed 's/\([^_]*\)_\([^_]*\)_\([^_]*\)_\(.*\)/\4/' Input_file
Or as per Bodo's nice suggestion:
sed 's/[^_]*_[^_]*_[^_]_\(.*\)/\1/' Input_file
This might work for you (GNU sed):
sed 's/_/\n/3;s/.*\n//;t;s/Test2/\n/;s/.*\n//;t;d' file
Replace the third _ by a newline and then remove everything upto and including the first newline. If this succeeds, bail out and print the result. Otherwise, try the same method with Test2 and if this fails delete the entire line.
I'm very much a junior when it comes to the sed command, and my Bruce Barnett guide sits right next to me, but one thing has been troubling me. With a file, can you filter it using sed to select only specific items? For example, in the following file:
alpha|november
bravo|october
charlie|papa
alpha|quebec
bravo|romeo
charlie|sahara
Would it be possible to set a command to return only the bravos, like:
bravo|october
bravo|romeo
With sed:
sed '/^bravo|/!d' filename
Alternatively, with grep (because it's sort of made for this stuff):
grep '^bravo|' filename
or with awk, which works nicely for tabular data,
awk -F '|' '$1 == "bravo"' filename
The first two use a regular expression, selecting those lines that match it. In ^bravo|, ^ matches the beginning of the line and bravo| the literal string bravo|, so this selects all lines that begin with bravo|.
The awk way splits the line across the field separator | and selects those lines whose first field is bravo.
You could also use a regex with awk:
awk '/^bravo|/' filename
...but I don't think this plays to awk's strengths in this case.
Another solution with sed:
sed -n '/^bravo|/p' filename
-n option => no printing by default.
If line begins with bravo|, print it (p)
2 way (at least) with sed
removing unwanted line
sed '/^bravo\|/ !d' YourFile
Printing only wanted lines
sed -n '/^bravo\|/ p' YourFile
if no other constraint or action occur, both are the same and a grep is better.
If there will be some action after, it could change the performance where a d cycle directly to the next line and a p will print then continue the following action.
Note the escape of pipe is needed for GNU sed, not on posix version