$ echo '' | sed -e '$a\new content' | cat -n
1
2 new content
but if we don't put new-line, there's no output at all:
$ echo -n '' | sed -e '$a\new content' | cat -n
$
Another questions (even more important) are: Can it be worked around? How?
According to
Eric Pement, this is
not possible with sed.
However awk can do this easily
$ printf '' | awk '{print} END {print "new content"}'
new content
Related
When I run this command:
bjobs -r -P xenon -W | awk '{print $7}' | grep -v JOB_NAME |
cut -f 1 -d ' ' | xargs
in a terminal, all running JOB_NAMES are coming, but when I do this in per_script only JOB_ID are coming.
Perl script code is below:
#dummy_jobs = qx/bjobs -r -P xenon -W | awk '{print $7}' | grep -v JOB_NAME | cut -f 1 -d ' ' | xargs/;
What needs to be changed in Perl?
qx/.../ literals are very much like double-quoted strings. Specifically, $7 is interpolated, so you end up passing ... | awk '{print }' | ....
Replace
qx/...$7.../
with
qx/...\$7.../
Or if you prefer, you can use
my $shell_cmd = <<'EOS'; # These single-quotes means you get exactly what follows.
bjobs -r -P xenon -W | awk '{print $7}' | grep -v JOB_NAME | cut -f 1 -d ' ' | xargs
EOS
my #dummy_jobs = qx/$shell_cmd/;
Another difference is that qx uses /bin/sh instead of whatever shell you were using, but that shouldn't be relevant here.
This is mostly by curiosity, I am trying to have the same behavior as:
echo -e "test1:test2:test3"| sed 's/:/\n/g' | grep 1
in a single sed command.
I already tried
echo -e "test1:test2:test3"| sed -e "s/:/\n/g" -n "/1/p"
But I get the following error:
sed: can't read /1/p: No such file or directory
Any idea on how to fix this and combine different types of commands into a single sed call?
Of course this is overly simplified compared to the real usecase, and I know I can get around by using multiple calls, again this is just out of curiosity.
EDIT: I am mostly interested in the sed tool, I already know how to do it using other tools, or even combinations of those.
EDIT2: Here is a more realistic script, closer to what I am trying to achieve:
arch=linux64
base=https://chromedriver.storage.googleapis.com
split="<Contents>"
curl $base \
| sed -e 's/<Contents>/<Contents>\n/g' \
| grep $arch \
| sed -e 's/^<Key>\(.*\)\/chromedriver.*/\1/' \
| sort -V > out
What I would like to simplify is the curl line, turning it into something like:
curl $base \
| sed 's/<Contents>/<Contents>\n/g' -n '/1/p' -e 's/^<Key>\(.*\)\/chromedriver.*/\1/' \
| sort -V > out
Here are some alternatives, awk and sed based:
sed -E "s/(.*:)?([^:]*1[^:]*).*/\2/" <<< "test1:test2:test3"
awk -v RS=":" '/1/' <<< "test1:test2:test3"
# or also
awk 'BEGIN{RS=":"} /1/' <<< "test1:test2:test3"
Or, using your logic, you would need to pipe a second sed command:
sed "s/:/\n/g" <<< "test1:test2:test3" | sed -n "/1/p"
See this online demo. The awk solution looks cleanest.
Details
In sed solution, (.*:)?([^:]*1[^:]*).* pattern matches an optional sequence of any 0+ chars and a :, then captures into Group 2 any 0 or more chars other than :, 1, again 0 or more chars other than :, and then just matches the rest of the line. The replacement just keeps Group 2 contents.
In awk solution, the record separator is set to : and then /1/ regex is used to only return the record having 1 in it.
This might work for you (GNU sed):
sed 's/:/\n/;/^[^\n]*1/P;D' file
Replace each : and if the first line in the pattern space contains 1 print it.
Repeat.
An alternative:
sed -Ez 's/:/\n/g;s/^[^1]*$//mg;s/\n+/\n/;s/^\n//' file
This slurps the whole file into memory and replaces all colons by newlines. All lines that do not contain 1 are removed and surplus newlines deleted.
An alternative to the really ugly sed is: grep -o '\w*2\w*'
$ printf "test1:test2:test3\nbob3:bob2:fred2\n" | grep -o '\w*2\w*'
test2
bob2
fred2
grep -o: only matching
Or: grep -o '[^:]*2[^:]*'
echo -e "test1:test2:test3" | sed -En 's/:/\n/g;/^[^\n]*2[^\n]*(\n|$)/P;//!D'
sed -n doesn't print unless told to
sed -E allows using parens to match (\n|$) which is newline or the end of the pattern space
P prints the pattern buffer up to the first newline.
D trims the pattern buffer up to the first newline
[^\n] is a character class that matches anything except a newline
// is sed shorthand for repeating a match
//! is then matching everything that didn't match previously
So, after you split into newlines, you want to make sure the 2 character is between the start of the pattern buffer ^ and the first newline.
And, if there is not the character you are looking for, you want to D delete up to the first newline.
At that point, it works for one line of input, with one string containing the character you're looking for.
To expand to several matches within a line, you have to ta, conditionally branch back to label :a:
$ printf "test1:test2:test3\nbob3:bob2:fred2\n" | \
sed -En ':a s/:/\n/g;/^[^\n]*2[^\n]*(\n|$)/P;D;ta'
test2
bob2
fred2
This is simply NOT a job for sed. With GNU awk for multi-char RS:
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' '/1/'
test1
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' 'NR%2'
test1
test3
test5
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' '!(NR%2)'
test2
test4
test6
$ echo "foo1:bar1:foo2:bar2:foo3:bar3" | awk -v RS='[:\n]' '/foo/ || /2/'
foo1
foo2
bar2
foo3
With any awk you'd just have to strip the \n from the final record before operating on it:
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS=':' '{sub(/\n$/,"")} /1/'
test1
Is there a way to do the following in a single sed command to improve performance?
cat some_file | sed -n '/^MODIFIED/p' | sed 's/^MODIFIED\s*//g'
You could try the below sed command. -n and p helps to print those lines where the replacement takesplace. [[:space:]]* POSIX notation which matches zero or more spaces.
sed -n 's/^MODIFIED[[:space:]]*//p' some_file
OR
sed -n 's/^MODIFIED\s*//p' some_file
Example:
$ cat ri
MODIFIED foo bar
apple
mango
$ cat ri | sed -n '/^MODIFIED/p' | sed 's/^MODIFIED\s*//g'
foo bar
$ sed -n 's/^MODIFIED[[:space:]]*//p' ri
foo bar
Here is an awk version:
awk 'gsub(/^MODIFIED\s*/,"")' file
Example:
cat file
test
MODIFIED data
more MODIFIED home
awk 'gsub(/^MODIFIED\s*/,"")' file
data
You can remove both cat and sed, just use last sed like:
sed -nr 's/^MODIFIED\s*//p' some_file
thanks in advance for the help.
I have the following line that does work on linux.
myfile (extract)
active_instance_count=
aq_tm_processes=1
archive_lag_target=0
audit_file_dest=?/rdbms/audit
audit_sys_operations=FALSE
audit_trail=NONE
background_core_dump=partial
background_dump_dest=/home1/oracle/app/oracle/admin/iopecom/bdump
...
cat myfile |sed -r 's/ {1,}//g'|sed -r 's/\t*//g' |grep -v "^#"|sed -s "/^$/d" |sed =|sed 'N;s/\n/\t/'|sed -r "s/#.*//g" | sed "s/\t/;/g"|sed "s/\t/;/g"|sed -e "s,',\o042,g"
The result will be:
1;O7_DICTIONARY_ACCESSIBILITY=TRUE
2;active_instance_count=
3;aq_tm_processes=1
4;archive_lag_target=0
5;audit_file_dest=?/rdbms/audit
6;audit_sys_operations=FALSE
7;audit_trail=NONE
8;background_core_dump=partial
9;background_dump_dest=/home1/oracle/app/oracle/admin/iopecom/bdump
But, I can't figure out, how to perform the same command on AIX server.
Help is very welcome.
Regards.
Antonio.
Unless you have a compelling reason to use sed, you could use alternate tools:
awk -v OFS=';' '{print NR,$0}' filename
would produce the desired output.
You could also use perl:
perl -ne 'print "$.;$_"' filename
It appears that your sed expression would skip lines beginning with a #. As such, you could say:
perl -ne '$,=";"; !/^#/ && print ++$i,$_' filename
or something like:
grep -v '^#' filename | awk ...
reformatting your pipeline:
cat myfile |
sed -r 's/ {1,}//g' | # strip all spaces (1)
sed -r 's/\t*//g' | # strip all tabs (2)
grep -v "^#" | # delete all lines beginning `#` (3)
sed -s "/^$/d" | # delete all empty lines (4)
sed = | # interleave with line numbers (5)
sed 'N;s/\n/\t/' | # join line number and line with `\t` (6)
sed -r "s/#.*//g" | # strip all `#` comments (7)
sed "s/\t/;/g" | # replace all tabs with `;` (8)
sed "s/\t/;/g" | # do it again (9)
sed -e "s,',\o042,g" # replace all ' with " (10)
Boiling that down and using cat -n to provide the line numbers up front gets:
cat -n myfile |
sed "$(print 's/\t/;/')
$(print 's/[ \t]*//g')
s/#.*//g
/^$/d
s/'/\"/g"
which behaves identically unless I'm misreading the aix docs. The $(...) construction is command substitution, it runs that command and substitutes its output. print would be printf on linux.
I try to extract digits with sed:
echo hgdfjg678gfdg kjg45nn | sed 's/.*\([0-9]\+\).*/\1/g'
but result is:
5
How to extract: 678 and 45?
Thanks in advance!
The problem is that the . in .* will match digits as well as non-digits, and it keeps on matching as long as it can -- that is as long as there's one digit left unconsumed that can match the [0-9].
Instead of extracting digits, just delete non-digits:
echo hgdfjg678gfdg kjg45nn | sed 's/[^0-9]//g'
or even
echo hgdfjg678gfdg kjg45nn | tr -d -c 0-9
You may use grep with option -o for this:
$ echo hgdfjg678gfdg kjg45nn | grep -E -o "[0-9]+"
678
45
Or use tr:
$ echo hgdfjg678gfdg kjg45nn | tr -d [a-z]
678 45
.* in sed is greedy. And there are no non-greedy option AFAIK.(You must use [^0-9]* in this case for non-greedy matching. But this works only once, so you will get only 678 without 45.)
If you must use only sed, it would not be easy to get the result.
I recommend to use gnu’s grep
$ echo hgdfjg678gfdg kjg45nn | grep -oP '\d+'
678
45
If you really want to stick to sed, this would be one of many possible answers.
$ echo hgdfjg678gfdg kjg45nn | \
sed -e 's/\([0-9^]\)\([^0-9]\)/\1\n\2/g' | \
sed -n 's/[^0-9]*\([0-9]\+\).*/\1/p’
678
45