I want to extract /battle/result from following the txt file
$ cat sample
user_id=1234 /battle/start
I run following the sed command
$ cat sample | sed 's|.*\(/.*\)|\1|g'
/start
But, result is deleting /battle, so I can't extract it as I want.
What is wrong with it?
You can remove all characters up to last space:
$ sed 's/.* //' <<< "user_id=1234 /battle/start"
/battle/start
or use cut:
$ cut -d' ' -f2 <<< "user_id=1234 /battle/start"
/battle/start
Sed tries to do a greedy (maximal) match, therefore .* matches your whole line up to but not including the second /.
Try:
< sample sed 's|.* \(/.*\)|\1|g'
or
< sample sed 's|[^/]*\(/.*\)|\1|g'
In your RE the .* is greedy and swallows the /battle part, you could try to invert the logic and delete everything in front of /:
cat sample | sed 's/[^/]*//'
Here [^/]* matches everthing that is not a / and replaces it with nothing.
echo user_id=1234 /battle/start |grep -oP '\s\K.*'
/battle/start
echo user_id=1234 /battle/start |sed -r 's/(^.*\s)(.*)/\2/g'
/battle/start
Related
can please somebody help me with this?
I have this line
test.txt
siemplog1.nw.lan / 172.31.180.22
I tried this command sed -Ei "s/^[a-z A-Z].*([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/" test.txt
result should be 172.31.180.22 but I got this 2.31.180.22
thank you
The .* matches as many chars as it can (it is "greedy") and since [0-9]{1,3} can match just 1 digit, the 17 is matched by the .* and 2 is matched by [0-9]{1,3}.
You may stop the .* before any non-digit:
sed -Ei 's~.*[^0-9]([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*~\1~' test.txt
Or, before /:
sed -Ei 's~.*/ *([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*~\1~' test.txt
See online sed demo:
s='siemplog1.nw.lan / 172.31.180.22'
sed -E 's~.*/ *([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*~\1~' <<< "$s"
# => 172.31.180.22
If you string is always in this format, you might simplify the sed command to
sed -E 's~.*/ *([0-9.]+)~\1~p'
sed -E 's~.*/ *([0-9.]+).*~\1~p'
If you have space before ip
$echo siemplog1.nw.lan / 172.31.180.22 | sed -E "s/.* ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/"
172.31.180.22
$
This is mostly by curiosity, I am trying to have the same behavior as:
echo -e "test1:test2:test3"| sed 's/:/\n/g' | grep 1
in a single sed command.
I already tried
echo -e "test1:test2:test3"| sed -e "s/:/\n/g" -n "/1/p"
But I get the following error:
sed: can't read /1/p: No such file or directory
Any idea on how to fix this and combine different types of commands into a single sed call?
Of course this is overly simplified compared to the real usecase, and I know I can get around by using multiple calls, again this is just out of curiosity.
EDIT: I am mostly interested in the sed tool, I already know how to do it using other tools, or even combinations of those.
EDIT2: Here is a more realistic script, closer to what I am trying to achieve:
arch=linux64
base=https://chromedriver.storage.googleapis.com
split="<Contents>"
curl $base \
| sed -e 's/<Contents>/<Contents>\n/g' \
| grep $arch \
| sed -e 's/^<Key>\(.*\)\/chromedriver.*/\1/' \
| sort -V > out
What I would like to simplify is the curl line, turning it into something like:
curl $base \
| sed 's/<Contents>/<Contents>\n/g' -n '/1/p' -e 's/^<Key>\(.*\)\/chromedriver.*/\1/' \
| sort -V > out
Here are some alternatives, awk and sed based:
sed -E "s/(.*:)?([^:]*1[^:]*).*/\2/" <<< "test1:test2:test3"
awk -v RS=":" '/1/' <<< "test1:test2:test3"
# or also
awk 'BEGIN{RS=":"} /1/' <<< "test1:test2:test3"
Or, using your logic, you would need to pipe a second sed command:
sed "s/:/\n/g" <<< "test1:test2:test3" | sed -n "/1/p"
See this online demo. The awk solution looks cleanest.
Details
In sed solution, (.*:)?([^:]*1[^:]*).* pattern matches an optional sequence of any 0+ chars and a :, then captures into Group 2 any 0 or more chars other than :, 1, again 0 or more chars other than :, and then just matches the rest of the line. The replacement just keeps Group 2 contents.
In awk solution, the record separator is set to : and then /1/ regex is used to only return the record having 1 in it.
This might work for you (GNU sed):
sed 's/:/\n/;/^[^\n]*1/P;D' file
Replace each : and if the first line in the pattern space contains 1 print it.
Repeat.
An alternative:
sed -Ez 's/:/\n/g;s/^[^1]*$//mg;s/\n+/\n/;s/^\n//' file
This slurps the whole file into memory and replaces all colons by newlines. All lines that do not contain 1 are removed and surplus newlines deleted.
An alternative to the really ugly sed is: grep -o '\w*2\w*'
$ printf "test1:test2:test3\nbob3:bob2:fred2\n" | grep -o '\w*2\w*'
test2
bob2
fred2
grep -o: only matching
Or: grep -o '[^:]*2[^:]*'
echo -e "test1:test2:test3" | sed -En 's/:/\n/g;/^[^\n]*2[^\n]*(\n|$)/P;//!D'
sed -n doesn't print unless told to
sed -E allows using parens to match (\n|$) which is newline or the end of the pattern space
P prints the pattern buffer up to the first newline.
D trims the pattern buffer up to the first newline
[^\n] is a character class that matches anything except a newline
// is sed shorthand for repeating a match
//! is then matching everything that didn't match previously
So, after you split into newlines, you want to make sure the 2 character is between the start of the pattern buffer ^ and the first newline.
And, if there is not the character you are looking for, you want to D delete up to the first newline.
At that point, it works for one line of input, with one string containing the character you're looking for.
To expand to several matches within a line, you have to ta, conditionally branch back to label :a:
$ printf "test1:test2:test3\nbob3:bob2:fred2\n" | \
sed -En ':a s/:/\n/g;/^[^\n]*2[^\n]*(\n|$)/P;D;ta'
test2
bob2
fred2
This is simply NOT a job for sed. With GNU awk for multi-char RS:
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' '/1/'
test1
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' 'NR%2'
test1
test3
test5
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS='[:\n]' '!(NR%2)'
test2
test4
test6
$ echo "foo1:bar1:foo2:bar2:foo3:bar3" | awk -v RS='[:\n]' '/foo/ || /2/'
foo1
foo2
bar2
foo3
With any awk you'd just have to strip the \n from the final record before operating on it:
$ echo "test1:test2:test3:test4:test5:test6"| awk -v RS=':' '{sub(/\n$/,"")} /1/'
test1
In my script, have a possible version number: 15.03.2 set to variable $STRING. These numbers always change. I want to strip it down to: 15.03 (or whatever it will be next time).
How do I remove everything after the second . using sed?
Something like:
$(echo "$STRING" | sed "s/\.^$\.//")
(I don't know what ^, $ and others do, but they look related, so I just guessed.)
I think the better tool here is cut
echo '15.03.2' | cut -d . -f -2
This might work for you (GNU sed):
sed 's/\.[^.]*//2g' file
Remove the second or more occurrence of a period followed by zero or non-period character(s).
$ echo '15.03.2' | sed 's/\([^.]*\.[^.]*\)\..*/\1/'
15.03
More generally to skip N periods:
$ echo '15.03.2.3.4.5' | sed -E 's/(([^.]*\.){2}[^.]*)\..*/\1/'
15.03.2
$ echo '15.03.2.3.4.5' | sed -E 's/(([^.]*\.){3}[^.]*)\..*/\1/'
15.03.2.3
$ echo '15.03.2.3.4.5' | sed -E 's/(([^.]*\.){4}[^.]*)\..*/\1/'
15.03.2.3.4
How to replace '.' with '_' within the part of the line before the '=' char in the input below
Need single sed command to do all three
echo "few.num.dots=/home/user/.hidden/folder.dot" | sed 's/\./_/g'
required output => few_num_dots=/home/user/.hidden/folder.dot
echo "var=nodot" | sed 's/\./_/g'
required output => var=nodot
echo "var.one=onedot.notthis" | sed 's/\./_/g'
required output => var_one=onedot.notthis
You can use a conditional branching using the t command. It does a loop until the substitution command fails, and the command replaces any . character followed by an equal sign:
echo "few.num.dots=/home/user/.hidden/folder.dot" |
sed ':a; s/\.\([^=]*=\)/_\1/; ta'
It yields:
few_num_dots=/home/user/.hidden/folder.dot
perl?
echo "few.num.dots=/home/user/.hidden/folder.dot" |
perl -pe 's/^[^=]+/ ($x=$&) =~ tr{.}{_}; $x /e'
few_num_dots=/home/user/.hidden/folder.dot
awk?
awk -F= -v OFS='=' '{gsub(/\./,"_",$1)} 1'
You can do it this way as well,
echo "few.num.dots=/home/user/.hidden/folder.dot" |
sed -e '1,/./s/\./_/' -e '1,/./s/\./_/'
few_num_dots=/home/user/.hidden/folder.dot
First -e replaces the first occurrence of the pattern ., next -e replaces the next one...
Using awk
$ echo "few.num.dots=/home/user/.hidden/folder.dot" |awk '/=/{gsub(/\./,"_",$1)}1' FS="=" OFS="="
few_num_dots=/home/user/.hidden/folder.dot
$ echo "var.one=onedot.notthis" |awk '/=/{gsub(/\./,"_",$1)}1' FS="=" OFS="="
var_one=onedot.notthis
This might work for you (GNU sed):
sed 's/=/\n&/;h;y/./_/;G;s/\n.*\n.*\n//' file
Insert a marker to divide the line, copy the line, translate the characters, append the original line and using the marker reconstitute the line.
Suppose I have a string like this
<start><a></a><a></a><a></a></start>
I want to replace values inside <start></start> like this
<start><ab></ab><ab></ab><ab></ab><more></more><vale></value></start>
How do I do this using Sed?
Try this :
sed 's#<start>.*</start>#<start><ab></ab><ab></ab><ab></ab></start>#' file
I get this line with gnu sed :
sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:a>:ab>:g")"\3"#ge'
see example:
kent$ echo "<start><a></a><a></a><a></a><foo></foo><bar></bar></start>"|sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:a>:ab>:g")"\3"#ge'
<start><ab></ab><ab></ab><ab></ab><foo></foo><bar></bar></start>
note
this will replace the tags between <start>s which ending with a . which worked for your example. but if you have <aaa></aaa>:
you could do: (I break it into lines for better reading)
sed -r 's#(<start>)(.*)(</start>)
#echo "\1"$(echo "\2"\|sed "s:<a>:<ab>:g;s:</a>:</ab>:g")"\3"
#ge'
e.g.
kent$ echo "<start><a></a><a></a><a></a><aaa></aaa><aba></aba></start>" \
|sed -r 's#(<start>)(.*)(</start>)#echo "\1"$(echo "\2"\|sed "s:<a>:<ab>:g;s:</a>:</ab>:g")"\3"#ge'
<start><ab></ab><ab></ab><ab></ab><aaa></aaa><aba></aba></start>
sed 's/(\<\/?)a\>/\1ab\>/g' yourfile, though that would get <a></a> that was outside <start> as well...
grep -rl 'abc' a.txt | xargs sed -i 's/abc/def/g'