I'd like to replace commas within brackets with spaces (and also remove the brackets). I used sed, but the solution I could come up to is dependent on the elements in the list.
sed 's/\[\(.*\), \(.*\)\]/\1 \2/g'
# [-0.0, 1.23] => -0.0 1.23 (works)
# [-0.0, 1.23, 4.56] => -0.0, 1.23 4.56 (doesn't work)
# foo=[12.3, 4.5, 3.0, 4.1], bar=123.0, xyz=6.7 => foo=12.3, 4.5, 3.0 4.1, bar=123.0, xyz=6.7` (doesn't work, expected: foo=12.3 4.5 3.0 4.1, bar=123.0, xyz=6.7)
Is there any way sed can be used to do what I want?
Consider this test file:
$ cat file
[-0.0, 1.23]
[-0.0, 1.23, 4.56]
foo=[12.3, 4.5, 3.0, 4.1], bar=123.0, xyz=6.7
[1,2,-3,4]
To remove any commas within square brackets and also the remove square brackets:
$ sed -E ':a; s/(\[[^],]*), */\1 /; ta; s/\[([^]]*)\]/\1/g' file
-0.0 1.23
-0.0 1.23 4.56
foo=12.3 4.5 3.0 4.1, bar=123.0, xyz=6.7
1 2 -3 4
How it works
:a
This defines a label a.
s/(\[[^],]*), */\1 /
This looks for the first comma within a square bracket and removes it.
[^],] matches any character except ] or ,. Thus, (\[[^],]*) matches [ followed by any number of characters not ] or , and stores the result in group 1.
ta
If the above substitution resulted in a change, jump back to label a so we can try the substitution again.
s/\[([^]]*)\]/\1/g
After we have finished removing commas, this removes the square brackets.
Note that [^]] matches any character that is not ]. Thus \[([^]]*)\] matches a [ followed by any number of any character except ] followed by ]. In other words, it matches a single bracketed expression and the contents of the expression, excluding the square brackets, are stored in group 1.
Related
My code has a ton of occurrences of something like:
idof(some_object)
I want to replace them with:
some_object["id"]
It sounds simple:
sed -i 's/idof(\([^)]\+\))/\1["id"]/g' source.py
The problem is that some_object might be something like idof(get_some_object()), or idof(my_class().get_some_object()), in which case, instead of getting what I want (get_some_object()["id"] or my_class().get_some_object()["id"]), I get get_some_object(["id"]) or my_class(["id"].get_some_object()).
Is there a way to have sed match closing bracket, so that it internally keeps track of any opening/closing brackets inside my (), and ignores those?
It needs to keep everything that's between those brackets: idof(ANYTHING) becomes ANYTHING["id"].
Using sed
$ sed -E 's/idof\(([[:alpha:][:punct:]]*)\)/\1["id"]/g' input_file
Using ERE, exclude idof and the first opening parenthesis.
As a literal closing parenthesis is also excluded, everything in-between the capture parenthesis including additional parenthesis will be captured.
[[:alpha:]] will match all alphabetic characters including upper and lower case while [[:punct:]] will capture punctuation characters including ().-{} and more.
The g option will make the substitution as many times as the pattern is found.
Theoretically, you can write a regex that will handle all combinations of idof(....) up to some limit of nested () calls inside ..... Such regex would have to list with all possible combinations of calls, like idof(one(two(three))) or idof(one(two(three)four(five)) you can match with an appropriate regex like idof([^()]*([^()]*([^()]*)[^()]*)[^()]*) or idof([^()]*([^()]*([^()]*)[^()]*([^()]*)[^()]*) respectively.
The following regex handles only some cases, but shows the complexity and general path. Writing a regex to handle all possible cases to "eat" everything in front of the trailing ) is left to OP as an exercise why it's better to use something else. Note that handling string literals ")" becomes increasingly complex.
The following Bash code:
sed '
: begin
# No idof? Just print the line!
/^\(.*\)idof(\([^)]*)\)/!n
# Note: regex is greedy - we start from the back!
# Note: using newline as a stack separator.
s//\1\n\2/
# hold the front
{ h ; x ; s/\n.*// ; x ; s/[^\n]*\n// ; }
: handle_brackets
# Eat everything before final ) up to some number of nested ((())) calls.
# Insert more jokes here.
: eat_brackets
/^[^()]*\(([^()]*\(([^()]*\(([^()]*\(([^()]*\(([^()]*\(([^()]*)\)\?[^()]*)\)\?[^()]*)\)\?[^()]*)\)\?[^()]*)\)\?[^()]*)\)/{
s//&\n/
# Hold the front.
{ H ; x ; s/\n\([^\n]*\)\n.*/\1/ ; x ; s/[^\n]*\n// ; }
b eat_brackets
}
/^\([^()]*\))/!{
s/^/ERROR: eating brackets did not work: /
q1
}
# Add the id after trailing ) and remove it.
s//\1["id"]/
# Join with hold space and clear the hold space for next round
{ H ; s/.*// ; x ; s/\n//g ; }
# Restart for another idof if in input.
b begin
' <<EOF
before idof(some_object) after
before idof(get_some_object()) after
before idof(my_class().get_some_object()) after
before idof(one(two(three)four)five) after
before idof(one(two(three)four)five) between idof(one(two(three)four)five) after
before idof( one(two(three)four)five one(two(three)four)five ) after
before idof(one(two(three(four)five)six(seven(eight)nine)ten) between idof(one(two(three(four)five)six(seven(eight)nine)ten) after
EOF
Will output:
before some_object["id"] after
before get_some_object()["id"] after
before my_class().get_some_object()["id"] after
before one(two(three)four)five["id"] after
before one(two(three)four)five["id"] between one(two(three)four)five["id"] after
before one(two(three)four)five one(two(three)four)five ["id"] after
ERROR: eating brackets did not work: one(two(three(four)five)six(seven(eight)nine)ten) after
The last line is not handled correctly, because (()()) case is not correctly handled. One would have to write a regex to match it.
This question already has answers here:
Remove Left and right square brackets using sed/bash
(1 answer)
Remove a pattern using sed which has square brackets and quotes
(2 answers)
How can I use sed to delete line with square brackets?
(2 answers)
Closed 1 year ago.
How can I remove the line "name['todo']['remove'] = 3456" from a text file?
[test.txt]
name['myname']['test'] = 12
name['todo']['remove'] = 3456
name['todo']['remove']['inspection'] = 34
My current approach is not working as expected. The line is still in my file.
sed -i "name\['todo'\]\['remove'\]" test.txt
The error message is "sed: -e expression #1, char 2: extra characters after command"
A simple grep -vF would work fine here that matches using fixed string without requiring escaping of special regex characters:
grep -ivF "name['todo']['remove'] " file
[test.txt]
name['myname']['test'] = 12
name['todo']['remove']['inspection'] = 34
You can use
sed -i "/name\['todo']\['remove'] =/d" test.txt
Note that the pattern is wrapped with / regex delimiters, and the d means the matched line will get removed.
See an online demo:
s="[test.txt]
name['myname']['test'] = 12
name['todo']['remove'] = 3456
name['todo']['remove']['inspection'] = 34"
sed "/name\['todo']\['remove'] =/d" <<< "$s"
yielding
[test.txt]
name['myname']['test'] = 12
name['todo']['remove']['inspection'] = 34
If you want to make sure you only match a whole line with digits after =, you may use "/^name\['todo']\['remove'] = [0-9]*$/d" command with sed.
I have the following string:
signal[i]
signal[bg]
output [10:0]
input [i:1]
what I want is to replace the letters between square brackets (by underscore for example) and to keep the other strings that represents table declaration:
signal[_]
signal[__]
output [10:0]
input [i:1]
thanks
try:
awk '{gsub(/\[[a-zA-Z]+\]/,"[_]")} 1' Input_file
Globally substituting the (bracket)alphabets till their longest match then with [_]. Mentioning 1 will print the lines(edited or without edited ones).
EDIT: Above will substitute all alphabets with one single _, so to get as many underscores as many characters are there following may help in same.
awk '{match($0,/\[[a-zA-Z]+\]/);VAL=substr($0,RSTART+1,RLENGTH-2);if(VAL){len=length(VAL);;while(i<len){q=q?q"_":"_";i++}};gsub(/\[[a-zA-Z]+\]/,"["q"]")}1' Input_file
OR
awk '{
match($0,/\[[a-zA-Z]+\]/);
VAL=substr($0,RSTART+1,RLENGTH-2);
if(VAL){
len=length(VAL);
while(i<len){
q=q?q"_":"_";
i++
}
};
gsub(/\[[a-zA-Z]+\]/,"["q"]")
}
1
' Input_file
Will add explanation soon.
EDIT2: Following is the one with explanation purposes for OP and users.
awk '{
match($0,/\[[a-zA-Z]+\]/); #### using match awk's built-in utility to match the [alphabets] as per OP's requirement.
VAL=substr($0,RSTART+1,RLENGTH-2); #### Creating a variable named VAL which has substr($0,RSTART+1,RLENGTH-2); which will have substring value, whose starting point is RSTART+1 and ending point is RLENGTH-2.
RSTART and RLENGTH are the variables out of the box which will be having values only when awk finds any match while using match.
if(VAL){ #### Checking if value of VAL variable is NOT NULL. Then perform following actions.
len=length(VAL); #### creating a variable named len which will have length of variable VAL in it.
while(i<len){ #### Starting a while loop which will run till the value of VAL from i(null value).
q=q?q"_":"_"; #### creating a variable named q whose value will be concatenated it itself with "_".
i++ #### incrementing the value of variable i with 1 each time.
}
};
gsub(/\[[a-zA-Z]+\]/,"["q"]") #### Now globally substituting the value of [ alphabets ] with [ value of q(which have all underscores in it) then ].
}
1 #### Mentioning 1 will print (edited or non-edited) lines here.
' Input_file #### Mentioning the Input_file here.
Alternative gawk solution:
awk -F'\\[|\\]' '$2!~/^[0-9]+:[0-9]$/{ gsub(/./,"_",$2); $2="["$2"]" }1' OFS= file
The output:
signal[_]
signal[__]
output [10:0]
-F'\\[|\\]' - treating [ and ] as field separators
$2!~/^[0-9]+:[0-9]$/ - performing action if the 2nd field does not represent table declaration
gsub(/./,"_",$2) - replace each character with _
This might work for you (GNU sed);
sed ':a;s/\(\[_*\)[[:alpha:]]\([[:alpha:]]*\]\)/\1_\2/;ta' file
Match on opening and closing square brackets with any number of _'s and at least one alpha character and replace said character by an underscore and repeat.
awk '{sub(/\[i\]/,"[_]")sub(/\[bg\]/,"[__]")}1' file
signal[_]
signal[__]
output [10:0]
input [i:1]
The explanation is as follows: Since bracket is as special character it has to be escaped to be handled literally then it becomes easy use sub.
I have some java code declaring a 2d array that I want to flip.
Content is like:
zData[0][0] = 198;
zData[0][1] = 198;
zData[0][2] = 198;
...
And I want to flip indices to have
zData[0][0] = 198;
zData[1][0] = 198;
zData[2][0] = 198;
So I tried doing it with sed:
sed -r 's#zData[([0-9]*)][([0-9]*)]#zData[\2][\1]#g' DataSample1.java
But unfortunately sed says:
sed: -e expression #1, char 43: Unmatched ) or \)
Might the string "zData" hold kind of flag or option?
I tried not using the -r option but I have the same kind of message for:
sed 's#zData[\(\[\0\-\9\]\*\)][\(\[\0\-\9\]\*\)]#zData[\2][\1]#g' DataSample1.java
Thanks for your help
Simples:
$ sed -r 's/(zData)(\[[^]]+])(\[[^]]+])/\1\3\2/' file
zData[0][0] = 198;
zData[1][0] = 198;
zData[2][0] = 198;
Regexplanation:
# Match
(zData) # Capture the variable name we want to transpose
( # Start capture group for first index
\[ # Opening bracket escaped to mean literal [
[^]]+ # One or more none ] characters i.e the digits
] # The closing literal ] doesn't need escaping here.
) # Close the capture
(\[[^]]+]) # Same regexp as before for the second index
# Replace
\1\3\2 # Switch the indexes but rearranging the 2nd and 3rd capture groups
Note: Switch \[[^]]+] to if it is clearer \[[0-9]+] for you, so instead of saying match an opening square bracket followed by one or more none-closing brackets followed by a closing bracket you are saying match an opening square bracket followed by one or more digit followed by a closing bracket.
Try that one:
sed 's#\([a-zA-Z0-9_-]\+\)\(\[[^]]*\]\)\(\[[^]*]\]\)\(.*$\)#\1\3\2\4#'
It adds four captures for the variable name, the first index, the second index and the rest and then switches order.
Edit: #Sudo_O's solution with extended regular expressions is much more readable. Thx for that! Nevertheless, on some systems sed -r may not be available, since it is not part of basic POSIX.
The tv.txt file is as following:
mms://live21.gztv.com/gztv_gz 广州台[可于Totem/VLC/MPlayer播放，记得把高宽比设置成4:3]
mms://live21.gztv.com/gztv_news 广州新闻台·直播广州(可于Totem/VLC/MPlayer播放，记得把高宽比设置成4:3)
mms://live21.gztv.com/gztv_kids 广州少儿台(可于Totem/VLC/MPlayer播放，记得把高宽比设置成4:3)
mms://live21.gztv.com/gztv_econ 广州经济台
I want to group it into three groups.
sed -r 's/([^ ]*)\s([^][()]*)((\(.+\))*|(\[.+\])*)/\3/' tv.txt
got the result:
[可于Totem/VLC/MPlayer播放，记得把高宽比设置成4:3]
(可于Totem/VLC/MPlayer播放，记得把高宽比设置成4:3)
(可于Totem/VLC/MPlayer播放，记得把高宽比设置成4:3)
When I write it into
sed -r 's/([^ ]*)\s([^][()]*)((\(.+\))*|(\[.+\])*)/\3/' tv.txt
It can't work.
The only difference is [^][()] and [^[]()]; neither of the [^\[\]()] ,escape characters can not make it run properly.
I want to know the reason.
The POSIX rules for getting ] into a character class are a little arcane, but they make sense when you think about it hard.
For a positive (non-negated) character class, the ] must be the first character:
[]and]
This recognizes any character a, n, d or ] as part of the character class.
For a negated character class, the ] must be the first character after the ^:
[^]and]
This recognizes any character except a, n, d or ] as part of the character class.
Otherwise, the first ] after the [ marks the end of the character class. Inside a character class, most of the normal regex special characters lose their special meaning, and others (notably - minus) acquire special meanings. (If you want a - in a character class, it has to be 'first' or last, where 'first' means 'after the optional ^ and only if ] is not present'.)
In your examples:
[^][()] — this is a negated character class that recognizes any character except [, ], ( or ), but
[^[]()] — this is a negated character class that recognizes any character except [, followed by whatever () symbolizes in the regex family you're using, and ] which represents itself.