I need to extract in linux bash from a string a set of characters which are included between a static string and the first number.
A simple example should be helpful:
Base string: hello-world-my_name-1.0.jar
Static string: hello-world-
Target: my_name
I'm trying with
ls *.jar | sed 's/(?<=hello-world-)(.+?(?=-[0-9]))/\1/'
but unfortunately I can't understand where I'm wrong
Match everything, capture your middle string in a group and replace everything with the captured value:
printf 'hello-world-my_name-1.0.jar\n' | sed 's/hello-world-\([^0-9]*\).*/\1/'
Output:
my_name-
If the hyphen needs to be replaced too, match it outside the group; or remove it with a second pattern:
printf 'hello-world-my_name-1.0.jar\n' | sed 's/hello-world-\([^0-9]*\)-.*/\1/'
# or
printf 'hello-world-my_name-1.0.jar\n' | sed 's/hello-world-\([^0-9]*\).*/\1/;s/-$//'
Alternatively, if the middle string cannot contain hyphens, add it to the character class:
printf 'hello-world-my_name-1.0.jar\n' | sed 's/hello-world-\([^0-9-]*\).*/\1/'
using sed, i would like to extract the first match between square brackets.
i couldn't come up with a matching regex, since it seems that sed is greedy in its regex. for instance, given the regex \[.*\] - sed will match everything between the first opening bracket and the last closing bracket, which is not what i am after (would appreciate your help on this).
but until i could come up with a regex for that, i made an assumption that there must be a space after the closing bracket, to come up with a regex that will let me continue my work \[[^ ]*\].
i have tried it with grep, e.g.
$ echo '++ *+ ++ + [SPAM] foo(): z.y.o ## [x.y.z]----- ' | grep -oE '\[[^ ]*\]'
[SPAM]
[x.y.z]
i would like to use the regex in sed (not in grep) and output the first match (i.e. [SPAM]). i have tried it as follows, but wasn't able to do that
$ echo '++ *+ ++ + [SPAM] foo(): z.y.o ## [x.y.z]----- ' | sed 's/\[[^ ]*\]/\1/'
sed: 1: "s/\[[^ ]*\]/\1/": \1 not defined in the RE
$ echo '++ *+ ++ + [SPAM] foo(): z.y.o ## [x.y.z]----- ' | sed 's/\(\[[^ ]*\]\)/\1/'
++ *+ ++ + [SPAM] foo(): z.y.o ## [x.y.z]-----
would appreciate if you could assist me in:
constructing a regex to match all text between every opening and closing square brackets (see grep example above)
use the regex in sed and output only the first occurrence of the match
You can use this sed:
s='++ *+ ++ + [SPAM] foo(): z.y.o ## [x.y.z]----- '
sed -E 's/[^[]*(\[[^]]*\]).*/\1/' <<< "$s"
[SPAM]
Here:
[^[]* match 0 or more of any non-[ character
(\[[^]]*\]) matches a [...] substring and captures in group #1
.* matches rest of the string till end
\1 in substitution puts value captured in group #1 back in output
An awk solution would be nice as well:
awk 'match($0, /\[[^]]*\]/){print substr($0, RSTART, RLENGTH)}' <<< "$s"
[SPAM]
You can use
grep -o '\[[^][]*]' <<< "$text"
sed -n 's/^[^[]*\(\[[^][]*]\).*/\1/p' <<< "$text"
See the online demo. Details:
grep -o '\[[^][]*]' - outputs only matching substrings that meet the pattern: [, then zero or more chars other than [ and ], and then a ] char
sed -n 's/^[^[]*\(\[[^][]*]\).*/\1/p':
-n - suppresses default line output
^[^[]*\(\[[^][]*]\).* - matches start of string, then zero or more chars other than [, then captures into Group 1 a [, then any zero or more chars other than [ and ] and then a ] char, and then matches the rest of the string
\1 - replaces the match with Group 1 value
p - prints the result of the replacement.
I want to extract an atomic symbols inside a parentheses using sed.
The data I have is in the form C(X12), and I only want the X symbol
EX: that a test command :
echo "C(Br12)" | sed 's/[0-9][0-9])$//g'
gives me C(Br.
You can use
sed -n 's/.*(\(.*\)[0-9]\{2\})$/\1/p'
See the online demo:
sed -n 's/.*(\(.*\)[0-9]\{2\})$/\1/p' <<< "c(Br12)"
# => Br
Details
-n - suppresses the default line output
.*(\(.*\)[0-9]\{2\})$ - a regex that matches
.* - any text
( - a ( char
\(.*\) - Capturing group 1: any text up to the last....
[0-9]\{2\} - two digits
)$ - a ) at the end of string
\1 - replaces with Group 1 value
p - prints the result of the substitution.
For example:
echo "C(Br12)" | sed 's/C(\(.\).*/\1/'
C( - match exactly literally C(
. match anything
\(.\) - match anythig - one character- and "remember" it in a backreference \1
.* ignore everything behind it
\1 - replace it by the stuff that was remembered. The first character.
Research sed, regex and backreferences for more information.
Try using the following command
echo "C(BR12)" | cut -d "(" -f2 | cut -d ")" -f1 | sed 's/[0-9]*//g'
The cut tool will split and get you the string in middle of the paranthesis.Then pass the string to a sed for replacing the numbers inside the string.
Not a fully sed solution but this will get you the output.
Working in bash. I have a file.txt with contents in the following format.
default=60.0,
default=False,
default=(0.2, 0.2, 0.3, 1.0),
default="blend",
default="/path/to/directory")
There is other info on each line before and after what i am showing but this is the part i want to edit. i am trying to replace the values following the equals sign and before the comma with values stored in a variable. i can make it work if i know all the info of line number, what to replace, and what replacing with beforehand but as soon as i start working with variables it all falls apart. i can use sed -i '4s/default="blend",/default="burn",/' file.txt to replace blend with burn on line 4 but how can i do this with variables? i cant seem to find the right way to quote this. and obviously the data i am trying to replace is not consistent, some numbers, some strings, some quoted, some not, some bracketed, some not, some followed by ,, some followed by ). focused on line 4 currently, if var1=4 and var2="burn" and default= could be any of "blend, burn, or bump". so say i want to replace whatever follows default= with "burn" on line 4, var1=4 var2="burn"
Desired output:
default=60.0,
default=False,
default=(0.2, 0.2, 0.3, 1.0),
default="burn",
default="/path/to/directory")
hopefully what i am trying to accomplish makes sense because i'm stuck. how can i make this work? doesn't have to use sed if there is another simple way.
edit:
contents of entire line are:
parse.add_argument("-j", "--blend_type", default="blend", choices=["blend", "burn", "bump"], help="type of blend the shader can do")
using: sed -r "$var1{s/(default=\")(.*)(\".*$)/\1$var2\3/p}" file.txt
my output is parse.add_argument("-j", "--blend_type", default="burn")
and i want it to be parse.add_argument("-j", "--blend_type", default="burn", choices=["blend", "burn", "bump"], help="type of blend the shader can do")
i'm close but can't seem to keep the end of the line intact.
i can use sed -i '4s/default="blend",/default="burn",/' file.txt to replace blend with burn on line 4 but how can i do this with variables?
linenumber=4
from=blend
to=burn
# a bit extra: escape special characters
# https://stackoverflow.com/questions/407523/escape-a-string-for-a-sed-replace-pattern
from=$(printf '%s\n' "$from" | sed -e 's/[]\/$*.^[]/\\&/g');
to=$(printf '%s\n' "$to" | sed -e 's/[\/&]/\\&/g')
# Note the quoting
sed -i "${line}s/default=\"${from}\",/default=\"${to}\",/"
# ^^^^^^ - used ${line} so that 's' is separate
# ^^ ^^ ^^ ^^ - double quotes within double quoting need to be escaped
# ^ ^ - double quotes so that ${..} are expanded
sed -i '"$var1"s/default="blend",/default="burn",/' $f2
Variables are not expanded within single quotes.
sed -i "$var1"s/default="*"/default="$var2"/ file.txt
Regex is not glob. "* matches zero or more " characters. It does not match anything, * matches zero or more of preceding character or group. A dot . matches anything, so you could default=".*" or better I would go with default="[^"]*".
sed -i ""$var1"s/default="*",/default="$var2",/" file.txt
Use an editor with syntax highlighting. Even stackoverflow syntax highlighting should hint you - " close the other ". To write " within double quotation, escape them "\"".
Assumptions/provisos:
patterns to be searched and replaced can contain any printable characters which means ...
said patterns may need to include escape characters to keep from clashing with sed delimters and constructs
default=<search_pattern> only shows up once in a line
Sample data:
$ cat file.txt
abcde default=60.0, ABC
12345 default=False, ABC
STUVW default=(0.2, 0.2, 0.3, 1.0), ABC
##%## default="blend", ABC
ABCDE default="/path/to/directory") ABC
One sed idea based on 2 variables (var1 == search / var2 == replacement):
sed -E "s/default=$var1/default=$var2/g" file.txt
Test case #1:
var1='"blend"'
var2='"burn"'
sed -E "s/default=$var1/default=$var2/g" "${infile}"
Generates:
abcde default=60.0, ABC
12345 default=False, ABC
STUVW default=(0.2, 0.2, 0.3, 1.0), ABC
##%## default="burn", ABC # modified line
ABCDE default="/path/to/directory") ABC
Test case #2:
var1='\(0.2, 0.2, 0.3, 1.0\)' # must escape the parens
var2='[something different = 3.22]'
sed -E "s/default=$var1/default=$var2/g" "${infile}"
Generates:
abcde default=60.0, ABC
12345 default=False, ABC
STUVW default=[something different = 3.22], ABC # modified line
##%## default="blend", ABC
ABCDE default="/path/to/directory") ABC
Test case #3:
var1='"\/path\/to\/directory"' # must escape the forward slashes
var2='"\/a\/new\/directory"' # must escape the forward slashes
sed -E "s/default=$var1/default=$var2/g" "${infile}"
Generates:
abcde default=60.0, ABC
12345 default=False, ABC
STUVW default=(0.2, 0.2, 0.3, 1.0), ABC
##%## default="blend", ABC
ABCDE default="/a/new/directory") ABC # modified line
To expand the variables, place the sed statement in double quotes as opposed to single quotes. Also since you are using var1 to hold the line to process, place the proceeding substitution statement in {}:
sed -r "$var1{s/(^default=\")(.*)(\",?.*$)/\1$var2\3/}" file1.txt
Also, enable the regular expression interpretation with -r or -E and then split the line into three sections specified in parenthesis and substitute the line for the first section, var2 and then then the third section. Escape any double quotes in the text to change where required.
I would be happy if anyone can suggest me command (sed or AWK one line command) to divide each line of file in equal number of part. For example divide each line in 4 part.
Input:
ATGCATHLMNPHLNTPLML
Output:
ATGCA THLMN PHLNT PLML
This should work using GNU sed:
sed -r 's/(.{4})/\1 /g'
-r is needed to use extended regular expressions
.{4} captures every four characters
\1 refers to the captured group which is surrounded by the parenthesis ( ) and adds a space behind this group
g makes sure that the replacement is done as many times as possible on each line
A test; this is the input and output in my terminal:
$ echo "ATGCATHLMNPHLNTPLML" | sed -r 's/(.{4})/\1 /g'
ATGC ATHL MNPH LNTP LML
I suspect awk is not the best tool for this, but:
gawk --posix '{ l = sprintf( "%d", 1 + (length()-1)/4);
gsub( ".{"l"}", "& " ) } 1' input-file
If you have a posix compliant awk you can omit the --posix, but --posix is necessary for gnu awk and since that seems to be the most commonly used implementation I've given the solution in terms of gawk.
This might work for you (GNU sed):
sed 'h;s/./X/g;s/^\(.*\)\1\1\1/\1 \1 \1 \1/;G;s/\n/&&/;:a;/^\n/bb;/^ /s/ \(.*\n.*\)\n\(.\)/\1 \n\2/;ta;s/^.\(.*\n.*\)\n\(.\)/\1\2\n/;ta;:b;s/\n//g' file
Explanation:
h copy the pattern space (PS) to the hold space (HS)
s/./X/g replace every character in the HS with the same non-space character (in this case X)
s/^\(.*\)\1\1\1/\1 \1 \1 \1/ split the line into 4 parts (space separated)
G append a newline followed by the contents of the HS to the PS
s/\n/&&/ double the newline (to be later used as markers)
:a introduce a loop namespace
/^\n/bb if we reach a newline we are done and branch to the b namespace
/^ /s/ \(.*\n.*\)\n\(.\)/\1 \n\2/;ta; if the first character is a space add a space to the real line at this point and repeat
s/^.\(.*\n.*\)\n\(.\)/\1\2\n/;ta any other character just bump along and repeat
:b;s/\n//g all done just remove the markers and print out the result
This work for any length of line, however is the line is not exactly divisible by 4 the last portion will contain the remainder as well.
perl
perl might be a better choice here:
export cols=4
perl -ne 'chomp; $fw = 1 + int length()/$ENV{cols}; while(/(.{1,$fw})/gm) { print $1 . " " } print "\n"'
This re-calculates field-width for every line.
coreutils
A GNU coreutils alternative, field-width is chosen based on the first line of infile:
cols=4
len=$(( $(head -n1 infile | wc -c) - 1 ))
fw=$(echo "scale=0; 1 + $len / 4" | bc)
cut_arg=$(paste -d- <(seq 1 $fw 19) <(seq $fw $fw $len) | head -c-1 | tr '\n' ',')
Value of cut_arg is in the above case:
1-5,6-10,11-15,16-
Now cut the line into appropriate chunks:
cut --output-delimiter=' ' -c $cut_arg infile