using sed with variables to replace quoted string in bash - sed

Working in bash. I have a file.txt with contents in the following format.
default=60.0,
default=False,
default=(0.2, 0.2, 0.3, 1.0),
default="blend",
default="/path/to/directory")
There is other info on each line before and after what i am showing but this is the part i want to edit. i am trying to replace the values following the equals sign and before the comma with values stored in a variable. i can make it work if i know all the info of line number, what to replace, and what replacing with beforehand but as soon as i start working with variables it all falls apart. i can use sed -i '4s/default="blend",/default="burn",/' file.txt to replace blend with burn on line 4 but how can i do this with variables? i cant seem to find the right way to quote this. and obviously the data i am trying to replace is not consistent, some numbers, some strings, some quoted, some not, some bracketed, some not, some followed by ,, some followed by ). focused on line 4 currently, if var1=4 and var2="burn" and default= could be any of "blend, burn, or bump". so say i want to replace whatever follows default= with "burn" on line 4, var1=4 var2="burn"
Desired output:
default=60.0,
default=False,
default=(0.2, 0.2, 0.3, 1.0),
default="burn",
default="/path/to/directory")
hopefully what i am trying to accomplish makes sense because i'm stuck. how can i make this work? doesn't have to use sed if there is another simple way.
edit:
contents of entire line are:
parse.add_argument("-j", "--blend_type", default="blend", choices=["blend", "burn", "bump"], help="type of blend the shader can do")
using: sed -r "$var1{s/(default=\")(.*)(\".*$)/\1$var2\3/p}" file.txt
my output is parse.add_argument("-j", "--blend_type", default="burn")
and i want it to be parse.add_argument("-j", "--blend_type", default="burn", choices=["blend", "burn", "bump"], help="type of blend the shader can do")
i'm close but can't seem to keep the end of the line intact.

i can use sed -i '4s/default="blend",/default="burn",/' file.txt to replace blend with burn on line 4 but how can i do this with variables?
linenumber=4
from=blend
to=burn
# a bit extra: escape special characters
# https://stackoverflow.com/questions/407523/escape-a-string-for-a-sed-replace-pattern
from=$(printf '%s\n' "$from" | sed -e 's/[]\/$*.^[]/\\&/g');
to=$(printf '%s\n' "$to" | sed -e 's/[\/&]/\\&/g')
# Note the quoting
sed -i "${line}s/default=\"${from}\",/default=\"${to}\",/"
# ^^^^^^ - used ${line} so that 's' is separate
# ^^ ^^ ^^ ^^ - double quotes within double quoting need to be escaped
# ^ ^ - double quotes so that ${..} are expanded
sed -i '"$var1"s/default="blend",/default="burn",/' $f2
Variables are not expanded within single quotes.
sed -i "$var1"s/default="*"/default="$var2"/ file.txt
Regex is not glob. "* matches zero or more " characters. It does not match anything, * matches zero or more of preceding character or group. A dot . matches anything, so you could default=".*" or better I would go with default="[^"]*".
sed -i ""$var1"s/default="*",/default="$var2",/" file.txt
Use an editor with syntax highlighting. Even stackoverflow syntax highlighting should hint you - " close the other ". To write " within double quotation, escape them "\"".

Assumptions/provisos:
patterns to be searched and replaced can contain any printable characters which means ...
said patterns may need to include escape characters to keep from clashing with sed delimters and constructs
default=<search_pattern> only shows up once in a line
Sample data:
$ cat file.txt
abcde default=60.0, ABC
12345 default=False, ABC
STUVW default=(0.2, 0.2, 0.3, 1.0), ABC
##%## default="blend", ABC
ABCDE default="/path/to/directory") ABC
One sed idea based on 2 variables (var1 == search / var2 == replacement):
sed -E "s/default=$var1/default=$var2/g" file.txt
Test case #1:
var1='"blend"'
var2='"burn"'
sed -E "s/default=$var1/default=$var2/g" "${infile}"
Generates:
abcde default=60.0, ABC
12345 default=False, ABC
STUVW default=(0.2, 0.2, 0.3, 1.0), ABC
##%## default="burn", ABC # modified line
ABCDE default="/path/to/directory") ABC
Test case #2:
var1='\(0.2, 0.2, 0.3, 1.0\)' # must escape the parens
var2='[something different = 3.22]'
sed -E "s/default=$var1/default=$var2/g" "${infile}"
Generates:
abcde default=60.0, ABC
12345 default=False, ABC
STUVW default=[something different = 3.22], ABC # modified line
##%## default="blend", ABC
ABCDE default="/path/to/directory") ABC
Test case #3:
var1='"\/path\/to\/directory"' # must escape the forward slashes
var2='"\/a\/new\/directory"' # must escape the forward slashes
sed -E "s/default=$var1/default=$var2/g" "${infile}"
Generates:
abcde default=60.0, ABC
12345 default=False, ABC
STUVW default=(0.2, 0.2, 0.3, 1.0), ABC
##%## default="blend", ABC
ABCDE default="/a/new/directory") ABC # modified line

To expand the variables, place the sed statement in double quotes as opposed to single quotes. Also since you are using var1 to hold the line to process, place the proceeding substitution statement in {}:
sed -r "$var1{s/(^default=\")(.*)(\",?.*$)/\1$var2\3/}" file1.txt
Also, enable the regular expression interpretation with -r or -E and then split the line into three sections specified in parenthesis and substitute the line for the first section, var2 and then then the third section. Escape any double quotes in the text to change where required.

Related

Extract substrings between strings

I have a file with text as follows:
###interest1 moreinterest1### sometext ###interest2###
not-interesting-line
sometext ###interest3###
sometext ###interest4### sometext othertext ###interest5### sometext ###interest6###
I want to extract all strings between ### .
My desired output would be something like this:
interest1 moreinterest1
interest2
interest3
interest4
interest5
interest6
I have tried the following:
grep '###' file.txt | sed -e 's/.*###\(.*\)###.*/\1/g'
This almost works but only seems to grab the first instance per line, so the first line in my output only grabs
interest1 moreinterest1
rather than
interest1 moreinterest1
interest2
Here is a single awk command to achieve this that makes ### field separator and prints each even numbered field:
awk -F '###' '{for (i=2; i<NF; i+=2) print $i}' file
interest1 moreinterest1
interest2
interest3
interest4
interest5
interest6
Here is an alternative grep + sed solution:
grep -oE '###[^#]*###' file | sed -E 's/^###|###$//g'
This assumes there are no # characters in between ### markers.
With GNU awk for multi-char RS:
$ awk -v RS='###' '!(NR%2)' file
interest1 moreinterest1
interest2
interest3
interest4
interest5
interest6
You can use pcregrep:
pcregrep -o1 '###(.*?)###' file
The regex - ###(.*?)### - matches ###, then captures into Group 1 any zero o more chars other than line break chars, as few as possible, and ### then matches ###.
o1 option will output Group 1 value only.
See the regex demo online.
sed 't x
s/###/\
/;D; :x
s//\
/;t y
D;:y
P;D' file
Replacing "###" with newline, D, then conditionally branching to P if a second replacement of "###" is successful.
This might work for you (GNU sed):
sed -n 's/###/\n/g;/[^\n]*\n/{s///;P;D}' file
Replace all occurrences of ###'s by newlines.
If a line contains a newline, remove any characters before and including the first newline, print the details up to and including the following newline, delete those details and repeat.

How to extract a specific character inside a parentheses using sed command?

I want to extract an atomic symbols inside a parentheses using sed.
The data I have is in the form C(X12), and I only want the X symbol
EX: that a test command :
echo "C(Br12)" | sed 's/[0-9][0-9])$//g'
gives me C(Br.
You can use
sed -n 's/.*(\(.*\)[0-9]\{2\})$/\1/p'
See the online demo:
sed -n 's/.*(\(.*\)[0-9]\{2\})$/\1/p' <<< "c(Br12)"
# => Br
Details
-n - suppresses the default line output
.*(\(.*\)[0-9]\{2\})$ - a regex that matches
.* - any text
( - a ( char
\(.*\) - Capturing group 1: any text up to the last....
[0-9]\{2\} - two digits
)$ - a ) at the end of string
\1 - replaces with Group 1 value
p - prints the result of the substitution.
For example:
echo "C(Br12)" | sed 's/C(\(.\).*/\1/'
C( - match exactly literally C(
. match anything
\(.\) - match anythig - one character- and "remember" it in a backreference \1
.* ignore everything behind it
\1 - replace it by the stuff that was remembered. The first character.
Research sed, regex and backreferences for more information.
Try using the following command
echo "C(BR12)" | cut -d "(" -f2 | cut -d ")" -f1 | sed 's/[0-9]*//g'
The cut tool will split and get you the string in middle of the paranthesis.Then pass the string to a sed for replacing the numbers inside the string.
Not a fully sed solution but this will get you the output.

sed substitute with quotes and wildcard

I need to replace if ($_SESSION['POST']['*']==1){ with if (isset($_SESSION['POST']['*']) && $_SESSION['POST']['*']==1){
(I'm using * as a wild card)
I've tried sed -i "s/if ($_SESSION['POST']['.*']/if (isset($_SESSION['POST']['.*']) && $_SESSION['POST']['.*']/g" *.php and a few other variations without success.
Here goes...
sed "s/\(if (\)\(\$_SESSION\['POST']\['\([^']*\)']\)==1/\1isset(\2) \&\& \$_SESSION['POST']['\3']==1/" file
Using double quotes means that the $ symbols must be escaped, otherwise they will be interpreted as shell variables. The square brackets need to be escaped, otherwise they will be interpreted as the beginning of a range. It's OK to leave the closing square brackets as they are.
In order to capture the key, I have used a character class [^']*. This means zero or more characters that are not a single quote.
In the replacement, the captured groups (the parts between parentheses in the match) are referred to using \1, \2, etc.
Testing it out:
$ cat file
if ($_SESSION['POST']['foo']==1){
// do something
}
if ($_SESSION['POST']['bar']==1){
// do something else
}
$ sed "s/\(if (\)\(\$_SESSION\['POST']\['\([^']*\)']\)==1/\1isset(\2) \&\& \$_SESSION['POST']['\3']==1/" file
if (isset($_SESSION['POST']['foo']) && $_SESSION['POST']['foo']==1){
// do something
}
if (isset($_SESSION['POST']['bar']) && $_SESSION['POST']['bar']==1){
// do something else
}
By the way it makes the command a few characters shorter if you use extended regexp mode (-r or -E). In extended mode, the parentheses enclosing capture groups don't have to be escaped but literal ones do, so your command would then be:
sed -r "s/(if \()(\$_SESSION\['POST']\['([^']*)'])==1/\1isset(\2) \&\& \$_SESSION['POST']['\3']==1/" file
This sed should work:
s="if (\$_SESSION['POST']['name']==1){"
sed -r 's/(if +)\((([^=]+)[^\)]+)/\1(isset(\3) \&\& \2/' <<< "$s"
if (isset($_SESSION['POST']['name']) && $_SESSION['POST']['name']==1){
PS: Use sed -E instead of sed -r on OSX.
Here's another.
This is what we need to produce:
Pattern: if (\$_SESSION\['POST'\]\['\([^']*\)'\]
Replacement: if (isset($_SESSION['POST']['\1']) \&\& $_SESSION['POST']['\1']
When quoted in shell level:
Pattern: "if (\\\$_SESSION\['POST'\]\['\([^']*\)'\]"
Replacement: "if (isset(\$_SESSION['POST']['\1']) \\&\\& \$_SESSION['POST']['\1']"
Putting it together:
sed -i "s|if (\\\$_SESSION\['POST'\]\['\([^']*\)'\]|if (isset(\$_SESSION['POST']['\1']) \\&\\& \$_SESSION['POST']['\1']|g" file
Test:
# sed "s|if (\\\$_SESSION\['POST'\]\['\([^']*\)'\]|if (isset(\$_SESSION['POST']['\1']) \\&\\& \$_SESSION['POST']['\1']|g" <<'EOF'
> if ($_SESSION['POST']['ABC']==1){
> EOF
if (isset($_SESSION['POST']['ABC']) && $_SESSION['POST']['ABC']==1){

divide each line in equal part

I would be happy if anyone can suggest me command (sed or AWK one line command) to divide each line of file in equal number of part. For example divide each line in 4 part.
Input:
ATGCATHLMNPHLNTPLML
Output:
ATGCA THLMN PHLNT PLML
This should work using GNU sed:
sed -r 's/(.{4})/\1 /g'
-r is needed to use extended regular expressions
.{4} captures every four characters
\1 refers to the captured group which is surrounded by the parenthesis ( ) and adds a space behind this group
g makes sure that the replacement is done as many times as possible on each line
A test; this is the input and output in my terminal:
$ echo "ATGCATHLMNPHLNTPLML" | sed -r 's/(.{4})/\1 /g'
ATGC ATHL MNPH LNTP LML
I suspect awk is not the best tool for this, but:
gawk --posix '{ l = sprintf( "%d", 1 + (length()-1)/4);
gsub( ".{"l"}", "& " ) } 1' input-file
If you have a posix compliant awk you can omit the --posix, but --posix is necessary for gnu awk and since that seems to be the most commonly used implementation I've given the solution in terms of gawk.
This might work for you (GNU sed):
sed 'h;s/./X/g;s/^\(.*\)\1\1\1/\1 \1 \1 \1/;G;s/\n/&&/;:a;/^\n/bb;/^ /s/ \(.*\n.*\)\n\(.\)/\1 \n\2/;ta;s/^.\(.*\n.*\)\n\(.\)/\1\2\n/;ta;:b;s/\n//g' file
Explanation:
h copy the pattern space (PS) to the hold space (HS)
s/./X/g replace every character in the HS with the same non-space character (in this case X)
s/^\(.*\)\1\1\1/\1 \1 \1 \1/ split the line into 4 parts (space separated)
G append a newline followed by the contents of the HS to the PS
s/\n/&&/ double the newline (to be later used as markers)
:a introduce a loop namespace
/^\n/bb if we reach a newline we are done and branch to the b namespace
/^ /s/ \(.*\n.*\)\n\(.\)/\1 \n\2/;ta; if the first character is a space add a space to the real line at this point and repeat
s/^.\(.*\n.*\)\n\(.\)/\1\2\n/;ta any other character just bump along and repeat
:b;s/\n//g all done just remove the markers and print out the result
This work for any length of line, however is the line is not exactly divisible by 4 the last portion will contain the remainder as well.
perl
perl might be a better choice here:
export cols=4
perl -ne 'chomp; $fw = 1 + int length()/$ENV{cols}; while(/(.{1,$fw})/gm) { print $1 . " " } print "\n"'
This re-calculates field-width for every line.
coreutils
A GNU coreutils alternative, field-width is chosen based on the first line of infile:
cols=4
len=$(( $(head -n1 infile | wc -c) - 1 ))
fw=$(echo "scale=0; 1 + $len / 4" | bc)
cut_arg=$(paste -d- <(seq 1 $fw 19) <(seq $fw $fw $len) | head -c-1 | tr '\n' ',')
Value of cut_arg is in the above case:
1-5,6-10,11-15,16-
Now cut the line into appropriate chunks:
cut --output-delimiter=' ' -c $cut_arg infile

sed - comment a matching line and x lines after it

I need help with using sed to comment a matching lines and 4 lines which follows it.
in a text file.
my text file is like this:
[myprocess-a]
property1=1
property2=2
property3=3
property4=4
[anotherprocess-b]
property1=gffgg
property3=gjdl
property2=red
property4=djfjf
[myprocess-b]
property1=1
property4=4
property2=2
property3=3
I want to prefix # to all the lines having text '[myprocess' and 4 lines that follows it
expected output:
#[myprocess-a]
#property1=1
#property2=2
#property3=3
#property4=4
[anotherprocess-b]
property1=gffgg
property3=gjdl
property2=red
property4=djfjf
#[myprocess-b]
#property1=1
#property4=4
#property2=2
#property3=3
Greatly appreciate your help on this.
You can do this by applying a regular expression to a set of lines:
sed -e '/myprocess/,+4 s/^/#/'
This matches lines with 'myprocess' and the 4 lines after them. For those 4 lines it then inserts a '#' at the beginning of the line.
(I think this might be a GNU extension - it's not in any of the "sed one liner" cheatsheets I know)
sed '/\[myprocess/ { N;N;N;N; s/^/#/gm }' input_file
Using string concatenation and default action in awk.
http://www.gnu.org/software/gawk/manual/html_node/Concatenation.html
awk '/myprocess/{f=1} f>5{f=0} f{f++; $0="#" $0} 1' foo.txt
or if the block always ends with empty line
awk '/myprocess/{f=1} !NF{f=0} f{$0="#" $0} 1' foo.txt