I'd like to replace the database server of a horde config file from "localhost" to a remote server (I use "database.contoso.com" as a placeholder).
The file in question is /var/www/horde/config/conf.php.
The line in the file looks like this:
$conf['sql']['hostspec'] = 'localhost';
Now I have created a sed line like so:
sed s/\$conf\[\'sql\'\]\[\'hostspec\'\]\ \=\ \'localhost\'\;/\$conf\[\'sql\'\]\[\
'hostspec\'\]\ \=\ \'database\.contoso\.com\'\;/ /var/www/horde/config/conf.php
But for whatever reason, it does not work -I spare out the -i option for later.
While trying to figure out, why it does not work, I did this:
echo "\$conf['sql']['hostspec'] = 'localhost';"|sed s/\$conf\[\'sql\'\]\[\'hostspec\'\]\ \=\ \'localhost\'\;/\$conf\[\'sql\'\]\[\'hostspec\'\]\ \=\ \'database\.contoso\.com\'\;/
which returns this:
$conf['sql']['hostspec'] = 'localhost';
but it should return:
$conf['sql']['hostspec'] = 'database.contoso.com';
What am I missing?
From Escape a string for a sed replace pattern in this case it would work:
KEYWORD="\$conf['sql']['hostspec'] = 'localhost';"
REPLACE="\$conf['sql']['hostspec'] = 'database.contoso.com';"
ESCAPED_REPLACE=$(printf '%s\n' "$REPLACE" | sed -e 's/[\/&]/\\&/g')
ESCAPED_KEYWORD=$(printf '%s\n' "$KEYWORD" | sed -e 's/[]\/$*.^[]/\\&/g');
sed "s/$ESCAPED_KEYWORD/$ESCAPED_REPLACE/"
The immediate problem is that you are not quoting enough. To match a regex metacharacter literally, you need to pass in a literal backslash \\ followed by a literal, like for example \[. But the simplest solution by far is to use single quotes around your expression, and then only backslash the characters which are regex metacharacters.
Literal single quotes inside single quotes are still challenging. Here, I have chosen to end the single-quoted string, insert a backslash-escaped but otherwise unquoted single quote, and add an opening single quote to continue with another single-quoted string. The shell glues these together into a single string.
echo "\$conf['sql']['hostspec'] = 'localhost';" |
sed 's/\$conf\['\''sql'\''\]\['\''hostspec'\''\] = '\''localhost'\'';/$conf['\''sql'\'']['\''hostspec'\''] = '\''database.contoso.com'\'';/'
A better solution generally is to use backreferences to quote back part of the matched string so you don't have to repeat it.
echo "\$conf['sql']['hostspec'] = 'localhost';" |
sed 's/\(\$conf\['\''sql'\''\]\['\''hostspec'\''\] = '\''\)[^'\'']*'\'';/\1database.contoso.com'\'';/'
Demo: https://ideone.com/RA0MSi
A much much much better solution is to change your PHP script so that this setting can be overridden with an option, an environment variable, and/or a configuration file.
This might work for you (GNU sed & shell):
sed -E 's/(\$conf\[('\'')sql\2]\[\2hostspec\2\] = )\2localhost\2;/\1\2database.contoso.com\2;/' file
Use pattern matching to match and replace.
N.B. Certain metacharacters must be escaped/quoted i.e. $,[,] and then because the sed commands are surrounded by single quotes, each single quote (within the substitution command) must be replaced by '\'' (see here for reasoning). Also, back references can be used both in the RHS and the LHS of the substitution command. The back references in the LHS especially allow for the shortening of the overall command and perhaps make the regexp more readable.
Related
I want to replace specific strings in php files automatically using sed. Some work, and some do not. I already investigated this is not an issue with the replacement string but with the string that is to be replaced. I already tried to escape [ and ] with no success. It seems to be the whitespace within the () - not whitespaces in general. The first whitespaces (around the = ) do not have any problems. Please can someone point me to the problem:
sed -e "1,\$s/$adm = substr($path . rawurlencode($upload['name']) , 16);/$adm = rawurlencode($upload['name']); # fix 23/g" -i administration/identify.php
I already tried to shorten the string which should be replaced and the result was if I cut it directly behind $path it works, with the following whitespace it does not. Escaping whitespace has no effect...
what must be escaped for sed
The following characters have special meaning in sed and have to be escaped with \ for the regex to be taken literally:
\
[
the character used in separating s command parts, ie. / here
.
*
& only replacement string
Newline character is handled specially as the end of the string, but can be replaced for \n.
So first escape all special characters in input and then pass it to sed:
rgx="$adm = substr($path . rawurlencode($upload['name']) , 16);"
rgx_escaped=$(sed 's/[\\\[\.\*\/&]/\\&/g' <<<"$rgx")
sed "s/$rgx_escaped/ etc."
See Escape a string for a sed replace pattern for a generic escaping solution.
You may use
sed -i 's/\$adm = substr(\$path \. rawurlencode(\$upload\['"'"'name'"'"']) , 16);/$adm = rawurlencode($upload['"'"'name'"'"']); # fix 23/g' administration/identify.php
Note:
the sed command is basically wrapped in single quotes, the variable expansion won't occur inside single quotes
In the POSIX BRE syntax, ( matches a literal (, you do not need to escape ) either, but you need to escape [ and . that must match themselves
The single quotes require additional quoting with concatenation.
I have one file named `config_3_setConfigPW.ldif? containing the following line:
{pass}
on terminal, I used following commands
SLAPPASSWD=Pwd&0011
sed -i "s#{pass}#$SLAPPASSWD#" config_3_setConfigPW.ldif
It should replace {pass} to Pwd&0011 but it generates Pwd{pass}0011.
The reason is that the SLAPPASSWD shell variable is expanded before sed sees it. So sed sees:
sed -i "s#{pass}#Pwd&0011#" config_3_setConfigPW.ldif
When an "&" is on the right hand side of a pattern it means "copy the matched input", and in your case the matched input is "{pass}".
The real problem is that you would have to escape all the special characters that might arise in SLAPPASSWD, to prevent sed doing this. For example, if you had character "#" in the password, sed would think it was the end of the substitute command, and give a syntax error.
Because of this, I wouldn't use sed for this. You could try gawk or perl?
eg, this will print out the modified file in awk (though it still assumes that SLAPPASSWD contains no " character
awk -F \{pass\} ' { print $1"'${SLAPPASSWD}'"$2 } ' config_3_setConfigPW.ldif
That's because$SLAPPASSWD contains the character sequences & which is a metacharacter used by sed and evaluates to the matched text in the s command. Meaning:
sed 's/{pass}/match: &/' <<< '{pass}'
would give you:
match: {pass}
A time ago I've asked this question: "Is it possible to escape regex metacharacters reliably with sed". Answers there show how to reliably escape the password before using it as the replacement part:
pwd="Pwd&0011"
pwdEscaped="$(sed 's/[&/\]/\\&/g' <<< "$pwd")"
# Now you can safely pass $pwd to sed
sed -i "s/{pass}/$pwdEscaped/" config_3_setConfigPW.ldif
Bear in mind that sed NEVER operates on strings. The thing sed searches for is a regexp and the thing it replaces it with is string-like but has some metacharacters you need to be aware of, e.g. & or \<number>, and all of it needs to avoid using the sed delimiters, / typically.
If you want to operate on strings you need to use awk:
awk -v old="{pass}" -v new="$SLAPPASSWD" 's=index($0,old){ $0 = substr($0,1,s-1) new substr($0,s+length(old))} 1' file
Even the above would need tweaked if old or new contained escape characters.
I need to replace text with single quotes with sed and I can't get it to work. Here is my code; can you help me?
I have a text file with this format:
#sometext
$configuration_TEstbk2_bk2_environment12 = 'lalala'
$configuration_TEstbk2_bk2_envoronment12 = 'lalala1'
$configuration_TEstbk2_bk2_staging12 = 'BACKUP 2'
$configuration_waq4faw4f_q4fq4qg4f = 'r234rq43rq4rqr'
$configuration_alice_StagingTEstBk_bk = 'testebk'
$configuration_deployment_overlays_alice_TEStStngDir = 'some'
$configuration_arefgqrqgrq_341q34tq34t = '134t135'
And I need to do something like:
sed s/$configuration_arefgqrqgrq_341q34tq34t ='134t135'/$configuration_arefgqrqgrq_341q34tq34t = 'NEWVALUE'/g
I have tried with many combinations with sed but I can't find one that works.
Would this work for you?
sed "/\$config_deployment_overlays_alice_arefgqrqgrq_341q34tq34t_staging/s/'134t135'/'NEWVALUE'/" file
I'd probably use:
sed '/$configuration_arefgqrqgrq_341q34tq34t *=/'"s/'134t135'/'NEWVALUE'/"
This uses a mix of single quotes and double quotes at the shell level to get the correct information to sed. The single quotes enclose the search condition for the lines containing $configuration_arefgqrqgrq_341q34tq34t followed by some blanks and an equals sign. This avoids the shell expanding what might be a shell variable name (but probably isn't, so the empty string would be substituted). I then switch to double quotes at the shell level, so that sed gets to see the single quotes: it substitutes the value in single quotes with the replacement value, but only on those lines that contain the given configuration parameter name.
I hope users never have to type those configuration parameter names.
I suppose your problem is with the quoting. You could use complex quoting to make sure everything is in single quotes, except the single quotes which need to be in double quotes:
sed 's/$configuration_arefgqrqgrq_341q34tq34t *= *'"'134t135'"'/$configuration_arefgqrqgrq_341q34tq34t = '"'NEWVALUE'/g"
... or you could use some temporary variables to make the whole thing more readable, and suitable for inclusion in a single pair of double quotes:
directive='$configuration_arefgqrqgrq_341q34tq34t'
oldvalue="'134t135'"
newvalue="'NEWVALUE'"
sed "s/$directive *= *$oldvalue/$directive = $newvalue/g"
(If you only expect one match, the /g flag is superfluous.)
You can also totally avoid matching quotes by capturing them:
sed '/$configuration_arefgqrqgrq_341q34tq34t/{
s/\(= *.\).*\(.\) *$/\1NEWVALUE\2/
}' input
This might work for you (GNU sed):
sed -r 's/^(\$\S+\s=\s'\'').*('\'')/\1NEWVALUE\2/' file
In my code, I need to replace variable assignments with addresses:
sed -i "s/^variable = .*$/variable = http://myaddress/"
Obviously, this does not work because the forward slashes in the address are recognized in the sed command.
I want to keep the $ at the end of the first expression for replacing anything to the end of the line. I also do not want to escape the dollar sign as such, \$ because it will search for a dollar sign.
Also, I don't want to just escape the forward slashes in the address as there are also variables in some places for the addresses.
I've tried using # instead of / but have to include what I don't want to - the \$.
Are there any alternate delimiters I can use that fit my situation?
The $ is interpreted by your shell. Wrap the whole argument to sed with ' to prevent this.
sed -i 's#^variable = .*$#variable = http://myaddress#'
sed -i "s#^variable = .*$#variable = http://myaddress#" file
should work for you.
Note that the $ in the first expression is not literature "dollar", but a regex expression, means, the end of the line.
sed -i 's#^variable = .*#variable = http://myaddress#'
I am using the Unix sed command on a string that can contain all types of characters (&, |, !, /, ?, etc).
Is there a complex delimiter (with two characters?) that can fix the error:
sed: -e expression #1, char 22: unknown option to `s'
The characters in the input file are of no concern - sed parses them fine. There may be an issue, however, if you have most of the common characters in your pattern - or if your pattern may not be known beforehand.
At least on GNU sed, you can use a non-printable character that is highly improbable to exist in your pattern as a delimiter. For example, if your shell is Bash:
$ echo '|||' | sed s$'\001''|'$'\001''/'$'\001''g'
In this example, Bash replaces $'\001' with the character that has the octal value 001 - in ASCII it's the SOH character (start of heading).
Since such characters are control/non-printable characters, it's doubtful that they will exist in the pattern. Unless, that is, you are doing something weird like modifying binary files - or Unicode files without the proper locale settings.
Another way to do this is to use Shell Parameter Substitution.
${parameter/pattern/replace} # substitute replace for pattern once
or
${parameter//pattern/replace} # substitute replace for pattern everywhere
Here is a quite complex example that is difficult with sed:
$ parameter="Common sed delimiters: [sed-del]"
$ pattern="\[sed-del\]"
$ replace="[/_%:\\#]"
$ echo "${parameter//$pattern/replace}"
result is:
Common sed delimiters: [/_%:\#]
However: This only work with bash parameters and not files where sed excel.
There is no such option for multi-character expression delimiters in sed, but I doubt
you need that. The delimiter character should not occur in the pattern, but if it appears in the string being processed, it's not a problem. And unless you're doing something extremely weird, there will always be some character that doesn't appear in your search pattern that can serve as a delimiter.
You need the nested delimiter facility that Perl offers. That allows to use stuff like matching, substituting, and transliterating without worrying about the delimiter being included in your contents. Since perl is a superset of sed, you should be able to use it for whatever you’re used sed for.
Consider this:
$ perl -nle 'print if /something/' inputs
Now if your something contains a slash, you have a problem. The way to fix this is to change delimiter, preferably to a bracketing one. So for example, you could having anything you like in the $WHATEVER shell variable (provided the backets are balanced), which gets interpolated by the shell before Perl is even called here:
$ perl -nle "print if m($WHATEVER)" /usr/share/dict/words
That works even if you have correctly nested parens in $WHATEVER. The four bracketing pairs which correctly nest like this in Perl are < >, ( ), [ ], and { }. They allow arbitrary contents that include the delimiter if that delimiter is balanced.
If it is not balanced, then do not use a delimiter at all. If the pattern is in a Perl variable, you don’t need to use the match operator provided you use the =~ operator, so:
$whatever = "some arbitrary string ( / # [ etc";
if ($line =~ $whatever) { ... }
With the help of Jim Lewis, I finally did a test before using sed :
if [ `echo $1 | grep '|'` ]; then
grep ".*$1.*:" $DB_FILE | sed "s#^.*$1*.*\(:\)## "
else
grep ".*$1.*:" $DB_FILE | sed "s|^.*$1*.*\(:\)|| "
fi
Thanks for help
Wow. I totally did not know that you could use any character as a delimiter.
At least half the time I use the sed and BREs its on paths, code snippets, junk characters, things like that. I end up with a bunch of horribly unreadable escapes which I'm not even sure won't die on some combination I didn't think of. But if you can exclude just some character class (or just one character even)
echo '#01Y $#1+!' | sed -e 'sa$#1+ashita' -e 'su#01YuHolyug'
> > > Holy shit!
That's so much easier.
Escaping the delimiter inline for BASH to parse is cumbersome and difficult to read (although the delimiter does need escaping for sed's benefit when it's first used, per-expression).
To pull together thkala's answer and user4401178's comment:
DELIM=$(echo -en "\001");
sed -n "\\${DELIM}${STARTING_SEARCH_TERM}${DELIM},\\${DELIM}${ENDING_SEARCH_TERM}${DELIM}p" "${FILE}"
This example returns all results starting from ${STARTING_SEARCH_TERM} until ${ENDING_SEARCH_TERM} that don't match the SOH (start of heading) character with ASCII code 001.
There's no universal separator, but it can be escaped by a backslash for sed to not treat it like separator (at least unless you choose a backslash character as separator).
Depending on the actual application, it might be handy to just escape those characters in both pattern and replacement.
If you're in a bash environment, you can use bash substitution to escape sed separator, like this:
safe_replace () {
sed "s/${1//\//\\\/}/${2//\//\\\/}/g"
}
It's pretty self-explanatory, except for the bizarre part.
Explanation to that:
${1//\//\\\/}
${ - bash expansion starts
1 - first positional argument - the pattern
// - bash pattern substitution pattern separator "replace-all" variant
\/ - literal slash
/ - bash pattern substitution replacement separator
\\ - literal backslash
\/ - literal slash
} - bash expansion ends
example use:
$ input="ka/pus/ta"
$ pattern="/pus/"
$ replacement="/re/"
$ safe_replace "$pattern" "$replacement" <<< "$input"
ka/re/ta