I need to modify some Windows paths.
For instance,
D:\usr
to
D:\first\usr
So, I have created a variable.
$path = "first\usr"
then used the following command:
sed -i -e 's!\\usr!${path}/g;' test.txt
However, this ends up with the following:
D:\firstSr
How do I escape \u in sed?
Assuming your path variable was assigned properly (without spaces in the assignment: path='first\usr'), fixing step by step for an input file test.txt with one example path:
$ cat test.txt
D:\usr
Your original command
$ sed 's!\\usr!${path}/g;' test.txt
sed: -e expression #1, char 18: unterminated `s' command
doesn't do much, as you've mixed ! and / as the delimiter.
Fixing delimiters:
$ sed 's!\\usr!${path}!g;' test.txt
D:${path}
Now no interpolation happens at all because of the single quotes. I suspect these are just copy-paste mistakes, as you obviously got some output.
Double quotes:
$ sed "s!\\usr!${path}!g" test.txt
bash: !\\usr!${path}!g: event not found
Now this clashes with history expansion. We could escape the !, or use a different delimiter.
/ as delimiter:
$ sed "s/\\usr/${path}/g" test.txt
D:\firstSr
Now we're where the question actually started. ${path} expands to first\usr, but \u has a special meaning in GNU sed in the replacement string: it uppercases the following character, hence the S.
Even without the special meaning, \u would most likely just expand to u and the backslash would be gone.
Escaping the backslash:
$ path='first\\usr'
$ sed "s/\\usr/${path}/g" test.txt
D:\first\usr
This works.
Depending on which shell you are using, you may be able to use parameter expansion to double \ in your substitution string and prevent the \u interpretation:
path="first\usr"
sed -e "s/\\usr/${path//\\/\\\\}/g" <<< "D:\usr"
The syntax for replacing a pattern with the shell parameter expansion is ${parameter/pattern/string} (one replacement) or ${parameter//pattern/string} (replace all matches).
This substitution is not specified by POSIX, but is available in Bash.
Where it is not available, you may need to filter $path through a process:
path=$(echo "$path" | sed 's/[][\\*.%$]/\\&/g')
(N.B. I have also quoted other sed metacharacters in this filter).
Related
Can you give the sed command that will find \" and replace with \\' in a file.
For example line:
LOG_FN=\"file_name\"
will become
LOG_FN=\\'file_name\\'
By using this template:
sed -i 's/old-text/new-text/g' input.txt
I tried following sed commands:
sed -i 's/\\\"/\\\\\'/g' input.txt
sed -i "s/\\\"/\\\\'/g" input.txt
Unfortunately they fail because what I am looking for is a string substitution for \" while commands I tried change individual " characters.
You can't escape a single quote inside single quotes. Your second attempt needs more backslashes: Remember, inside double quotes, the shell processes one layer of backslashes, so you have to double each backslash which should make it through to sed.
sed "s/\\\\\"/\\\\\\\\'/g" input.txt
After the shell has processed the double-quoted string, the script which ends up being executed is
s/\\"/\\\\'/g
where the first pair of backslashes produce a literal backslash in the matching regex, and each pair of backslashes in the replacement produce one literal backslash in the output.
Demo: https://ideone.com/XqfwbV
I need to replace text in a file with a Windows-style directory path containing backslash (REVERSE SOLIDUS) characters. I am already using an alternative expression delimiter. The backslashes appear to be treated as escape characters.
How can I keep the backslashes in the output?
$ echo DIR=foobar | sed -e "s#DIR=.*#$(cygpath -w $(pwd))#"
C:gwin64homelit
The desired output is:
C:\cygwin64\home\lit
You'll have to escape metacharacters in sed replacement pattern. Fortunately, there are only three of those: &, \, and a delimiter / (see this question and this). In your case, since you're using # for delimiter, you'll have to escape # instead of /.
You can create a helper shell function (like here):
escapeSubst() { sed 's/[&#\]/\\&/g'; }
and then pass your string through it before giving it to sed, like this:
$ echo DIR=foobar | sed -e "s#DIR=.*#$(cygpath -w $(pwd) | escapeSubst)#"
C:\cygwin64\home\lit
I need to comment out a line in a crontab file through a script, so it contains directories, spaces and symbols. This specific line is stored in a variable and I am starting to get mixed up on how to escape the variable. Since the line changes on a regular basis I dont want any escaping in there. I don't want to simply add # in front of it, since I also need to switch it around and replace the line again with the original without the #.
So the goal is to replace $line with #$line (comment) with the possibility to do it the other way around (uncomment).
So I have a variable:
line="* * * hello/this/line & /still/this/line"
This is a line that occurs in a file, file.txt. Wich needs to get comment out.
First try:
sed -i "s/^${line}/#${line}/" file.txt
Second try:
sed -i 's|'${line}'|'"#${line}"'|g' file.txt
choroba's helpful answer shows an effective solution using perl.
sed solution
If you want to use sed, you must use a separate sed command just to escape the $line variable value, because sed has no built-in way to escape strings for use as literals in a regex context:
lineEscaped=$(sed 's/[^^]/[&]/g; s/\^/\\^/g' <<<"$line") # escape $line for use in regex
sed -i "s/^$lineEscaped\$/#&/" file.txt # Note the \$ to escape the end-of-line anchor $
With BSD/macOS sed, use -i '' instead of just -i for in-place updating without backup.
And the reverse (un-commenting):
sed -i "s/^#\($lineEscaped\)\$/\1/" file.txt
See this answer of mine for an explanation of the sed command used for escaping, which should work with any input string.
Also note how variable $lineEscaped is only referenced once, in the regex portion of the s command, whereas the substitution-string portion simply references what the regex matched (which avoids the need to escape the variable again, using different rules):
& in the substitution string represents the entire match, and \1 the first capture group (parenthesized subexpression, \(...\)).
For simplicity, the second sed command uses double quotes in order to embed the value of shell variable $lineEscaped in the sed script, but it is generally preferable to use single-quoted scripts so as to avoid confusion between what the shell interprets up front vs. what sed ends up seeing.
For instance, $ is special to both the shell and sed, and in the above script the end-of-line anchor $ in the sed regex must therefore be escaped as \$ to prevent the shell from interpreting it.
One way to avoid confusion is to selectively splice double-quoted shell-variable references into the otherwise single-quoted script:
sed -i 's/^'"$lineEscaped"'$/#&/' file.txt
awk solution
awk offers literal string matching, which obviates the need for escaping:
awk -v line="$line" '$0 == line { $0 = "#" $0 } 1' file.txt > $$.tmp && mv $$.tmp file.txt
If you have GNU Awk v4.1+, you can use -i inplace for in-place updating.
And the reverse (un-commenting):
awk -v line="#$line" '$0 == line { $0 = substr($0, 2) } 1' file.txt > $$.tmp &&
mv $$.tmp file.txt
Perl has ways to do the quoting/escaping for you:
line=$line perl -i~ -pe '$regex = quotemeta $ENV{line}; s/^$regex/#$ENV{line}/' -- input.txt
I’m having some issues trying to edit a file with SED using a variable that has forward slashes in it.
The file I’m trying to edit “checksum” has the following contents…
758f9775093bf885dbc963cba1aec732,/bin/bash
1b45027142521f2a1c3d95891770eb47,/etc/grub.conf
2a34dc288f7994cc7b1b338605ef8775,/etc/hosts
03f670845fe199955f9a0babecbda9a0,/etc/networks
I want to search for the string “/bin/bash” using a variable (file_path=“/bin/bash”) and when I find that string, remove everything up to the “,” and replace with the work “VALID”.
Here’s my sed syntax and attempts below, however it fails when I try to use the variable. Any suggestions?
Success without variable
# sed '\/bin\/bash/ s/^.*,/VALID,/g' checksum
VALID,/bin/bash
1b45027142521f2a1c3d95891770eb47,/etc/grub.conf
2a34dc288f7994cc7b1b338605ef8775,/etc/hosts
03f670845fe199955f9a0babecbda9a0,/etc/networks
Failure with variable
# file_path="/bin/bash"
# echo $file_path
/bin/bash
# sed '/$file_path/ s/^.*,/VALID,/g' checksum
758f9775093bf885dbc963cba1aec732,/bin/bash
1b45027142521f2a1c3d95891770eb47,/etc/grub.conf
2a34dc288f7994cc7b1b338605ef8775,/etc/hosts
03f670845fe199955f9a0babecbda9a0,/etc/networks
I’ve tried wrapping the variable in quotes, {} and still had the same failures:
# sed '/"$file_path"/ s/^.*,/VALID,/g' checksum
758f9775093bf885dbc963cba1aec732,/bin/bash
1b45027142521f2a1c3d95891770eb47,/etc/grub.conf
2a34dc288f7994cc7b1b338605ef8775,/etc/hosts
03f670845fe199955f9a0babecbda9a0,/etc/networks
# sed '/{$file_path}/ s/^.*,/VALID,/g' checksum
758f9775093bf885dbc963cba1aec732,/bin/bash
1b45027142521f2a1c3d95891770eb47,/etc/grub.conf
2a34dc288f7994cc7b1b338605ef8775,/etc/hosts
03f670845fe199955f9a0babecbda9a0,/etc/networks
I’ve also used “|” as the escape character.
# sed '|{$file_path}| s|^.*,|VALID,|g' checksum
sed: -e expression #1, char 1: unknown command: `|'
Any suggestions on how to accomplish this?
# sed --version
GNU sed version 4.2.1
The single quotes inhibit variable substitution. Try this:
sed "\:$file_path: s/^.*,/VALID,/" checksum
Note that I've changed the pattern delimiter to a colon so that the slashes in the variable's value won't end the pattern.
To complete #ooga's good answer. It can be interesting to keep single quotes to avoid ambiguities that may occur with special characters that bash can interpret. To do that, you can simply close the sed block with a single quote to reopen it later:
sed '\~'$file_path'~s~^[^,]*~VALID~' checksum
Just use awk for a simple life:
awk -v fp="$file_path" 'BEGIN{FS=OFS=","} $2==fp{$1="Valid"} 1' file
That will work no matter what the contents of file_path, including / or an RE metacharacters or shell globbing characters and it won't produce a false match if it's contents partially match any part of line in the file.
I want to replace word with \word{sth} with sed.
I type in
sed -i s#word#\\word{sth}
but i am getting is word{sth} instead of \word{sth}
I tried with 1 slash also in the command
you should add four backslashes.
you need two to escape the backslash by the terminal, and two to escape it for sed. 2*2=4.
$ echo word|sed s#word#\\\\word{sth}#gi
\word{sth}
Consider enclosing sed expression with single-quotes '
sed -i 's#word#\\word{sth}#' file