I'm trying to remove some text from multiple files using sed. This is the text I'm trying to delete:
\once override TupletBracket #'stencil = ##f
I've tried this line in sed but I can't get it to work:
sed -i '' -e 's/\\once \\override TupletBracket #'stencil = ##f//g' *ily
I've tried escaping the # symbols, the ' and the = but still no joy. Could anyone please point me in the right direction?
I think it's better to use single quotes here rather than double quotes to avoid the extra \s and other possible expansions (e.g. variables). Where you want a literal single quote, you close the quotation, add \', and then start a new quotation for the remainder.
$ cat in
before \once override TupletBracket #'stencil = ##f after
$ sed 's/\\once override TupletBracket #'\''stencil = ##f//g' in
before after
you can't use ' directly inside sed command that is quoted using '. Use a double quotes instead and to match \ you'll need to use \\\ to have \\ i.e \.
$ sed "s/\\\once override TupletBracket #'stencil = ##f//g"
\once override TupletBracket #'stencil = ##f
hello \once override TupletBracket #'stencil = ##f xyz
hello xyz
$
# and = are not RE metacharacters nor do they have any other special meaning to sed within a regexp (= does outside of a regexp) unless the regexp is delimited with one of them so there's no reason to escape them in your script. ' only has significance if the whole script is delimited with 's since in shell no script that's delimited by a given character can include that character. So here's your choices:
$ echo "seab'cd" | sed 's/b'\''c/foo/'
seafood
$ echo "seab'cd" | sed "s/b'c/foo/"
seafood
Note that if you use the second (double quotes) version then you're allowing shell variables to expand inside the script and would require double-backslashes to escape chars.
I expected using the octal representation of a ' (i.e. \047) would work too like it does in awk:
$ echo "seab'cd" | awk '{sub(/b\047c/,"foo")}1'
seafood
but it didn't:
$ echo "seab'cd" | sed 's/b\047c/foo/'
seab'cd
and I suspect that's because sed is treating \0 as a backreference. It does work with the hex representation:
$ echo "seab'cd" | sed 's/b\x27c/foo/'
seafood
but that's dangerous and should be avoided (see http://awk.freeshell.org/PrintASingleQuote).
Related
I'm trying to run the command below to replace every char in DECEMBER by itself followed by $n question marks. I tried both escaping {$n} like so {$n} and leaving it as is. Yet my output just keeps being D?{$n}E?{$n}... Is it just not possible to do this with a sed?
How should i got about this.
echo 'DECEMBER' > a.txt
sed -i "s%\(.\)%\1\(?\){$n}%g" a.txt
cat a.txt
This might work for you (GNU sed):
n=5
sed -E ':a;s/[^\n]/&\n/g;x;s/^/x/;/x{'"$n"'}/{z;x;y/\n/?/;b};x;ba' file
Append a newline to each non-newline character in a line $n times then replace all newlines by the intended character ?.
N.B. The newline is chosen as the initial substitute character as it is not possible for it to be within a line (sed uses newlines to separate lines) and if the final substitution character already exists within the current line, the substitutions are correct.
Range (also, interval or limiting quantifiers), like {3} / {3,} / {3,6}, are part of regex, and not replacement patterns.
You can use
sed -i "s/./&$(for i in {1..7}; do echo -n '?'; done)/g" a.txt
See the online demo:
#!/bin/bash
sed "s/./&$(for i in {1..7}; do echo -n '?'; done)/g" <<< "DECEMBER"
# => D???????E???????C???????E???????M???????B???????E???????R???????
Here, . matches any char, and & in the replacement pattern puts it back and $(for i in {1..7}; do echo -n '?'; done) adds seven question marks right after it.
This one-liner should do the trick:
sed 's/./&'$(printf '%*s' "$n" '' | tr ' ' '?')'/g' a.txt
with the assumption that $n expands to a positive integer and the command is executed in a POSIX shell.
Efficiently using any awk in any shell on every Unix box after setting n=2:
$ awk -v n="$n" '
BEGIN {
new = sprintf("%*s",n,"")
gsub(/./,"?",new)
}
{
gsub(/./,"&"new)
print
}
' a.txt
D??E??C??E??M??B??E??R??
To make the changes "inplace" use GNU awk with -i inplace just like GNU sed has -i.
Caveat - if the character you want to use in the replacement text is & then you'd need to use gsub(/./,"\\\\\\&",new) in the BEGIN section to make it is treated as literal instead of a backreference metachar. You'd have that issue and more (e.g. handling \1 or /) with any sed solution and any solution that uses double quotes around the script would have more issues with handling $s and the solutions that have a shell script expanding unquoted would have even more issues with globbing chars.
I have to collect a select query data to a CSV file. I want to use a sed command to replace \n from the data by a space.
I'm using this:
query | sed "s/\n/ /g" > file.csv .......
But it is not working. Only \ is getting removed, while it should also remove n and add a space. Please suggest something.
You want to replace newline with space, not necessarily using sed.
Use tr:
tr '\n' ' '
\n is special to sed: it stands for the newline character. To replace a literal \n, you have to escape the backslash:
sed 's/\\n/ /g'
Notice that I've used single quotes. If you use double quotes, the backslash has a special meaning if followed by any of $, `, ", \, or newline, i.e., "\n" is still \n, but "\\n" would become \n.
Since we want sed to see \\n, we'd have to use one of these:
sed "s/\\\n/ /g" – the first \\ becomes \, and \n doesn't change, resulting in \\n
sed "s/\\\\n/ /g" – both pairs of \\ are reduced to \ and sed gets \\n as well
but single quotes are much simpler:
$ sed 's/\\n/ /g' <<< 'my\nname\nis\nrohinee'
my name is rohinee
From comments on the question, it became apparent that sed had nothing to do with removing the backslashes; the OP tried
echo my\nname\nis | sed 's/\n/ /g'
but the backslashes are removed by the shell:
$ echo my\nname\nis
mynnamenis
so even if the correct \\n were used, sed wouldn't find any matches. The correct way is
$ echo 'my\nname\nis' | sed 's/\\n/ /g'
my name is
How do you escape line beginning and line end in bracket expressions in sed?
For example, let's say I want to replace both comma, line beginning, and line end in each line with pipe:
echo "a,b,c" | sed 's/,/|/g'
# a|b|c
echo "a,b,c" | sed 's/^/|/g'
# |a,b,c
echo "a,b,c" | sed 's/$/|/g'
# a,b,c|
echo "a,b,c" | sed 's/[,^$]/|/g'
# a|b|c
I would expect the last command to produce |a|b|c|. I also tried escaping the line beginning and line end via backslash, with no change.
With GNU sed with extended regular expressions, you can do:
$ echo "a,b,c" | /opt/gnu/bin/sed -E 's/^|,|$/|/g'
|a|b|c|
$
The -E option enables the extended regular expressions, as does -r, but -E is also used by other sed variants for the same purpose, unlike -r.
However, for reasons which elude me, the BSD (macOS) variant of sed produces:
$ echo "a,b,c" | sed -E 's/^|,|$/|/g'
|a|b|c
$
I can't think why.
If this variability is unacceptable, go with the three-substitution solution:
$ echo "a,b,c" | sed -e "s/^/|/" -e "s/$/|/" -e "s/,/|/g"
|a|b|c|
$
which should work with any variant of sed. However, note that echo "" | sed …3 subs… produces || whereas the -E variant produces |. I'm not sure if there's an easy fix for that.
You tried this, but it didn't do what you wanted:
$ echo "a,b,c" | sed 's/[,^$]/|/g'
a|b|c
$
This is what should be expected. Inside character classes, most special characters lose their special-ness. There is nothing special about $ (or , but it isn't a metacharacter anyway) in a character class; ^ is only special at the start of the class and it negates the character class. That means that what follows shows the correct, expected behaviour from this permutation of the contents of your character class:
$ echo "a,b\$\$b,c" | sed 's/[^,$]/|/g'
|,|$$|,|
$
It mapped all the non-comma, non-dollar characters to pipes. I should be using single quotes around the echo; then the backslashes wouldn't be necessary. I just followed the question's code quietly.
Following sed may help you in same.
echo "a,b,c" | sed 's/^/|/;s/,/|/g;s/$/|/'
Output will be as follows.
|a|b|c|
I need to replace text in a file with a Windows-style directory path containing backslash (REVERSE SOLIDUS) characters. I am already using an alternative expression delimiter. The backslashes appear to be treated as escape characters.
How can I keep the backslashes in the output?
$ echo DIR=foobar | sed -e "s#DIR=.*#$(cygpath -w $(pwd))#"
C:gwin64homelit
The desired output is:
C:\cygwin64\home\lit
You'll have to escape metacharacters in sed replacement pattern. Fortunately, there are only three of those: &, \, and a delimiter / (see this question and this). In your case, since you're using # for delimiter, you'll have to escape # instead of /.
You can create a helper shell function (like here):
escapeSubst() { sed 's/[&#\]/\\&/g'; }
and then pass your string through it before giving it to sed, like this:
$ echo DIR=foobar | sed -e "s#DIR=.*#$(cygpath -w $(pwd) | escapeSubst)#"
C:\cygwin64\home\lit
How to escape a single quote in a sed expression that is already surrounded by quotes?
For example:
sed 's/ones/one's/' <<< 'ones thing'
Quote sed codes with double quotes:
$ sed "s/ones/one's/"<<<"ones thing"
one's thing
I don't like escaping codes with hundreds of backslashes – hurts my eyes. Usually I do in this way:
$ sed 's/ones/one\x27s/'<<<"ones thing"
one's thing
One trick is to use shell string concatenation of adjacent strings and escape the embedded quote using shell escaping:
sed 's/ones/two'\''s/' <<< 'ones thing'
two's thing
There are 3 strings in the sed expression, which the shell then stitches together:
sed 's/ones/two'
\'
's/'
Escaping single quote in sed: 3 different ways:
From fragile to solid...
Note: This answer is based on GNU sed!!
1. Using double-quotes to enclose sed script:
Simpliest way:
sed "s/ones/one's/" <<< 'ones thing'
But using double-quote lead to shell variables expansion and backslashes to be considered as shell escape before running sed.
1.1. Specific case without space and special chars
In this specific case, you could avoid enclosing at shell level (command line):
sed s/ones/one\'s/ <<<'ones thing'
will work until whole sedscript don't contain spaces, semicolons, special characters and so on... (fragile!)
2. Using octal or hexadecimal representation:
This way is simple and efficient, if not as readable as next one.
sed 's/ones/one\o047s/' <<< 'ones thing'
sed 's/ones/one\x27s/' <<< 'ones thing'
And as following character (s) is not a digit, you coul write octal with only 2 digits:
sed 's/ones/one\o47s/' <<< 'ones thing'
3. Creating a dedicated sed script
cat <<eosedscript >sampleSedWithQuotes.sed
#!$(which sed) -f
s/ones/one's/;
eosedscript
chmod +x sampleSedWithQuotes.sed
From there, you could run:
./sampleSedWithQuotes.sed <<<'ones thing'
one's thing
This is the strongest and simpliest solution as your script is the most readable:$ cat sampleSedWithQuotes.sed
#!/bin/sed -f
s/ones/one's/;
3.1 You coud use -i sed flag:
As this script use sed in shebang, you could use sed flags on command line. For editing file.txt in place, with the -i flag:
echo >file.txt 'ones thing'
./sampleSedWithQuotes.sed -i file.txt
cat file.txt
one's thing
3.2 Mixing quotes AND double quotes
Using dedicated script may simplify mixing quotes and double quotes in same script.
Adding a new operation in our script to enclose the word thing in double quotes:
echo >>sampleSedWithQuotes.sed 's/\bthing\b/"&"/;'
( now our script look like:
#!/bin/sed -f
s/ones/one's/;
s/\bthing\b/"&"/;
)
then
./sampleSedWithQuotes.sed <<<'ones thing'
one's "thing"
The best way is to use $'some string with \' quotes \''
eg:
sed $'s/ones/two\'s/' <<< 'ones thing'
Just use double quotes on the outside of the sed command.
$ sed "s/ones/one's/" <<< 'ones thing'
one's thing
It works with files too.
$ echo 'ones thing' > testfile
$ sed -i "s/ones/one's/" testfile
$ cat testfile
one's thing
If you have single and double quotes inside the string, that's ok too. Just escape the double quotes.
For example, this file contains a string with both single and double quotes. I'll use sed to add a single quote and remove some double quotes.
$ cat testfile
"it's more than ones thing"
$ sed -i "s/\"it's more than ones thing\"/it's more than one's thing/" testfile
$ cat testfile
it's more than one's thing
This is kind of absurd but I couldn't get \' in sed 's/ones/one\'s/' to work. I was looking this up to make a shell script that will automatically add import 'hammerjs'; to my src/main.ts file with Angular.
What I did get to work is this:
apost=\'
sed -i '' '/environments/a\
import '$apost'hammerjs'$apost';' src/main.ts
So for the example above, it would be:
apost=\'
sed 's/ones/one'$apost's/'
I have no idea why \' wouldn't work by itself, but there it is.
Some escapes on AppleMacOSX terminals fail so:
sed 's|ones|one'$(echo -e "\x27")'s|1' <<<'ones thing'
use an alternative string seperator like ":" to avoid confusion with different slashes
sed "s:ones:one's:" <<< 'ones thing'
or if you wish to highligh the single quote
sed "s:ones:one\'s:" <<< 'ones thing'
both return
one's thing