This question already has an answer here:
sed fails with "unknown option to `s'" error [closed]
(1 answer)
Closed 7 years ago.
I'm trying to use sed to update a config file using a bash script. I have a similar sed command right above this one in the script that runs fine. I can't seem to figure out why this is breaking:
sed -i.bak \
-e s/"socketPath:'https://localhost:9091'"/"socketPath:'/socket'"/g \
$WEB_CONF
Any ideas?
The quotes and double quotes are causing problems.
You are using them in view of the slashes in the string.
In sed you can use another delimiter, such as a #.
sed -e 's#socketPath:https://localhost:9091#socketPath:/socket#g' \
$WEB_CONF
Escape your slashes in the pattern or use a different delimiter like this:
sed -i.bak \
-e s%"socketPath:'https://localhost:9091'"%"socketPath:'/socket'"%g \
$WEB_CONF
I am confused looking at the delimiters you have used so you can't blame sed for goofing up. When your dataset has / in them, it is often advised to use a different delimiters like # or _ (yes, sed supports various delimiters).
Also, your quoting looks abit off. The syntax should be:
sed -i.bak 's#substitute#replacement#g' "$WEB_CONF"
If your substitute and/or replacement has variables use " instead ' which will allow it to interpolate.
Related
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 1 year ago.
I have a Visual Studio project, which is developed locally. Code files have to be deployed to a remote server. The only problem is the URLs they contain, which are hard-coded.
The project contains URLs such as ?page=one. For the link to be valid on the server, it must be /page/one .
I've decided to replace all URLs in my code files with sed before deployment, but I'm stuck on slashes.
I know this is not a pretty solution, but it's simple and would save me a lot of time. The total number of strings I have to replace is fewer than 10. A total number of files which have to be checked is ~30.
An example describing my situation is below:
The command I'm using:
sed -f replace.txt < a.txt > b.txt
replace.txt which contains all the strings:
s/?page=one&/pageone/g
s/?page=two&/pagetwo/g
s/?page=three&/pagethree/g
a.txt:
?page=one&
?page=two&
?page=three&
Content of b.txt after I run my sed command:
pageone
pagetwo
pagethree
What I want b.txt to contain:
/page/one
/page/two
/page/three
The easiest way would be to use a different delimiter in your search/replace lines, e.g.:
s:?page=one&:pageone:g
You can use any character as a delimiter that's not part of either string. Or, you could escape it with a backslash:
s/\//foo/
Which would replace / with foo. You'd want to use the escaped backslash in cases where you don't know what characters might occur in the replacement strings (if they are shell variables, for example).
The s command can use any character as a delimiter; whatever character comes after the s is used. I was brought up to use a #. Like so:
s#?page=one&#/page/one#g
A very useful but lesser-known fact about sed is that the familiar s/foo/bar/ command can use any punctuation, not only slashes. A common alternative is s#foo#bar#, from which it becomes obvious how to solve your problem.
add \ before special characters:
s/\?page=one&/page\/one\//g
etc.
In a system I am developing, the string to be replaced by sed is input text from a user which is stored in a variable and passed to sed.
As noted earlier on this post, if the string contained within the sed command block contains the actual delimiter used by sed - then sed terminates on syntax error. Consider the following example:
This works:
$ VALUE=12345
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345
This breaks:
$ VALUE=12345/6
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
sed: -e expression #1, char 21: unknown option to `s'
Replacing the default delimiter is not a robust solution in my case as I did not want to limit the user from entering specific characters used by sed as the delimiter (e.g. "/").
However, escaping any occurrences of the delimiter in the input string would solve the problem.
Consider the below solution of systematically escaping the delimiter character in the input string before having it parsed by sed.
Such escaping can be implemented as a replacement using sed itself, this replacement is safe even if the input string contains the delimiter - this is since the input string is not part of the sed command block:
$ VALUE=$(echo ${VALUE} | sed -e "s#/#\\\/#g")
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345/6
I have converted this to a function to be used by various scripts:
escapeForwardSlashes() {
# Validate parameters
if [ -z "$1" ]
then
echo -e "Error - no parameter specified!"
return 1
fi
# Perform replacement
echo ${1} | sed -e "s#/#\\\/#g"
return 0
}
this line should work for your 3 examples:
sed -r 's#\?(page)=([^&]*)&#/\1/\2#g' a.txt
I used -r to save some escaping .
the line should be generic for your one, two three case. you don't have to do the sub 3 times
test with your example (a.txt):
kent$ echo "?page=one&
?page=two&
?page=three&"|sed -r 's#\?(page)=([^&]*)&#/\1/\2#g'
/page/one
/page/two
/page/three
replace.txt should be
s/?page=/\/page\//g
s/&//g
please see this article
http://netjunky.net/sed-replace-path-with-slash-separators/
Just using | instead of /
Great answer from Anonymous. \ solved my problem when I tried to escape quotes in HTML strings.
So if you use sed to return some HTML templates (on a server), use double backslash instead of single:
var htmlTemplate = "<div style=\\"color:green;\\"></div>";
A simplier alternative is using AWK as on this answer:
awk '$0="prefix"$0' file > new_file
You may use an alternative regex delimiter as a search pattern by backs lashing it:
sed '\,{some_path},d'
For the s command:
sed 's,{some_path},{other_path},'
I'm trying to replace a string AUD A0-FX.20 with AUD/USD.20 using sed for windows through the windows cmd shell.
I dont think windows shell handles spaces in strings well. Here is what I'm running -
SED -e s{AUD A0-FX.20{AUD/USD.20{ "C:\sed\bin\text.txt" > "C:\sed\bin\text1.txt"
but I get an error SED: -e expression #1, char 5: unterminateds' command`
I'm using { as the delimiter because i already have a / in the replacement string. Any help would be appreciated. I'm using sed for windows from http://gnuwin32.sourceforge.net/packages/sed.htm
I don't know if this will solve your question because can't test in Windows, but I will give two suggestions:
. is special for sed and matches any character. Escape it like \.
Quote your sed instructions, with single or double quotes, like "s{...{...{"
Try:
sed "s|AUD A0-FX\.20|AUD/USD.20|"
I'm trying to find some text in an XML file and delete only a part of a line that has it.
I found this format to try: perl -p -i -e "s/$1/$2/g" $3 after some code searches.
So I'm using this code:
perl -p -i.bak -e "s/\'../../../specialText/\'//g" "C:/box/fileName.XML";
What I want to do is delete everything from the inner single quotes as in:
'../../../specialText/', but using q() or \' to escape the quote doesn't work and I'm not sure the ..'s aren't messing things up either. I'm guessing that not putting anything in as a text replacement will delete it properly, but I'm not sure.
The errors are:
Backslash found where operator expected at -e line 1, near "/specialText/\"
(Missing operator before \?)
syntax error at -e line 1, near "/specialText/\"
Can't find string terminator "'" anywhere before EOF at -e line 1.
How do rewrite this one liner to accomplish this?
This works.
C:\box>perl -p -i.bak -e s/Copyright/bar/g Test.txt
I tried it on another file, so now I just have to play with it to modify my original.
You can escape the . and / characters in the search string by putting a backslash (\) before each of them.
However, to avoid acute leaning toothpick syndrome, I'd recommend instead using alternative regexp delimiters and the \Q and \E escape sequences, like this:
perl -p -i.bak -e "s(\Q'../../../specialText/'\E)()g" "C:/box/fileName.XML"
And what's wrong with using another set of delimiters?
perl -p -i.bak -e "s{'../../../specialText/'}{}g" "C:/box/fileName.XML"
I am trying to write a sed expression that can remove urls from a file
example
http://samgovephotography.blogspot.com/ updated my blog just a little bit ago. Take a chance to check out my latest work. Hope all is well:)
Meet Former Child Star & Author Melissa Gilbert 6/15/09 at LA's B&N https://hollywoodmomblog.com/?p=2442 Thx to HMB Contributor #kdpartak :)
But I dont get it:
sed 's/[\w \W \s]*http[s]*:\/\/\([\w \W]\)\+[\w \W \s]*/ /g' posFile
FIXED!!!!!
handles almost all cases, even malformed URLs
sed 's/[\w \W \s]*http[s]*[a-zA-Z0-9 : \. \/ ; % " \W]*/ /g' positiveTweets | grep "http" | more
The following removes http:// or https:// and everything up until the next space:
sed -e 's!http\(s\)\{0,1\}://[^[:space:]]*!!g' posFile
updated my blog just a little bit ago. Take a chance to check out my latest work. Hope all is well:)
Meet Former Child Star & Author Melissa Gilbert 6/15/09 at LA's B&N Thx to HMB Contributor #kdpartak :)
Edit:
I should have used:
sed -e 's!http[s]\?://\S*!!g' posFile
"[s]\?" is a far more readable way of writing "an optional s" compared to "\(s\)\{0,1\}"
"\S*" a more readable version of "any non-space characters" than "[^[:space:]]*"
I must have been using the sed that came installed with my Mac at the time I wrote this answer (brew install gnu-sed FTW).
There are better URL regular expressions out there (those that take into account schemes other than HTTP(S), for instance), but this will work for you, given the examples you give. Why complicate things?
The accepted answer provides the approach that I used to remove URLs, etc. from my files. However it left "blank" lines. Here is a solution.
sed -i -e 's/http[s]\?:\/\/\S*//g ; s/www\.\S*//g ; s/ftp:\S*//g' input_file
perl -i -pe 's/^'`echo "\012"`'${2,}//g' input_file
The GNU sed flags, expressions used are:
-i Edit in-place
-e [-e script] --expression=script : basically, add the commands in script
(expression) to the set of commands to be run while processing the input
^ Match start of line
$ Match end of line
? Match one or more of preceding regular expression
{2,} Match 2 or more of preceding regular expression
\S* Any non-space character; alternative to: [^[:space:]]*
However,
sed -i -e 's/http[s]\?:\/\/\S*//g ; s/www\.\S*//g ; s/ftp:\S*//g'
leaves nonprinting character(s), presumably \n (newlines). Standard sed-based approaches to remove "blank" lines, tabs and spaces, e.g.
sed -i 's/^[ \t]*//; s/[ \t]*$//'
do not work, here: if you do not use a "branch label" to process newlines, you cannot replace them using sed (which reads input one line at a time).
The solution is to use the following perl expression:
perl -i -pe 's/^'`echo "\012"`'${2,}//g'
which uses a shell substitution,
'`echo "\012"`'
to replace an octal value
\012
(i.e., a newline, \n), that occurs 2 or more times,
{2,}
(otherwise we would unwrap all lines), with something else; here:
//
i.e., nothing.
[The second reference below provides a wonderful table of these values!]
The perl flags used are:
-p Places a printing loop around your command,
so that it acts on each line of standard input
-i Edit in-place
-e Allows you to provide the program as an argument,
rather than in a file
References:
perl flags: Perl flags -pe, -pi, -p, -w, -d, -i, -t?
ASCII control codes: https://www.cyberciti.biz/faq/unix-linux-sed-ascii-control-codes-nonprintable/
remove URLs: sed to remove URLs from a file
branch labels: How can I replace a newline (\n) using sed?
GNU sed manual: https://www.gnu.org/software/sed/manual/sed.html
quick regex guide: https://www.gnu.org/software/sed/manual/html_node/Regular-Expressions.html
Example:
$ cat url_test_input.txt
Some text ...
https://stackoverflow.com/questions/4283344/sed-to-remove-urls-from-a-file
https://www.google.ca/search?dcr=0&ei=QCsyWtbYF43YjwPpzKyQAQ&q=python+remove++citations&oq=python+remove++citations&gs_l=psy-ab.3...1806.1806.0.2004.1.1.0.0.0.0.61.61.1.1.0....0...1c.1.64.psy-ab..0.0.0....0.-cxpNc6youY
http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html
https://bbengfort.github.io/tutorials/2016/05/19/text-classification-nltk-sckit-learn.html
http://datasynce.org/2017/05/sentiment-analysis-on-python-through-textblob/
https://www.google.ca/?q=halifax&gws_rd=cr&dcr=0&ei=j7UyWuGKM47SjwOq-ojgCw
http://www.google.ca/?q=halifax&gws_rd=cr&dcr=0&ei=j7UyWuGKM47SjwOq-ojgCw
www.google.ca/?q=halifax&gws_rd=cr&dcr=0&ei=j7UyWuGKM47SjwOq-ojgCw
ftp://ftp.ncbi.nlm.nih.gov/
ftp://ftp.ncbi.nlm.nih.gov/1000genomes/ftp/alignment_indices/20100804.alignment.index
Some more text.
$ sed -e 's/http[s]\?:\/\/\S*//g ; s/www\.\S*//g ; s/ftp:\S*//g' url_test_input.txt > a
$ cat a
Some text ...
Some more text.
$ perl -i -pe 's/^'`echo "\012"`'${2,}//g' a
Some text ...
Some more text.
$
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 1 year ago.
I'm writing a shell script with this command:
sed -e 's/OLD_ITEM/NEW_ITEM/g'
But I actually want to do something that includes a directory:
sed -e 's/FOLDER/OLD_ITEM/NEW_ITEM/g'
How do ignore the forward slash so that the entire line FOLDER/OLD_ITEM is read properly?
You don't have to use / as delimiter in sed regexps. You can use whatever character you like, as long as it doesn't appear in the regexp itself:
sed -e 's#FOLDER/OLD_ITEM#NEW_ITEM#g'
or
sed -e 's|FOLDER/OLD_ITEM|NEW_ITEM|g'
You need to escape the / as \/.
The escape (\) preceding a character tells the shell to interpret that character literally.
So use FOLDER\/OLD_ITEM
Escape it !
sed -e 's/FOLDER\/OLD_ITEM/NEW_ITEM/g'