sed replace forward slash with backslash - sed

I have to convert filenames having a forward slash to filenames with a back slash in a makefile using mingw32. I used the following sed command to store the result in a variable:
ORIGINAL=./a/b/main1.c ./a/b/main2.c ./a/b/main3.c
sed command:
RESULT=$(shell echo $(ORIGINAL) | sed 's/\//\\/g')
And the resulting output is:
.\a\b\main1.c .abmain2.c .abmain3.c
It works fine if I run it directly on bash. Can anyone tell me whats wrong?
Thanks!


I'm trying to duplicate your test in a makefile, but I don't have your environment.
But I would not use '/' as a command separator if I am searching for the same character. Use another character, like ':'
sed 's:/:\\:g'


RESULT=$(sed 's/\//\\/g' <<< "$ORIGINAL")
$ ORIGINAL='./a/b/main1.c ./a/b/main2.c ./a/b/main3.c'
$ echo "$ORIGINAL"
./a/b/main1.c ./a/b/main2.c ./a/b/main3.c
$ RESULT=$(sed 's/\//\\/g' <<< "$ORIGINAL")
$ echo $RESULT
.\a\b\main1.c .\a\b\main2.c .\a\b\main3.c


Most likely the shell that make is invoking is not bash. However, I find the behavior you're seeing very strange for any shell (it replaces in the first word but not the rest?!?!)
Have you considered using GNU make's $(subst ...) function, instead of the shell?

Related

sed conditionally + append to end of line if condition applied

Playing with makefile and bash shell scripting where I ended-up having a variable containing:
--global-option=build_ext --global-option=--b2-args=libtorrent-python-pic=on --global-option=--b2-args=address-model=32
I need to convert it so double-quotes gets appended at the right place such as:
--global-option=build_ext --global-option=--b2-args="libtorrent-python-pic=on" --global-option=--b2-args="address-model=32"
I tried the following without success:
echo $myvar | sed -e 's/ /\n/' | sed -z '{s/=/="/2;t;s/$/"/}'
--global-option=build_ext
--global-option="--b2-args=libtorrent-python-pic=on
EDIT: Note that this time it's --b2-args= but this could be a conjunction of --anything=, and the reason why I was focussing on the second instance of = to change for =" and if true append = at the end of word.

Since your question doesn't discuss anything about prepending --global-option= if it's missing as in the final --b2-args... string on the provided input line, I think your input was really supposed to be:
$ cat file
--global-option=build_ext --global-option=--b2-args=libtorrent-python-pic=on --global-option=--b2-args=address-model=32
in which case using any sed in any shell on every Unix box:
$ sed 's/\([^ =]*=[^= ]*=\)\([^ ]*\)/\1"\2"/g' file
--global-option=build_ext --global-option=--b2-args="libtorrent-python-pic=on" --global-option=--b2-args="address-model=32"

Makefile $shell into variable sed expanded incorrectly?

I'm having an issue trying to capture the output of a sed command in a makefile variable.
JS_SRC:=$(shell sed -n 's#.*src="\([^"]*\.js\).*#\1#p' index.html)
Which gives me
sed: -e expression #1, char 34: unknown option tos'
`
I've been trying to escape things and the like, but am always given that error.
All variations of escaping I have run, run fine from the terminal.
How does a makefile call the shell command?. /usr/bin/sh -c "cmd?" or something different?.
Somethings being interpolated but I have no idea what.

JS_SRC:=$(shell sed -n "s/.*src=\"\\([^\"]*\\.js\\).*/\\1/p" index.html)
Appears to work. I figured this out via running make -d and seeing the process it was creating.
What was baffling is that it did different things with ' vs " in the sed argument. " is run with /bin/sh -c "args" so I was able to tweak the escaping to get what I needed to appear there. Using ' seems to invoke sed directly.
There is a whole heap of escaping, that i imagine is unnecessary (I don't need to interpolate variables in the sed expression, but it sends it to a shell I understand. So it will have to do ! :)

Strange sed command

I was trying to install docker in Ubuntu, and following the instructions I have come across a very strange sed command which I do not understand (this seems to be used to set-up bash autocompletion for docker):
sudo sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io
What is that command doing? The -i command means in-place editing, but what does the $acomplete -F _docker docker mean? The $ is matching the last line, but what is it doing? I do not even recognize any sed command there! For example, a substitution command would look like:
$s/in/out/
Could somebody explain that expression for me?

It appends complete -F _docker docker to the file.
From sed's manual:
a \
text Append text, which has each embedded newline preceded by a backslash.

How to insert strings containing slashes with sed? [duplicate]

This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 1 year ago.
I have a Visual Studio project, which is developed locally. Code files have to be deployed to a remote server. The only problem is the URLs they contain, which are hard-coded.
The project contains URLs such as ?page=one. For the link to be valid on the server, it must be /page/one .
I've decided to replace all URLs in my code files with sed before deployment, but I'm stuck on slashes.
I know this is not a pretty solution, but it's simple and would save me a lot of time. The total number of strings I have to replace is fewer than 10. A total number of files which have to be checked is ~30.
An example describing my situation is below:
The command I'm using:
sed -f replace.txt < a.txt > b.txt
replace.txt which contains all the strings:
s/?page=one&/pageone/g
s/?page=two&/pagetwo/g
s/?page=three&/pagethree/g
a.txt:
?page=one&
?page=two&
?page=three&
Content of b.txt after I run my sed command:
pageone
pagetwo
pagethree
What I want b.txt to contain:
/page/one
/page/two
/page/three

The easiest way would be to use a different delimiter in your search/replace lines, e.g.:
s:?page=one&:pageone:g
You can use any character as a delimiter that's not part of either string. Or, you could escape it with a backslash:
s/\//foo/
Which would replace / with foo. You'd want to use the escaped backslash in cases where you don't know what characters might occur in the replacement strings (if they are shell variables, for example).

The s command can use any character as a delimiter; whatever character comes after the s is used. I was brought up to use a #. Like so:
s#?page=one&#/page/one#g

A very useful but lesser-known fact about sed is that the familiar s/foo/bar/ command can use any punctuation, not only slashes. A common alternative is s#foo#bar#, from which it becomes obvious how to solve your problem.

add \ before special characters:
s/\?page=one&/page\/one\//g
etc.

In a system I am developing, the string to be replaced by sed is input text from a user which is stored in a variable and passed to sed.
As noted earlier on this post, if the string contained within the sed command block contains the actual delimiter used by sed - then sed terminates on syntax error. Consider the following example:
This works:
$ VALUE=12345
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345
This breaks:
$ VALUE=12345/6
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
sed: -e expression #1, char 21: unknown option to `s'
Replacing the default delimiter is not a robust solution in my case as I did not want to limit the user from entering specific characters used by sed as the delimiter (e.g. "/").
However, escaping any occurrences of the delimiter in the input string would solve the problem.
Consider the below solution of systematically escaping the delimiter character in the input string before having it parsed by sed.
Such escaping can be implemented as a replacement using sed itself, this replacement is safe even if the input string contains the delimiter - this is since the input string is not part of the sed command block:
$ VALUE=$(echo ${VALUE} | sed -e "s#/#\\\/#g")
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345/6
I have converted this to a function to be used by various scripts:
escapeForwardSlashes() {
# Validate parameters
if [ -z "$1" ]
then
echo -e "Error - no parameter specified!"
return 1
fi
# Perform replacement
echo ${1} | sed -e "s#/#\\\/#g"
return 0
}

this line should work for your 3 examples:
sed -r 's#\?(page)=([^&]*)&#/\1/\2#g' a.txt
I used -r to save some escaping .
the line should be generic for your one, two three case. you don't have to do the sub 3 times
test with your example (a.txt):
kent$ echo "?page=one&
?page=two&
?page=three&"|sed -r 's#\?(page)=([^&]*)&#/\1/\2#g'
/page/one
/page/two
/page/three

replace.txt should be
s/?page=/\/page\//g
s/&//g

please see this article
http://netjunky.net/sed-replace-path-with-slash-separators/
Just using | instead of /

Great answer from Anonymous. \ solved my problem when I tried to escape quotes in HTML strings.
So if you use sed to return some HTML templates (on a server), use double backslash instead of single:
var htmlTemplate = "<div style=\\"color:green;\\"></div>";

A simplier alternative is using AWK as on this answer:
awk '$0="prefix"$0' file > new_file

You may use an alternative regex delimiter as a search pattern by backs lashing it:
sed '\,{some_path},d'
For the s command:
sed 's,{some_path},{other_path},'

Perl command is not behaving as expected?

I have a file with below contents:
[TEMP.s_m_update_BUS_spec]
$$SRC_STAT_RA=WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
$InputFile_RA_SPE=/edwload/rqt/workingdir/status_spe/WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
[TEMP.s_m_upd_salions_rqthk]
$$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901094550
$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
I am using below perl command to just replace WHG_STATUS_SITEENTSEQCHAIN_20110901094550 with WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat in the section [TEMP.s_m_upd_salions_rqthk] But somehow its not giving me expected result. Even the WHG_STATUS_SITEENTSEQCHAIN_20110901094550 under section [TEMP.s_m_update_BUS_spec] is getting replaced.
perl -p -i -e "s|\$\$SRC_STAT_RN=.*|\$\$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|g;s|\$InputFile_RN_RQT=\/edwload\/rqt\/workingdir\/restriction\/.*|\$InputFile_RN_RQT=\/edwload\/rqt\/workingdir\/restriction\/WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|g" Input_File
Please let me know the modifications required in command above.Same subsitute commands works fine with SED command. But i wud want to use perl.

The program you run is
s|$$SRC_STAT_RN=.*|$$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|g; s|$InputFile_RN_RQT=\/edwload\/rqt\/workingdir\/restriction\/.*|$InputFile_RN_RQT=\/edwload\/rqt\/workingdir\/restriction\/WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|g
There are a fair number of $ that should be escaped but aren't. It would be simpler if you used single quotes instead of double quotes. You were probably trying for:
perl -i -pe'
s{\$\$SRC_STAT_RN=.*}{\$\$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat}g;
s{\$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/.*}{\$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat}g;
' Input_File

What exactly is not working as you want? On my machine, after running your perl code, the file looks like:
[TEMP.s_m_update_BUS_spec] $$SRC_STAT_RA=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat
[TEMP.s_m_upd_salions_rqthk] $$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat
Ain't this what you expected?
Edit
Try modifying your command to:
perl -p -i -e "s|\$\$SRC_STAT_RN=.*?|\$\$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|gmx;s|\$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/.*?|\$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|gmx" Input_File
and see if the result is as expected:
[TEMP.s_m_update_BUS_spec]
$$SRC_STAT_RA=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.datWHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
$InputFile_RA_SPE=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat/edwload/rqt/workingdir/status_spe/WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
[TEMP.s_m_upd_salions_rqthk]
$$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.datWHG_STATUS_SITEENTSEQCHAIN_20110901094550
$InputFile_RN_RQT=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat/edwload/rqt/workingdir/restriction/WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat