I am trying to replace ${dbPassword} in a property file with a password including $ signs.
My command is the following, but I have no idea how to replace this properly.
run: #cat $(FILE_PATH) | sed -i .bak 's/$${dbPassword}/$(VARIABLE_WITH_PWD)/g' $(FILE_PATH)
Let's say my dbPassword is: 123$456$789
With this, I am getting the result: 123${dbPassword}456${dbPassword}789
Three problems:
Target and recipe on the same line. The recipe must be on the next line, indented with a hard tab.
You pass $(FILE_PATH) on stdin, and as a file argument to sed. Remove the cat pipe.
Shell variables (as opposed to make variables) are not substituted inside single quotes.
So you might want to try this instead:
.PHONY: run
run:
sed -i .bak "s/$${dbPassword}/$(VARIABLE_WITH_PWD)/g" $(FILE_PATH)
This assumes you have an environment variable, dbPassword and exported it prior to running make. If that is not the case, please provide your complete makefile.
I also have removed the # so you actually see what command make is executing. There's no point in wearing a blindfold while debugging your makefile.
Related
Running Fedora 25 server edition. sed --version gives me sed (GNU sed) 4.2.2 along with the usual copyright and contact info. I've create a text file sudo vi ./potential_sed_bug. Vi shows the contents of this file (with :set list enabled) as:
don't$
delete$
me$
please$
I then run the following command:
sudo sed -n -i.bak /please/a\testing ./potential_sed_bug
Before we discuss the results; here is what the sed man page says:
-n, --quiet, --silent
suppress automatic printing of pattern space
and
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if extension supplied). The default operation mode is to break symbolic and hard links. This can be changed with --follow-symlinks and --copy.
I've also looked other sed command references to learn how to append with sed. Based on my understanding from the research I've done; the resulting file content should be:
don't
delete
me
please
testing
However, running sudo cat ./potential_sed_bug gives me the following output:
testing
In light of this discrepancy, is my understanding of the command I ran incorrect or is there a bug with sed/the environment?
tl;dr
Don't use -n with -i: unless you use explicit output commands in your sed script, nothing will be written to your file.
Using -i produces no stdout (terminal) output, so there's nothing extra you need to do to make your command quiet.
By default, sed automatically prints the (possibly modified) input lines to whatever its output target is, whether implied or explicitly specified: by default, to stdout (the terminal, unless redirected); with -i, to a temporary file that ultimately replaces the input file.
In both cases, -n suppresses this automatic printing, so that - unless you use explicit output functions such as p or, in your case, a - nothing gets printed to stdout / written to the temporary file.
Note that the automatic printing applies to the so-called pattern space, which is where the (possibly modified) input is held; explicit output functions such as p, a, i and c do not print to the pattern space (for potential subsequent modification), they print directly to the target stream / file, which is why a\testing was able to produce output, despite the use of -n.
Note that with -i, sed's implicit printing / explicit output commands only print to the temporary file, and not also to stdout, so a command using -i is invariably quiet with respect to stdout (terminal) output - there's nothing extra you need to do.
To give a concrete example (GNU sed syntax).
Since the use of -i is incidental to the question, I've omitted it for simplicity. Note that -i prints to a temporary file first, which, on completion, replaces the original. This comes with pitfalls, notably the potential destruction of symlinks; see the lower half of this answer of mine.
# Print input (by default), and append literal 'testing' after
# lines that contain 'please'.
$ sed '/please/ a testing' <<<$'yes\nplease\nmore'
yes
please
testing
more
# Adding `-n` suppresses the default printing, so only `testing` is printed.
# Note that the sequence of processing is exactly the same as without `-n`:
# If and when a line with 'please' is found, 'testing' is appended *at that time*.
$ sed -n '/please/ a testing' <<<$'yes\nplease\nmore'
testing
# Adding an unconditional `p` (print) call undoes the effect of `-n`.
$ sed -n 'p; /please/ a testing' <<<$'yes\nplease\nmore'
yes
please
testing
more
I want to modify src/some_file.txt before building my executable:
cmake_minimum_required(VERSION 3.5.1)
project(MyProject)
add_custom_target(run ALL
COMMAND sed -i "s#MY_PATH=\\(.*\\)#MY_PATH=${CMAKE_BINARY_DIR}/\\1#" ${CMAKE_CURRENT_SOURCE_DIR}/some_file.txt
)
add_executable(e main.cpp)
add_dependencies(e run)
src/some_file.txt has content:
MY_PATH=something
Targets run and e get build but src/some_file.txt remains unchanged. Why?
Either you're not using GNU sed or the file doesn't match the pattern.
My guess would be that you need to escape the backslashes again, but you don't need them anyway, just use:
sed -i 's#MY_PATH=#MY_PATH=${CMAKE_BINARY_DIR}/#'
Or simply
sed -i 's#MY_PATH=#&${CMAKE_BINARY_DIR}/#'
where & expands to the matched pattern. You should use single-quotes not double-quotes unless you specifically want the shell to expand variables.
I'm having an issue trying to capture the output of a sed command in a makefile variable.
JS_SRC:=$(shell sed -n 's#.*src="\([^"]*\.js\).*#\1#p' index.html)
Which gives me
sed: -e expression #1, char 34: unknown option tos'
`
I've been trying to escape things and the like, but am always given that error.
All variations of escaping I have run, run fine from the terminal.
How does a makefile call the shell command?. /usr/bin/sh -c "cmd?" or something different?.
Somethings being interpolated but I have no idea what.
JS_SRC:=$(shell sed -n "s/.*src=\"\\([^\"]*\\.js\\).*/\\1/p" index.html)
Appears to work. I figured this out via running make -d and seeing the process it was creating.
What was baffling is that it did different things with ' vs " in the sed argument. " is run with /bin/sh -c "args" so I was able to tweak the escaping to get what I needed to appear there. Using ' seems to invoke sed directly.
There is a whole heap of escaping, that i imagine is unnecessary (I don't need to interpolate variables in the sed expression, but it sends it to a shell I understand. So it will have to do ! :)
I have the following line in my proftpd log (line 78 to be precise)
Deny from 1.2.3.4
I also have a script which rolls through my logs for people using brute force attacks and then stores their IP (ready for a black listing). What i'm struggling with is inserting (presume with sed) at the end of that specific line - this is what I've got so far:
sed "77i3.4.5.6" /opt/etc/proftpd.conf >> /opt/etc/proftpd.conf
Now one would presume this would work perfectly, however it actually does the following (lines 77 through 78):
3.4.5.6
Deny from 1.2.3.4
I suspect this is due to my dated version of sed, are there any other ways of acheiving the same thing? Also the >> causes the config to be duplicated at the end of the fole (again i'm sure this is a limitation of my version of sed). This is running a homebrew linux kernel on my nas. Sed options below:
root#NAS:~# sed BusyBox v1.7.0
(2009-04-29 19:12:57 JST) multi-call
binary
Usage: sed [-efinr] pattern [files...]
Options:
-e script Add the script to the commands to be executed
-f scriptfile Add script-file contents to the
commands to be executed
-i Edit files in-place
-n Suppress automatic printing of pattern space
-r Use extended regular expression syntax
If no -e or -f is given, the first
non-option argument is taken as the
sed script to interpret. All remaining
arguments are names of input files; if
no input files are specified, then the
standard input is read. Source files
will not be modified unless -i option
is given.
Cheers for your help guys.
This has nothing to do with the version of sed; this is just plain old Doing It Wrong.
sed -i '77s/$/,3.4.5.6/' /opt/etc/proftpd.conf
I have a file named check.txt which has the below contents:
$ cat check.txt
~/bin/tibemsadmin -server $URL-user $USER -password $PASWRD
$
I have a main script where the values of $URL, $USER, $PASWRD are obtained from the main script. I want to use the SED utility to replace the $URL, $USER, $PASWRD to the actual values in the check.txt.
I am trying like this but it fails.
emsurl=tcp://myserver:3243
emsuser=test
emspasswd=new
sed s/$URL/${emsurl}/g check.txt >> check_new.txt
sed s/$USER/${emsuser}/g check.txt_new.txt >> check_new_1.txt
sed s/PASWRD/${emspasswd}/g check_new_1.txt >> final.txt
My final.txt output is desired as below:
~/bin/tibemsadmin -server tcp://myserver:3243 -user test -password new
Could you please help me?
You have to be rather careful with your use of quotes. You also need to learn how to do multiple operations in a single pass, and/or how to use pipes.
emsurl=tcp://myserver:3243
emsuser=test
emspasswd=new
sed -e "s%\$URL%${emsurl}%g" \
-e "s%\$USER%${emsuser}%g" \
-e "s%\$PASWRD%${emspasswd}%g" check.txt >final.txt
Your problem is that the shell expanded the '$URL' in your command line (probably to nothing), meaning that sed got to see something other than what you intended. By escaping the $ with the \, sed gets to see what you intended.
Note that I initially used / as the separator in the substitute operations; however, as DarkDust rightly points out, that won't work since there are slashes in the URLs. My normal fallback character is % - as now shown - but that can appear in some URLs and might not be appropriate. I'd probably use a control character, such as control-A, if I needed to worry about that - or I'd use Perl which would be able to play without getting confused.
You can also combine the three separate -e expressions into one with semi-colons replacing them. However, I prefer the clarity of the three operations clearly separated.
You could take a slightly different approach by modifying your main script as follows :-
export URL="tcp://myserver:3243"
export USER=test
export PASWRD=new
. ./check.txt
This sets up the variables and then runs check.txt within the context of your main script
Although you don't say what's failing I guess I see the problems.
I suggest you do this:
sed "s|\$URL|${emsurl}|g"
That is, the first $ needs to be escaped because you want it literally. Then, instead of / I suggest you use | (pipe) as delimiter since it's not used in your strings. Finally, use " to ensure the content is interpreted as string by the shell.
You can then pipe everything together to not need any temporary files:
sed "s|\$URL|${emsurl}|g" | sed "s|\$USER|${emsuser}|g" | sed "s|\$PASSWRD|${emspasswd}|g"
Variable substitution should be outside sed expression and '$' should be escaped; in your case something like this:
sed -e 's/\$URL/'$emsurl'/g' -e 's/\$USER/'$emsuser'/g' -e 's/\$PASSWORD/'$emaspasswd'/g'
Anyway in your place I would avoid using $ to match placeholders in a template file, because it's causing confusion with BASH variables, use a different pattern instead (for instance #URL#).