I'm having an issue trying to capture the output of a sed command in a makefile variable.
JS_SRC:=$(shell sed -n 's#.*src="\([^"]*\.js\).*#\1#p' index.html)
Which gives me
sed: -e expression #1, char 34: unknown option tos'
`
I've been trying to escape things and the like, but am always given that error.
All variations of escaping I have run, run fine from the terminal.
How does a makefile call the shell command?. /usr/bin/sh -c "cmd?" or something different?.
Somethings being interpolated but I have no idea what.
JS_SRC:=$(shell sed -n "s/.*src=\"\\([^\"]*\\.js\\).*/\\1/p" index.html)
Appears to work. I figured this out via running make -d and seeing the process it was creating.
What was baffling is that it did different things with ' vs " in the sed argument. " is run with /bin/sh -c "args" so I was able to tweak the escaping to get what I needed to appear there. Using ' seems to invoke sed directly.
There is a whole heap of escaping, that i imagine is unnecessary (I don't need to interpolate variables in the sed expression, but it sends it to a shell I understand. So it will have to do ! :)
Related
#!/bin/bash
jobname="job_201312161447_0003"
jobname_pre=${jobname:0:16}
jobname_post=${jobname:17}
This bash script gives me Bad substitution error on ubuntu. Any help will be highly appreciated.
The default shell (/bin/sh) under Ubuntu points to dash, not bash.
me#pc:~$ readlink -f $(which sh)
/bin/dash
So if you chmod +x your_script_file.sh and then run it with ./your_script_file.sh, or if you run it with bash your_script_file.sh, it should work fine.
Running it with sh your_script_file.sh will not work because the hashbang line will be ignored and the script will be interpreted by dash, which does not support that string substitution syntax.
I had the same problem. Make sure your script didnt have
#!/bin/sh
at the top of your script. Instead, you should add
#!/bin/bash
For others that arrive here, this exact message will also appear when using the env variable syntax for commands, for example ${which sh} instead of the correct $(which sh)
Your script syntax is valid bash and good.
Possible causes for the failure:
Your bash is not really bash but ksh or some other shell which doesn't understand bash's parameter substitution. Because your script looks fine and works with bash.
Do ls -l /bin/bash and check it's really bash and not sym-linked to some other shell.
If you do have bash on your system, then you may be executing your script the wrong way like: ksh script.sh or sh script.sh (and your default shell is not bash). Since you have proper shebang, if you have bash ./script.sh or bash ./script.sh should be fine.
Try running the script explicitly using bash command rather than just executing it as executable.
Also, make sure you don't have an empty string for the first line of your script.
i.e. make sure #!/bin/bash is the very first line of your script.
Not relevant to your example, but you can also get the Bad substitution error in Bash for any substitution syntax that Bash does not recognize. This could be:
Stray whitespace. E.g. bash -c '${x }'
A typo. E.g. bash -c '${x;-}'
A feature that was added in a later Bash version. E.g. bash -c '${x#Q}' before Bash 4.4.
If you have multiple substitutions in the same expression, Bash may not be very helpful in pinpointing the problematic expression. E.g.:
$ bash -c '"${x } multiline string
$y"'
bash: line 1: ${x } multiline string
$y: bad substitution
Both - bash or dash - work, but the syntax needs to be:
FILENAME=/my/complex/path/name.ext
NEWNAME=${FILENAME%ext}new
I was adding a dollar sign twice in an expression with curly braces in bash:
cp -r $PROJECT_NAME ${$PROJECT_NAME}2
instead of
cp -r $PROJECT_NAME ${PROJECT_NAME}2
I have found that this issue is either caused by the marked answer or you have a line or space before the bash declaration
Looks like "+x" causes problems:
root#raspi1:~# cat > /tmp/btest
#!/bin/bash
jobname="job_201312161447_0003"
jobname_pre=${jobname:0:16}
jobname_post=${jobname:17}
root#raspi1:~# chmod +x /tmp/btest
root#raspi1:~# /tmp/btest
root#raspi1:~# sh -x /tmp/btest
+ jobname=job_201312161447_0003
/tmp/btest: 4: /tmp/btest: Bad substitution
in my case (under ubuntu 18.04), I have mixed $( ${} ) that works fine:
BACKUPED_NB=$(ls ${HOST_BACKUP_DIR}*${CONTAINER_NAME}.backup.sql.gz | wc --lines)
full example here.
I used #!bin/bash as well tried all approaches like no line before or after #!bin/bash.
Then also tried using +x but still didn't work.
Finally i tried running the script ./script.sh it worked fine.
#!/bin/bash
jobname="job_201312161447_0003"
jobname_post=${jobname:17}
root#ip-10-2-250-36:/home/bitnami/python-module/workflow_scripts# sh jaru.sh
jaru.sh: 3: jaru.sh: Bad substitution
root#ip-10-2-250-36:/home/bitnami/python-module/workflow_scripts# ./jaru.sh
root#ip-10-2-250-36:/home/bitnami/python-module/workflow_scripts#
I am trying to execute sed command inside TCL script . Basically i wanted to remove all empty lines from the input file before reading the file using TCL. so i tried following in my script
exec sed -i '/^\s*$/d' .tmp.PG_Ring
set fid [open ".tmp.PG_Ring" r]
But the script is dumping following Error .
sed: -e expression #1, char 1: unknown command: `''
while executing
"exec sed -i '/^\s*$/d' .tmp.PG_Ring"
(file "pg_ring.tcl" line 1)
could you please provide me work around for this & help me with best way to do this?
That won't work, as single quotes have no special meaning to Tcl at all. Tcl uses braces to mean the same sort of thing (except they nest nicely), so instead you can use this:.
exec sed -i {/^\s*$/d} .tmp.PG_Ring
I have to convert filenames having a forward slash to filenames with a back slash in a makefile using mingw32. I used the following sed command to store the result in a variable:
ORIGINAL=./a/b/main1.c ./a/b/main2.c ./a/b/main3.c
sed command:
RESULT=$(shell echo $(ORIGINAL) | sed 's/\//\\/g')
And the resulting output is:
.\a\b\main1.c .abmain2.c .abmain3.c
It works fine if I run it directly on bash. Can anyone tell me whats wrong?
Thanks!
I'm trying to duplicate your test in a makefile, but I don't have your environment.
But I would not use '/' as a command separator if I am searching for the same character. Use another character, like ':'
sed 's:/:\\:g'
RESULT=$(sed 's/\//\\/g' <<< "$ORIGINAL")
$ ORIGINAL='./a/b/main1.c ./a/b/main2.c ./a/b/main3.c'
$ echo "$ORIGINAL"
./a/b/main1.c ./a/b/main2.c ./a/b/main3.c
$ RESULT=$(sed 's/\//\\/g' <<< "$ORIGINAL")
$ echo $RESULT
.\a\b\main1.c .\a\b\main2.c .\a\b\main3.c
Most likely the shell that make is invoking is not bash. However, I find the behavior you're seeing very strange for any shell (it replaces in the first word but not the rest?!?!)
Have you considered using GNU make's $(subst ...) function, instead of the shell?
So, I have looked around for an answer to this, and indeed I have found some, but none seem to work...
I have a folder full of bash scripts that I need to modify. specifically, I need to replace the line
INPUT=/data/scratch02/mpgussert/HAWC-30/${RUN}_reco
with
INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/${RUN}
I have tried this
perl -w -i -p -e "s'INPUT=/data/scratch02/mpgussert/HAWC-30/${RUN}_reco'INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/${RUN}'g" *.sh
which executes without error, but does not find and replace the desired text. From my understanding, using ' to deliminate the regex should search without special character replacement. Is that correct? If so, any ideas why it fails?
I have also tried
perl -w -i -p -e "s/INPUT=\/data\/scratch02\/mpgussert\/HAWC-30\/\$\{RUN\}_reco/INPUT=\/data\/hawc01\/hawcroot\/data\/hawc\/reconstructed\/quesadilla\/\$\{RUN\}/g" *.sh
the backslash should ignore special character replacement, but this returns the following error.
Backslash found where operator expected at -e line 1, near "RUN\"
syntax error at -e line 1, near "RUN\"
Execution of -e aborted due to compilation errors.
So it's searching for RUN\, which is not what I want... Any thoughts? I would appreciate any help you can give.
Thanks!
You want the pattern to be ...\$\{RUN\}..., but that's not what you're passing:
$ echo "...\$\{RUN\}..."
...$\{RUN\}...
You either need do more escaping, or switch to single quotes.
$ echo '...\$\{RUN\}...'
...\$\{RUN\}...
All together:
perl -i -wpe'
s{INPUT=/data/scratch02/mpgussert/HAWC-30/\$\{RUN\}_reco}
{INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/\${RUN}}g
' *.sh
I have a file with below contents:
[TEMP.s_m_update_BUS_spec]
$$SRC_STAT_RA=WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
$InputFile_RA_SPE=/edwload/rqt/workingdir/status_spe/WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
[TEMP.s_m_upd_salions_rqthk]
$$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901094550
$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
I am using below perl command to just replace WHG_STATUS_SITEENTSEQCHAIN_20110901094550 with WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat in the section [TEMP.s_m_upd_salions_rqthk] But somehow its not giving me expected result. Even the WHG_STATUS_SITEENTSEQCHAIN_20110901094550 under section [TEMP.s_m_update_BUS_spec] is getting replaced.
perl -p -i -e "s|\$\$SRC_STAT_RN=.*|\$\$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|g;s|\$InputFile_RN_RQT=\/edwload\/rqt\/workingdir\/restriction\/.*|\$InputFile_RN_RQT=\/edwload\/rqt\/workingdir\/restriction\/WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|g" Input_File
Please let me know the modifications required in command above.Same subsitute commands works fine with SED command. But i wud want to use perl.
The program you run is
s|$$SRC_STAT_RN=.*|$$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|g; s|$InputFile_RN_RQT=\/edwload\/rqt\/workingdir\/restriction\/.*|$InputFile_RN_RQT=\/edwload\/rqt\/workingdir\/restriction\/WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|g
There are a fair number of $ that should be escaped but aren't. It would be simpler if you used single quotes instead of double quotes. You were probably trying for:
perl -i -pe'
s{\$\$SRC_STAT_RN=.*}{\$\$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat}g;
s{\$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/.*}{\$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat}g;
' Input_File
What exactly is not working as you want? On my machine, after running your perl code, the file looks like:
[TEMP.s_m_update_BUS_spec] $$SRC_STAT_RA=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat
[TEMP.s_m_upd_salions_rqthk] $$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat
Ain't this what you expected?
Edit
Try modifying your command to:
perl -p -i -e "s|\$\$SRC_STAT_RN=.*?|\$\$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|gmx;s|\$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/.*?|\$InputFile_RN_RQT=/edwload/rqt/workingdir/restriction/WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat|gmx" Input_File
and see if the result is as expected:
[TEMP.s_m_update_BUS_spec]
$$SRC_STAT_RA=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.datWHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
$InputFile_RA_SPE=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat/edwload/rqt/workingdir/status_spe/WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat
[TEMP.s_m_upd_salions_rqthk]
$$SRC_STAT_RN=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.datWHG_STATUS_SITEENTSEQCHAIN_20110901094550
$InputFile_RN_RQT=WHG_STATUS_SITEENTSEQCHAIN_20110901999999.dat/edwload/rqt/workingdir/restriction/WHG_STATUS_SITEENTSEQCHAIN_20110901094550.dat