Environment variables in Compose using Powershell with only 1 line [duplicate] - powershell

This question already has answers here:
PowerShell: Setting an environment variable for a single command only
(10 answers)
Closed 4 years ago.
I know that you can pass environment variables to docker-compose.
docker-compose.yml
. . .
mysql:
image: mariadb:10.2
ports:
- "${DB_PORT}:3306"
. . .
$ DB_PORT=3396 docker-compose up
However this only works using bash. I am using PowerShell and am trying to find an equivalent that is only a one line command.
PS> $env:DB_PORT:3306 docker-compose up does not work. Neither does
multiline
$env:DB_PORT=3396 `
>> docker-compose -up
The error I get is
Unexpected token 'docker-compose' in expression or statement.
If I do it one at a time it does work...
PS> $env:DB_PORT=3396
PS> docker-compose -up
Is there not way to do this in PowerShell when the equivalent in bash is ridiculously simple?

POSIX-like shells such as bash offer a way to set environment variables in a command-scoped way, simply by prepending <varName>=<value> pairs directly to a command, as the following example demonstrates:
$ foo=bar bash -c 'echo "[$foo]"'; echo "[$foo]"
[bar]
[]
foo=bar defines environment variable foo for the bash -c '...' child process only; the next command - echo ... - does not see this variable.
PowerShell has NO equivalent construct.
The best you can do is to define the environment variable of interest first, in a separate statement, using ;, PowerShell's statement separator. Any external utility you invoke thereafter - which invariably runs in a child process - will see it, but note that the environment variable will remain in effect in the current PowerShell session, unless you manually remove it:
# Set the env. variable, call the command that should see it,
# remove it afterwards.
PS> $env:foo = 'bar'; bash -c 'echo "[$foo]"'; $env:foo = $null
[bar]
Note how $env:foo = $null i.e., setting the environment variable to $null is the same as removing it; alternatively, you could all Remove-Item env:foo
If you also want to restore a pre-existing value afterwards:
$env:foo = 'original'
# Temporarily change $env:foo to a different value, invoke the
# program that should see it, then restore the previous value.
& { $org, $env:foo = $env:foo, 'bar'; bash -c 'echo "[$foo]"'; $env:foo = $org }
$env:foo
The above yields:
[bar]
original
showing that while the bash process saw the temporary value, bar, the original value of $env:foo was restored afterwards.
Also note another important difference:
In POSIX-like shells, environment variables are implicitly surfaced as shell variables - they share the one and only namespace the shell has for variables.
By contrast, PowerShell surfaces environment variables only via the $env:<varName> namespace (e.g., $env:foo), which is distinct from the (prefix-less) namespace for PowerShell's own variables (e.g., $foo).

Related

How to run command with preceding variable [duplicate]

This question already has answers here:
PowerShell: Setting an environment variable for a single command only
(10 answers)
Closed 4 years ago.
I know that you can pass environment variables to docker-compose.
docker-compose.yml
. . .
mysql:
image: mariadb:10.2
ports:
- "${DB_PORT}:3306"
. . .
$ DB_PORT=3396 docker-compose up
However this only works using bash. I am using PowerShell and am trying to find an equivalent that is only a one line command.
PS> $env:DB_PORT:3306 docker-compose up does not work. Neither does
multiline
$env:DB_PORT=3396 `
>> docker-compose -up
The error I get is
Unexpected token 'docker-compose' in expression or statement.
If I do it one at a time it does work...
PS> $env:DB_PORT=3396
PS> docker-compose -up
Is there not way to do this in PowerShell when the equivalent in bash is ridiculously simple?
POSIX-like shells such as bash offer a way to set environment variables in a command-scoped way, simply by prepending <varName>=<value> pairs directly to a command, as the following example demonstrates:
$ foo=bar bash -c 'echo "[$foo]"'; echo "[$foo]"
[bar]
[]
foo=bar defines environment variable foo for the bash -c '...' child process only; the next command - echo ... - does not see this variable.
PowerShell has NO equivalent construct.
The best you can do is to define the environment variable of interest first, in a separate statement, using ;, PowerShell's statement separator. Any external utility you invoke thereafter - which invariably runs in a child process - will see it, but note that the environment variable will remain in effect in the current PowerShell session, unless you manually remove it:
# Set the env. variable, call the command that should see it,
# remove it afterwards.
PS> $env:foo = 'bar'; bash -c 'echo "[$foo]"'; $env:foo = $null
[bar]
Note how $env:foo = $null i.e., setting the environment variable to $null is the same as removing it; alternatively, you could all Remove-Item env:foo
If you also want to restore a pre-existing value afterwards:
$env:foo = 'original'
# Temporarily change $env:foo to a different value, invoke the
# program that should see it, then restore the previous value.
& { $org, $env:foo = $env:foo, 'bar'; bash -c 'echo "[$foo]"'; $env:foo = $org }
$env:foo
The above yields:
[bar]
original
showing that while the bash process saw the temporary value, bar, the original value of $env:foo was restored afterwards.
Also note another important difference:
In POSIX-like shells, environment variables are implicitly surfaced as shell variables - they share the one and only namespace the shell has for variables.
By contrast, PowerShell surfaces environment variables only via the $env:<varName> namespace (e.g., $env:foo), which is distinct from the (prefix-less) namespace for PowerShell's own variables (e.g., $foo).

Powershell Variable replacement not working from command line

I have the following command I want to run from PowerShell:
docker run -v C:\src\docker_certs:/root/.dotnet/https -it MyContainer:MyTag /bin/bash
When I run that it works perfectly. (It mounts a volume using the source folder at the destination folder.)
But when I run this:
docker run -v $env:DOCKER_CERTS_PATH:/root/.dotnet/https -it MyContainer:MyTag /bin/bash
The volume does not get mounted.
I run this to check the value:
echo $env:DOCKER_CERTS_PATH
And it returns:
C:\src\docker_certs
As I understood things, it should have replaced the value of $env:DOCKER_CERTS_PATH with C:\src\docker_certs in the second command.
How can I get the PowerShell reference to an environment variable to replace when I run a command?
Enclose the environment-variable reference in {...}:
docker run -v ${env:DOCKER_CERTS_PATH}:/root/.dotnet/https ...
Alternatively, `-escape the : char. following the env.-var. reference:
docker run -v $env:DOCKER_CERTS_PATH`:/root/.dotnet/https ...
As for what you tried:
docker run -v $env:DOCKER_CERTS_PATH:/root/.dotnet/https ...
If you don't use {...} to explicitly delineate a variable name, PowerShell may have a different idea of where the variable name ends than you do.
As an alternative to using {...}, you can `-escape the first character you don't want to be considered part of the variable name.
Note that your command argument is in this case implicitly treated as if it were enclosed in "...", so the above applies to expandable strings ("...") too.
For an comprehensive discussion of how unquoted tokens are parsed as command arguments, see this answer.
In the case at hand, the : that follows $env:DOCKER_CERTS_PATH is not considered the end of the variable reference; instead, it is considered part of the variable name, so that PowerShell looks for an environment variable (env:) literally named DOCKER_CERTS_PATH: (sic).
Since no such environment variable (presumably) exists, $env:DOCKER_CERTS_PATH: expands to the empty string and all that is passed to docker is /root/.dotnet/https.
You can verify that DOCKER_CERTS_PATH: is a valid environment variable name as follows:
PS> $env:DOCKER_CERTS_PATH: = 'hi'; $env:DOCKER_CERTS_PATH:
hi
By contrast, a regular (shell) variable is not permitted to contain :, because that : - in the absence of a namespace prefix such as env: - is itself considered a namespace prefix, which fails, because then the variable-name part is missing:
PS> $DOCKER_CERTS_PATH: = 'hi' # BREAKS, even with {...}
Variable reference is not valid. ':' was not followed by a valid variable name character. Consider using ${} to delimit the name.
The first : in a variable identifier is invariably considered the end of the namespace identifier, which must refer to an existing PowerShell drive name, as reported by Get-PSDrive.
This notation is called namespace variable notation, as explained in this answer.

Running executable with command line parameters in PowerShell

This has to be possible. I am able to open a command prompt in windows and do the following:
<some exe> <some exe command line parameters>
There must be an equivalent way to do this in PowerShell or even a standard windows batch file. For example, from the windows command prompt I can start a docker container with:
docker run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
however if I try something like this with PowerShell
& "docker" run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
it fails with an generic error:
invalid reference format.
Perhaps PowerShell is not suited for this type of advanced use case. Any suggestions would be appreciated!
Is there a better scripting language for advanced usages like this?
I think Start-Process cmdlet will be useful. ArgumentList can be single or double quoted.
Start-Process docker -ArgumentList "run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0"
By and large, external programs in PowerShell are called the same way as from cmd.exe - there are differences, due to PowerShell having additional metacharacters such as $ and #, but they do not come into play in your specific case.
(Your & "docker" ... variant would work in principle too, but the use of & is only necessary if you must use a quoted or variable-based command name or path).
The problem is that your original command line contains two instances of – (EN DASH, U+2013) instead of the expected ASCII-range - dash (hyphen), which docker doesn't recognize.
A quick way to discover the problem:
# Print the code points of characters outside the ASCII range.
PS> [int[]] [char[]] '& "docker" run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0' -gt 127
8211
8211
Decimal 8211 is hex. 0x2013, the code point of en-dash, whereas the code point of the regular - is 45 (0x2d).
All that is needed is to replace these – instances with - (and, since docker needn't be quoted, there is no need for &):
docker run --net=kafka -d --name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
Your own answer shows a variable-based implementation of the command that is effectively the same as the command above - if all the arguments are known in advance, there is never a need to use variables.
If you do want to use variables, it is much simpler to use a single array variable for all the arguments and pass that:
$dockerExe = 'docker'
$dockerArgs = 'run',
'--net=kafka',
'-d',
'--name=zookeeper',
'-e',
'ZOOKEEPER_CLIENT_PORT=2181',
'confluentinc/cp-zookeeper:4.1.0'
& $dockerExe $dockerArgs
Note:
The executable name/path must always be specified separately, and if it is quoted or involves variable references (as in this case), &, the call operator must be used for invocation, for syntactic reasons.
Passing the arguments as an array this way works with external programs; for PowerShell commands, you'd create a hashtable variable that you pass with sigil # instead of $, a feature known as splatting.
There is a lot of complexities in powershell escaping. I wrote this module to assist with running of external commands:
https://github.com/choovick/ps-invoke-externalcommand
Demo:
https://terminalizer.com/view/49acb54a979
Install-Module -Name ExternalCommand -Scope CurrentUser
Invoke-ExternalCommand -Command "docker" -Arguments #("run","-d","--name=zookeeper","--net=kafka","-e","ZOOKEEPER_CLIENT_PORT=2181", "confluentinc/cp-zookeeper:4.1.0")
Here's how to do it.
$app = 'docker'
$a1 = 'run'
$a2 = '--net=kafka'
$a3 = '-d'
$a4 = '--name=zookeeper'
$a5 = '-e'
$a6 = 'ZOOKEEPER_CLIENT_PORT=2181'
$a7 = 'confluentinc/cp-zookeeper:4.1.0'
& $app $a1 $a2 $a3 $a4 $a5 $a6 $a7

Passing optional parameters to rundeck script

I have a python script that I would like to run using rundeck that is invoked as follows:
createInstance.py [-n <name>] <env> <version>
Where name is optional and env and version are required.
e.g. if I want to call the script with a name I would call:
createInstance.py -n test staging 1.2.3.4
If I want to default the name, I would call:
createInstance.py staging 1.2.3.4
The problem i have is that I dont know how to specify the script arguments string in rundeck. I have a job, with 3 options, one for env, version and name and if I define the arguments string as:
-n ${option.name} ${option.env} ${option.version}
Whenever the name is unset, rundeck calls:
createInstance.py -n staging 1.2.3.4
Instead I would like it to omit the -n. Is there any way of doing this? Right now my only option is to change the script to be more forgiving in how it handles the -n, and to always ensure its at the end, e.g.:
createInstance.py staging 1.2.3.4 -n
createInstance.py staging 1.2.3.4 -n test
I would like to avoid making this change though, as I want to be able to use the scripts standalone as well.
Rather than use a command step, try an inline script step. Your inline script can count the number of arguments and if they are set. Then with that logic you can choose how to set the creteInstance.py args.
As #Alex-SF suggests, I've also used an inline script for this, along with a Key Value Data log filter. The script is:
#!/bin/bash
# Parse optional parameters
# https://stackoverflow.com/questions/41233996/passing-optional-parameters-to-rundeck-script
# Arguments to this script should be in the format "flag" "value", eg "-p" ${option.name}
# If value is not missing then return will be "flag value", otherwise blank
echo -n "RUNDECK:DATA:"
while (( "$#" )); do
flag="$1"
value="$2"
if [[ -z "$value" ]] || [[ $value =~ ^\- ]]; then
# no value for this parameter (empty or picking up the next flag)
echo -n ""
shift
else
# value provided for this parameter
echo -n "$flag $value "
shift
shift
fi
done
And the key value data filter uses the pattern ^RUNDECK:DATA:(.*)$ and the name data args. Then I use ${data.args*} as the input for the real command.
It's all rather messy, and I can't find any open issue requesting this as a feature (yet).
Use an inline script and use conditional variable expansion feature from bash.
createInstance.py ${RD_OPTION_NAME:+-n $RD_OPTION_NAME} $RD_OPTION_ENV $RD_OPTION_VERSION
This will omit the first option altogether if it is empty ("").

How to set environment variables for a shell command

I often see this command in node.js programs: NODE_ENV=test node app.js which sets the NODE_ENV variable to test and works. I also read here https://en.wikipedia.org/wiki/Environment_variable that this should work for any shell command, but running some tests on my own, here is what I see
$ HELLO="WORLD"
$ HELLO="MARS" echo "$HELLO"
WORLD
$
I would expect this to print MARS. Is there something I am missing here?
The syntax VAR=value command means that the command will be invoked with the environment variable VAR set to VALUE, and this will apply only for the scope of that command.
However, when you are using the command line:
HELLO="MARS" echo "$HELLO"
The shell first interprets the "$HELLO" parameter, determines that it is WORLD, and then what it actually does is run:
HELLO="MARS" echo "WORLD"
So the echo may have the HELLO variable set, but it doesn't affect what it prints - it has already been interpreted before.
Doing
HELLO="MARS"; echo "$HELLO"
does something else entirely. First it sets HELLO to MARS in the current shell, and then it goes on to interpret the echo command. By this time HELLO contains MARS, not WORLD. But this is an entirely different effect - the variable HELLO stays with the value MARS, which is not the case in the command without the ;.
Your problem is that echo is just a poor choice for a demonstartion of this. You can do other demonstrations to prove that HELLO is changed properly:
HELLO="MARS" eval 'echo $HELLO'
In this case, the shell will not interpret the $HELLO because it is within a string in single quotes. It will first put MARS in HELLO, and then call the eval 'echo $HELLO' with that variable set. The eval command with then run echo $HELLO, and you'll get the output you were expecting.
This syntax is best used for things that don't use the given variable as part of the command line, but rather use it internally.
Other answers are correct, but here a refinement :
There are 2 cases in fact when defining a list of variable separated by spaces in bash whether it ends or not with a command.
VAR1=value1 VAR2=value2 ... VARn=valuen command arg1 arg2 ... argn
and
VAR1=value1 VAR2=value2 ... VARn=valuen
don't export VAR1 ... VARn the same way.
In first case VAR1 ... VARn will be set only for command and will then not be exported to current shell.
In second case VAR1 ... VARn will alter current shell.
then ( remark that ';' is very same of using a new line )
HELLO=WORLD
HELLO=MARS echo "i don't export HELLO."
echo "HELLO=$HELLO"
will display
i don't export HELLO.
HELLO=WORLD
and
HELLO=WORLD
HELLO=MARS ; echo "i did export HELLO."
echo "HELLO=$HELLO"
will display
i did export HELLO.
HELLO=MARS