Powershell Invoke-Expression mangling command - powershell

I am trying to invoke a command from a powershell script. The command works fine when run from a normal command line. Here is the full command (sorry, it's long, but if I truncate it I'm afraid I might leave out something significant.)
C:\Users\Dave.Work\Desktop\wix36-binaries\candle.exe C:\Users\Dave.Work\Developer\MapCreator\install\win\product.wxs -arch x64 -dPlatform=x64 -dProductVersion=0.9.1.0 -dKarteReleaseBinDir=C:\Users\Dave.Work\Developer\MapCreator\karte-build-release\release -out C:\Users\Dave.Work\Developer\MapCreator\install\win\obj\
If I invoke this exact same command using Invoke-Expression, it fails. It gives an error from the executable (candle.exe), but since it works fine from the command line, the problem is clearly that powershell is mangling the string somehow. Here is my call:
Invoke-Expression 'C:\Users\Dave.Work\Desktop\wix36-binaries\candle.exe C:\Users\Dave.Work\Developer\MapCreator\install\win\product.wxs -arch x64 -dPlatform=x64 -dProductVersion=0.9.1.0 -dKarteReleaseBinDir=C:\Users\Dave.Work\Developer\MapCreator\karte-build-release\release -out C:\Users\Dave.Work\Developer\MapCreator\install\win\obj\'
This results in the following error from candle.exe:
candle.exe : error CNDL0103 : The system cannot find the file '.9.1.0' with type 'Source'.
Somehow the numeric version number is getting mangled? Again, it works fine from a command line.
How to I pass this command to Powershell?
[NOTE] Ultimately, this command is generated from variables, i.e. the actual command will be something like:
$WixDir\candle.exe $ScriptDir\product.wxs -arch $Platform -dPlatform=$Platform -dProductVersion=$ProductVersion -dKarteReleaseBinDir=$KarteReleaseBinDir -out $ScriptDir\obj\
I'm using Invoke-Expression because I was having major problems expanding the variables using, for example, the call operator. But I can't get it to work even without the variable expansion. Thus, if the solution is to escape certain parts of the command string, I would need to also know how to apply those escapes to variables.

Try "-dProductVersion=0.9.1.0" enclosed with " or with '. My Get-Arg utility shows that the original command is parsed so that -dProductVersion=0 and .9.1.0 are different arguments.
Just in case, my Get-Arg.exe is (C#):
using System;
class GetArg
{
static void Main(string[] args)
{
foreach(string s in args)
{
Console.WriteLine(s);
}
}
}
It is useful for checking such cases.
There is even simpler illustration. This code
function Get-Argument {
$args
}
Get-Argument -dProductVersion=0.9.1.0
gets this output:
-dProductVersion=0
.9.1.0
That is PowerShell treats such a command as having two arguments.
Explanation
In PowerShell 2 0 Language Specification we can see for parameters
parameter-char:
Any Unicode character except
{ } ( ) ; , | & . [
colon
whitespace
new-line-character
Note: '=' is included, '.' is excluded. In our case we have
-dProductVersion=0.9.1.0
According to the specification all characters before '.' are valid parameter
characters. So we get, the first result argument is -dProductVersion=0
Then parser chokes at '.'. Technically, it looks like we do something against
the rules including '.' into the argument that starts with '-'. That is why we
should enclose the whole argument with ' or ".
This is probably true for the other excluded characters as well.

Related

Run program from relative path [duplicate]

In GNU/Linux I would do:
PROGPATH=/long/and/complicated/path/to/some/bin
$PROGPATH/program args...
but in Powershell if I try this:
$PROGPATH=\long\and\complicated\path\to\some\bin
$PROGPATH\program args...
I get:
At script.ps1:2 char:...
+ $PROGPATH\program args ...
+ ~~~~~~~~
Unexpected token '\program' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : UnexpectedToken
So how do I do this simple thing I know how to do in bash, in Powershell?
js2010's helpful answer shows the correct solution:
Because your command name/path contains a variable reference ($PROGPATH/...), you must invoke it with &.
The same applies if a grouping expression, (...) is used, or a subexpression, $(...) is involved.
Additionally, the same applies if a command name/path is quoted ('...' or "...")[1], as is required if the path contains spaces, for instance.
To put it differently: Direct invocation is only supported if the command name/path is a verbatim, unquoted string[1]; in all other cases, & must be used.
As for why:
&, the call operator is necessary to force interpretation of a statement as a command, i.e. to have it parsed in argument mode (see below), so as to result in command execution rather than expression evaluation.
PowerShell has two fundamental parsing modes:
argument mode, which works like a traditional shell, where the first token is a command name/path, such as a cmdlet or an external program, with subsequent tokens representing the arguments, which only require quoting if they contain shell metacharacters (chars. with special meaning to PowerShell, such as spaces to separate tokens).
expression mode, which works like expressions in programming languages.
PowerShell decides based on a statement's first token what parsing mode to apply:
If, among other things, the first token starts with a variable reference or is a quoted string, PowerShell parses in expression mode.
In expression mode, \ starts a new token, and unrecognized token \program results in the syntax error you saw.
(If you had used /, it would have been interpreted as the division operator, and program wouldn't be a valid divisor operand.)
[1] Note that if your executable path is a literal string (doesn't contain variable references of expressions) you may alternatively `-escape individual characters (spaces) in lieu of enclosing entire string in '...' or "...", in which case & is then not necessary; e.g.:
C:\Program` Files\Notepad++\notepad++.exe
With a literal string you can even employ partial single- or double-quoting as long as the first token is unquoted; e.g.:
C:\"Program Files"\Notepad++\notepad++.exe
Use the call operator "&". https://ss64.com/ps/call.html
Related: Executing a command stored in a variable from PowerShell
$progpath = 'c:\windows\system32'
& $progpath\notepad somefile.txt
Something with a space:
& 'C:\Program Files\internet explorer\iexplore' yahoo.com
Other options, adding to the path:
$env:path += ';C:\Program Files\internet explorer'
iexplore yahoo.com
And backquoting the spaces:
C:\Program` Files\internet` explorer\iexplore yahoo.com

Launching python.exe in PowerShell not working when using username environment variable [duplicate]

In GNU/Linux I would do:
PROGPATH=/long/and/complicated/path/to/some/bin
$PROGPATH/program args...
but in Powershell if I try this:
$PROGPATH=\long\and\complicated\path\to\some\bin
$PROGPATH\program args...
I get:
At script.ps1:2 char:...
+ $PROGPATH\program args ...
+ ~~~~~~~~
Unexpected token '\program' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : UnexpectedToken
So how do I do this simple thing I know how to do in bash, in Powershell?
js2010's helpful answer shows the correct solution:
Because your command name/path contains a variable reference ($PROGPATH/...), you must invoke it with &.
The same applies if a grouping expression, (...) is used, or a subexpression, $(...) is involved.
Additionally, the same applies if a command name/path is quoted ('...' or "...")[1], as is required if the path contains spaces, for instance.
To put it differently: Direct invocation is only supported if the command name/path is a verbatim, unquoted string[1]; in all other cases, & must be used.
As for why:
&, the call operator is necessary to force interpretation of a statement as a command, i.e. to have it parsed in argument mode (see below), so as to result in command execution rather than expression evaluation.
PowerShell has two fundamental parsing modes:
argument mode, which works like a traditional shell, where the first token is a command name/path, such as a cmdlet or an external program, with subsequent tokens representing the arguments, which only require quoting if they contain shell metacharacters (chars. with special meaning to PowerShell, such as spaces to separate tokens).
expression mode, which works like expressions in programming languages.
PowerShell decides based on a statement's first token what parsing mode to apply:
If, among other things, the first token starts with a variable reference or is a quoted string, PowerShell parses in expression mode.
In expression mode, \ starts a new token, and unrecognized token \program results in the syntax error you saw.
(If you had used /, it would have been interpreted as the division operator, and program wouldn't be a valid divisor operand.)
[1] Note that if your executable path is a literal string (doesn't contain variable references of expressions) you may alternatively `-escape individual characters (spaces) in lieu of enclosing entire string in '...' or "...", in which case & is then not necessary; e.g.:
C:\Program` Files\Notepad++\notepad++.exe
With a literal string you can even employ partial single- or double-quoting as long as the first token is unquoted; e.g.:
C:\"Program Files"\Notepad++\notepad++.exe
Use the call operator "&". https://ss64.com/ps/call.html
Related: Executing a command stored in a variable from PowerShell
$progpath = 'c:\windows\system32'
& $progpath\notepad somefile.txt
Something with a space:
& 'C:\Program Files\internet explorer\iexplore' yahoo.com
Other options, adding to the path:
$env:path += ';C:\Program Files\internet explorer'
iexplore yahoo.com
And backquoting the spaces:
C:\Program` Files\internet` explorer\iexplore yahoo.com

Double dash in powershell seems to cause 'Unexpected token' error [duplicate]

In GNU/Linux I would do:
PROGPATH=/long/and/complicated/path/to/some/bin
$PROGPATH/program args...
but in Powershell if I try this:
$PROGPATH=\long\and\complicated\path\to\some\bin
$PROGPATH\program args...
I get:
At script.ps1:2 char:...
+ $PROGPATH\program args ...
+ ~~~~~~~~
Unexpected token '\program' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : UnexpectedToken
So how do I do this simple thing I know how to do in bash, in Powershell?
js2010's helpful answer shows the correct solution:
Because your command name/path contains a variable reference ($PROGPATH/...), you must invoke it with &.
The same applies if a grouping expression, (...) is used, or a subexpression, $(...) is involved.
Additionally, the same applies if a command name/path is quoted ('...' or "...")[1], as is required if the path contains spaces, for instance.
To put it differently: Direct invocation is only supported if the command name/path is a verbatim, unquoted string[1]; in all other cases, & must be used.
As for why:
&, the call operator is necessary to force interpretation of a statement as a command, i.e. to have it parsed in argument mode (see below), so as to result in command execution rather than expression evaluation.
PowerShell has two fundamental parsing modes:
argument mode, which works like a traditional shell, where the first token is a command name/path, such as a cmdlet or an external program, with subsequent tokens representing the arguments, which only require quoting if they contain shell metacharacters (chars. with special meaning to PowerShell, such as spaces to separate tokens).
expression mode, which works like expressions in programming languages.
PowerShell decides based on a statement's first token what parsing mode to apply:
If, among other things, the first token starts with a variable reference or is a quoted string, PowerShell parses in expression mode.
In expression mode, \ starts a new token, and unrecognized token \program results in the syntax error you saw.
(If you had used /, it would have been interpreted as the division operator, and program wouldn't be a valid divisor operand.)
[1] Note that if your executable path is a literal string (doesn't contain variable references of expressions) you may alternatively `-escape individual characters (spaces) in lieu of enclosing entire string in '...' or "...", in which case & is then not necessary; e.g.:
C:\Program` Files\Notepad++\notepad++.exe
With a literal string you can even employ partial single- or double-quoting as long as the first token is unquoted; e.g.:
C:\"Program Files"\Notepad++\notepad++.exe
Use the call operator "&". https://ss64.com/ps/call.html
Related: Executing a command stored in a variable from PowerShell
$progpath = 'c:\windows\system32'
& $progpath\notepad somefile.txt
Something with a space:
& 'C:\Program Files\internet explorer\iexplore' yahoo.com
Other options, adding to the path:
$env:path += ';C:\Program Files\internet explorer'
iexplore yahoo.com
And backquoting the spaces:
C:\Program` Files\internet` explorer\iexplore yahoo.com

Run command line in PowerShell

I know there are lots of posts regarding this, but nothing worked for me.
I am trying to run this command line in PowerShell:
C:/Program Files (x86)/ClamWin/bin/clamd.exe --install
I have this in PowerShell:
&"C:/Program Files (x86)/ClamWin/bin/clamd.exe --install"
But all this does is execute clamd.exe, ignoring the --install parameter
How can I get the full command line to run?
Josef Z's comment on the question provides the solution:
& "C:/Program Files (x86)/ClamWin/bin/clamd.exe" --install # double-quoted exe path
or, given that the executable path is a literal (contains no variable references or subexpressions), using a verbatim (single-quoted) string ('...'):
& 'C:/Program Files (x86)/ClamWin/bin/clamd.exe' --install # single-quoted exe path
As for why your own solution attempt failed: The call operator, &, expects only a command name/path as an argument, not a full command line.
Invoke-Expression accepts an entire command line, but that complicates things further and can be a security risk.
As for why this is the solution:
The need for quoting stands to reason: you need to tell PowerShell that C:/Program Files (x86)/ClamWin/bin/clamd.exe is a single token (path), despite containing embedded spaces.
&, the so-called call operator, is needed, because PowerShell has two fundamental parsing modes:
argument mode, which works like a traditional shell, where the first token is a command name, with subsequent tokens representing the arguments, which only require quoting if they contain shell metacharacters (chars. with special meaning to PowerShell, such as spaces to separate tokens);
that is why --install need not, but can be quoted (PowerShell will simply remove the quotes for you before passing the argument to the target executable.)
expression mode, which works like expressions in programming languages.
PowerShell decides based on a statement's first token what parsing mode to apply:
If the first token is a quoted string - which we need here due to the embedded spaces in the executable path - or a variable reference (e.g., $var ...), PowerShell parses in expression mode by default.
A quoted string or a variable reference as an expression would simply output the string / variable value.
However, given that we want to execute the executable whose path is stored in a quoted string, we need to force argument mode, which is what the & operator ensures.
Generally, it's important to understand that PowerShell performs nontrivial pre-processing of the command line before the target executable is invoked, so what the command line looks like in PowerShell code is generally not directly what the target executable sees.
If you reference a PowerShell variable on the command line and that variable contains embedded spaces, PowerShell will implicitly enclose the variable's value in double quotes before passing it on - this is discussed in this answer to the linked question.
PowerShell's metacharacters differ from that of cmd.exe and are more numerous (notably, , has special meaning in PowerShell (array constructor), but not cmd.exe - see this answer).
To simplify reuse of existing, cmd.exe-based command lines, PowerShell v3 introduced the special stop-parsing symbol, --%, which turns off PowerShell's normal parsing of the remainder of the command line and only interpolates cmd.exe-style environment-variable references (e.g., %USERNAME%).

Why does robocopy use its own command line parser?

If I execute the following command on a Windows 8.1 machine:
robocopy "C:\Temp\A\" "C:\Temp\B\"
Robocopy fails due to the following problem:
Source : C:\Temp\A" C:\Temp\B"\
Dest -
...
ERROR : No Destination Directory Specified.
It looks like \ is used as some kind of escape character (which is not normal behavior in the windows command line) The final \" is even transformed to "\ which I do not understand at all. Why's that so?
Note: this is not the default behavior of the command line, if they would have used argv[1] and argv[2] within robocopy, they would've retrieved the correct arguments.
Why are they using their own command line parsing? It really confused me for the last hour...
You should omit the trailing backslashes.
From http://ss64.com/nt/robocopy.html :
If either the source or destination are a "quoted long foldername" do
not include a trailing backslash as this will be treated as an escape
character, i.e. "C:\some path\" will fail but "C:\some path\\" or
"C:\some path." or "C:\some path" will work.
robocopy is not an exception. Any executable uses its own line parser to determine the arguments that were sent to it. The OS just uses the API to create the process and pass to it a string to be handled as arguments. The process can handle the string as it wants.
In the case of robocopy, the parser used is the standard Microsoft C startup code. This parser follow the rules described here, and in the full list you can found
A double quotation mark preceded by a backslash, \", is interpreted as
a literal double quotation mark (").