How to escape the contents of a dynamic variable in PowerShell? - powershell

Let's say I have the variable $password = Get-Passwd ACME\bob where I get the password of a given user. The password contains all sorts of special characters including $, that is used by PowerShell.
I need to dynamically use the password in the following command:
cmdkey.exe /add:$hostname /user:"$user" /pass:"`"$password`""
I need to escape the " character on both sides so the command is parsed as pass:"123qwe" and not pass:123qwe. The issue is that it breaks when the password includes a $ character. How can I pass the password to this command without breaking it?

Running Executables in PowerShell
For most executables, it's not necessary to manually quote the parameters in PowerShell. Just write the command using any variables, and the PowerShell parser will automatically quote the arguments for the program's command line. This usually "just works."
cmdkey.exe /add:hostname /user:username /pass:$pass
You can inspect the actual command line that PowerShell passes to an executable by using a small program I wrote called showargs.exe, available at the following page:
https://www.itprotoday.com/management-mobility/running-executables-powershell
Example:
showargs cmdkey.exe /add:hostname /user:username /pass:$pass
The showargs.exe program simply echoes its command line to standard output so you can see the literal command line that PowerShell actually passes to the executable.
An embedded $ character shouldn't be a problem. If that character is in a PowerShell string variable, PowerShell will pass it along as a part of the command line string. Example:
$pass = 'my$pass'
cmdkey.exe /add:hostname /user:username /password:$pass
No matter how you pass the command line to the program, it's important to note that the interpretation of that command line is up to that individual program. (For example, if an executable's parser doesn't support certain characters, no amount of quoting or parsing will allow you to work around the limitation.)
Cmdkey.exe Uses a Non-Standard Parser
In my prefunctory testing of of cmdkey.exe, it seems it does not have a way of "escaping" the " character on its command line. Since this seems to be the case, you will not be able to use cmdkey.exe to store a credential that contains an embedded " character.
Embedding a Space in a Cmdkey.exe Command Line Argument
Because cmdkey.exe uses a non-standard command-line parser, you can't use a variable on its command line that contains embedded spaces. For example:
PS C:\> $pass = "my pass"
PS C:\> showargs cmdkey.exe /add:hostname /user:username /password:$pass
cmdkey.exe /add:hostname /user:username "/password:my pass"
The "/password:my pass" evidently confuses the cmdkey.exe parser, so we have to work around this behavior by bypassing PowerShell's default parsing behavior. The simplest way to do this is by escaping the quotes around the argument containing the space:
PS C:\> showargs.exe cmdkey.exe /add:hostname /user:username /password:`"$pass`"
cmdkey.exe /add:hostname /user:username /password:"my pass"
In any case, you can use showargs.exe to diagnose the trouble and work out a solution appropriate to the executable you need to run.

tl;dr
Your command should work - except if $password contains " chars.
Embedded $ chars., by contrast, should not be a problem.
Your /pass:"`"$password`"" technique - i.e., explicit, embedded double-quoting - also handles values with embedded spaces correctly, unlike the /pass:$password technique (also) suggested in Bill Stewart's helpful answer.
You can simplify the command by omitting the outer "..." quoting, as also suggested in Bill's answer:
/pass:`"$password`"
Caveat: If PowerShell's argument-passing worked correctly, these techniques shouldn't work and if it ever gets fixed, such techniques will stop working - see this answer for background.
As for supporting " chars. in values: even \-escaping them doesn't always work, namely when spaces are also involved - see details below.
The issue is that it breaks when the password includes a $ character.
Your command passes any $ characters embedded in the value of $password through to the target program as-is.
Therefore, if there is a problem, the implication is that your target program - cmdkey.exe - interprets $ characters, but note that the docs make no mention of that.
If it indeed does, however, you would have to escape $ characters as required by the target program in order to pass them literally.
Caveat: There is a fundamental limitation, however:
Commands typically break if the argument value contains spaces AND embedded " chars, whether the latter are properly escaped for the target program or not.
Normally, you'd escape value-internal " as \" so as not to break the enclosing double-quoting of the value:
# !! Only works if:
# !! * $password contains NO "
# !! * $password contains " but NOT ALSO SPACES
# !! * $password contains PAIRS of " and what is between each
# !! pair does not contain spaces
cmdkey.exe /add:$hostname /user:"$user" /pass:`"$($password -replace '"', '\"')`"
Note:
Escaping embedded " as \" is a not a standard per se, but it is recognized by most external programs; the only notable exceptions are batch files - see this answer for details.
Arguably, PowerShell should handle this escaping automatically - see this GitHub issue for details.
If Windows PowerShell thinks it cannot pass the resulting token as-is as a single argument to the target program, it blindly applies double-quoting around the entire token[1], which, in combination with the escaped ", can result in invalid syntax:
E.g., if $password literally contains a " b and is manually escaped as a \" b by the command above, PowerShell ends up passing the following behind the scenes:
... "/pass:"a \" b""
That is, the resulting literal token that PowerShell saw - /pass:"a \" b" - was blindly enclosed in double quotes, as a whole, even though most target programs would parse /pass:"a \" b" correctly as-is.
As a result, the explicitly provided double-quoting is invalidated (as it would then require another level of escaping) - and short of using --%, the stop-parsing symbol, which then limits you to literals (or %...%-style environment-variable references), there is no way around that.
If the target program recognizes an argument if it is double-quoted as a whole (e.g.,
"/pass:a b" instead of /pass:"a b"), you can omit the explicit double-quoting around the argument value.
Do note, however, that some target programs - including cmdkey.exe and notably msiexec - do not recognize the argument if it is double-quoted as a whole.
... /pass:$($password -replace '"', '\"')
With $password literally containing a " b, PowerShell then passes behind the scenes:
... "/pass:a \" b"
This is syntactically valid - for target programs that recognize \" as an escaped ", which is the norm - but, as stated, the fact that the entire argument is enclosed in `"...", and not just the value may not be supported by the target program.
[1] Windows PowerShell ignores any \-escaping in the resulting token and considers all embedded " to have syntactic function: From that perspective, if the token is not composed of any mix of directly concatenated unquoted and double-quoted strings, enclosing double-quoting is blindly applied - which may break the command.
This behavior is not only obscure, it prevents robust, predictable parameter passing.
PowerShell Core now recognizes \" as as escaped; however, quotes that aren't pre-escaped as \" still result in broken quoting; e.g., 'a "b c" d' is passed as "a "b c" d", which target programs parse as 2 arguments, a b and c d (after quote removal).

Related

What is the difference between the escape backtick "`" and backslash "\" in PowerShell?

I know that both are used in PowerShell but for different contexts.
On the internet there is a paucity of information on this topic and the only site that talks about it (without making me understand the concept) is:
https://www.rlmueller.net/PowerShellEscape.htm
I am a beginner with PowerShell, I am recently approaching it.
A use case of the \ escape came up in the answer to this other topic of mine:
PowerShell removes multiple consecutive whitespaces when I pass arguments to a nested Start-Process command
Is there anyone who can explain to me in detail the difference between the escape backtick ` and backslash \ in PowerShell, with examples and use cases?
At least one source is welcome, but it is not mandatory.
vonPryz's helpful answer covers the PowerShell-internal angle well; let me attempt a systematic summary that includes the PowerShell CLI angle, as well as passing arguments with embedded " to external programs:
Inside a PowerShell session, the only escape character is ` (the so-called backtick), in the following contexts:
Inside an expandable string ("...", double quoted), but not inside a verbatim string ('...', single-quoted); for the supported escape sequences, see the conceptual about_Special_Characters help topic:
# " must be escaped; escape sequence `n expands to a newline.
"3`" of`nrain"
In unquoted command arguments:
# > must be escaped to write verbatim 'a>b',
# since outside a quoted string an unescaped > is a redirection.
Write-Output a`>b
For line-continuations:
# Spread the Get-Date call across two lines.
# Important: ` must be the *very last char* on the line.
Get-Date `
-Year 2525
Note: Various subsystems, whether PowerShell-specific or not, may have their own escaping rules, such as \ in regular expressions and ` in wildcard expressions. Since arguments to those subsystems are delivered via PowerShell strings, it's best to use verbatim string literals, so as to avoid confusion between PowerShell's own string interpolation and what the target subsystem ends up seeing; e.g. 'A $50 fine.' -match '\$50' (\ is needed to treat regex metacharacter $ literally).
When PowerShell is called from the outside, via its CLI, different rules apply, possibly in addition:
In order to adhere to widely used convention for CLIs (command-line interfaces, programs that accept arguments via a command line) on Windows:
In calls to powershell.exe, the Windows PowerShell CLI, " characters must be escaped with a backslash - i.e. as \" - in order to be preserved during parsing of the original command line.
pwsh.exe, the CLI of the cross-platform, install-on-demand PowerShell (Core) 7+ edition, now commendably alternatively accepts ""[1] in lieu of \", which makes calls from cmd.exe more robust. To get the same robustness in Windows PowerShell - from cmd.exe only - use "^"" (sic).
Note that - unlike \" - these escape sequences only work inside an (unescaped) "..." string (e.g., pwsh.exe -c " ""you & I"" " or powershell.exe -c " "^""you & I"^"" "
By contrast, unescaped " have syntactic function on the command line and tell PowerShell where the boundaries between arguments are; these " instances are removed during command-line parsing.
This ensures that outside callers that merely want to invoke a PowerShell script file (.ps1) with arguments, using the -File parameter, can use the conventional syntax and needn't special-case calls to PowerShell's CLI.
However, if you pass a string containing PowerShell code to the CLI, using the -Command parameter, what PowerShell ends up interpreting obviously must be syntactically valid PowerShell code.
Caveat: If you specify neither -Command nor -File:
powershell.exe defaults to -Command
pwsh.exe now defaults to -File
For the differences between -File and -Command calls and when to use which, see this answer.
If you use -Command, there are two, sequential parsing stages:
The command-line parsing stage, where syntactic (unescaped) " are removed, and escaped \" (or "") turn into literal ".
The result of this stage is then parsed as PowerShell code, as it would be from inside a PowerShell session.
Therefore, you may situationally have to combine \ and `-escaping; e.g. (call from cmd.exe):
C:>powershell.exe -Command " \"3`\" of snow as of $(Get-Date)\" "
3" of snow as of 11/04/2021 14:13:41
Note the use of `\" in order to make PowerShell see `", i.e. a properly escaped " inside a "..." string, after command-line parsing.
Alternatively, depending on the specifics of the command(s) you pass to -Command, using embedded '...' quoting may be an option, which simplifies matters, because ' chars. don't require escaping:
C:>powershell.exe -Command " '3\" of snow as of today.' "
3" of snow as of today.
Given that '...' strings in PowerShell are verbatim strings, use of '...' is only an option if you don't require string interpolation (such as the $(Get-Date) subexpression in the prior example).
Escaping " when calling external programs from PowerShell:
As a shell, it is PowerShell's job to pass the arguments that were passed based on its syntax rules to a target executable, so that the verbatim values that result from PowerShell's parsing are passed in a way that makes the target executable see them as such. In other words: PowerShell should perform any required escaping automatically, behind the scenes. (Unlike cmd.exe, PowerShell cannot just pass its own argument syntax through as-is, because external CLIs cannot be expected to understand '...' strings (single-quoting) or `-escaping).
To use a simply example: Passing '3" of snow' should be passed as "3\" of snow" behind the scenes, based on the most widely used escaping convention.
Sadly, up to PowerShell 7.2.x, this is not the case, and embedded " characters in arguments for external programs must additionally, manually be \-escaped in order to be passed through properly.
This has been fixed in PowerShell 7.3, with selective exceptions on Windows - see the new $PSNativeCommandArgumentPassing preference variable
# Broken behavior up to PS v7.2.x
PS> cmd /c echo '{ "foo": "bar" }'
"{ "foo": "bar" }" # !! Embedded " aren't escaped.
PS> choice.exe /d Y /t 0 /m '{ "foo": "bar" }'
{ foo: bar } [Y,N]?Y # !! When parsed by an executable,
# !! embedded " are effectively LOST.
# Manual escaping required.
PS> choice.exe /d Y /t 0 /m '{ \"foo\": \"bar\" }'
{ "foo": "bar" } [Y,N]?Y # OK
This bug has existed since v1, and has never been fixed so as to avoid breaking existing workarounds.
[1] Inside PowerShell, in "..." strings only, you may also use "" to escape an embedded ", as an alternative to `"
The backtick ` is used as the escape character in Powershell. That is, escaping quotes, tabs and whatnot. Unlike in many other environments, Powershell's newline is `n, not \n. This has the benefit of simplifying paths, since those use backslash in Windows.
As for a practical example, in many programming languages one needs to escape the backslashes in paths. If you have an application located at
c:\program files\newApplication\myApp.exe
Its path must be written as
c:\\program files\\newApplication\\myApp.exe
The double-backslash notation means that there is actaully a backslash, not a metacharacter such as tab or newline. Note that the path contains \newApplication. Now, \n usually means a newline, but it clearly isn't the case. The file does not reside in
c:\program files\
ewApplication\myApp.exe
after all, doesn't it?
The escaping is not needed in Powershell, as the backslash doesn't have special meaning by itself. When Powershell sees c:\program files\newApplication\myApp.exe, it doesn't assign any special meaning to the forementioned \n part, it is just a string literal.
The backslash \ is used as the escape character in regular expressions and to signify metacharacters. That is, to match a literal * in regex, it must be escaped, lest it means the Kleene star (zero or more matches).

How to escape a string to pass it to another command?

I have an array of arguments $p = "a", "b c", "d`"e" to be passed to command.
I can do this like so:
& my_command $p
Now, I need to call that same command inside some external wrapper (binary), which works like this using fixed parameters:
& my_wrapper /Command=my_command "/CommandParameters=a 'b c' 'd`"e'"
But how can I pass the /CommandParameters from the array $p?
One approach that works for some inputs is to pre-process the array like so:
$p = $p | Foreach {"\`"$_\`""}
& my_wrapper /Command=my_command "/CommandParameters=$p"
but this seems pretty fragile. Values in $p do contain spaces and quotes.
In Bash, I could use printf %q to properly escape the parameters.
The following encloses all strings in the array in embedded "..." and additionally escapes preexisting embedded " chars. as \", which is what external programs typically expect:
Note: In Windows PowerShell and in PowerShell (Core) up to at least v7.1, an additional round of \-escaping is - unfortunately - needed when passing arguments with embedded " to external programs:
$p = "a", "b c", "d`"e"
$pEscaped = $p.ForEach({ '\"{0}\"' -f ($_ -replace '"', '\\\"') })
& my_wrapper /Command=my_command "/CommandParameters=$pEscaped"
Note: While $pEscaped is a collection (array) of strings, using it inside an expandable string ("...") automatically creates a space-separated single-line representation; e.g. "$('foo', 'bar')" yields verbatim foo bar
This longstanding problem - the unexpected need for manual \-escaping of embedded " chars. when passing arguments to external programs - is summarized in this answer.
Preview versions of v7.2 now come with experimental feature PSNativeCommandArgumentPassing, which is an attempted fix, but, unfortunately, it looks like it will lack important accommodations for high-profile CLIs on Windows - see this summary from GitHub issue #15143. However, the fix is effective for executables that expect " to be escaped as \", so the solution simplifies to (using string concatenation (+) inside an expression ((...)) to construct the argument in-line):
# Note: Requires experimental feature PSNativeCommandArgumentPassing
# to be turned on, available in preview versions of v7.2
& my_wrapper /Command=my_command ('/CommandParameters=' + $p.ForEach({ '"{0}"' -f ($_ -replace '"', '\"') }))
Note:
Assuming this experimental feature becomes an official one (which is in general never guaranteed), the corrected behavior will most likely be opt-in, via a preference variable: $PSNativeCommandArgumentPassing = 'Standard' ('Legacy' selects the old behavior).
If you don't mind installing a third-party module (authored by me), the Native module (Install-Module Native) comes with a backward- and forward-compatible helper function, ie, which too obviates the need for the extra escaping, while also containing the important accommodations for high-profile CLIs on Windows missing from the experimental feature:
# Note: Assumes `Install-Module Native` was called.
# Just use `ie` instead of `&`
ie my_wrapper /Command=my_command ('/CommandParameters=' + $p.ForEach({ '"{0}"' -f ($_ -replace '"', '\"') }))
As for what you tried:
[Enclosing the value in embedded "] seems pretty fragile
If you additionally escape any preexisting " as \" (\\\", if the argument-passing bug must be compensated for), as shown above, it works robustly.
Ultimately, the following command line must be executed, which PowerShell constructs behind the scenes:
my_wrapper /Command=my_command "/CommandParameters=\"a\" \"b c\" \"d\\\"e\""
When my_wrapper parses its command line, it ends up seeing the following verbatim string as the last argument:
/CommandParameters="a" "b c" "d\"e"
In Bash, I could use printf %q to properly escape the parameters.
Unfortunately, PowerShell has no equivalent feature, but it's not hard to provide it ad hoc:
Meta-quoting strings in Powershell:
Note:
By meta-quoting I mean the functionality that Bash's printf %q provides: It formats a given string value to become usable as a string literal in source code. For instance (this example illustrates the general principle, not printf %q's actual behavior), verbatim string value a b is transformed to verbatim string value "a b", and the latter can be used as an argument when constructing a command line stored in a string.
The required approach depends on whether the meta-quoted string is to be used in the string representation of a PowerShell command (such as a cmdlet call) or that of a call to an external program, given their different escaping needs. Additionally, while most external programs on Windows (also) understand \" to be an escaped ", some - notably batch files and msiexec.exe - only understand "".
The commands below use the following sample input input string, which contains both a ' and a " (constructed via a verbatim here-string for quoting convenience):
$str = #'
6'1" tall
'#
The solutions below use the -f operator to synthesize the result strings, not just for conceptual clarity, but also to work around string interpolation bugs that can cause subexpressions embedded in expandable strings to yield incorrect results (e.g., "a$('""')b" and "a$('`"')b" both yield verbatim a"b - one " / the ` is missing); an alternative is to use simple string concatenation with +.
Unfortunately, it looks like these bugs will not be fixed, so as to preserve backward-compatibility; see GitHub issue #3039
The verbatim content of the resulting strings is shown in source-code comments, enclosed in «...»; e.g. «"6'1\""» (this representation is just for illustration; PowerShell does not support such delimiters).
Meta-quoting for external programs:
Note:
On Windows, console applications typically only recognize double quotes as string delimiters in their command lines, so that's what the solutions below focus on. To create a single-quoted representation that is understood by POSIX-compatible shells such as bash, use the following:
"'{0}'" -f ($str -replace "'", "'\''"), which yields verbatim '6'\''1" tall' (sic).
In edge cases on Windows, you may have to bypass PowerShell's command-line parsing altogether, so as to fully control the command line that is used for actual process creation behind the scenes, either via --%, the stop-parsing symbol, which has severe limitations or by delegating the invocation to cmd.exe, by passing an entire command line to /c - again, see this answer.
For external programs that require \"-escaping (typical):
# With the experimental, pre-v7.2 PSNativeCommandArgumentPassing
# feature turned on, you can directly pass the result
# as an external-program argument.
# Ditto with the `ie` helper function.
'"{0}"' -f ($str -replace '"', '\"') # -> «"6'1\" tall"»
# With additional \-escaping to compensate for PowerShell's
# argument-passing bug, required up to at least v7.1
'\"{0}\"' -f ($str -replace '"', '\\\"') # -> «\"6'1\\\" tall\"»
For external programs that require ""-escaping (e.g., batch files, msiexec - Windows-only):
# CAVEAT: With the experimental, pre-v7.2 PSNativeCommandArgumentPassing
# feature turned on, passing the result as an external-program argument
# will NOT work as intended, because \" rather than "" is invariably used.
# By contrast, the `ie` helper function automatically
# switches to "" for batch files, msiexec.exe and msdeploy.exe
# and WSH scripts.
'"{0}"' -f ($str -replace '"', '""') # -> «"6'1"" tall"»
# With additional escaping to compensate for PowerShell's
# argument-passing bug, required up to at least v7.1
'""{0}""' -f ($str -replace '"', '""""') # -> «""6'1"""" tall""»
Meta-quoting for PowerShell commands:
Note:
Fortunately, no extra escaping is ever needed in this case; the argument-passing bug discussed above only affects calls to external programs.
Creating a double-quoted representation ("..."):
'"{0}"' -f ($str -replace '"', '`"') # -> «"6'1`" tall"»
Creating a single-quoted representation ('...'):
"'{0}'" -f ($str -replace "'", "''") # -> «'6''1" tall'»

Is there a way to escape quotes in ripgrep for MS Windows (Powershell or CMD)?

I want to find a string "Hello (Hello starts with double quote) in text files using ripgrep.
Normally, in Bash or ZSH, this would work by escaping with backslash or surrounding with single quote:
rg \"Hello
rg '"Hello'
However, in MS Windows (Powershell and CMD), I've tried these but none of these worked:
rg \"Hello
rg '"Hello'
rg `"Hello
rg '`"Hello'
Is there any way to escape single or double quotes using ripgrep in MS Windows?
Verbatim string "Hello must ultimately be passed as \"Hello to rg ("\"Hello" would work too). That is, the verbatim " char. must be \-escaped:
From cmd.exe:
rg \^"Hello
^, cmd.exe's escape character, ensures that the " is treated verbatim and is removed by cmd.exe before calling rg.
Note that ^ isn't strictly necessary here, but it prevents the " from being considered the start of a double-quoted argument, which could make a difference if there were additional arguments.
From PowerShell:
rg \`"Hello
`, PowerShell's escape character, ensures that the " is treated verbatim and is removed by PowerShell before calling rg.
Arguably, the explicit \-escaping shouldn't be necessary, because it is the duty of a shell to properly pass arguments to target executables after the user has satisfied the shell's own escaping requirements (escaping the verbatim " with ^ in cmd.exe, and with ` in PowerShell).
In the context of PowerShell, this problematic behavior is summarized in this answer.
Note that in PowerShell this extra escaping is only needed if you call external programs; it isn't needed PowerShell-internally - such as when you call Select-String, as shown in js2010's answer.
You can use rg -F \"Hello
-F, --fixed-strings
Treat the pattern as a literal string instead of a regular expression. When this flag is used, special regular expression meta characters such as .(){}*+ do not
need to be escaped.
This flag can be disabled with --no-fixed-strings.
If you're in powershell you might as well do:
get-childitem file | select-string '"hello'
file:1:"hello

Run command line in PowerShell

I know there are lots of posts regarding this, but nothing worked for me.
I am trying to run this command line in PowerShell:
C:/Program Files (x86)/ClamWin/bin/clamd.exe --install
I have this in PowerShell:
&"C:/Program Files (x86)/ClamWin/bin/clamd.exe --install"
But all this does is execute clamd.exe, ignoring the --install parameter
How can I get the full command line to run?
Josef Z's comment on the question provides the solution:
& "C:/Program Files (x86)/ClamWin/bin/clamd.exe" --install # double-quoted exe path
or, given that the executable path is a literal (contains no variable references or subexpressions), using a verbatim (single-quoted) string ('...'):
& 'C:/Program Files (x86)/ClamWin/bin/clamd.exe' --install # single-quoted exe path
As for why your own solution attempt failed: The call operator, &, expects only a command name/path as an argument, not a full command line.
Invoke-Expression accepts an entire command line, but that complicates things further and can be a security risk.
As for why this is the solution:
The need for quoting stands to reason: you need to tell PowerShell that C:/Program Files (x86)/ClamWin/bin/clamd.exe is a single token (path), despite containing embedded spaces.
&, the so-called call operator, is needed, because PowerShell has two fundamental parsing modes:
argument mode, which works like a traditional shell, where the first token is a command name, with subsequent tokens representing the arguments, which only require quoting if they contain shell metacharacters (chars. with special meaning to PowerShell, such as spaces to separate tokens);
that is why --install need not, but can be quoted (PowerShell will simply remove the quotes for you before passing the argument to the target executable.)
expression mode, which works like expressions in programming languages.
PowerShell decides based on a statement's first token what parsing mode to apply:
If the first token is a quoted string - which we need here due to the embedded spaces in the executable path - or a variable reference (e.g., $var ...), PowerShell parses in expression mode by default.
A quoted string or a variable reference as an expression would simply output the string / variable value.
However, given that we want to execute the executable whose path is stored in a quoted string, we need to force argument mode, which is what the & operator ensures.
Generally, it's important to understand that PowerShell performs nontrivial pre-processing of the command line before the target executable is invoked, so what the command line looks like in PowerShell code is generally not directly what the target executable sees.
If you reference a PowerShell variable on the command line and that variable contains embedded spaces, PowerShell will implicitly enclose the variable's value in double quotes before passing it on - this is discussed in this answer to the linked question.
PowerShell's metacharacters differ from that of cmd.exe and are more numerous (notably, , has special meaning in PowerShell (array constructor), but not cmd.exe - see this answer).
To simplify reuse of existing, cmd.exe-based command lines, PowerShell v3 introduced the special stop-parsing symbol, --%, which turns off PowerShell's normal parsing of the remainder of the command line and only interpolates cmd.exe-style environment-variable references (e.g., %USERNAME%).

Powershell: passing json string to curl

I'm trying to pass a JSON string from within a powershell script to the build.phonegap.com api, using curl.
According to phonegap's forum, when running on a Windows machine, the JSON data has to be formatted as:
curl.exe -ku user#email:mypass -X PUT -d "data={\"password\":\"keypass\"}" https://build.phonegap.com/api/v1/key
Indeed, this does run fine when invoked from the command line.
However, when I try to invoke this from within a powershell script, the double quotes seem to be stripped.
So far, I have tried:
Putting the JSON in single quoted string:
curl.exe -ku user#email:mypass -X PUT -d '"data={\"password\":\"keypass\"}"' https://build.phonegap.com/api/v1/key
Putting the JSON in single quoted string, without the DOS escape backslashes:
curl.exe -ku user#email:mypass -X PUT -d '"data={"password":"keypass"}"' https://build.phonegap.com/api/v1/key
Putting the JSON in single quoted string, escaping the double quotes and backslashes (DOS style with a backslash):
curl.exe -ku user#email:mypass -X PUT -d '\"data={\\\"password\\\":\\\"keypass\\\"}\"' https://build.phonegap.com/api/v1/key
Putting the JSON in a double quoted string, escaping the double quotes with the powershell backtick character (`):
curl.exe -ku user#email:mypass -X PUT -d "`"data={\`"password\`":\`"build*2014`\`"}`"" https://build.phonegap.com/api/v1/key
Any idea how to achieve this?
Thanks for your time,
Koen
Try using the --% operator to put PowerShell into simple (dumb) argument parsing mode:
curl.exe --% -ku user#email:mypass -X PUT -d "data={\"password\":\"keypass\"}" https://build.phonegap.com/api/v1/key
This is quite often useful for invoking exes with argument syntax that runs afoul of PowerShell's argument syntax. This does require PowerShell V3 or higher.
Update:
PowerShell 7.3.0 mostly fixed the problem, with selective exceptions on Windows, though a future version may requires opt-in - see this answer for details.
For cross-version, cross-edition code, the Native module discussed below may still be of interest.
tl;dr:
Up to at least PowerShell 7.2.x, manual \-escaping of " characters inside PowerShell strings is - unfortunately - required.
# From inside PowerShell:
# Note the outer '...' quoting and the unexpected need to escape
# the embedded " chars. as \" (unexpected, because PowerShell itself doesn't
# require " inside '...' to be escaped; also, PowerShell's escape char. is `).
# If outer "..." quoting must be used, use \`" (sic) to escape the embeded "
curl.exe -ku user#email:mypass -X PUT -d 'data={\"password\":\"keypass\"}' https://build.phonegap.com/api/v1/key
Read on for why that is necessary, and about a potential future fix.
PowerShell's escape character is ` (the so-called backtick), so in order to embed " characters in a "..." (double-quoted, interpolating) string, use `" (or "") rather than \"; by contrast, inside a '...' (single-quoted, verbatim) string, " need not be escaped.
In your attempt, PowerShell didn't see \" as an escaped " and therefore saw multiple "..." strings, which ultimately - when PowerShell of necessity applied its on demand re-quoting behind the scenes, passed two separate string arguments that individually didn't need double-quoting, due to not containing spaces, namely: verbatim data={\ and password\:\keypass\}
Using PowerShell's quoting rules, you should have used:
either:
"data={`"password`":`"keypass`"}"
or, more simply, given that no string interpolation is needed, via a verbatim, single-quoted string, inside of which " chars. don't require escaping:
'data={"password":"keypass"}'
Unfortunately, however, as of PowerShell 7.2.x this is NOT enough, though the experimental PSNativeCommandArgumentPassing feature available since PowerShell Core 7.2.0-preview.5 may fix this at least for some external programs, including curl.exe; read on for details.
As of PowerShell 7.2.x, an unexpected extra layer of escaping of embedded " characters is needed, using \-escaping when calling (most) external programs:
In Windows PowerShell there are edge cases where this approach doesn't work, in which case use of --% is required (see below). Notably, escaping '"foo bar"' as '\"foo bar\"' doesn't work, due to the enclosing \" being at the very start and end of the string - see this answer for details.
Also, some external programs on Windows understand ""-escaping only (e.g. msiexec); for them, use -replace '"', '""' in order to programmatically perform the extra escaping, assuming the value contains at least one space. Do the same for programs that do not support embedded " chars. at all (WSH), so that the embedded " at least do not break argument boundaries (but they will be stripped).
For programs that expect \"-escaping, use the following -replace operation to robustly perform the extra escaping programmatically:
'...' -replace '([\\]*)"', '$1$1\"'
If the input string contains no preexisting verbatim \" sequences, you can simplify to '...' -replace '"', '\"'
# Note: Escaping the embedded " chars. as `" is enough for PowerShell itself,
# but, unfortunately, not when calling *external programs*.
# The `-replace` operation performs the necessary additional \-escaping.
$passwd = 'foo'
curl.exe -ku user#email:mypass -X PUT -d (
"data={`"password`": `"$passwd`"}" -replace '([\\]*)"', '$1$1\"'
) https://build.phonegap.com/api/v1/key
This shouldn't be required, but is due to a bug since v1 that hasn't been fixed for fear of breaking backward compatibility - see this answer.
A - presumably - opt-in fix is now being considered for some future version, post v7.2 - see GitHub issue #14747 - and using it, once available, would obviate the need for the manual escaping.
Since PowerShell Core 7.2.0-preview.5, experimental feature PSNativeCommandArgumentPassing with an attempted fix is available, but, unfortunately, it looks like it will lack important accommodations for high-profile CLIs on Windows (though curl.exe wouldn't be affected) - see this summary from GitHub issue #15143.
A backward- and forward-compatible helper function is the ie function from the Native module (Install-Module Native), which obviates the need for the extra escaping, contains important accommodations for high-profile CLIs on Windows, and will continue to work as expected even with the opt-in fix in place:
ie curl.exe ... -d "data={`"password`": `"$passwd`"}" ... )
Using --%, the stop-parsing symbol, as in Keith Hill's answer is a suboptimal workaround that also doesn't require the extra \-escaping, however:
--% has inherent limitations - see GitHub docs issue #6149 - and is virtually useless on Unix-like platforms - see GitHub docs issue #4963.
The only - awkward and side effect-producing - way to embed PowerShell variable values in the arguments following --% is to (a) define them as environment variables (e.g., $env:passwd = 'foo') and (b) to reference these variables cmd.exe-style, even on Unix (e.g., %passwd%).
An alternative workaround - especially if you need to include the values of PowerShell variables or expressions in your calls - is to call via cmd /c with a single argument containing the entire command line; for quoting convenience, the following example uses a here-string (see the bottom section of this answer for an overview of PowerShell's string literals):
# Use #"<newline>...<newline>"# if you need to embed PowerShell variables / expressions.
cmd /c #'
curl.exe -ku user#email:mypass -X PUT -d "data={\"password\":"\keypass\"}" https://build.phonegap.com/api/v1/key
'#
Here's how I did manage to send a json request in PowerShell 7.2
Finally found the solution. Istead of using one " use 3 of them ("""), and thats it. So it would be:
data={"""password""":"""keypass"""}
Set the content type:
curl -H "Content-Type: application/json" -d '{"password":"keypass"}' https://build.phonegap.com/api/v1/key