Please could anyone explain me why the following happens:
"Fancy string - Hor""ray"
# outputs correctly (only one double quote): Fancy string - Hor"ray
'Hor"ray'.Replace('"', '""')
# outputs correctly (two double quotes): Hor""ray
"Fancy string - $('Hor"ray'.Replace('"', '"'+'"'))"
#outputs correctly (two double quotes): Hor""ray
"Fancy string - $('Hor"ray'.Replace('"', '""'))"
# outputs INCORRECTLY (only one double quote): Fancy string - Hor"ray
In my opinion, developers would intuitively expect, that within "$(inline expressions)" Powershell would treat text as statements and won't interfere with the last argument of Replace('"', '""') converting it into '"' (unless the statement interpreter decides to do so).
Do I miss something here?
This appears to be a bug in how PowerShell parses expandable string literals.
From ยง2.3.5.2 on string literals in the PowerShell 3.0 Language Specification, the expandable-string-literal explicitly rejects the $( sequence so that sub-expression parsing will occur instead.
So it seems reasonable to expect $('""') to parse consistently, whether or not it happens to be embedded in a string literal. And clearly sub-expressions are parsed separately, since they support values that would be illegal on their own in an expandable string (e.g. you can write "$('"')" or "$('`""')", where " '"' " or " '`""' " would fail).
However, comparing the AST from [Management.Automation.Language.Parser]::ParseInput for both $('""') and "$('""')", we get two different results. Both have a final StringConstantExpressionAst element with an Extent of '""', but the Value for the stand-alone sub-expression is "" while the Value for the embedded sub-expression is ".
Its because the inline expression is evaluated and its string value is then placed in the string and evaluated.
#Breaking "Fancy string - $('Hor"ray'.Replace('"', '""'))" down
#This inline expression is evaluated first
$('Hor"ray'.Replace('"', '""'))
#giving
Hor""ray
#That value is then interpreted as part of the string
"Fancy string - Hor""ray"
#giving
Fancy string - Hor"ray
This is exactly what I would expect to see. The inline expression evaluated and its resulting value then being used.
Could this not be done by simply using things like below:
`'$($RemFiles[$i].FullName)`'
`"$($RemFiles[$i].FullName)`"
Use the backtick and either a single or double quote to then prevent powershell from using this as an open comment, thus putting the actual symbol in...
The above outputs:
'F:\portable\adobe'
or
"F:\portable\adobe"
I noticed that it seems you are telling it to literally add two double quotes rather than just using the backtick to force it. Therefore telling it to add nothing surely :S So could you change this to something like this:
"Fancy string - $('Hor"ray'.Replace('"', '`"`"'))"
Though that may make 3, as you have one present in Hor"ray anyway.
Just got to test it, was busy with something:
PS D:\> "Fancy string - $('Hor"ray'.Replace('"', '`"`"'))"
Fancy string - Hor""ray
Scroll down on the site below to find out about the backtick and how/where it can be used.
http://www.neolisk.com/techblog/powershell-specialcharactersandtokens
Related
I see this problem in several area, but here is an example
I read an xml document like this and print out a value
[xml]$pom = get-content -path pom.xml
PS C:\> $pom.project.artifactId
nexus-peter-test-service
However, if I put the value in double quotes, I get this
"$pom.project.artifactId"
System.Xml.XmlDocument.project.artifactId
I need the value in double quotes because it's part of a long string. In my case, a url. So I'm using it like this:
"/$pom.project.artifactId/"
Why does Powershell change the meaning of the variable when in it's double quotes? And how can I fix this?
The problem is that the interpolation stops at the period. It interpolates "$pom" - which stringifies as the class name - followed by the literal string ".project.artifactId".
To interpolate anything more complex than a simple variable name, you need to wrap $(...) around the expression:
"$($pom.project.artifactId)"
Note: A summary of this question has since been posted at the PowerShell GitHub repository, since superseded by this more comprehensive issue.
Arguments passed to a command in PowerShell are parsed in argument mode (as opposed to expression mode - see Get-Help about_Parsing).
Conveniently, (double-)quoting arguments that do not contain whitespace or metacharacters is usually optional, even when these arguments involve variable references (e.g. $HOME\sub) or subexpressions (e.g., version=$($PsVersionTable.PsVersion).
For the most part, such unquoted arguments are treated as if they were double-quoted strings, and the usual string-interpolation rules apply (except that metacharacters such as , need escaping).
I've tried to summarize the parsing rules for unquoted tokens in argument mode in this answer, but there are curious edge cases:
Specifically (as of Windows PowerShell v5.1), why is the unquoted argument token in each of the following commands NOT recognized as a single, expandable string, and results in 2 arguments getting passed (with the variable reference / subexpression retaining its type)?
$(...) at the start of a token:
Write-Output $(Get-Date)/today # -> 2 arguments: [datetime] obj. and string '/today'
Note that the following work as expected:
Write-Output $HOME/sub - simple var. reference at the start
Write-Output today/$(Get-Date) - subexpression not at the start
.$ at the start of a token:
Write-Output .$HOME # -> 2 arguments: string '.' and value of $HOME
Note that the following work as expected:
Write-Output /$HOME - different initial char. preceding $
Write-Output .-$HOME - initial . not directly followed by $
Write-Output a.$HOME - . is not the initial char.
As an aside: As of PowerShell Core v6.0.0-alpha.15, a = following a simple var. reference at the start of a token also seems to break the token into 2 arguments, which does not happen in Windows PowerShell v5.1; e.g., Write-Output $HOME=dir.
Note:
I'm primarily looking for a design rationale for the described behavior, or, as the case may be, confirmation that it is a bug. If it's not a bug, I want something to help me conceptualize the behavior, so I can remember it and avoid its pitfalls.
All these edge cases can be avoided with explicit double-quoting, which, given the non-obvious behavior above, may be the safest choice to use routinely.
Optional reading: The state of the documentation and design musings
As of this writing, the v5.1 Get-Help about_Parsing page:
incompletely describes the rules
uses terms that aren't neither defined in the topic nor generally in common use in the world of PowerShell ("expandable string", "value expression" - though one can guess their meaning)
From the linked page (emphasis added):
In argument mode, each value is treated as an expandable string unless it begins with one of the following special characters: dollar sign ($), at sign (#), single quotation mark ('), double quotation mark ("), or an opening parenthesis (().
If preceded by one of these characters, the value is treated as a value expression.
As an aside: A token that starts with " is, of course, by definition, also an expandable string (interpolating string).
Curiously, the conceptual help topic about quoting, Get-Help about_Quoting_Rules, manages to avoid both the terms "expand" and "interpolate".
Note how the passage does not state what happens when (non-meta)characters directly follow a token that starts with these special characters, notably $.
However, the page contains an example that shows that a token that starts with a variable reference is interpreted as an expandable string too:
With $a containing 4, Write-Output $a/H evaluates to (single string argument) 4/H.
Note that the passage does imply that variable references / subexpressions in the interior of an unquoted token (that doesn't start with a special char.) are expanded as if inside a double-quoted string ("treated as an expandable string").
If these work:
$a = 4
Write-Output $a/H # -> '4/H'
Write-Output H/$a # -> 'H/4'
Write-Output H/$(2 + 2) # -> 'H/4'
why shouldn't Write-Output $(2 + 2)/H expand to '4/H' too (instead of being treated as 2 arguments?
Why is a subexpression at the start treated differently than a variable reference?
Such subtle distinctions are hard to remember, especially in the absence of a justification.
A rule that would make more sense to me is to unconditionally treat a token that starts with $ and has additional characters following the variable reference / subexpression as an expandable string as well.
(By contrast, it makes sense for a standalone variable reference / subexpression to retain its type, as it does now.)
Note that the case of a token that starts with .$ getting split into 2 arguments is not covered in the help topic at all.
Even more optional reading: following a token that starts with one of the other special characters with additional characters.
Among the other special token-starting characters, the following unconditionally treat any characters that follow the end of the construct as a separate argument (which makes sense):
( ' "
Write-Output (2 + 2)/H # -> 2 arguments: 4 and '/H'
Write-Output "2 + $a"/H # -> 2 arguments: '2 + 4' and '/H', assuming $a equals 4
Write-Output '2 + 2'/H # -> 2 arguments: '2 + 2' and '/H'
As an aside: This shows that bash-style string concatenation - placing any mix of quoted and unquoted tokens right next to each other - is not generally supported in PowerShell; it only works if the 1st substring / variable reference happens to be unquoted. E.g., Write-Output H/'2 + 2', unlike the substrings-reversed example above, produces only a single argument.
The exception is #: while # does have special meaning (see Get-Help about_Splatting) when followed by just a syntactically valid variable name (e.g., #parms), anything else causes the token to be treated as an expandable string again:
Write-Output #parms # splatting (results in no arguments if $parms is undefined)
Write-Output #parms$a # *expandable string*: '#parms4', if $a equals 4
I think what you're sort of hitting here is more the the type "hinting" than anything else.
You're using Write-Output which specifies in it's Synopsis that it
Sends the specified objects to the next command in the pipeline.
This command is designed to take in an array. When it hits the first item as a string like today/ it treats it like a string. When the first item ends up being the result of a function call, that may or may not be a string, so it starts up an array.
It's telling that if you run the same command to Write-Host (which is designed to take in a string to output) it works as you'd expect it to:
Write-Host $(Get-Date)/today
Outputs
7/25/2018 1:30:43 PM /today
So I think you're edge cases you're running up against are less about the parsing, and mor about the typing that powershell uses (and tries to hide).
I have the following json code in my powershell script.
I set the $variable to 1111111111
$jsonfile = '{"Version": "2012-10-17","Statement": {"Effect": "Allow","Action": "sts:AssumeRole","Resource": "arn:aws:iam::$variable:role/xxxxxx"}}'
The output gives ....arn:aws:iam::$variable:role/xxxxxx..... instead of ....arn:aws:iam::1111111111:role/xxxxxx
The problem is that I must use the single quote for the json string otherwise I will get an error. If I use single quote I wont be able to put the variables inside the string. How do I workaround this problem?
There are various ways to solve your problem, but perhaps the easiest approach is to use PowerShell's string interpolation:
use a double-quoted string overall to enable interpolation of embedded variable references and subexpressions ($(...)).
escape embedded " chars. as `" (using backticks)
disambiguate variable references by enclosing the variable name in {...}.
Simplified example:
PS> $variable='111'
PS> "{`"Version`": `"arn:aws:iam::${variable}:role/xxxxxx`"}}"
{"Version": "arn:aws:iam::111:role/xxxxxx"}}
Note that enclosing variable names in {...} in interpolated strings is only necessary if the following char. could be misinterpreted as part of the variable name.
A : following the variable name - as is the case here - is such a case, because PS variables can have a scope specifier preceding the variable name that is separated from the variable name with :, such as in $env:USERNAME.
DAXaholic's helpful answer shows an alternative based on PowerShell's binary -f operator, which is essentially the same as the .NET framework's String.Format method; as such:
it introduces additional complexity, such as needing to know what its escaping rules are ({ chars. must be escape as {{, and how to format its arguments specified on the RHS of -r ({0} refers to the 1st RHS argument, ...)
on the flip side, -f offers many sophisticated formatting options.
Also, consider use of the Convert*-Json cmdlets his answer demonstrates: even though they're less performant, they ultimately make manipulation of JSON much easier and more robust.
Alternatives in the realm of native PowerShell code:
String concatenation with the binary + operator:
'{"Version": "arn:aws:iam::' + $variable + ':role/xxxxxx"}}'
String templating with $ExecutionContext.InvokeCommand.ExpandString():
$variable='111'
$tmpl = '{"Version": "arn:aws:iam::${variable}:role/xxxxxx"}}' # string template *literal*
$ExecutionContext.InvokeCommand.ExpandString($tmpl) # performs on-demand interpolation
Another solution would be
$jsonfile = '{{"Version": "2012-10-17","Statement": {{"Effect": "Allow","Action": "sts:AssumeRole","Resource": "arn:aws:iam::{0}:role/xxxxxx"}}}}' -f $variable
So you have to escape the braces with another brace but in your case you have fewer braces than quotes so it is "less obfuscation" :)
In your case, maybe the simplest solution is just concatenating the strings together instead of using string formatting / interpolation.
In addition you could also go the way with the JSON cmdlets:
$jsonfile |
ConvertFrom-Json |
% { $_.Statement.Resource = "arn:aws:iam::${variable}:role/xxxxxx"; $_ } |
ConvertTo-Json
The Coffeescript docs contain the following blurb
String Interpolation, Block Strings, and Block Comments
Ruby-style string interpolation is included in CoffeeScript. Double-quoted strings allow for interpolated values, using #{ ... }, and single-quoted strings are literal.
Can someone explain what "single-quoted strings are literal" means here?
Does it just mean that there will be no interpretation? Is there a more javascripty meaning?
I ask because I was seeing a difference when using a single quoted string vs a double quoted as a object key.
name = "George"
"My Name: #{name}" ====> "My Name: George"
'My Name: #{name}' ====> "My Name: #{name}"
Literal in this case means that it is literally what you wrote in the string
For this reason (and a few others) I like to use a convention of double quotes when the string is natural language that is meaningful to an end user (for example an error message). And single quotes for symbols that are meaningful to the program (like property names, flags, module names, etc).
And no, that's not your fault, the word 'literal' has like 20 definitions in cs.
Powershell tab expansion functions take 2 parameters, the line so far, and the "current word". The function should return a replacement for the current word.
From experiment, it seems to me that the current word is passed to the function without any quotes, and the returned word is inserted into the line with the same quoting as the original. So, for example, if I type
PS> foo "bar"<TAB>
I will get the string bar passed to my tab expansion function (without quotes), and my returned value will be placed back on the line in double quotes.
This behaviour causes problems in certain cases. For example, partial completion of file names, where I might type C:\Pro<TAB> to get "C:\Program Files", but I then need to delete the final quote to expand further (say, by typing \Micro and then hitting TAB again.
Also, returning an expanded value containing quotes can be very messy:
PS> function TabExpansion($line, $lastword) {
PS> "looks like '" + $lastword + "' when quoted"
PS> }
PS>
PS> Silly 'example'<TAB>
This results in unbalanced quotes.
Is there any way of avoiding or working around this behaviour?
Paul.
First off, this is not true:
This behaviour causes problems in
certain cases. For example, partial
completion of file names, where I
might type C:\Pro to get
"C:\Program Files", but I then need to
delete the final quote to expand
further (say, by typing \Micro and
then hitting TAB again.
You can continue typing the \Micro after the quote and it will take care of it for you.
If you really need to return a value containing quotes, you can inject the escape character (`) into your string. Note that you will need to escape the escape character itself so it doesn't get eaten:
function TabExpansion($line, $lastword){
"looks like ``'" + $lastword + "``' when quoted"
}
After Tab expansion, your example will look like:
Silly "looks like `'example`' when quoted"
and the parser should have no problem with it.