This question already has an answer here:
PowerShell :: Microsoft.Azure.Commands.Sql.Database.Model.AzureSqlDatabaseModel.DatabaseName [duplicate]
(1 answer)
Closed 5 months ago.
I am attempting to make a string the contains the bellow hash table object strings and the $today variable and resulting with System.Collections.Hashtable.allshift.
$today = (Get-Date).ToString('yyyyMMdd')
$saturday = #{
allshift = "All_Shifts_Checklist_Saturday.xlsx";
elements = "Elements_Weekend.xlsx";
night = "Night_Shift_Saturday.xlsx"
}
Write-Host $today$saturday.allshift
###ACTUAL 20220923System.Collections.Hashtable.allshift
###EXPECTED 20220923All_Shifts_Checklist_Saturday.xlsx
Building on the helpful comments:
Because argument $today$saturday.allshift is implicitly treated as if it were enclosed in "...", the interpolation rules of expandable strings apply.
Inside "..." strings, only stand-alone variable references are expanded as-is by default, e.g., $saturday, and not also expressions, which includes property access, e.g. $saturday.allshift. In the latter case, $saturday is expanded as a whole, and .allshift is printed verbatim.
In order to embed expressions such as $saturday.allshift inside "...", you need $(...), the subexpression operator:
# "..." isn't strictly necessary, but is conceptually clearer.
Write-Host "$today$($saturday.allshift)"
See this answer for more information, and this answer for a comprehensive overview of the string-interpolation rules.
What makes your example tricky are the subtleties around how PowerShell decides whether a given command argument is implicitly treated as if it were "..."-enclosed:
A few examples (Write-Output is used instead of Write-Host, to make it more obvious when a compound token is actually passed as two arguments):
# Argument is treated as an *expression*
# -> 'All_Shifts_Checklist_Saturday.xlsx'
Write-Output $saturday.allshift
# The compound token treats $saturday.allshift as an *expression*,
# and $today therefore becomes a *separate argument*.
# -> 'All_Shifts_Checklist_Saturday.xlsx', '20220923'
Write-Output $saturday.allshift$today
# Your case:
# Because the compound token starts with a stand-alone variable reference
# *followed by additional characters* that do not constitute a property
# access or method call, the whole token is considered a single, expandable
# string:
# * $today expands as intended
# * $saturday expands *as a whole* - resulting in just the *type name* of
# the hashtable, 'System.Collections.Hashtable' (try `#{}.ToString()`)
# * and '.allshift' is treated as a literal.
# -> '20220923System.Collections.Hashtable.allshift'
Write-Output $today$saturday.allshift
That is, if a compound token starts with a variable-based expression that is a property access or method call, the expression is recognized as such, and whatever follows becomes a separate argument.
The exact rules are complex; this answer discusses them.
Therefore, it is best to form a habit to enclose compound tokens that are meant to be a single string argument in "...", and use $(...) to enclose any embedded expressions or commands.
Related
I always used the following syntax to be sure that variable were expanded in a string:
"my string with a $($variable)"
I recently ran into the following syntax:
"my string with a ${variable}"
Are they equivalent? Any difference?
To complement marsze's helpful answer:
${...} (enclosing the variable name in { and }) is indeed always necessary if a variable name contains special characters, such as spaces, ., or -.
Not special are _ and - surprisingly and problematically - ?.
Note: : is invariably interpreted as terminating a PowerShell drive reference, in the context of namespace variable notation, or a scope specifier, irrespective of whether {...} enclosure is used or required (e.g., in $env:USERNAME or ${env:USERNAME}, env refers to the PowerShell drive representing all environment variables; in $script:foo or ${script:foo}, script refers to the script's scope and its variables).
Note:
${...} - the syntax for disambiguating a variable name - is not to be confused with $(...), which is the subexpression operator, needed to embed any expression or command that goes beyond a stand-alone variable reference in an expandable string ("..."). As such, the two syntax forms are independent of one another and may need to be combined in a given situation; e.g. "$var" / "${var}" work fine, but "$var.someProperty" / "${var}.someProperty" do not: you need "$($var.someProperty)" / "$(${var}.someProperty)"
In the context of string expansion (interpolation) inside "...", there is another reason to use ${...}, even if the variable name itself doesn't need it:
If you need to delineate the variable name from directly following non-whitespace characters, notably including ::
$foo = 'bar' # example variable
# INCORRECT: PowerShell assumes that the variable name is 'foobarian', not 'foo'
PS> "A $foobarian."
A . # Variable $foobarian doesn't exist -> reference expanded to empty string.
# CORRECT: Use {...} to delineate the variable name:
PS> "A ${foo}barian."
A barbarian.
# INCORRECT: PowerShell assumes that 'foo:' is a *namespace* (drive) reference
# (such as 'env:' in $env:PATH) and FAILS:
PS> "$foo: bar"
Variable reference is not valid. ':' was not followed by a valid variable name character.
Consider using ${} to delimit the name.
# CORRECT: Use {...} to delineate the variable name:
PS> "${foo}: bar"
bar: bar
See this answer for a comprehensive overview of PowerShell string-expansion rules.
Note that you need the same technique when string expansion is implicitly applied, in the context of passing an unquoted argument to a command; e.g.:
# INCORRECT: The argument is treated as if it were enclosed in "...",
# so the same rules apply.
Write-Output $foo:/bar
# CORRECT
Write-Output ${foo}:/bar
Finally, a somewhat obscure alternative is to `-escape the first character after the variable name, but the problem is that this only works as expected with characters that aren't part of escape sequences (see about_Special_Characters):
# OK: because `: is not an escape sequence.
PS> "$foo`: bar"
bar: bar
# NOT OK, because `b is the escape sequence for a backspace character.
PS> "$foo`bar"
baar # The `b "ate" the trailing 'r' of the variable value
# and only "ar" was the literal part.
${variable} is the syntax for variable names that include special characters.
(See about_Variables -> Variable names that include special characters
)
Example:
${var with spaces} = "value"
"var with spaces: ${var with spaces}"
So in your case it's basically the same as simply writing $variable
Note that $() is helpful for json objects:
"My json property is $($jsonObj.property)"
I found some strange behavior in PowerShell surrounding arrays and double quotes. If I create and print the first element in an array, such as:
$test = #('testing')
echo $test[0]
Output:
testing
Everything works fine. But if I put double quotes around it:
echo "$test[0]"
Output:
testing[0]
Only the $test variable was evaluated and the array marker [0] was treated literally as a string. The easy fix is to just avoid interpolating array variables in double quotes, or assign them to another variable first. But is this behavior by design?
So when you are using interpolation, by default it interpolates just the next variable in toto. So when you do this:
"$test[0]"
It sees the $test as the next variable, it realizes that this is an array and that it has no good way to display an array, so it decides it can't interpolate and just displays the string as a string. The solution is to explicitly tell PowerShell where the bit to interpolate starts and where it stops:
"$($test[0])"
Note that this behavior is one of my main reasons for using formatted strings instead of relying on interpolation:
"{0}" -f $test[0]
EBGreen's helpful answer contains effective solutions, but only a cursory explanation of PowerShell's string expansion (string interpolation):
Only variables by themselves can be embedded directly inside double-quoted strings ("...") (by contrast, single-quoted strings ('...'), as in many other languages, are for literal contents).
This applies to both regular variables and variables referencing a specific namespace; e.g.:
"var contains: $var", "Path: $env:PATH"
If the first character after the variable name can be mistaken for part of the name - which notably includes : - use {...} around the variable name to disambiguate; e.g.:
"${var}", "${env:PATH}"
To use a $ as a literal, you must escape it with `, PowerShell's escape character; e.g.:
"Variable `$var"
Any character after the variable name - including [ and . is treated as a literal part of the string, so in order to index into embedded variables ($var[0]) or to access a property ($var.Count), you need $(...), the subexpression operator (in fact, $(...) allows you to embed entire statements); e.g.:
"1st element: $($var[0])"
"Element count: $($var.Count)"
"Today's date: $((Get-Date -DisplayHint Date | Out-String).Trim())"
Stringification (to-string conversion) is applied to any variable value / evaluation result that isn't already a string:
Caveat: Where culture-specific formatting can be applied, PowerShell chooses the invariant culture, which largely coincides with the US-English date and number formatting; that is, dates and numbers will be represented in US-like format (e.g., month-first date format and . as the decimal mark).
In essence, the .ToString() method is called on any resulting non-string object or collection (strictly speaking, it is .psobject.ToString(), which overrides .ToString() in some cases, notably for arrays / collections and PS custom objects)
Note that this is not the same representation you get when you output a variable or expression directly, and many types have no meaningful default string representations - they just return their full type name.
However, you can embed $(... | Out-String) in order to explicitly apply PowerShell's default output formatting.
For a more comprehensive discussion of stringification, see this answer.
As stated, using -f, the string-formatting operator (<format-string> -f <arg>[, ...]) is an alternative to string interpolation that separates the literal parts of a string from the variable parts:
'1st element: {0}; count: {1:x}' -f $var[0], $var.Count
Note the use of '...' on the LHS, because the format string (the template) is itself a literal. Using '...' in this case is a good habit to form, both to signal the intent of using literal contents and for the ability to embed $ characters without escaping.
In addition to simple positional placeholders ({0} for the 1st argument. {1} for the 2nd, ...), you may optionally exercise more formatting control over the to-string conversion; in the example above, x requests a hex representation of the number.
For available formats, see the documentation of the .NET framework's String.Format method, which the -f operator is based on.
Pitfall: -f has high precedence, so be sure to enclose RHS expressions other than simple index or property access in (...); e.g., '{0:N2}' -f 1/3 won't work as intended, only '{0:N2}' -f (1/3) will.
Caveats: There are important differences between string interpolation and -f:
Unlike expansion inside "...", the -f operator is culture-sensitive:
Therefore, the following two seemingly equivalent statements do not
yield the same result:
PS> [cultureinfo]::CurrentCulture='fr'; $n=1.2; "expanded: $n"; '-f: {0}' -f $n
expanded: 1.2
-f: 1,2
Note how only the -f-formatted command respected the French (fr) decimal mark (,).
Again, see the previously linked answer for a comprehensive look at when PowerShell is and isn't culture-sensitive.
Unlike expansion inside "...", -f stringifies arrays as <type-name>[]:
PS> $arr = 1, 2, 3; "`$arr: $arr"; '$arr: {0}' -f (, $arr)
$arr: 1 2 3
$arr: System.Object[]
Note how "..." interpolation created a space-separated list of the stringification of all array elements, whereas -f-formatting only printed the array's type name.
(As discussed, $arr inside "..." is equivalent to:
(1, 2, 3).psobject.ToString() and it is the generally invisible helper type [psobject] that provides the friendly representation.)
Also note how (, ...) was used to wrap array $arr in a helper array that ensures that -f sees the expression as a single operand; by default, the array's elements would be treated as individual operands.
In such cases you have to do:
echo "$($test[0])"
Another alternative is to use string formatting
echo "this is {0}" -f $test[0]
Note that this will be the case when you are accessing properties in strings as well. Like "$a.Foo" - should be written as "$($a.Foo)"
Note: A summary of this question has since been posted at the PowerShell GitHub repository, since superseded by this more comprehensive issue.
Arguments passed to a command in PowerShell are parsed in argument mode (as opposed to expression mode - see Get-Help about_Parsing).
Conveniently, (double-)quoting arguments that do not contain whitespace or metacharacters is usually optional, even when these arguments involve variable references (e.g. $HOME\sub) or subexpressions (e.g., version=$($PsVersionTable.PsVersion).
For the most part, such unquoted arguments are treated as if they were double-quoted strings, and the usual string-interpolation rules apply (except that metacharacters such as , need escaping).
I've tried to summarize the parsing rules for unquoted tokens in argument mode in this answer, but there are curious edge cases:
Specifically (as of Windows PowerShell v5.1), why is the unquoted argument token in each of the following commands NOT recognized as a single, expandable string, and results in 2 arguments getting passed (with the variable reference / subexpression retaining its type)?
$(...) at the start of a token:
Write-Output $(Get-Date)/today # -> 2 arguments: [datetime] obj. and string '/today'
Note that the following work as expected:
Write-Output $HOME/sub - simple var. reference at the start
Write-Output today/$(Get-Date) - subexpression not at the start
.$ at the start of a token:
Write-Output .$HOME # -> 2 arguments: string '.' and value of $HOME
Note that the following work as expected:
Write-Output /$HOME - different initial char. preceding $
Write-Output .-$HOME - initial . not directly followed by $
Write-Output a.$HOME - . is not the initial char.
As an aside: As of PowerShell Core v6.0.0-alpha.15, a = following a simple var. reference at the start of a token also seems to break the token into 2 arguments, which does not happen in Windows PowerShell v5.1; e.g., Write-Output $HOME=dir.
Note:
I'm primarily looking for a design rationale for the described behavior, or, as the case may be, confirmation that it is a bug. If it's not a bug, I want something to help me conceptualize the behavior, so I can remember it and avoid its pitfalls.
All these edge cases can be avoided with explicit double-quoting, which, given the non-obvious behavior above, may be the safest choice to use routinely.
Optional reading: The state of the documentation and design musings
As of this writing, the v5.1 Get-Help about_Parsing page:
incompletely describes the rules
uses terms that aren't neither defined in the topic nor generally in common use in the world of PowerShell ("expandable string", "value expression" - though one can guess their meaning)
From the linked page (emphasis added):
In argument mode, each value is treated as an expandable string unless it begins with one of the following special characters: dollar sign ($), at sign (#), single quotation mark ('), double quotation mark ("), or an opening parenthesis (().
If preceded by one of these characters, the value is treated as a value expression.
As an aside: A token that starts with " is, of course, by definition, also an expandable string (interpolating string).
Curiously, the conceptual help topic about quoting, Get-Help about_Quoting_Rules, manages to avoid both the terms "expand" and "interpolate".
Note how the passage does not state what happens when (non-meta)characters directly follow a token that starts with these special characters, notably $.
However, the page contains an example that shows that a token that starts with a variable reference is interpreted as an expandable string too:
With $a containing 4, Write-Output $a/H evaluates to (single string argument) 4/H.
Note that the passage does imply that variable references / subexpressions in the interior of an unquoted token (that doesn't start with a special char.) are expanded as if inside a double-quoted string ("treated as an expandable string").
If these work:
$a = 4
Write-Output $a/H # -> '4/H'
Write-Output H/$a # -> 'H/4'
Write-Output H/$(2 + 2) # -> 'H/4'
why shouldn't Write-Output $(2 + 2)/H expand to '4/H' too (instead of being treated as 2 arguments?
Why is a subexpression at the start treated differently than a variable reference?
Such subtle distinctions are hard to remember, especially in the absence of a justification.
A rule that would make more sense to me is to unconditionally treat a token that starts with $ and has additional characters following the variable reference / subexpression as an expandable string as well.
(By contrast, it makes sense for a standalone variable reference / subexpression to retain its type, as it does now.)
Note that the case of a token that starts with .$ getting split into 2 arguments is not covered in the help topic at all.
Even more optional reading: following a token that starts with one of the other special characters with additional characters.
Among the other special token-starting characters, the following unconditionally treat any characters that follow the end of the construct as a separate argument (which makes sense):
( ' "
Write-Output (2 + 2)/H # -> 2 arguments: 4 and '/H'
Write-Output "2 + $a"/H # -> 2 arguments: '2 + 4' and '/H', assuming $a equals 4
Write-Output '2 + 2'/H # -> 2 arguments: '2 + 2' and '/H'
As an aside: This shows that bash-style string concatenation - placing any mix of quoted and unquoted tokens right next to each other - is not generally supported in PowerShell; it only works if the 1st substring / variable reference happens to be unquoted. E.g., Write-Output H/'2 + 2', unlike the substrings-reversed example above, produces only a single argument.
The exception is #: while # does have special meaning (see Get-Help about_Splatting) when followed by just a syntactically valid variable name (e.g., #parms), anything else causes the token to be treated as an expandable string again:
Write-Output #parms # splatting (results in no arguments if $parms is undefined)
Write-Output #parms$a # *expandable string*: '#parms4', if $a equals 4
I think what you're sort of hitting here is more the the type "hinting" than anything else.
You're using Write-Output which specifies in it's Synopsis that it
Sends the specified objects to the next command in the pipeline.
This command is designed to take in an array. When it hits the first item as a string like today/ it treats it like a string. When the first item ends up being the result of a function call, that may or may not be a string, so it starts up an array.
It's telling that if you run the same command to Write-Host (which is designed to take in a string to output) it works as you'd expect it to:
Write-Host $(Get-Date)/today
Outputs
7/25/2018 1:30:43 PM /today
So I think you're edge cases you're running up against are less about the parsing, and mor about the typing that powershell uses (and tries to hide).
I have the following json code in my powershell script.
I set the $variable to 1111111111
$jsonfile = '{"Version": "2012-10-17","Statement": {"Effect": "Allow","Action": "sts:AssumeRole","Resource": "arn:aws:iam::$variable:role/xxxxxx"}}'
The output gives ....arn:aws:iam::$variable:role/xxxxxx..... instead of ....arn:aws:iam::1111111111:role/xxxxxx
The problem is that I must use the single quote for the json string otherwise I will get an error. If I use single quote I wont be able to put the variables inside the string. How do I workaround this problem?
There are various ways to solve your problem, but perhaps the easiest approach is to use PowerShell's string interpolation:
use a double-quoted string overall to enable interpolation of embedded variable references and subexpressions ($(...)).
escape embedded " chars. as `" (using backticks)
disambiguate variable references by enclosing the variable name in {...}.
Simplified example:
PS> $variable='111'
PS> "{`"Version`": `"arn:aws:iam::${variable}:role/xxxxxx`"}}"
{"Version": "arn:aws:iam::111:role/xxxxxx"}}
Note that enclosing variable names in {...} in interpolated strings is only necessary if the following char. could be misinterpreted as part of the variable name.
A : following the variable name - as is the case here - is such a case, because PS variables can have a scope specifier preceding the variable name that is separated from the variable name with :, such as in $env:USERNAME.
DAXaholic's helpful answer shows an alternative based on PowerShell's binary -f operator, which is essentially the same as the .NET framework's String.Format method; as such:
it introduces additional complexity, such as needing to know what its escaping rules are ({ chars. must be escape as {{, and how to format its arguments specified on the RHS of -r ({0} refers to the 1st RHS argument, ...)
on the flip side, -f offers many sophisticated formatting options.
Also, consider use of the Convert*-Json cmdlets his answer demonstrates: even though they're less performant, they ultimately make manipulation of JSON much easier and more robust.
Alternatives in the realm of native PowerShell code:
String concatenation with the binary + operator:
'{"Version": "arn:aws:iam::' + $variable + ':role/xxxxxx"}}'
String templating with $ExecutionContext.InvokeCommand.ExpandString():
$variable='111'
$tmpl = '{"Version": "arn:aws:iam::${variable}:role/xxxxxx"}}' # string template *literal*
$ExecutionContext.InvokeCommand.ExpandString($tmpl) # performs on-demand interpolation
Another solution would be
$jsonfile = '{{"Version": "2012-10-17","Statement": {{"Effect": "Allow","Action": "sts:AssumeRole","Resource": "arn:aws:iam::{0}:role/xxxxxx"}}}}' -f $variable
So you have to escape the braces with another brace but in your case you have fewer braces than quotes so it is "less obfuscation" :)
In your case, maybe the simplest solution is just concatenating the strings together instead of using string formatting / interpolation.
In addition you could also go the way with the JSON cmdlets:
$jsonfile |
ConvertFrom-Json |
% { $_.Statement.Resource = "arn:aws:iam::${variable}:role/xxxxxx"; $_ } |
ConvertTo-Json
I found some strange behavior in PowerShell surrounding arrays and double quotes. If I create and print the first element in an array, such as:
$test = #('testing')
echo $test[0]
Output:
testing
Everything works fine. But if I put double quotes around it:
echo "$test[0]"
Output:
testing[0]
Only the $test variable was evaluated and the array marker [0] was treated literally as a string. The easy fix is to just avoid interpolating array variables in double quotes, or assign them to another variable first. But is this behavior by design?
So when you are using interpolation, by default it interpolates just the next variable in toto. So when you do this:
"$test[0]"
It sees the $test as the next variable, it realizes that this is an array and that it has no good way to display an array, so it decides it can't interpolate and just displays the string as a string. The solution is to explicitly tell PowerShell where the bit to interpolate starts and where it stops:
"$($test[0])"
Note that this behavior is one of my main reasons for using formatted strings instead of relying on interpolation:
"{0}" -f $test[0]
EBGreen's helpful answer contains effective solutions, but only a cursory explanation of PowerShell's string expansion (string interpolation):
Only variables by themselves can be embedded directly inside double-quoted strings ("...") (by contrast, single-quoted strings ('...'), as in many other languages, are for literal contents).
This applies to both regular variables and variables referencing a specific namespace; e.g.:
"var contains: $var", "Path: $env:PATH"
If the first character after the variable name can be mistaken for part of the name - which notably includes : - use {...} around the variable name to disambiguate; e.g.:
"${var}", "${env:PATH}"
To use a $ as a literal, you must escape it with `, PowerShell's escape character; e.g.:
"Variable `$var"
Any character after the variable name - including [ and . is treated as a literal part of the string, so in order to index into embedded variables ($var[0]) or to access a property ($var.Count), you need $(...), the subexpression operator (in fact, $(...) allows you to embed entire statements); e.g.:
"1st element: $($var[0])"
"Element count: $($var.Count)"
"Today's date: $((Get-Date -DisplayHint Date | Out-String).Trim())"
Stringification (to-string conversion) is applied to any variable value / evaluation result that isn't already a string:
Caveat: Where culture-specific formatting can be applied, PowerShell chooses the invariant culture, which largely coincides with the US-English date and number formatting; that is, dates and numbers will be represented in US-like format (e.g., month-first date format and . as the decimal mark).
In essence, the .ToString() method is called on any resulting non-string object or collection (strictly speaking, it is .psobject.ToString(), which overrides .ToString() in some cases, notably for arrays / collections and PS custom objects)
Note that this is not the same representation you get when you output a variable or expression directly, and many types have no meaningful default string representations - they just return their full type name.
However, you can embed $(... | Out-String) in order to explicitly apply PowerShell's default output formatting.
For a more comprehensive discussion of stringification, see this answer.
As stated, using -f, the string-formatting operator (<format-string> -f <arg>[, ...]) is an alternative to string interpolation that separates the literal parts of a string from the variable parts:
'1st element: {0}; count: {1:x}' -f $var[0], $var.Count
Note the use of '...' on the LHS, because the format string (the template) is itself a literal. Using '...' in this case is a good habit to form, both to signal the intent of using literal contents and for the ability to embed $ characters without escaping.
In addition to simple positional placeholders ({0} for the 1st argument. {1} for the 2nd, ...), you may optionally exercise more formatting control over the to-string conversion; in the example above, x requests a hex representation of the number.
For available formats, see the documentation of the .NET framework's String.Format method, which the -f operator is based on.
Pitfall: -f has high precedence, so be sure to enclose RHS expressions other than simple index or property access in (...); e.g., '{0:N2}' -f 1/3 won't work as intended, only '{0:N2}' -f (1/3) will.
Caveats: There are important differences between string interpolation and -f:
Unlike expansion inside "...", the -f operator is culture-sensitive:
Therefore, the following two seemingly equivalent statements do not
yield the same result:
PS> [cultureinfo]::CurrentCulture='fr'; $n=1.2; "expanded: $n"; '-f: {0}' -f $n
expanded: 1.2
-f: 1,2
Note how only the -f-formatted command respected the French (fr) decimal mark (,).
Again, see the previously linked answer for a comprehensive look at when PowerShell is and isn't culture-sensitive.
Unlike expansion inside "...", -f stringifies arrays as <type-name>[]:
PS> $arr = 1, 2, 3; "`$arr: $arr"; '$arr: {0}' -f (, $arr)
$arr: 1 2 3
$arr: System.Object[]
Note how "..." interpolation created a space-separated list of the stringification of all array elements, whereas -f-formatting only printed the array's type name.
(As discussed, $arr inside "..." is equivalent to:
(1, 2, 3).psobject.ToString() and it is the generally invisible helper type [psobject] that provides the friendly representation.)
Also note how (, ...) was used to wrap array $arr in a helper array that ensures that -f sees the expression as a single operand; by default, the array's elements would be treated as individual operands.
In such cases you have to do:
echo "$($test[0])"
Another alternative is to use string formatting
echo "this is {0}" -f $test[0]
Note that this will be the case when you are accessing properties in strings as well. Like "$a.Foo" - should be written as "$($a.Foo)"