Change nested property of an object dynamically (eg: $_.Property.SubProperty) [duplicate] - powershell

Say I have JSON like:
{
"a" : {
"b" : 1,
"c" : 2
}
}
Now ConvertTo-Json will happily create PSObjects out of that. I want to access an item I could do $json.a.b and get 1 - nicely nested properties.
Now if I have the string "a.b" the question is how to use that string to access the same item in that structure? Seems like there should be some special syntax I'm missing like & for dynamic function calls because otherwise you have to interpret the string yourself using Get-Member repeatedly I expect.

No, there is no special syntax, but there is a simple workaround, using iex, the built-in alias[1] for the Invoke-Expression cmdlet:
$propertyPath = 'a.b'
# Note the ` (backtick) before $json, to prevent premature expansion.
iex "`$json.$propertyPath" # Same as: $json.a.b
# You can use the same approach for *setting* a property value:
$newValue = 'foo'
iex "`$json.$propertyPath = `$newValue" # Same as: $json.a.b = $newValue
Caveat: Do this only if you fully control or implicitly trust the value of $propertyPath.
Only in rare situation is Invoke-Expression truly needed, and it should generally be avoided, because it can be a security risk.
Note that if the target property contains an instance of a specific collection type and you want to preserve it as-is (which is not common) (e.g., if the property value is a strongly typed array such as [int[]], or an instance of a list type such as [System.Collections.Generic.List`1]), use the following:
# "," constructs an aux., transient array that is enumerated by
# Invoke-Expression and therefore returns the original property value as-is.
iex ", `$json.$propertyPath"
Without the , technique, Invoke-Expression enumerates the elements of a collection-valued property and you'll end up with a regular PowerShell array, which is of type [object[]] - typically, however, this distinction won't matter.
Note: If you were to send the result of the , technique directly through the pipeline, a collection-valued property value would be sent as a single object instead of getting enumerated, as usual. (By contrast, if you save the result in a variable first and the send it through the pipeline, the usual enumeration occurs). While you can force enumeration simply by enclosing the Invoke-Expression call in (...), there is no reason to use the , technique to begin with in this case, given that enumeration invariably entails loss of the information about the type of the collection whose elements are being enumerated.
Read on for packaged solutions.
Note:
The following packaged solutions originally used Invoke-Expression combined with sanitizing the specified property paths in order to prevent inadvertent/malicious injection of commands. However, the solutions now use a different approach, namely splitting the property path into individual property names and iteratively drilling down into the object, as shown in Gyula Kokas's helpful answer. This not only obviates the need for sanitizing, but turns out to be faster than use of Invoke-Expression (the latter is still worth considering for one-off use).
The no-frills, get-only, always-enumerate version of this technique would be the following function:
# Sample call: propByPath $json 'a.b'
function propByPath { param($obj, $propPath) foreach ($prop in $propPath.Split('.')) { $obj = $obj.$prop }; $obj }
What the more elaborate solutions below offer: parameter validation, the ability to also set a property value by path, and - in the case of the propByPath function - the option to prevent enumeration of property values that are collections (see next point).
The propByPath function offers a -NoEnumerate switch to optionally request preserving a property value's specific collection type.
By contrast, this feature is omitted from the .PropByPath() method, because there is no syntactically convenient way to request it (methods only support positional arguments). A possible solution is to create a second method, say .PropByPathNoEnumerate(), that applies the , technique discussed above.
Helper function propByPath:
function propByPath {
param(
[Parameter(Mandatory)] $Object,
[Parameter(Mandatory)] [string] $PropertyPath,
$Value, # optional value to SET
[switch] $NoEnumerate # only applies to GET
)
Set-StrictMode -Version 1
# Note: Iteratively drilling down into the object turns out to be *faster*
# than using Invoke-Expression; it also obviates the need to sanitize
# the property-path string.
$props = $PropertyPath.Split('.') # Split the path into an array of property names.
if ($PSBoundParameters.ContainsKey('Value')) { # SET
$parentObject = $Object
if ($props.Count -gt 1) {
foreach ($prop in $props[0..($props.Count-2)]) { $parentObject = $parentObject.$prop }
}
$parentObject.($props[-1]) = $Value
}
else { # GET
$value = $Object
foreach ($prop in $props) { $value = $value.$prop }
if ($NoEnumerate) {
, $value
} else {
$value
}
}
}
Instead of the Invoke-Expression call you would then use:
# GET
propByPath $obj $propertyPath
# GET, with preservation of the property value's specific collection type.
propByPath $obj $propertyPath -NoEnumerate
# SET
propByPath $obj $propertyPath 'new value'
You could even use PowerShell's ETS (extended type system) to attach a .PropByPath() method to all [pscustomobject] instances (PSv3+ syntax; in PSv2 you'd have to create a *.types.ps1xml file and load it with Update-TypeData -PrependPath):
'System.Management.Automation.PSCustomObject',
'Deserialized.System.Management.Automation.PSCustomObject' |
Update-TypeData -TypeName { $_ } `
-MemberType ScriptMethod -MemberName PropByPath -Value { #`
param(
[Parameter(Mandatory)] [string] $PropertyPath,
$Value
)
Set-StrictMode -Version 1
$props = $PropertyPath.Split('.') # Split the path into an array of property names.
if ($PSBoundParameters.ContainsKey('Value')) { # SET
$parentObject = $this
if ($props.Count -gt 1) {
foreach ($prop in $props[0..($props.Count-2)]) { $parentObject = $parentObject.$prop }
}
$parentObject.($props[-1]) = $Value
}
else { # GET
# Note: Iteratively drilling down into the object turns out to be *faster*
# than using Invoke-Expression; it also obviates the need to sanitize
# the property-path string.
$value = $this
foreach ($prop in $PropertyPath.Split('.')) { $value = $value.$prop }
$value
}
}
You could then call $obj.PropByPath('a.b') or $obj.PropByPath('a.b', 'new value')
Note: Type Deserialized.System.Management.Automation.PSCustomObject is targeted in addition to System.Management.Automation.PSCustomObject in order to also cover deserialized custom objects, which are returned in a number of scenarios, such as using Import-CliXml, receiving output from background jobs, and using remoting.
.PropByPath() will be available on any [pscustomobject] instance in the remainder of the session (even on instances created prior to the Update-TypeData call [2]); place the Update-TypeData call in your $PROFILE (profile file) to make the method available by default.
[1] Note: While it is generally advisable to limit aliases to interactive use and use full cmdlet names in scripts, use of iex to me is acceptable, because it is a built-in alias and enables a concise solution.
[2] Verify with (all on one line) $co = New-Object PSCustomObject; Update-TypeData -TypeName System.Management.Automation.PSCustomObject -MemberType ScriptMethod -MemberName GetFoo -Value { 'foo' }; $co.GetFoo(), which outputs foo even though $co was created before Update-TypeData was called.

This workaround is maybe useful to somebody.
The result goes always deeper, until it hits the right object.
$json=(Get-Content ./json.json | ConvertFrom-Json)
$result=$json
$search="a.c"
$search.split(".")|% {$result=$result.($_) }
$result

You can have 2 variables.
$json = '{
"a" : {
"b" : 1,
"c" : 2
}
}' | convertfrom-json
$a,$b = 'a','b'
$json.$a.$b
1

Related

How to extend and override Add in a collection class

Background
I have a data object in PowerShell with 4 properties, 3 of which are strings and the 4th a hashtable. I would like to arrange for a new type that is defined as a collection of this data object.
In this collection class, I wish to enforce a particular format that will make my code elsewhere in the module more convenient. Namely, I wish to override the add method with a new definition, such that unique combinations of the 3 string properties add the 4th property as a hashtable, while duplicates of the 3 string properties simply update the hashtable property of the already existing row with the new input hashtable.
This will allow me to abstract the expansion of the collection and ensure that when the Add method is called on it, it will retain my required format of hashtables grouped by unique combinations of the 3 string properties.
My idea was to create a class that extends a collection, and then override the add method.
Code so far
As a short description for my code below, there are 3 classes:
A data class for a namespace based on 3 string properties (which I can reuse in my script for other things).
A class specifically for adding an id property to this data class. This id is the key in a hashtable with values that are configuration parameters in the namespace of my object.
A 3rd class to handle a collection of these objects, where I can define the add method. This is where I am having my issue.
Using namespace System.Collections.Generic
Class Model_Namespace {
[string]$Unit
[string]$Date
[string]$Name
Model_Namespace([string]$unit, [string]$date, [string]$name) {
$this.Unit = $unit
$this.Date = $date
$this.Name = $name
}
}
Class Model_Config {
[Model_Namespace]$namespace
[Hashtable]$id
Model_Config([Model_Namespace]$namespace, [hashtable]$config) {
$this.namespace = $namespace
$this.id = $config
}
Model_Config([string]$unit, [string]$date, [string]$name, [hashtable]$config) {
$this.namespace = [Model_Namespace]::new($unit, $date, $name)
$this.id = $config
}
}
Class Collection_Configs {
$List = [List[Model_Config]]#()
[void] Add ([Model_Config]$newConfig ){
$checkNamespaceExists = $null
$u = $newConfig.Unit
$d = $newConfig.Date
$n = $newConfig.Name
$id = $newConfig.id
$checkNamespaceExists = $this.List | Where { $u -eq $_.Unit -and $d -eq $_.Date -and $n -eq $_.Name }
If ($checkNamespaceExists){
($this.List | Where { $u -eq $_.Unit -and $d -eq $_.Date -and $n -eq $_.Name }).id += $id
}
Else {
$this.List.add($newConfig)
}
}
}
Problem
I would like the class Collection_Configs to extend a built-in collection type and override the Add method. Like a generic List<> type, I could simply output the variable referencing my collection and automatically return the collection. This way, I won't need to dot into the List property to access the collection. In fact I wouldn't need the List property at all.
However, when I inherit from System.Array, I need to supply a fixed array size in the constructor. I'd like to avoid this, as my collection should be mutable. I tried inheriting from List, but I can't get the syntax to work; PowerShell throws a type not found error.
Is there a way to accomplish this?
Update
After mklement's helpful answer, I modified the last class as:
Using namespace System.Collections.ObjectModel
Class Collection_Configs : System.Collections.ObjectModel.Collection[Object]{
[void] Add ([Model_Config]$newConfig ){
$checkNamespaceExists = $null
$newConfigParams = $newConfig.namespace
$u = $newConfigParams.Unit
$d = $newConfigParams.Date
$n = $newConfigParams.Name
$id = $newConfig.id
$checkNamespaceExists = $this.namespace | Where { $u -eq $_.Unit -and $d -eq $_.Date -and $n -eq $_.Name }
If ($checkNamespaceExists){
($this | Where { $u -eq $_.namespace.Unit -and $d -eq $_.namespace.Date -and $n -eq $_.namespace.Name }).id += $id
}
Else {
([Collection[object]]$this).add($newConfig)
}
}
}
Which seems to work. In addition to the inheritance, had to do some other corrections regarding how I dotted into my input types, and I also needed to load the collection class separately after the other 2 classes as well as use the base class's add method in my else statement.
Going forward, I will have to do some other validation to ensure that a model_config type is entered. Currently the custom collection accepts any input, even though I auto-convert the add parameter to model_config, e.g.,
$config = [model_config]::new('a','b','c',#{'h'='t'})
$collection = [Collection_Configs]::new()
$collection.Add($config)
works, but
$collection.Add('test')
also works when it should fail validation. Perhaps it is not overriding correctly and using the base class's overload?
Last update
Everything seems to be working now. The last update to the class is:
using namespace System.Collections.ObjectModel
Class Collection_Configs : Collection[Model_Config]{
[void] Add ([Model_Config]$newConfig ){
$checkNamespaceExists = $null
$namespace = $newConfig.namespace
$u = $namespace.Unit
$d = $namespace.Date
$n = $namespace.Name
$id = $newConfig.id
$checkNamespaceExists = $this.namespace | Where { $u -eq $_.Unit -and $d -eq $_.Date -and $n -eq $_.Name }
If ($checkNamespaceExists){
($this | Where { $u -eq $_.namespace.Unit -and $d -eq $_.namespace.Date -and $n -eq $_.namespace.Name }).id += $id
}
Else {
[Collection[Model_Config]].GetMethod('Add').Invoke($this, [Model_Config[]]$newConfig)
}
}
}
Notice in the else statement that ....GetMethod('Add')... is necessary for Windows PowerShell, as pointed out in the footnote of mklement0's super useful and correct answer. If you are able to work with Core, then mklement0's syntax will work (I tested).
Also mentioned by mklement0, the types need to be loaded separately. FYI this can be done on the commandline for quick provisional testing by typing in the model_namespace and model_config classes and pressing enter before doing the same for Collection_Configs.
In summary this will create a custom collection type with custom methods in PowerShell.
It is possible to subclass System.Collections.Generic.List`1, as this simplified example, which derives from a list with [regex] elements, demonstrates:[1]
using namespace System.Collections.Generic
# Subclass System.Collections.Generic.List`1 with [regex] elements.
class Collection_Configs : List[regex] {
# Override the .Add() method.
# Note: You probably want to override .AddRange() too.
Add([regex] $item) {
Write-Verbose -Verbose 'Doing custom things...'
# Call the base-class method.
([List[regex]] $this).Add($item)
}
}
# Sample use.
$list = [Collection_Configs]::new()
$list.Add([regex] 'foo')
$list
However, as you note, it is recommended to derive custom collections from base class System.Collections.ObjectModel.Collection`1:
using namespace System.Collections.ObjectModel
# Subclass System.Collections.ObjectModel`1 with [regex] elements.
class Collection_Configs : Collection[regex] {
# Override the .Add() method.
# Note: Unlike with List`1, there is no .AddRange() method.
Add([regex] $item) {
Write-Verbose -Verbose 'Doing custom things...'
# Call the base-class method.
([Collection[regex]] $this).Add($item)
}
}
As for the pros and cons:
List`1 has more built-in functionality (methods) than ObjectModel`1, such as .Reverse(), Exists(), and .ForEach().
In the case of .ForEach() that actually works to the advantage of ObjectModel`1: not having such a method avoids a clash with PowerShell's intrinsic .ForEach() method.
Note that in either case it is important to use the specific type that your collection should be composed of as the generic type argument for the base class: [regex] in the example above, [Model_Config] in your real code (see next section).
If you use [object] instead, your collection won't be type-safe, because it'll have a void Add(object item) method that PowerShell will select whenever you call the .Add() method with an instance of a type that is not the desired type (or cannot be converted to it).
However, there's an additional challenge in your case:
As of PowerShell 7.3.1, because the generic type argument that determines the list element type is another custom class, that other class must unexpectedly be loaded beforehand, in a separate script, the script that defines the dependent Collection_Configs class.
This requirement is unfortunate, and at least conceptually related to the general (equally unfortunate) need to ensure that .NET types referenced in class definitions have been loaded before the enclosing script executes - see this post, whose accepted answer demonstrates workarounds.
However, given that all classes involved are part of the same script file in your case, a potential fix should be simpler than the one discussed in the linked post - see GitHub issue #18872.
[1] Note: There appears to be a bug in Windows PowerShell, where calling the base class' .Add() method fails if the generic type argument (element type) happens to be [pscustomobject] aka [psobject]: That is, while ([List[pscustomobject]] $this).Add($item) works as expected in PowerShell (Core) 7+, an error occurs in Windows PowerShell, which requires the following reflection-based workaround: [List[pscustomobject]].GetMethod('Add').Invoke($this, [object[]] $item)
There were a few issues with the original code:
The Using keyword was spelled incorrectly. It should be using.
The $List variable in the Collection_Configs class was not declared with a type. It should be [List[Model_Config]]$List.
The Add method in the Collection_Configs class was missing its return type. It should be [void] Add ([Model_Config]$newConfig).
The Add method was missing its opening curly brace.

Powershell passing multiple parameters from one script to another [duplicate]

I've seen the # symbol used in PowerShell to initialise arrays.
What exactly does the # symbol denote and where can I read more about it?
In PowerShell V2, # is also the Splat operator.
PS> # First use it to create a hashtable of parameters:
PS> $params = #{path = "c:\temp"; Recurse= $true}
PS> # Then use it to SPLAT the parameters - which is to say to expand a hash table
PS> # into a set of command line parameters.
PS> dir #params
PS> # That was the equivalent of:
PS> dir -Path c:\temp -Recurse:$true
PowerShell will actually treat any comma-separated list as an array:
"server1","server2"
So the # is optional in those cases. However, for associative arrays, the # is required:
#{"Key"="Value";"Key2"="Value2"}
Officially, # is the "array operator." You can read more about it in the documentation that installed along with PowerShell, or in a book like "Windows PowerShell: TFM," which I co-authored.
While the above responses provide most of the answer it is useful--even this late to the question--to provide the full answer, to wit:
Array sub-expression (see about_arrays)
Forces the value to be an array, even if a singleton or a null, e.g. $a = #(ps | where name -like 'foo')
Hash initializer (see about_hash_tables)
Initializes a hash table with key-value pairs, e.g.
$HashArguments = #{ Path = "test.txt"; Destination = "test2.txt"; WhatIf = $true }
Splatting (see about_splatting)
Let's you invoke a cmdlet with parameters from an array or a hash-table rather than the more customary individually enumerated parameters, e.g. using the hash table just above, Copy-Item #HashArguments
Here strings (see about_quoting_rules)
Let's you create strings with easily embedded quotes, typically used for multi-line strings, e.g.:
$data = #"
line one
line two
something "quoted" here
"#
Because this type of question (what does 'x' notation mean in PowerShell?) is so common here on StackOverflow as well as in many reader comments, I put together a lexicon of PowerShell punctuation, just published on Simple-Talk.com. Read all about # as well as % and # and $_ and ? and more at The Complete Guide to PowerShell Punctuation. Attached to the article is this wallchart that gives you everything on a single sheet:
You can also wrap the output of a cmdlet (or pipeline) in #() to ensure that what you get back is an array rather than a single item.
For instance, dir usually returns a list, but depending on the options, it might return a single object. If you are planning on iterating through the results with a foreach-object, you need to make sure you get a list back. Here's a contrived example:
$results = #( dir c:\autoexec.bat)
One more thing... an empty array (like to initialize a variable) is denoted #().
The Splatting Operator
To create an array, we create a variable and assign the array. Arrays are noted by the "#" symbol. Let's take the discussion above and use an array to connect to multiple remote computers:
$strComputers = #("Server1", "Server2", "Server3")<enter>
They are used for arrays and hashes.
PowerShell Tutorial 7: Accumulate, Recall, and Modify Data
Array Literals In PowerShell
I hope this helps to understand it a bit better.
You can store "values" within a key and return that value to do something.
In this case I have just provided #{a="";b="";c="";} and if not in the options i.e "keys" (a, b or c) then don't return a value
$array = #{
a = "test1";
b = "test2";
c = "test3"
}
foreach($elem in $array.GetEnumerator()){
if ($elem.key -eq "a"){
$key = $elem.key
$value = $elem.value
}
elseif ($elem.key -eq "b"){
$key = $elem.key
$value = $elem.value
}
elseif ($elem.key -eq "c"){
$key = $elem.key
$value = $elem.value
}
else{
Write-Host "No other value"
}
Write-Host "Key: " $key "Value: " $value
}

Tab-complete a parameter value based on another parameter's already specified value

This question addresses the following scenario:
Can custom tab-completion for a given command dynamically determine completions based on the value previously passed to another parameter on the same command line, using either a parameter-level [ArgumentCompleter()] attribute or the Register-ArgumentCompleter cmdlet?
If so, what are the limitations of this approach?
Example scenario:
A hypothetical Get-Property command has an -Object parameter that accepts an object of any type, and a -Property parameter that accepts the name of a property whose value to extract from the object.
Now, in the course of typing a Get-Property call, if a value is already specified for -Object, tab-completing -Property should cycle through the names of the specified object's (public) properties.
$obj = [pscustomobject] #{ foo = 1; bar = 2; baz = 3 }
Get-Property -Object $obj -Property # <- pressing <tab> here should cycle
# through 'foo', 'bar', 'baz'
#mklement0, regarding first limitation stated in your answer
The custom-completion script block ({ ... }) invoked by PowerShell fundamentally only sees values specified via parameters, not via the pipeline.
I struggled with this, and after some stubbornness I got a working solution.
At least good enough for my tooling, and I hope it can make life easier for many others out there.
This solution has been verified to work with PowerShell versions 5.1 and 7.1.2.
Here I made use of $cmdAst (called $commandAst in the docs), which contains information about the pipeline. With this we can get to know the previous pipeline element and even differentiate between it containing only a variable or a command. Yes, A COMMAND, which with help of Get-Command and the command's OutputType() member method, we can get (suggested) property names for such as well!
Example usage
PS> $obj = [pscustomobject] #{ foo = 1; bar = 2; baz = 3 }
PS> $obj | Get-Property -Property # <tab>: bar, baz, foo
PS> "la", "na", "le" | Select-String "a" | Get-Property -Property # <tab>: Chars, Context, Filename, ...
PS> 2,5,2,2,6,3 | group | Get-Property -Property # <tab>: Count, Values, Group, ...
Function code
Note that apart from now using $cmdAst, I also added [Parameter(ValueFromPipeline=$true)] so we actually pick the object, and PROCESS {$Object.$Property} so that one can test and see the code actually working.
param(
[Parameter(ValueFromPipeline=$true)]
[object] $Object,
[ArgumentCompleter({
param($cmdName, $paramName, $wordToComplete, $cmdAst, $preBoundParameters)
# Find out if we have pipeline input.
$pipelineElements = $cmdAst.Parent.PipelineElements
$thisPipelineElementAsString = $cmdAst.Extent.Text
$thisPipelinePosition = [array]::IndexOf($pipelineElements.Extent.Text, $thisPipelineElementAsString)
$hasPipelineInput = $thisPipelinePosition -ne 0
$possibleArguments = #()
if ($hasPipelineInput) {
# If we are in a pipeline, find out if the previous pipeline element is a variable or a command.
$previousPipelineElement = $pipelineElements[$thisPipelinePosition - 1]
$pipelineInputVariable = $previousPipelineElement.Expression.VariablePath.UserPath
if (-not [string]::IsNullOrEmpty($pipelineInputVariable)) {
# If previous pipeline element is a variable, get the object.
# Note that it can be a non-existent variable. In such case we simply get nothing.
$detectedInputObject = Get-Variable |
Where-Object {$_.Name -eq $pipelineInputVariable} |
ForEach-Object Value
} else {
$pipelineInputCommand = $previousPipelineElement.CommandElements[0].Value
if (-not [string]::IsNullOrEmpty($pipelineInputCommand)) {
# If previous pipeline element is a command, check if it exists as a command.
$possibleArguments += Get-Command -CommandType All |
Where-Object Name -Match "^$pipelineInputCommand$" |
# Collect properties for each documented output type.
ForEach-Object {$_.OutputType.Type} | ForEach-Object GetProperties |
# Group properties by Name to get unique ones, and sort them by
# the most frequent Name first. The sorting is a perk.
# A command can have multiple output types. If so, we might now
# have multiple properties with identical Name.
Group-Object Name -NoElement | Sort-Object Count -Descending |
ForEach-Object Name
}
}
} elseif ($preBoundParameters.ContainsKey("Object")) {
# If not in pipeline, but object has been given, get the object.
$detectedInputObject = $preBoundParameters["Object"]
}
if ($null -ne $detectedInputObject) {
# The input object might be an array of objects, if so, select the first one.
# We (at least I) are not interested in array properties, but the object element's properties.
if ($detectedInputObject -is [array]) {
$sampleInputObject = $detectedInputObject[0]
} else {
$sampleInputObject = $detectedInputObject
}
# Collect property names.
$possibleArguments += $sampleInputObject | Get-Member -MemberType Properties | ForEach-Object Name
}
# Refering to about_Functions_Argument_Completion documentation.
# The ArgumentCompleter script block must unroll the values using the pipeline,
# such as ForEach-Object, Where-Object, or another suitable method.
# Returning an array of values causes PowerShell to treat the entire array as one tab completion value.
$possibleArguments | Where-Object {$_ -like "$wordToComplete*"}
})]
[string] $Property
)
PROCESS {$Object.$Property}
Update: See betoz's helpful answer for a more complete solution that also supports pipeline input.
The part of the answer below that clarifies the limitations of pre-execution detection of the input objects' data type still applies.
The following solution uses a parameter-specific [ArgumentCompleter()] attribute as part of the definition of the Get-Property function itself, but the solution analogously applies to separately defining custom-completion logic via the Register-CommandCompleter cmdlet.
Limitations:
[See betoz's answer for how to overcome this limitation] The custom-completion script block ({ ... }) invoked by PowerShell fundamentally only sees values specified via parameters, not via the pipeline.
That is, if you type Get-Property -Object $obj -Property <tab>, the script block can determine that the value of $obj is to be bound to the -Object parameter, but that wouldn't work with
$obj | Get-Property -Property <tab> (even if -Object is declared as pipeline-binding).
Fundamentally, only values that can be evaluated without side effects are actually accessible in the script block; in concrete terms, this means:
Literal values (e.g., -Object ([pscustomobject] #{ foo = 1; bar = 2; baz = 3 })
Simple variable references (e.g., -Object $obj) or property-access or index-access expressions (e.g., -Object $obj.Foo or -Object $obj[0])
Notably, the following values are not accessible:
Method-call results (e.g., -Object $object.Foo())
Command output (via (...), $(...), or #(...), e.g.
-Object (Invoke-RestMethod http://example.org))
The reason for this limitation is that evaluating such values before actually submitting the command could have undesirable side effects and / or could take a long time to complete.
function Get-Property {
param(
[object] $Object,
[ArgumentCompleter({
# A fixed list of parameters is passed to an argument-completer script block.
# Here, only two are of interest:
# * $wordToComplete:
# The part of the value that the user has typed so far, if any.
# * $preBoundParameters (called $fakeBoundParameters
# in the docs):
# A hashtable of those (future) parameter values specified so
# far that are side effect-free (see above).
param($cmdName, $paramName, $wordToComplete, $cmdAst, $preBoundParameters)
# Was a side effect-free value specified for -Object?
if ($obj = $preBoundParameters['Object']) {
# Get all property names of the objects and filter them
# by the partial value already typed, if any,
# interpreted as a name prefix.
#($obj.psobject.Properties.Name) -like "$wordToComplete*"
}
})]
[string] $Property
)
# ...
}

Make select -unique on arraylist return arraylist instead of string

I have three arraylists in below class. I want to keep them unique. However if there's only one item (string) in the arraylist and you use select -unique (or any other method to achieve this) it will return the string instead of a list of strings. Surrounding it with #() also doesn't work because that transforms it to an array instead of an arraylist, which I can't add stuff to.
Any suggestions that are still performant? I tried HashSets before but somehow had horrible experiences with those. See my previous post for that.. Post on hashset issue
Code below:
Class OrgUnit
{
[String]$name
$parents
$children
$members
OrgUnit($name){
$this.name = $name
$this.parents = New-Object System.Collections.ArrayList
$this.children = New-Object System.Collections.ArrayList
$this.members = New-Object System.Collections.ArrayList
}
addChild($child){
# > $null to supress output
$tmp = $this.children.Add($child)
$this.children = $this.children | select -Unique
}
addParent($parent){
# > $null to supress output
$tmp = $this.parents.Add($parent)
$this.parents = $this.parents | select -Unique
}
addMember($member){
# > $null to supress output
$tmp = $this.members.Add($member)
$this.members = $this.members | select -Unique
}
}
You're adding a new item to the array, then selecting unique items from it, and reassingning it every time you add a member. This is extremely inefficient, maybe try the following instead:
if (-not $this.parents.Contains($parent)) {
$this.parents.Add($parent) | out-null
}
Would be much faster even with least efficient output supressing by out-null.
Check with .Contains() if the item is already added, so you don't have to eliminate duplicates with Select-Object -Unique afterwards all the time.
if (-not $this.children.Contains($child))
{
[System.Void]($this.children.Add($child))
}
As has been pointed out, it's worth changing your approach due to its inefficiency:
Instead of blindly appending and then possibly removing the new element if it turns out to be duplicate with Select-Object -Unique, use a test to decide whether an element needs to be appended or is already present.
Patrick's helpful answer is a straightforward implementation of this optimized approach that will greatly speed up your code and should perform acceptably unless the array lists get very large.
As a side effect of this optimization - because the array lists are only ever modified in-place with .Add() - your original problem goes away.
To answer the question as asked:
Simply type-constrain your (member) variables if you want them to retain a given type even during later assignments.
That is, just as you did with $name, place the type you want the member to be constrained to the left of the member variable declarations:
[System.Collections.ArrayList] $parents
[System.Collections.ArrayList] $children
[System.Collections.ArrayList] $members
However, that will initialize these member variables to $null, which means you won't be able to just call .Add() in your .add*() methods; therefore, construct an (initially empty) instance as part of the declaration:
[System.Collections.ArrayList] $parents = [System.Collections.ArrayList]::new()
[System.Collections.ArrayList] $children = [System.Collections.ArrayList]::new()
[System.Collections.ArrayList] $members = [System.Collections.ArrayList]::new()
Also, you do have to use #(...) around your Select-Object -Unique pipeline; while that indeed outputs an array (type [object[]]), the type constraint causes that array to be converted to a [System.Collections.ArrayList] instance, as explained below.
The need for #(...) is somewhat surprising - see bottom section.
Notes on type constraints:
If you assign a value that isn't already of the type that the variable is constrained to, PowerShell attempts to convert it to that type; you can think of it as implicitly performing a cast to the constraining type on every assignment:
This can fail, if the assigned value simply isn't convertible; PowerShell's type conversions are generally very flexible, however.
In the case of collection-like types such as [System.Collections.ArrayList], any other collection-like type can be assigned, such as the [object[]] arrays returned by #(...) (PowerShell's array-subexpression operator). Note that, of necessity, this involves constructing a new [System.Collections.ArrayList] every time, which becomes, loosely speaking, a shallow clone of the input collection.
Pitfalls re assigning $null:
If the constraining type is a value type (if its .IsValueType property reports $true), assigning $null will result in the type's default value; e.g., after executing [int] $i = 42; $i = $null, $i isn't $null, it is 0.
If the constraining type is a reference type (such as [System.Collections.ArrayList]), assigning $null will truly store $null in the variable, though later attempts to assign non-null values will again result in conversion to the constraining type.
In essence, this is the same technique used in parameter variables, and can also be used in regular variables.
With regular variables (local variables in a function or script), you must also initialize the variable in order for the type constraint to work (for the variable to even be created); e.g.:
[System.Collections.ArrayList] $alist = 1, 2
Applied to a simplified version of your code:
Class OrgUnit
{
[string] $name
# Type-constrain $children too, just like $name above, and initialize
# with an (initially empty) instance.
[System.Collections.ArrayList] $children = [System.Collections.ArrayList]::new()
addChild($child){
# Add a new element.
# Note the $null = ... to suppress the output from the .Add() method.
$null = $this.children.Add($child)
# (As noted, this approach is inefficient.)
# Note the required #(...) around the RHS (see notes in the last section).
# Due to its type constraint, $this.children remains a [System.Collections.ArrayList] (a new instance is created from the
# [object[]] array that #(...) outputs).
$this.children = #($this.children | Select-Object -Unique)
}
}
With the type constraint in place, the .children property now remains a [System.Collections.ArrayList]:
PS> $ou = [OrgUnit]::new(); $ou.addChild(1); $ou.children.GetType().Name
ArrayList # Proof that $children retained its type identity.
Note: The need for #(...) - to ensure an array-valued assignment value in order to successfully convert to [System.Collections.ArrayList] - is somewhat surprising, given that the following works with the similar generic list type, [System.Collections.Generic.List[object]]:
# OK: A scalar (single-object) input results in a 1-element list.
[System.Collections.Generic.List[object]] $list = 'one'
By contrast, this does not work with [System.Collections.ArrayList]:
# !! FAILS with a scalar (single object)
# Error message: Cannot convert the "one" value of type "System.String" to type "System.Collections.ArrayList".
[System.Collections.ArrayList] $list = 'one'
# OK
# Forcing the RHS to an array ([object[]]) fixes the problem.
[System.Collections.ArrayList] $list = #('one')
Try this one:
Add-Type -AssemblyName System.Collections
Class OrgUnit
{
[String]$name
$parents
$children
$members
OrgUnit($name){
$this.name = $name
$this.parents = [System.Collections.Generic.List[object]]::new()
$this.children = [System.Collections.Generic.List[object]]::new()
$this.members = [System.Collections.Generic.List[object]]::new()
}
addChild($child){
# > $null to supress output
$tmp = $this.children.Add($child)
$this.children = [System.Collections.Generic.List[object]]#($this.children | select -Unique)
}
addParent($parent){
# > $null to supress output
$tmp = $this.parents.Add($parent)
$this.parents = [System.Collections.Generic.List[object]]#($this.parents | select -Unique)
}
addMember($member){
# > $null to supress output
$tmp = $this.members.Add($member)
$this.members = [System.Collections.Generic.List[object]]#($this.members | select -Unique)
}
}

Powershell pitfalls

What Powershell pitfalls you have fall into? :-)
Mine are:
# -----------------------------------
function foo()
{
#("text")
}
# Expected 1, actually 4.
(foo).length
# -----------------------------------
if(#($null, $null))
{
Write-Host "Expected to be here, and I am here."
}
if(#($null))
{
Write-Host "Expected to be here, BUT NEVER EVER."
}
# -----------------------------------
function foo($a)
{
# I thought this is right.
#if($a -eq $null)
#{
# throw "You can't pass $null as argument."
#}
# But actually it should be:
if($null -eq $a)
{
throw "You can't pass $null as argument."
}
}
foo #($null, $null)
# -----------------------------------
# There is try/catch, but no callstack reported.
function foo()
{
bar
}
function bar()
{
throw "test"
}
# Expected:
# At bar() line:XX
# At foo() line:XX
#
# Actually some like this:
# At bar() line:XX
foo
Would like to know yours to walk them around :-)
My personal favorite is
function foo() {
param ( $param1, $param2 = $(throw "Need a second parameter"))
...
}
foo (1,2)
For those unfamiliar with powershell that line throws because instead of passing 2 parameters it actually creates an array and passes one parameter. You have to call it as follows
foo 1 2
Another fun one. Not handling an expression by default writes it to the pipeline. Really annoying when you don't realize a particular function returns a value.
function example() {
param ( $p1 ) {
if ( $p1 ) {
42
}
"done"
}
PS> example $true
42
"done"
$files = Get-ChildItem . -inc *.extdoesntexist
foreach ($file in $files) {
"$($file.Fullname.substring(2))"
}
Fails with:
You cannot call a method on a null-valued expression.
At line:3 char:25
+ $file.Fullname.substring <<<< (2)
Fix it like so:
$files = #(Get-ChildItem . -inc *.extdoesntexist)
foreach ($file in $files) {
"$($file.Fullname.substring(2))"
}
Bottom line is that the foreach statement will loop on a scalar value even if that scalar value is $null. When Get-ChildItem in the first example returns nothing, $files gets assinged $null. If you are expecting an array of items to be returned by a command but there is a chance it will only return 1 item or zero items, put #() around the command. Then you will always get an array - be it of 0, 1 or N items. Note: If the item is already an array putting #() has no effect - it will still be the very same array (i.e. there is no extra array wrapper).
# The pipeline doesn't enumerate hashtables.
$ht = #{"foo" = 1; "bar" = 2}
$ht | measure
# Workaround: call GetEnumerator
$ht.GetEnumerator() | measure
Here are my top 5 PowerShell gotchas
Here is something Ive stumble upon lately (PowerShell 2.0 CTP):
$items = "item0", "item1", "item2"
$part = ($items | select-string "item0")
$items = ($items | where {$part -notcontains $_})
what do you think that $items be at the end of the script?
I was expecting "item1", "item2" but instead the value of $items is: "item0", "item1", "item2".
Say you've got the following XML file:
<Root>
<Child />
<Child />
</Root>
Run this:
PS > $myDoc = [xml](Get-Content $pathToMyDoc)
PS > #($myDoc.SelectNodes("/Root/Child")).Count
2
PS > #($myDoc.Root.Child).Count
2
Now edit the XML file so it has no Child nodes, just the Root node, and run those statements again:
PS > $myDoc = [xml](Get-Content $pathToMyDoc)
PS > #($myDoc.SelectNodes("/Root/Child")).Count
0
PS > #($myDoc.Root.Child).Count
1
That 1 is annoying when you want to iterate over a collection of nodes using foreach if and only if there actually are any. This is how I learned that you cannot use the XML handler's property (dot) notation as a simple shortcut. I believe what's happening is that SelectNodes returns a collection of 0. When #'ed, it is transformed from an XPathNodeList to an Object[] (check GetType()), but the length is preserved. The dynamically generated $myDoc.Root.Child property (which essentially does not exist) returns $null. When $null is #'ed, it becomes an array of length 1.
On Functions...
The subtleties of processing pipeline input in a function with respect to using $_ or $input and with respect to the begin, process, and end blocks.
How to handle the six principal equivalence classes of input delivered to a function (no input, null, empty string, scalar, list, list with null and/or empty) -- for both direct input and pipeline input -- and get what you expect.
The correct calling syntax for sending multiple arguments to a function.
I discuss these points and more at length in my Simple-Talk.com article Down the Rabbit Hole- A Study in PowerShell Pipelines, Functions, and Parameters and also provide an accompanying wallchart--here is a glimpse showing the various calling syntax pitfalls for a function taking 3 arguments:
On Modules...
These points are expounded upon in my Simple-Talk.com article Further Down the Rabbit Hole: PowerShell Modules and Encapsulation.
Dot-sourcing a file inside a script using a relative path is relative to your current directory -- not the directory where the script resides!
To be relative to the script use this function to locate your script directory: [Update for PowerShell V3+: Just use the builtin $PSScriptRoot variable!]
function Get-ScriptDirectory
{ Split-Path $script:MyInvocation.MyCommand.Path }
Modules must be stored as ...Modules\name\name.psm1 or ...\Modules\any_subpath\name\name.psm1. That is, you cannot just use ...Modules\name.psm1 -- the name of the immediate parent of the module must match the base name of the module. This chart shows the various failure modes when this rule is violated:
2015.06.25 A Pitfall Reference Chart
Simple-Talk.com just published the last of my triumvirate of in-depth articles on PowerShell pitfalls. The first two parts are in the form of a quiz that helps you appreciate a select group of pitfalls; the last part is a wallchart (albeit it would need a rather high-ceilinged room) containing 36 of the most common pitfalls (some adapted from answers on this page), giving concrete examples and workarounds for most. Read more here.
There are some tricks to building command lines for utilities that were not built with Powershell in mind:
To run an executable who's name starts with a number, preface it with an Ampersand (&).
& 7zip.exe
To run an executable with a space anywhere in the path, preface it with an Ampersand (&) and wrap it in quotes, as you would any string. This means that strings in a variable can be executed as well.
# Executing a string with a space.
& 'c:\path with spaces\command with spaces.exe'
# Executing a string with a space, after first saving it in a variable.
$a = 'c:\path with spaces\command with spaces.exe'
& $a
Parameters and arguments are passed to legacy utilities positionally. So it is important to quote them the way the utility expects to see them. In general, one would quote when it contains spaces or does not start with a letter, number or dash (-).
C:\Path\utility.exe '/parameter1' 'Value #1' 1234567890
Variables can be used to pass string values containing spaces or special characters.
$b = 'string with spaces and special characters (-/&)'
utility.exe $b
Alternatively array expansion can be used to pass values as well.
$c = #('Value #1', $Value2)
utility.exe $c
If you want Powershell to wait for an application to complete, you have to consume the output, either by piping the output to something or using Start-Process.
# Saving output as a string to a variable.
$output = ping.exe example.com | Out-String
# Piping the output.
ping stackoverflow.com | where { $_ -match '^reply' }
# Using Start-Process affords the most control.
Start-Process -Wait SomeExecutable.com
Because of the way they display their output, some command line utilities will appear to hang when ran inside of Powershell_ISE.exe, particularly when awaiting input from the user. These utilities will usually work fine when ran within Powershell.exe console.
PowerShell Gotchas
There are a few pitfall that repeatedly reappear on StackOverflow. It is recommend to do some research if you are not familiar with these PowerShell gotchas before asking a new question. It might even be a good idea to investigate in these PowerShell gotchas before answering a PowerShell question to make sure that you teach the questioner the right thing.
TLDR: In PowerShell:
the comparison equality operator is: -eq
(Stackoverflow example: Powershell simple syntax if condition not working)
parentheses and commas are not used with arguments
(Stackoverflow example: How do I pass multiple parameters into a function in PowerShell?)
output properties are based on the first object in the pipeline
(Stackoverflow example: Not all properties displayed)
the pipeline unrolls
(Stackoverflow example: Pipe complete array-objects instead of array items one at a time?)
a. single item collections
(Stackoverflow example: Powershell ArrayList turns a single array item back into a string)
b. embedded arrays
(Stackoverflow example: Return Multidimensional Array From Function)
c. output collections
(Stackoverflow example: Why does PowerShell flatten arrays automatically?)
$Null should be on the left side of the equality comparison operator
(Stackoverflow example: Should $null be on the left side of the equality comparison)
parentheses and assignments choke the pipeline
(Stackoverflow example: Importing 16MB CSV Into Variable Creates >600MB's Memory Usage)
the increase assignment operator (+=) might become expensive
Stackoverflow example: Improve the efficiency of my PowerShell scrip
The Get-Content cmdlet returns separate lines
Stackoverflow example: Multiline regex to match config block
Examples and explanations
Some of the gotchas might really feel counter-intuitive but often can be explained by some very nice PowerShell features along with the pipeline, expression/argument mode and type casting.
1. The comparison equality operator is: -eq
Unlike the Microsoft scripting language VBScript and some other programming languages, the comparison equality operator differs from the assignment operator (=) and is: -eq.
Note: assigning a value to a variable might pass through the value if needed:
$a = $b = 3 # The value 3 is assigned to both variables $a and $b.
This implies that following statement might be unexpectedly truthy or falsy:
If ($a = $b) {
# (assigns $b to $a and) returns a truthy if $b is e.g. 3
} else {
# (assigns $b to $a and) returns a falsy if $b is e.g. 0
}
2. Parentheses and commas are not used with arguments
Unlike a lot of other programming languages and the way a primitive PowerShell function is defined, calling a function doesn't require parentheses or commas for their related arguments. Use spaces to separate the parameter arguments:
MyFunction($Param1, $Param2 $Param3) {
# ...
}
MyFunction 'one' 'two' 'three' # assigns 'one' to $Param1, 'two' to $Param2, 'three' to $Param3
Parentheses and commas are used for calling (.Net) methods.
Commas are used to define arrays. MyFunction 'one', 'two', 'three' (or MyFunction('one', 'two', 'three')) will load the array #('one', 'two', 'three') into the first parameter ($Param1).
Parentheses will intepret the containing contents as a single collection into memory (and choke the PowerShell pipeline) and should only be used as such, e.g. to call an embedded function, e.g.:
MyFunction (MyOtherFunction) # passes the results MyOtherFunction to the first positional parameter of MyFunction ($Param1)
MyFunction One $Two (getThree) # assigns 'One' to $Param1, $Two to $Param2, the results of getThree to $Param3
Note: that quoting text arguments (as the word one in the later example) is only required when it contains spaces or special characters.
3. Output properties are based on the first object in the pipeline
In a PowerShell pipeline each object is processed and passed on by a cmdlet (that is implemented for the middle of a pipeline) similar to how objects are processed and passed on by workstations in an assembly line. Meaning each cmdlet processes one item at the time while the prior cmdlet (workstation) simultaneously processes the upcoming one. This way, the objects aren't loaded into memory at once (less memory usage) and could already be processed before the next one is supplied (or even exists). The disadvantage of this feature is that there is no oversight of what (or how many) objects are expected to follow.
Therefore most PowerShell cmdlets assume that all the objects in the pipeline correspond to the first one and have the same properties which is usually the case, but not always...
$List =
[pscustomobject]#{ one = 'a1'; two = 'a2' },
[pscustomobject]#{ one = 'b1'; two = 'b2'; three = 'b3' }
$List |Select-Object *
one two
--- ---
a1 a2
b1 b2
As you see, the third column three is missing from the results as it didn't exists in the first object and the PowerShell was already outputting the results prior it was aware of the exists of the second object.
On way to workaround this behavior is to explicitly define the properties (of all the following objects) at forehand:
$List |Select-Object one, two, three
one two three
--- --- -----
a1 a2
b1 b2 b3
See also proposal: #13906 Add -UnifyProperties parameter to Select-Object
4. The pipeline unrolls
This feature might come in handy if it complies with the straightforward expectation:
$Array = 'one', 'two', 'three'
$Array.Length
3
a. single item collections
But it might get confusing:
$Selection = $Array |Select-Object -First 2
$Selection.Length
2
$Selection[0]
one
when the collection is down to a single item:
$Selection = $Array |Select-Object -First 1
$Selection.Length
3
$Selection[0]
o
Explanation
When the pipeline outputs a single item which is assigned to a variable, it is not assigned as a collection (with 1 item, like: #('one')) but as a scalar item (the item itself, like: 'one').
Which means that the property .Length (which is in fact an alias for the property .Count for an array) is no longer applied on the array but on the string: 'one'.length which equals 3. And in case of the index $Selection[0] , the first character of the string 'one'[0] (which equals the character o) is returned .
Workaround
To workaround this behavior, you might force the scalar item to an array using the Array subexpression operator #( ):
$Selection = $Array |Select-Object -First 1
#($Selection).Length
1
#($Selection)[0]
one
Knowing that in the case the $Selection is already an array, it will will not be further increased in depth (#(#('one', 'two')), see the next section 4b. Embedded collections are flattened).
b. embedded arrays
When an array (or a collection) includes embedded arrays, like:
$Array = #(#('a', 'b'), #('c', 'd'))
$Array.Count
2
All the embedded items will be processed in the pipeline and consequently returns a flat array when displayed or assigned to a new variable:
$Processed = $Array |ForEach-Object { $_ }
$Processed.Count
4
$Processed
a
b
c
d
To iterate the embedded arrays, you might use the foreach statement:
foreach ($Item in $Array) { $Item.Count }
2
2
Or a simply for loop:
for ($i = 0; $i -lt $Array.Count; $i++) { $Array[$i].Count }
2
2
c. output collections
Collections are usually unrolled when they are placed on the pipeline:
function GetList {
[Collections.Generic.List[String]]#('a', 'b')
}
(GetList).GetType().Name
Object[]
To output the collection as a single item, use the comma operator ,:
function GetList {
,[Collections.Generic.List[String]]#('a', 'b')
}
(GetList).GetType().Name
List`1
5. $Null should be on the left side of the equality comparison operator
This gotcha is related to this comparison operators feature:
When the input of an operator is a scalar value, the operator returns a Boolean value. When the input is a collection, the operator returns the elements of the collection that match the right-hand value of the expression. If there are no matches in the collection, comparison operators return an empty array.
This means for scalars:
'a' -eq 'a' # returns $True
'a' -eq 'b' # returns $False
'a' -eq $Null # returns $False
$Null -eq $Null # returns $True
and for collections, the matching elements are returned which evaluates to either a truthy or falsy condition:
'a', 'b', 'c' -eq 'a' # returns 'a' (truthy)
'a', 'b', 'c' -eq 'd' # returns an empty array (falsy)
'a', 'b', 'c' -eq $Null # returns an empty array (falsy)
'a', $Null, 'c' -eq $Null # returns $Null (falsy)
'a', $Null, $Null -eq $Null # returns #($Null, $Null) (truthy!!!)
$Null, $Null, $Null -eq $Null # returns #($Null, $Null, $Null) (truthy!!!)
In other words, to check whether a variable is $Null (and exclude a collection containing multiple $Nulls), put $Null at the LHS (left hand side) of the equality comparison operator:
if ($Null -eq $MyVariable) { ...
6. Parentheses and assignments choke the pipeline
The PowerShell Pipeline is not just a series of commands connected by pipeline operators (|) (ASCII 124). It is a concept to simultaneously stream individual objects through a sequence of cmdlets. If a cmdlet (or function) is written according to the Strongly Encouraged Development Guidelines and implemented for the middle of a pipeline, it takes each single object from the pipeline, processes it and passes the results to the next cmdlet just before it takes and processes the next object in the pipeline. Meaning that for a simple pipeline as:
Import-Csv .\Input.csv |Select-Object -Property Column1, Column2 |Export-Csv .\Output.csv
As the last cmdlet writes an object to the .\Output.csv file, the Select-Object cmdlet selects the properties of the next object and the Import-Csv reads the next object from the .\input.csv file (see also: Pipeline in Powershell). This will keep the memory usage low (especially where there are lots of object/records to process) and therefore might result in a faster throughput. To facilitate the pipeline, the PowerShell objects are quiet fat as each individual object contains all the property information (along with e.g. the property name).
Therefore it is not a good practice to choke the pipeline for no reason. There are two senarios that choke the pipeline:
Parentheses, e.g.:
(Import-Csv .\Input.csv) |Select-Object -Property Column1, Column2 |Export-Csv .\Output.csv
Where all the .\Input.csv records are loaded as an array of PowerShell objects into memory before passing it on to the Select-Object cmdlet.
Assignments, e.g.:
$Objects = Import-Csv .\Input.csv
$Objects |Select-Object -Property Column1, Column2 |Export-Csv .\Output.csv
Where all the .\Input.csv records are loaded as an array of PowerShell objects into $Objects (memory as well) before passing it on to the Select-Object cmdlet.
7. the increase assignment operator (+=) might become expensive
The increase assignment operator (+=) is syntactic sugar to increase and assign primitives as .e.g. $a += $b where $a is assigned $b + 1. The increase assignment operator can also be used for adding new items to a collection (or to String types and hash tables) but might get pretty expensive as the costs increases with each iteration (the size of the collection). The reason for this is that objects as array collections are immutable and the right variable in not just appended but *appended and reassigned to the left variable. For details see also: avoid using the increase assignment operator (+=) to create a collection
8. The Get-Content cmdlet returns separate lines
There are probably quite some more cmdlet gotchas, knowing that there exist a lot of (internal and external) cmdlets. In contrast to engine related gotchas, these gotchas are often easier to highlight (with e.g. a warning) as happend with ConvertTo-Json (see: Unexpected ConvertTo-Json results? Answer: it has a default -Depth of 2) or "fix". But there is very clasic gotcha in Get-Content which tight into the PowerShell general concept of streaming objects (in this case lines) rather than passing everything (the whole contents of the file) in once:
Get-Content .\Input.txt -Match '\r?\n.*Test.*\r?\n'
Will never work because, by default, Get-Contents returns a stream of objects where each object contains a single string (a line without any line breaks).
(Get-Content .\Input.txt).GetType().Name
Object[]
(Get-Content .\Input.txt)[0].GetType().Name
String
In fact:
Get-Content .\Input.txt -Match 'Test'
Returns all the lines with the word Test in it as Get-Contents puts every single line on the pipeline and when the input is a collection, the operator returns the elements of the collection that match the right-hand value of the expression.
Note: since PowerShell version 3, Get-Contents has a -Raw parameter that reads all the content of the concerned file at once, Meaning that this: Get-Content -Raw .\Input.txt -Match '\r?\n.*Test.*\r?\n' will work as it loads the whole file into memory.
alex2k8, I think this example of yours is good to talk about:
# -----------------------------------
function foo($a){
# I thought this is right.
#if($a -eq $null)
#{
# throw "You can't pass $null as argument."
#}
# But actually it should be:
if($null -eq $a)
{
throw "You can't pass $null as argument."
}
}
foo #($null, $null)
PowerShell can use some of the comparators against arrays like this:
$array -eq $value
## Returns all values in $array that equal $value
With that in mind, the original example returns two items (the two $null values in the array), which evalutates to $true because you end up with a collection of more than one item. Reversing the order of the arguments stops the array comparison.
This functionality is very handy in certain situations, but it is something you need to be aware of (just like array handling in PowerShell).
Functions 'foo' and 'bar' looks equivalent.
function foo() { $null }
function bar() { }
E.g.
(foo) -eq $null
# True
(bar) -eq $null
# True
But:
foo | %{ "foo" }
# Prints: foo
bar | %{ "bar" }
# PRINTS NOTHING
Returning $null and returning nothing is not equivalent dealing with pipes.
This one is inspired by Keith Hill example...
function bar() {}
$list = #(foo)
$list.length
# Prints: 0
# Now let's try the same but with a temporal variable.
$tmp = foo
$list = #($tmp)
$list.length
# Prints: 1
Another one:
$x = 2
$y = 3
$a,$b = $x,$y*5
because of operators precedence there is not 25 in $b; the command is the same as ($x,$y)*5
the correct version is
$a,$b = $x,($y*5)
The logical and bitwise operators don't follow standard precedence rules. The operator -and should have a higher priority than -or yet they're evaluated strictly left-to-right.
For example, compare logical operators between PowerShell and Python (or virtually any other modern language):
# PowerShell
PS> $true -or $false -and $false
False
# Python
>>> True or False and False
True
...and bitwise operators:
# PowerShell
PS> 1 -bor 0 -band 0
0
# Python
>>> 1 | 0 & 0
1
This works. But almost certainly not in the way you think it's working.
PS> $a = 42;
PS> [scriptblock]$b = { $a }
PS> & $b
42
This one has tripped me up before, using $o.SomeProperty where it should be $($o.SomeProperty).
# $x is not defined
[70]: $x -lt 0
True
[71]: [int]$x -eq 0
True
So, what's $x..?
Another one I ran into recently: [string] parameters that accept pipeline input are not strongly typed in practice. You can pipe anything at all and PS will coerce it via ToString().
function Foo
{
[CmdletBinding()]
param (
[parameter(Mandatory=$True, ValueFromPipeline=$True)]
[string] $param
)
process { $param }
}
get-process svchost | Foo
Unfortunately there is no way to turn this off. Best workaround I could think of:
function Bar
{
[CmdletBinding()]
param (
[parameter(Mandatory=$True, ValueFromPipeline=$True)]
[object] $param
)
process
{
if ($param -isnot [string]) {
throw "Pass a string you fool!"
}
# rest of function goes here
}
}
edit - a better workaround I've started using...
Add this to your custom type XML -
<?xml version="1.0" encoding="utf-8" ?>
<Types>
<Type>
<Name>System.String</Name>
<Members>
<ScriptProperty>
<Name>StringValue</Name>
<GetScriptBlock>
$this
</GetScriptBlock>
</ScriptProperty>
</Members>
</Type>
</Types>
Then write functions like this:
function Bar
{
[CmdletBinding()]
param (
[parameter(Mandatory=$True, ValueFromPipelineByPropertyName=$True)]
[Alias("StringValue")]
[string] $param
)
process
{
# rest of function goes here
}
}
Forgetting that $_ gets overwritten in blocks made me scratch my head in confusion a couple times, and similarly for multiple reg-ex matches and the $matches array. >.<
Remembering to explicitly type pscustom objects from imported data tables as numeric so they can be sorted correctly:
$CVAP_WA=foreach ($i in $C){[PSCustomObject]#{ `
County=$i.county; `
TotalVote=[INT]$i.TotalBallots; `
RegVoters=[INT]$i.regvoters; `
Turnout_PCT=($i.TotalBallots/$i.regvoters)*100; `
CVAP=[INT]($B | ? {$_.GeoName -match $i.county}).CVAP_EST }}
PS C:\Politics> $CVAP_WA | sort -desc TotalVote |ft -auto -wrap
County TotalVote RegVoters Turnout_PCT CVAP CVAP_TV_PCT CVAP_RV_PCT
------ --------- --------- ----------- ---- ----------- -----------
King 973088 1170638 83.189 1299290 74.893 90.099
Pierce 349377 442985 78.86 554975 62.959 79.837
Snohomish 334354 415504 80.461 478440 69.832 86.81
Spokane 227007 282442 80.346 342060 66.398 82.555
Clark 193102 243155 79.453 284190 67.911 85.52
Mine are both related to file copying...
Square Brackets in File Names
I once had to move a very large/complicated folder structure using Move-Item -Path C:\Source -Destination C:\Dest. At the end of the process there were still a number of files in source directory. I noticed that every remaining file had square brackets in the name.
The problem was that the -Path parameter treats square brackets as wildcards.
EG. If you wanted to copy Log001 to Log200, you could use square brackets as follows:
Move-Item -Path C:\Source\Log[001-200].log.
In my case, to avoid square brackets being interpreted as wildcards, I should have used the -LiteralPath parameter.
ErrorActionPreference
The $ErrorActionPreference variable is ignored when using Move-Item and Copy-Item with the -Verbose parameter.
Treating the ExitCode of a Process as a Boolean.
eg, with this code:
$p = Start-Process foo.exe -NoNewWindow -Wait -PassThru
if ($p.ExitCode) {
# handle error
}
things are good, unless say foo.exe doesn't exist or otherwise fails to launch.
in that case $p will be $null, and [bool]($null.ExitCode) is False.
a simple fix is to replace the logic with if ($p.ExitCode -ne 0) {},
however for clarity of code imo the following is better: if (($p -eq $null) -or ($p.ExitCode -ne 0)) {}