Powershell Add-Members Dynamically - powershell

Long story short - it seems impossible to get a pipeline value to resolve to plain text in a ScriptProperty and so I'm looking for a way to either do that, or to somehow parameterize when I'm adding properties to an object. I would be perfectly happy to add the values as NotePropertyMembers rather than a ScriptProperty; but I can't find a way to do that without looping through the entire object multiple times. The advantage of the ScriptProperty is the ability to use the $this variable so that I don't need to call Add-Member on each member of the object itself.
Here's what an example of what I'm trying to accomplish and where it is breaking down.
# This sample file contains the values that I'm interested in
#'
ID,CATEGORY,AVERAGE,MEDIAN,MIN,MAX
100,"",52,50,10,100
100,1,40,40,20,60
100,2,41,35,15,85
'# > values.csv
# To access the values, I decided to create a HashTable
$map = #{}
(Import-Csv values.csv).ForEach({$map[$_.ID + $_.CATEGORY] = $_})
# This is my original object. In reality, it has several additional columns of data - but it is
# missing the above values which is why I want to pull them in
#'
ID,CATEGORY
100,""
100,1
100,2
'# > object.csv
$object = Import-Csv object.csv
# This is the meat of my attempt. Loop through all of the properties in the HashTable that
# I need to pull into my object and add them
$map.Values | Get-Member -MemberType NoteProperty | Where-Object {$_.Name -NotIn 'ID', 'CATEGORY'} | ForEach-Object {$object | Add-Member -MemberType ScriptProperty -Name "$($_.Name)" -Value {$map[$this.ID + $this.CATEGORY]."$($_.Name)"}}
In theory, I know that it works because I can display the values for a particular member of my object using the code below
$map.Values | Get-Member -MemberType NoteProperty | Where-Object {$_.Name -NotIn 'ID', 'CATEGORY'} | ForEach-Object {"$($_.Name) = " + $map[$object[0].ID + $object[0].CATEGORY]."$($_.Name)"}
Displays
AVERAGE = 52
MAX = 100
MEDIAN = 50
MIN = 10
However - it is clear that $_.Name and "$($_.Name)" do not resolve to their plain text description when they're in the ScriptProperty
PS> $object | GM -MemberType ScriptProperty
# Displays
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
AVERAGE ScriptProperty System.Object AVERAGE {get=$map[$this.ID + $this.CATEGORY]."$($_.Name)";}
MAX ScriptProperty System.Object MAX {get=$map[$this.ID + $this.CATEGORY]."$($_.Name)";}
MEDIAN ScriptProperty System.Object MEDIAN {get=$map[$this.ID + $this.CATEGORY]."$($_.Name)";}
MIN ScriptProperty System.Object MIN {get=$map[$this.ID + $this.CATEGORY]."$($_.Name)";}
To me, an obvious workaround is to individually add each column which still allows me to take advantage of using Add-Member on the object itself instead of each individual member. However, the entire idea is to be able to dynamically add values without knowing ahead of time what their names are - in order to do that, I need to find a way to force the name to resolve within the ScriptProperty
# Workaround for this incredibly simple example
PS> $object | Add-Member -MemberType ScriptProperty -Name AVERAGE -Value {$map[$this.ID + $this.CATEGORY]."AVERAGE"}
PS> $object | Add-Member -MemberType ScriptProperty -Name MEDIAN -Value {$map[$this.ID + $this.CATEGORY]."MEDIAN"}
PS> $object | Add-Member -MemberType ScriptProperty -Name MIN -Value {$map[$this.ID + $this.CATEGORY]."MIN"}
PS> $object | Add-Member -MemberType ScriptProperty -Name MAX -Value {$map[$this.ID + $this.CATEGORY]."MAX"}
PS> $object | GM -MemberType ScriptProperty
# Displays
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
AVERAGE ScriptProperty System.Object AVERAGE {get=$map[$this.ID + $this.CATEGORY]."AVERAGE";}
MAX ScriptProperty System.Object MAX {get=$map[$this.ID + $this.CATEGORY]."MAX";}
MEDIAN ScriptProperty System.Object MEDIAN {get=$map[$this.ID + $this.CATEGORY]."MEDIAN";}
MIN ScriptProperty System.Object MIN {get=$map[$this.ID + $this.CATEGORY]."MIN";}

Complementing the helpful answer from Mathias which creates new objects, here is how you can approach dynamically updating the object itself:
$exclude = 'ID', 'CATEGORY'
$map = #{}
$properties = $values[0].PSObject.Properties.Name | Where-Object { $_ -notin $exclude }
$values.ForEach{ $map[$_.ID + $_.CATEGORY] = $_ | Select-Object $properties }
foreach($line in $object) {
$newValues = $map[$line.ID + $line.CATEGORY]
foreach($property in $properties) {
$line.PSObject.Properties.Add(
[psnoteproperty]::new($property, $newValues.$property)
)
}
}
Above code assumes that both Csvs ($values and $object) are loaded in memory and that you know which properties from the $values Csv should be excluded.

Use Select-Object instead of Add-Member:
$object |Select *,#{Name='Average';Expression={$map[$_.ID + $_.Category].Average}},#{Name='Median';Expression={$map[$_.ID + $_.Category].Median}},#{Name='Min';Expression={$map[$_.ID + $_.Category].Min}},#{Name='Max';Expression={$map[$_.ID + $_.Category].Max}}
Select-Object will evaluate the Expression block against every single individual input item, thereby resolving the mapping immediately, as opposed to deferring it until someone references the property

One additional option that I stumbled upon while working with Santiago's great solution. Using Add-Member rather than the Properties.Add() method would allow for a hash table of NotePropertyMembers to used, which could be advantageous although I haven't started testing performance of different methods yet. It makes things a bit simpler if you already have a hash table of the properties and values that you want to add to your object.
$map = #{}
# This will create a hash table nested in the hash table, rather than a hash table whose values are PSCustomObjects
$values.ForEach{$map[$_.ID + $_.CATEGORY] = $_.PSObject.Properties | ?{$_.Name -notin $exclude} | % {$ht = #{}} {$ht[$_.Name] = $_.Value} {$ht}}
ForEach($line in $object){
If($map.Contains($line.ID + $line.CATEGORY){
$line | Add-Member -NotePropertyMembers $map[$line.ID + $line.CATEGORY]
}
}

Santiago's answer and your own, based on static NoteProperty members rather than the originally requested dynamic ScriptProperty members, turned out to be the better solution in the case at hand.
As for how you could have made your dynamic ScriptProperty approach work:
$map.Values |
Get-Member -MemberType NoteProperty |
Where-Object Name -NotIn ID, CATEGORY |
ForEach-Object {
# Cache the property name at hand, so it can be referred to
# in the pipeline below.
$propName = $_.Name
# .GetNewClosure() is necessary for capturing the value of $amp
# as well as the current value of $propName as part of the script block.
$object |
Add-Member -MemberType ScriptProperty `
-Name $propName `
-Value { $map[$this.ID + $this.CATEGORY].$propName }.GetNewClosure()
}
Note that use of .GetNewClosure() is crucial, so as to capture:
each iteration's then-current $propName variable value
the $map variable's hashtable
as part of the script block.
Note:
.GetNewClosure() is expensive, as it creates a dynamic module behind the scenes in which the captured values are cached and to which the script block gets bound.
Generally, it's better to restrict ScriptProperty members to calculations that are self-contained, i.e. rely only on the state of the object itself (accessible via automatic $this variable), not also on outside values; a simple example:
$o = [pscustomobject] #{ A = 1; B = 2}
$o | Add-Member -Type ScriptProperty -Name C { $this.A + $this.B }
$o.C # -> 3
$o.A = 2; $o.C # -> 4

Related

Powershell How to extract hashtable value from nested hashtable

I've made an API request to the ManageEngine tech support database which returns a JSON response which is returned as a custom object of three hash tables: $result.response_status, $result.list_info, and $result.requests The third of these contains the data I'm looking for.
$results.requests.id an integer
$results.requests.subject a string
$results.requester a hashtable.
The requester hash table is #requester{name= 'Martin Zimmerman'; email='mzimmer#company.com'}
What I'd like to be able to do is:
$results.requests | select id, subject, requester.name
to get a single line displaying the id, subject and requester's name
id subject name
-- -------------- --------------
3329 Can't open file Martin Zimmerman
However, I cannot figure out the nomenclature to extract the value of the name key in the requester hash table.
I think this is what you want. You need to build a Calculated Property (Example 10).
$results.requests | Select-Object id,subject,#{l='name';e={$_.requester['name']}}
This approach allows you to call the name key from the hashtable.
EDIT
If your requester item is a PSCustomObject, then try this.
$results.requests | Select-Object id,subject,#{l='name';e={$_.requester.name}}
Test with similar object structure.
$results = New-Object psobject
$requests = New-Object psobject
$requests | Add-Member -MemberType NoteProperty -Name id -Value 2
$requests | Add-Member -MemberType NoteProperty -Name subject -Value "lol"
$hash = #{}
$hash.Add("email","mzimmerman#company.com")
$hash.Add("name","Martin Zimmerman")
$requests | Add-Member -MemberType NoteProperty -Name requester -Value $hash
$results | Add-Member -MemberType NoteProperty -Name requests -Value $requests
$results.requests | select id,subject,#{l='name';e={$_.requester['name']}}
id subject name
-- ------- ----
2 lol Martin Zimmerman

Powershell PSCustomObject Extracting values

When creating and working with PSCustomObjects which result in a NoteProperty member with a 'Definition' (as shown below), is there any easy, programmatic way to select the values from the definition fields without resorting to splitting the strings?
For example below, is there a 'good' way to extract the value 'silver' from the field of name 'token' that does not requre traditional string manipulations? I've been messing around with select and -ExpandProperty but getting no-where fast and would appreciate a nudge in the right direction.
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
bsw NoteProperty decimal bsw=3.14
name NoteProperty string name=chris
token NoteProperty string token=silver
volume NoteProperty decimal volume=17.22
Thanks.
Update: Following guidance from Thomas I came up with this function to extract Noteproperty members from a PSObject and return a Hashtable with the names of the fields and the values:
function convertObjectToHash($psObj) {
$hashBack = #{}
try {
$psObjFieldNames = $psObj | get-member -type NoteProperty | select "Name"
$psObjFieldNames | foreach-object {
$hashBack.Add($_.Name,$psObj.$($_.Name)) }
}catch{ "Error: $_" }
return $hashBack
}
Thanks!
You can access the members of a custom object like in any other object:
$myCustomObject.token
Reproduction:
$myCustomObject = New-Object -TypeName psobject
$myCustomObject | Add-Member -MemberType NoteProperty -Name bsw -Value 3.14
$myCustomObject | Add-Member -MemberType NoteProperty -Name name -Value "chris"
$myCustomObject | Add-Member -MemberType NoteProperty -Name token -Value "silver"
$myCustomObject | Add-Member -MemberType NoteProperty -Name volume -Value 17.22
$myCustomObject | Get-Member -MemberType NoteProperty
$myCustomObject.token
Output:
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
bsw NoteProperty System.Double bsw=3.14
name NoteProperty string name=chris
token NoteProperty string token=silver
volume NoteProperty System.Double volume=17.22
silver
Since in your objects you store a string value, I see no other way to extract just the value silver then to use a string method to get only the part after the equals sign.
($obj.token -split '=', 2)[-1] --> silver
Why not create the custom objects with an extra property called value and add the value you seek in there, taken from the Definition property? like
$obj = [PsCustomObject]#{'token' = 'string token=silver'; 'value' = 'silver'}

Correct way to add a string property to existing Powershell object?

So for example, I have a custom object for $server. I want to create a new property, $server.output to contain multi line output of varying size. What is the best approach to take? There doesn't seem to be any straight forward information on the web around how to add simple string properties to objects?
FYI I am using powershell v4, but it would be nice to have a solution for v5 too.
Outside of re-creating the object or ensuring the property exists on object initialization, you have to use Add-Member to create new properties on a pscustomobject:
# alternatively, use `-Force` on `Add-Member`
if (-not ($obj | Get-Member -Name output))
{
$obj | Add-Member -MemberType NoteProperty -Name output -Value ''
}
Supported in v3+:
$obj | Add-Member -NotePropertyName output -NotePropertyValue '' -Force
Add multiple members:
$obj | Add-Member -NotePropertyMembers #{output = ''; output1 = ''} -Force
Add-Member

Dealing with (members of) heterogeneous arrays

Let's create books
$a = New-Object –TypeName PSObject
$a | Add-Member –MemberType NoteProperty –Name Title –Value "Journey to the West"
$a | Add-Member –MemberType NoteProperty –Name Price –Value 12
$b = New-Object –TypeName PSObject
$b | Add-Member –MemberType NoteProperty –Name Title –Value "Faust"
$b | Add-Member –MemberType NoteProperty –Name Author –Value "Goethe"
$array1 = $a,$b
$array2 = $b,$a
Now let's display these two arrays
PS D:\Developpement\Powershell> $array1
Title Price
----- -----
Journey to the West 12
Faust
PS D:\Developpement\Powershell> $array2
Title Author
----- ------
Faust Goethe
Journey to the West
So as far as I understand this basically means that what powershell consider to be properties of an array are the properties of its first element (in fact that's not even true because if the first element is $null the next one will be considered). Now that also implies that :
if you call Get-Member on the array, you will only get members of the first element
if you call Convert-ToCvs on the array, you will only export property values for properties defined by the first element
etc
I hardly understand the rationals behind that and this behaviour has made it infuriatingly painful for me to work with heterogeneous arrays in powershell.
I'd like to import data from various external sources, process them and then export them to a cvs file. Items are similar but most of them miss some properties unpredictably. Is there any obvious way to handle that in Powershell without reprogramming the wheel?
This is the way it has to be because PowerShell uses pipelines. When you run ex. $array1 | Export-CSV ...., PowerShell starts to write to the CSV-file as soon as the first object arrives. At that point it needs to know what the header will look like as that is the first line in a csv-file. So PowerShell has to assume that the class/properties of the first object represents all the remaining objects in the pipeline. The same goes for Format-Table and similar commands that need to set a style/view before outputting any objects.
The usual workaround to this is to specify the header manually using Select-Object. It will add all missing properties to all objects with a value of $null. This way, all the objects sent to ex. Export-CSV will have all the same properties defined.
To get the header, you need to receive all unique property-names from all objects in your array. Ex.
$array1 |
ForEach-Object { $_.PSObject.Properties} |
Select-Object -ExpandProperty Name -Unique
Title
Price
Author
Then you can specify that as the header using Select-Object -Properties Title,Price,Author before sending the objects to Export-CSV Ex:
$a = New-Object –TypeName PSObject
$a | Add-Member –MemberType NoteProperty –Name Title –Value "Journey to the West"
$a | Add-Member –MemberType NoteProperty –Name Price –Value 12
$b = New-Object –TypeName PSObject
$b | Add-Member –MemberType NoteProperty –Name Title –Value "Faust"
$b | Add-Member –MemberType NoteProperty –Name Author –Value "Goethe"
$array = $a,$b
$AllProperties = $array |
ForEach-Object { $_.PSObject.Properties} |
Select-Object -ExpandProperty Name -Unique
$array | Select-Object -Property $AllProperties | Export-CSV -Path "mycsv.out" -NoTypeInformation
This will create this CSV-file:
"Title","Price","Author"
"Journey to the West","12",
"Faust",,"Goethe"
If you have mulltiple arrays you can combine them like this $array = $array1 + $array2

Method to export objects with varying properties?

I hit a common problem with my scripting lately and decided to throw it into the wild to see how other people deal with this problem.
tl;dr; I want to export objects which have a varying number of properties. eg; object 1 may have 3 IP address but object 2 has 7 IP addresses.
I've evolved to creating a new object with custom properties and then injecting these objects into an array as my method of catching results - happy to hear if there is a better way but this is how I roll. This method works 100% when outputting to the screen as the objects are shown in list format - I've tried a number of export/out-file methods to no avail when I want to capture and store the output for reading in something like Excell.
The following is an example of me building an object and storing it (the function of the code is not important here - just the results it generates):
add-pssnapin Quest.ActiveRoles.ADManagement
$Groups = get-qadgroup AmbigousGroupNameHere-
$UserInfo = #()
ForEach ( $Group in $Groups ) {
$CurrentGroupMembers = get-qadgroupmember $Group
write-host "Processing group:" $Group
ForEach ( $GroupMember in $CurrentGroupMembers ) {
If ( $GroupMember.type -eq "User" ) {
$counter = 1
write-host "Processing member:" $GroupMember
$UserObject = get-qaduser $GroupMember | select SamAccountName,FirstName,LastName
$objUserInfo = New-Object System.Object
$objUserInfo | Add-Member -MemberType NoteProperty -Name SamAccountName -Value $UserObject.SamAccountName
$objUserInfo | Add-Member -MemberType NoteProperty -Name FirstName -Value $UserObject.FirstName
$objUserInfo | Add-Member -MemberType NoteProperty -Name LastName -Value $UserObject.LastName
$GroupMembership = get-qadgroup -ContainsMember $GroupMember | where name -like "AmbigousGroupNameHere-*"
ForEach ( $GroupName in $GroupMembership ) {
$objUserInfo | Add-Member -MemberType NoteProperty -Name CtxGroup$counter -Value $GroupName.SamAccountName
$counter++
}
$UserInfo += $objUserInfo
} else {
write-host "This is a group - we are ignoring it."
}
}
}
$UserInfo | Export-Csv UsersOutput.csv -NoType
From the above - you can see I scale the object property name by 1 for each group. CtxGroup$counter allows me to scale an object for the correct number of groups each user has. Confirmed this works great when outputting to the screen by default. The object is listed and I can see a new property for each group that matches for that user.
Now for the problem. When I export-csv or out-file the file is generated with enough headers based off the first object - so it creates the headings based on the amount of properties the first object has. So lets say the first user has 3 matching groups, it will create heading CtxGroup1, CtxGroup2, CtxGroup3. Great! No.
If the next user has 5 matching groups - only the first three are included in the output and the additional 2 are discarded as we don't have headings for CtxGroup4, CtxGroup5.
How on earth do other people deal with this?
side note; I considered creating my first object as a dummy with a massive amount of object (and hence headings) but well - that is not cool and really makes me feel inefficient.
You can obtain what you want ordering $UserInfo array by the number of properties, it can be done, but it's not so simple, in your case I would add another propertie with the count of groups added:
...
ForEach ( $GroupName in $GroupMembership ) {
$objUserInfo | Add-Member -MemberType NoteProperty -Name CtxGroup$counter -Value $GroupName.SamAccountName
$counter++
}
$objUserInfo | Add-Member -MemberType NoteProperty -Name NbCtxGroup -Value ($counter - 1)
$UserInfo += $objUserInfo
...
And then order descending the array on this propertie :
$UserInfo | Sort-Object -Property NbCtxGroup -Descending | Export-Csv .\F.csv
It's not so nice but it will do the trick.
Have a look at :
$objUserInfo.psobject.Properties | Measure-Object