Powershell. Combine properties of objects in array - powershell

I'm not very good at powershell and I need help.
I am trying to combine properties of objects. For two objects, everything is fine. This is looks like.
$obj1 = [PSCustomObject]#{
'ip' = '10.10.10.11'
't1' = 'show'
't2' = 'ab'
}
$obj2 = [PSCustomObject]#{
'ip' = '10.10.10.11'
't1' = 'me'
't2' = 'cd'
}
foreach ($prop in $obj2 | Get-Member -MemberType NoteProperty | Where-Object {$_.name -ne 'ip'} | select -ExpandProperty name)
{
$obj1.$prop += $obj2.$prop
}
$obj1
Result:
ip t1 t2
-- -- --
10.10.10.11 showme abcd
But I can't do this if many objects in an array. How to do it?
Example arrays:
Array1:
ip t1 t2
-- -- --
10.10.10.11 show ab
10.10.10.12 show ab
Array2:
ip t1 t2
-- -- --
10.10.10.11 me cd
10.10.10.12 me cd

Basically, all you need is to add an outer loop that iterates over the array elements.
Additionally, you can use the hidden .psobject.Properties member to streamline your solution:
# Sample arrays
$array1 = [pscustomobject]#{ 'ip' = '10.10.10.11'; 't1' = 'show1'; 't2' = 'ab1' },
[pscustomobject]#{ 'ip' = '10.10.10.12'; 't1' = 'show2'; 't2' = 'ab2' }
$array2 = [pscustomobject]#{ 'ip' = '10.10.10.11'; 't1' = 'me1'; 't2' = 'cd1' },
[pscustomobject]#{ 'ip' = '10.10.10.12'; 't1' = 'me2'; 't2' = 'cd2' }
# Assuming all objects in the arrays have the same property structure,
# get the list of property names up front.
$mergeProperties = $array1[0].psobject.Properties.Name -ne 'ip'
# Loop over the array elements in pairs and merge their property values.
$i = 0
[array] $mergedArray =
foreach ($o1 in $array1) {
$o2 = $array2[$i++]
foreach ($p in $mergeProperties) {
$o1.$p += $o2.$p
}
$o1 # output the merged object
}
Outputting $mergedArray then yields:
ip t1 t2
-- -- --
10.10.10.11 show1me1 ab1cd1
10.10.10.12 show2me2 ab2cd2
Note:
As in your attempt, one of the original objects is modified in place.
The assumption is that matching objects in the two arrays are in the same position, respectively (merge the first object from the first array with the first object from the second, ....).

Related

Convert multiple txt file to one single csv file [duplicate]

When we're trying to export data to other functions via the pipeline, we observe some strange behavior in PowerShell.
Example code:
$Array = #()
$Obj1 = [PSCustomObject]#{
Member1 = 'First'
Member2 = 'Second'
}
$Obj2 = [PSCustomObject]#{
Member1 = 'First'
Member2 = 'Second'
Member3 = 'Third'
}
$Array = $Obj1, $Obj2
$Array | Out-GridView -Title 'Not showing Member3'
$Array = $Obj2, $Obj1
$Array | Out-GridView -Title 'All members correctly displayed'
In the example above you can see that when the first object only contains 2 properties, the Out-GridView CmdLet (and others) only show 2 properties, even though the second object has 3 properties. However, when the first object in the array has 3 properties it does display them all correctly.
Is there a way around this? Because it's not possible to predict up front how many properties on an object there will be and if the object with the most properties will be the first one in the array.
I had the same experience once and created the following reusable 'Union' function:
# 2021-08-25 Removed Union function
Usage:
$Obj1, $Obj2 | Union | Out-GridView -Title 'Showing all members'
It is also supposed to work with complex objects. Some standard cmdlets output multiple object types at once and if you view them (e.g. Out-GridView) or dump them in a file (e.g. Export-Csv) you might miss a lot of properties. Take as another example:
Get-WmiObject -Namespace root/hp/instrumentedBIOS -Class hp_biosSetting | Union | Export-Csv ".\HPBIOS.csv"
Added 2014-09-19:
Maybe this is already between the lines in the comments $Array | Select * | … will not resolve the issue but specifically selecting the properties $Array | Select Member1, Member2, Member3 | … does.
Besides, although in most cases the Union function will work, there are some exceptions to that as it will only align the first object with the rest.
Consider the following object:
$List = #(
New-Object PSObject -Property #{Id = 2}
New-Object PSObject -Property #{Id = 1}
New-Object PSObject -Property #{Id = 3; Name = "Test"}
)
If you Union this object everything appears to be fine and if you e.g. ExportTo-CSV and work with the export .csv file from then on you will never have any issue.
$List | Union
Id Name
-- ----
2
1
3 Test
Still there is a catch as only the first object is aligned. If you e.g. sort the result on Id (Sort Id) or take just the last 2 (Select -Last 2) entries, the Name is not listed because the second object doesn’t contain the Name property:
$List | Union | Sort Id
Id
--
1
2
3
Therefor I have rewritten the Union-Object (Alias Union) function`):
Union-Object
# 2021-08-25 Removed Union-Object function
Syntax:
$Array | Union | Out-GridView -Title 'All members correctly displayed'
Update 2021-08-25
Based on az1d helpful feedback on an error caused by equal property names with different casing, I have created a new UnifyProperties function.
(I will no longer use the name UnionObject for his)
function UnifyProperties {
$Names = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase)
$InputCollected = #($Input)
$InputCollected.ForEach({
foreach ($Name in $_.psobject.Properties.Name) { $Null = $Names.Add($Name) }
})
$inputCollected | Select-Object #($Names)
}
Usage:
[pscustomobject] #{ one = 1; two = 2; three = 3 },
[pscustomobject] #{ ONE = 10; THREE = 30; FOUR = 4 } |
UnifyProperties
one two three FOUR
--- --- ----- ----
1 2 3
10 30 4
See also: #13906 Add -UnifyProperties parameter to Select-Object

Select some properties of an array of object and rename it

I have an array of objects, which have the property of p1, p2, p3, p4.
$objectArray = [PSCustomObject]#{
p1 = "Value1"; p2 = "Value2"; p3 = "Value3"; p4 = "Value4"
},[PSCustomObject]#{
p1 = "Value5"; p2 = "Value6"; p3 = "Value7"; p4 = "Value8"
}
And I have a property-name-mapping hashtable, for example
$propertyReplacements = #{
"p1" = "x1"
"p2" = "y1"
}
How to select only the properties which exist in the keys of hash table $propertyReplacements; and rename the property name to the values in the hash table?
The result will be
[PSCustomObject]#{
x1 = "Value1"; y1 = "Value2";
},[PSCustomObject]#{
x1 = "Value5"; y1 = "Value6";
}
The properties filtering can be
$objectArray | select -Property #($propertyReplacements.Keys) |
....
The following uses an ordered dictionary and casting [pscustomobject] to create new objects, I really don't think Select-Object is the proper tool for this.
# use a temporary ordered dictionary
$tmp = [ordered]#{}
# enumerate each object
foreach($obj in $objectArray) {
# store the PSMemberInfoIntegratingCollection for later access
$properties = $obj.PSObject.Properties
# enumerate the hash keys
foreach($key in $propertyReplacements.Keys) {
# if the hash key exists as property Name in this object
if($prop = $properties.Item($key)) {
# create a new property Name using the Value of the hashtable
# and keep the current Value of the value
$tmp[$propertyReplacements[$prop.Name]] = $prop.Value
}
}
# create a new object casting this type accelerator
[pscustomobject] $tmp
# and clear the dictionary for the next object
$tmp.Clear()
}
Here is another solution
foreach($k in $propertyReplacements.Keys) {
if ($k -ne $propertyReplacements.$k) {
$objectArray = $objectArray | select *,{n=$propertyReplacements.$k; $e={$_.$k}}
$objectArray = $objectArray | select -ExcludeProperty $k
}
}
# $objectArray has updated property names now

Inner Join in PowerShell (without SQL)

How do we make Inner-Join or something a Cross-Join in PowerShell or PowerCLI?
Even though im new to PowerCLI/PowerShell , I do have a basic grasp on them, yet have practically spent 2 days trying to figure this, going through numerous documentations and blogs to no avail.
All I really want to know is if after typing my command
Get-Content File.txt
and getting:
Output1 or Table1 is
Name: Abc
Group: Bad
Policy: Great
Name: redi
Group: Good
Policy: MAD
etc. etc.
100s of these, and obviously more than just the 3 elements of Name, Group, Policy each.
Table2/Output2
Name: Abc
Limit: 10
used: 5
Name: redi
Limit: 20
used: 1
etc. etc.
100s of these.
and like 13 more of these text file tables, all with the "Name" as unique.
How can I combine it into one output at the end using Name with all the other elements?
My most obvious thought was something akin to joins, even if I had to do them 1 at a time, but even that I cant figure out how to do.
Is there anyway to do this in PowerShell itself without me having to go into Python or SQL?
If yes is there a method that is able to combine fields in spots where it's null?
If its not clear what type of result I am hoping for it will look something akin to this:
Name: Abc
Group: Bad
Policy: Great
Limit: 10
used: 5
Name: redi
Group: Good
Policy: MAD
Limit: 20
used: 1
Paweł Dyl provided you a solution
based on your two tables. However you probably need a generic solution where you don't have to specify each property by name yourself.
I would combine each table to a an array. Group the tables on the Name property using the Group-Object cmdlet. Iterate over each group and create a PsObject using the properties:
$table1 = [PSCustomObject]#{ Name = 'Abc'; Group = 'Bad'; Policy = 'Great'}, [PSCustomObject]#{ Name = 'redi'; Group = 'Good'; Policy = 'MAD'}
$table2 = [PSCustomObject]#{ Name = 'Abc'; Limit = '10'; used = '5'}, [PSCustomObject]#{ Name = 'redi'; Limit = '20'; used = '1'}
$allTables = $table1 + $table2
$allTables | group Name | Foreach {
$properties = #{}
$_.Group | Foreach {
$_.PsObject.Properties | Where Name -ne 'Name' | Foreach {
$properties += #{
"$($_.Name)" = "$($_.Value)"
}
}
}
$properties += #{Name = $_.Name}
New-Object PSObject –Property $properties
}
Output:
Group : Bad
Policy : Great
Name : Abc
Limit : 10
used : 5
Group : Good
Policy : MAD
Name : redi
Limit : 20
used : 1
You can use simple loop join as follows:
$table1 = [pscustomobject]#{Name='Abc';Group='Bad';Policy='Great'},[pscustomobject]#{Name='redi';Group='Good ';Policy='MAD'}
$table2 = [pscustomobject]#{Name='Abc';Limit=10;used=5},[pscustomobject]#{Name='redi';Limit=20;used=1}
$table1 | % {
foreach ($t2 in $table2) {
if ($_.Name -eq $t2.Name) {
[pscustomobject]#{Name=$_.Name;Group=$_.Group;Policy=$_.Policy;Limit=$t2.Limit;Used=$t2.Used}
}
}
}
Assuming uniqueness of keys you can also use faster, hashtable approach:
$hashed = $table1 | group Name -AsHashTable
$table2 | % {
$matched = $hashed[$_.Name]
if ($matched) {
[pscustomobject]#{Name=$matched.Name;Group=$matched.Group;Policy=$matched.Policy;Limit=$_.Limit;Used=$_.Used}
}
}
You can also use generic solution and wrap it in function. It matches records by their property names:
function Join-Records($tab1, $tab2){
$prop1 = $tab1 | select -First 1 | % {$_.PSObject.Properties.Name} #properties from t1
$prop2 = $tab2 | select -First 1 | % {$_.PSObject.Properties.Name} #properties from t2
$join = $prop1 | ? {$prop2 -Contains $_}
$unique1 = $prop1 | ?{ $join -notcontains $_}
$unique2 = $prop2 | ?{ $join -notcontains $_}
if ($join) {
$tab1 | % {
$t1 = $_
$tab2 | % {
$t2 = $_
foreach ($prop in $join) {
if (!$t1.$prop.Equals($t2.$prop)) { return; }
}
$result = #{}
$join | % { $result.Add($_,$t1.$_) }
$unique1 | % { $result.Add($_,$t1.$_) }
$unique2 | % { $result.Add($_,$t2.$_) }
[PSCustomObject]$result
}
}
}
}
$table1 = [pscustomobject]#{Name='Abc';Group='Bad';Policy='Great'},
[pscustomobject]#{Name='redi';Group='Good ';Policy='MAD'},
[pscustomobject]#{Name='Not joined';Group='Very bad';Policy='Great'}
$table2 = [pscustomobject]#{Name='Abc';Limit=10;used=5},
[pscustomobject]#{Name='redi';Limit=20;used=1},
[pscustomobject]#{Name='redi';Limit=20;used=2}
#name is only common property, records joined by name
Join-Records $table1 $table2
#example2
$test1 = [pscustomobject]#{A=1;B=1;C='R1'},
[pscustomobject]#{A=1;B=2;C='R2'},
[pscustomobject]#{A=2;B=2;C='R3'}
$test2 = [pscustomobject]#{A=1;B=1;D='R4'},
[pscustomobject]#{A=3;B=2;D='R5'},
[pscustomobject]#{A=4;B=2;D='R6'}
Join-Records $test1 $test2 #joined by two common columns - A and B
You can also cascade calls:
$test1 = [pscustomobject]#{A=1;B=1;C='R1'},
[pscustomobject]#{A=1;B=2;C='R2'},
[pscustomobject]#{A=2;B=2;C='R3'}
$test2 = [pscustomobject]#{A=1;B=1;D='R4'},
[pscustomobject]#{A=3;B=2;D='R5'},
[pscustomobject]#{A=4;B=2;D='R6'}
$test3 = [pscustomobject]#{B=1;E='R7'},
[pscustomobject]#{B=2;E='R8'},
[pscustomobject]#{B=3;E='R9'}
#first join by common A and B, then join result by common B
Join-Records (Join-Records $test1 $test2) $test3
So I found an Answer which was more suitable and it uses the join-Object function which was defined below:
you can access it at https://github.com/RamblingCookieMonster/PowerShell/blob/master/Join-Object.ps1
All I really had to do was Define my outputs as $A and $B and $C and so on, and just
$Join1= Join-Object -Left $A -Right $B -LeftJoinProperty Name - RightJoinProperty Name
made $Join2 then 3 so on until I got it all done
$Join2 = Join-Object -Left $Join1 -Right $C -LeftJoinProperty Name -RightJoinProperty Name
$Join3 = Join-Object -Left $Join2 -Right $D -LeftJoinProperty Name -RightJoinProperty Name
$Join4 = Join-Object -Left $Join3 -Right $E -LeftJoinProperty Name -RightJoinProperty Name
Until I got it all done
$Table1 | Join $Table2 -Using Name
$Table1 | Join $Table2 #Cross Join
See: In Powershell, what's the best way to join two tables into one?

Powershell v2.0 substitute null values from a Hash table

I have a hash table as below:
$Hash = #{
Team1=$Team1.count
Team2=$Team2.count
Team3=$Team3.count
}
$GroupByTeam = New-Object psobject -Property $Hash |
Select 'Team1','Team2','Team3' | ConvertTo-Html -Fragment
This is fine and each "team" returns their own value. However, teams may have a null value and I wish to substitute this for "0".
In an attempt to work this out, I have tried to select the null value first but can't seem to do this:
$Hash.values | select -property Values
Values
------
{1, 2}
But
$Hash.values | select -property Values | where {$_.Values is $null}
doesn't pull back anything. Also tried:
$Hash.values | select -expandproperty Values | where {$_.Values is $null}
Any ideas?
thanks
Your best option is to cast the values to int when creating the hashtable:
$Hash = #{
Team1 = [int]$Team1.Count
Team2 = [int]$Team2.Count
Team3 = [int]$Team3.Count
}
If that's not possible for some reason you could go with an enumerator:
($Hash.GetEnumerator()) | ForEach-Object {
if ($_.Value -eq $null) { $Hash[$_.Name] = 0 }
}
or (as Mathias suggested) use the Keys property to the same end:
($Hash.Keys) | ForEach-Object {
if ($Hash[$_] -eq $null) { $Hash[$_] = 0 }
}
Note that either way you need to use a subexpression (or assign the enumerated objects/keys to a variable) otherwise you'll get an error because you're modifying a data structure while it's being enumerated.
What you'll want to do is collect the keys that refer to null values, and then populate those with 0s:
# Create and populate hashtable
$HashTable = #{
Team1 = 123
Team2 = $null
Team3 = 456
}
# Find keys of `$null` values
$nullKeys = $HashTable.Keys |Where-Object { $HashTable[$_] -eq $null }
# Populate appropriate indices with 0
$nullKeys |ForEach-Object { $HashTable[$_] = 0 }

Looping through a hash, or using an array in PowerShell

I'm using this (simplified) chunk of code to extract a set of tables from SQL Server with BCP.
$OutputDirectory = 'c:\junk\'
$ServerOption = "-SServerName"
$TargetDatabase = "Content.dbo."
$ExtractTables = #(
"Page"
, "ChecklistItemCategory"
, "ChecklistItem"
)
for ($i=0; $i -le $ExtractTables.Length – 1; $i++) {
$InputFullTableName = "$TargetDatabase$($ExtractTables[$i])"
$OutputFullFileName = "$OutputDirectory$($ExtractTables[$i])"
bcp $InputFullTableName out $OutputFullFileName -T -c $ServerOption
}
It works great, but now some of the tables need to be extracted via views, and some don't. So I need a data structure something like this:
"Page" "vExtractPage"
, "ChecklistItemCategory" "ChecklistItemCategory"
, "ChecklistItem" "vExtractChecklistItem"
I was looking at hashes, but I'm not finding anything on how to loop through a hash. What would be the right thing to do here? Perhaps just use an array, but with both values, separated by space?
Or am I missing something obvious?
Shorthand is not preferred for scripts; it is less readable. The %{} operator is considered shorthand. Here's how it should be done in a script for readability and reusability:
Variable Setup
PS> $hash = #{
a = 1
b = 2
c = 3
}
PS> $hash
Name Value
---- -----
c 3
b 2
a 1
Option 1: GetEnumerator()
Note: personal preference; syntax is easier to read
The GetEnumerator() method would be done as shown:
foreach ($h in $hash.GetEnumerator()) {
Write-Host "$($h.Name): $($h.Value)"
}
Output:
c: 3
b: 2
a: 1
Option 2: Keys
The Keys method would be done as shown:
foreach ($h in $hash.Keys) {
Write-Host "${h}: $($hash.$h)"
}
Output:
c: 3
b: 2
a: 1
Additional information
Be careful sorting your hashtable...
Sort-Object may change it to an array:
PS> $hash.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Hashtable System.Object
PS> $hash = $hash.GetEnumerator() | Sort-Object Name
PS> $hash.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
This and other PowerShell looping are available on my blog.
Christian's answer works well and shows how you can loop through each hash table item using the GetEnumerator method. You can also loop through using the keys property. Here is an example how:
$hash = #{
a = 1
b = 2
c = 3
}
$hash.Keys | % { "key = $_ , value = " + $hash.Item($_) }
Output:
key = c , value = 3
key = a , value = 1
key = b , value = 2
You can also do this without a variable
#{
'foo' = 222
'bar' = 333
'baz' = 444
'qux' = 555
} | % getEnumerator | % {
$_.key
$_.value
}
I prefer this variant on the enumerator method with a pipeline, because you don't have to refer to the hash table in the foreach (tested in PowerShell 5):
$hash = #{
'a' = 3
'b' = 2
'c' = 1
}
$hash.getEnumerator() | foreach {
Write-Host ("Key = " + $_.key + " and Value = " + $_.value);
}
Output:
Key = c and Value = 1
Key = b and Value = 2
Key = a and Value = 3
Now, this has not been deliberately sorted on value, the enumerator simply returns the objects in reverse order.
But since this is a pipeline, I now can sort the objects received from the enumerator on value:
$hash.getEnumerator() | sort-object -Property value -Desc | foreach {
Write-Host ("Key = " + $_.key + " and Value = " + $_.value);
}
Output:
Key = a and Value = 3
Key = b and Value = 2
Key = c and Value = 1
Here is another quick way, just using the key as an index into the hash table to get the value:
$hash = #{
'a' = 1;
'b' = 2;
'c' = 3
};
foreach($key in $hash.keys) {
Write-Host ("Key = " + $key + " and Value = " + $hash[$key]);
}
About looping through a hash:
$Q = #{"ONE"="1";"TWO"="2";"THREE"="3"}
$Q.GETENUMERATOR() | % { $_.VALUE }
1
3
2
$Q.GETENUMERATOR() | % { $_.key }
ONE
THREE
TWO
A short traverse could be given too using the sub-expression operator $( ), which returns the result of one or more statements.
$hash = #{ a = 1; b = 2; c = 3}
forEach($y in $hash.Keys){
Write-Host "$y -> $($hash[$y])"
}
Result:
a -> 1
b -> 2
c -> 3
If you're using PowerShell v3, you can use JSON instead of a hashtable, and convert it to an object with Convert-FromJson:
#'
[
{
FileName = "Page";
ObjectName = "vExtractPage";
},
{
ObjectName = "ChecklistItemCategory";
},
{
ObjectName = "ChecklistItem";
},
]
'# |
Convert-FromJson |
ForEach-Object {
$InputFullTableName = '{0}{1}' -f $TargetDatabase,$_.ObjectName
# In strict mode, you can't reference a property that doesn't exist,
#so check if it has an explicit filename firest.
$outputFileName = $_.ObjectName
if( $_ | Get-Member FileName )
{
$outputFileName = $_.FileName
}
$OutputFullFileName = Join-Path $OutputDirectory $outputFileName
bcp $InputFullTableName out $OutputFullFileName -T -c $ServerOption
}