Powershell: Filter Hashtable - and get back a Hastable - powershell

Filtering a Hashtable using GetEnumerator always returns a object[] instead of a Hashtable:
# Init Hashtable
$items = #{ a1 = 1; a2 = 2; b1 = 3; b2 = 4}
# apply a filter
$filtered = $items.GetEnumerator() | ?{ $_.Key -match "a.*" }
# The result looks great
$filtered
Name Value
---- -----
a2 2
a1 1
# … but it is not a Hashtable :-(
$filtered.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
Is there a nice solution to this problem?
Thanks a lot for any Help!,
kind regards,
Tom

$filtered is an array of dictionary entries. There's no single cast or ctor for this as far as I know.
You can construct a hash though:
$hash = #{}
$filtered | ForEach-Object { $hash.Add($_.Key, $_.Value) }
Another workflow:
# Init Hashtable
$items = #{ a1 = 1; a2 = 2; b1 = 3; b2 = 4}
# Copy keys to an array to avoid enumerating them directly on the hashtable
$keys = #($items.Keys)
# Remove elements not matching the expected pattern
$keys | ForEach-Object {
if ($_ -notmatch "a.*") {
$items.Remove($_)
}
}
# $items is filtered

Here's an even simpler function, it even has include and exclude functionality
function Select-HashTable {
[CmdletBinding()]
param (
[Parameter(Mandatory,ValueFromPipeline)][Hashtable]$Hashtable,
[String[]]$Include = ($HashTable.Keys),
[String[]]$Exclude
)
if (-not $Include) {$Include = $HashTable.Keys}
$filteredHashTable = #{}
$HashTable.keys.where{
$PSItem -in $Include
}.where{
$PSItem -notin $Exclude
}.foreach{
$filteredHashTable[$PSItem] = $HashTable[$PSItem]
}
return $FilteredHashTable
}
Examples:
$testHashtable = #{a=1;b=2;c=3;d=4}
$testHashTable | Select-HashTable -Include a
Name Value
---- -----
a 1
$testHashTable | Select-HashTable -Exclude b
Name Value
---- -----
c 3
d 4
a 1
$testHashTable | Select-HashTable -Include a,b,c -Exclude b
Name Value
---- -----
a 1
c 3

As the accepted answer was resulting in a BadEnumeration exception for me (but still worked), I modified it to not throw an exception and also made sure that the original HashTable is not modified by cloning it first:
# Init Hashtable
$items = #{ a1 = 1; a2 = 2; b1 = 3; b2 = 4}
$filtered = $items.Clone()
$items.Keys | ForEach-Object {
if ($_ -notmatch "a.*") {
$filtered.Remove($_)
}
}

On a modern PowerShell (5+ as far as I remember) you can use reduce pattern. For that you need to use this form of ForEach-Object:
$Hashtable.Keys | ForEach-Object {$FilteredHashtable = #{}} {
if ($_ -eq 'Example') {
$FilteredHashtable[$_] = $Hashtable[$_];
}
} {$FilteredHashtable}
Yes, this snippet will return Hashtable.

Related

How to display all properties with Format-Table cmdlet

I have few [pscustomobject] objects that can have not all properties.
For example:
PS> $1 = [pscustomobject]#{ A='a1'; B='b1' }
PS> $2 = [pscustomobject]#{ A='a2'; C='c2' }
And I try to display all properties with Format-Table like this:
PS> $1,$2 | Format-Table
A B
- -
a1 b1
a2
PS> $2,$1 | Format-Table
A C
- -
a2 c2
a1
But every time it displays only properties from first object in collection.
I want to display all properties like if I set -Property argument explicitly.
PS> $1,$2 | Format-Table -Property A,B,C
A B C
- - -
a1 b1
a2 c2
Setting -Property argument is good if:
All set of properties is known in advance
Collection is small and I can get all properties with Get-Member -MemberType Properties
But I have a huge collection (above 10000 objects) with unknown properties so I need help with it.
REMARK: Format-Table will be used only for small slices (10-100 elements).
For that, you can use below function to merge all properties into the first object:
function Complete-ObjectHeaders {
# function to add properties to the first item in a collection of PSObjects
# when this object is missing properties from items further down the array.
# you may need this if you have such a collection and want to export it
# to Csv, since Export-Csv (and also Format-Table) only looks at the FIRST
# item to create the csv column headers.
[CmdletBinding()]
param (
[Parameter(Mandatory = $true, Position = 0)]
[PSObject[]]$Collection,
[int]$MaxItemsToTest = -1, # < 0 --> test all items in the collection
[switch]$SortHeaders
)
# Try and find all headers by looping over the items in the collection.
# The headers will be captured in the order in which they are found.
if ($MaxItemsToTest -gt 0) {
$MaxItemsToTest = [math]::Min($MaxItemsToTest, $Collection.Count)
$headers = for($i = 0; $i -lt $MaxItemsToTest; $i++) {
($Collection[$i].PSObject.Properties).Name
}
$headers = $headers | Select-Object -Unique
}
else {
$headers = $Collection | ForEach-Object {($_.PSObject.Properties).Name} | Select-Object -Unique
}
if ($SortHeaders) { $headers = $headers | Sort-Object }
# update the first object in the collection to contain all headers
$Collection[0] = $Collection[0] | Select-Object $headers
,$Collection
}
Use like this:
$1 = [pscustomobject]#{ A='a1'; B='b1' }
$2 = [pscustomobject]#{ A='a2'; C='c2' }
# just output to console
Complete-ObjectHeaders -Collection $1,$2 | Format-Table -AutoSize
# or capture the merged array of objects in a new variable you can save as CSV file for instance
$merged = Complete-ObjectHeaders -Collection $1,$2
$merged | Export-Csv -Path 'D:\Test\Merged.csv' -NoTypeInformation
Output:
A B C
- - -
a1 b1
a2 c2
Thanks #theo for the answer.
I used it to write my own version of a function that supports pipelining.
function Expand-Properties {
[Cmdletbinding()]
param(
[Parameter(ValueFromPipeline)]
$InputObject,
[Parameter()]
[Alias('All')]
[switch]
$ExpandAll,
[Parameter()]
[switch]
$SortHeaders
)
begin {
$collection = [System.Collections.ArrayList]::new()
$properties = [System.Collections.ArrayList]::new()
}
process {
[void]$collection.Add($InputObject)
$properties.AddRange((($InputObject.PSObject.Properties).Name))
}
end {
if ($SortHeaders) {
$properties = $properties | Sort-Object -Unique
} else {
$properties = $properties | Select-Object -Unique
}
if ($ExpandAll) {
for ($i = 0; $i -lt $collection.Count; ++$i) {
$collection[$i] = $collection[$i] | Select-Object -Property $properties
}
} else {
$collection[0] = $collection[0] | Select-Object -Property $properties
}
$collection
}
}
EXAMPLE:
PS> $1 = [pscustomobject]#{ A='a1'; B='b1' }
PS> $2 = [pscustomobject]#{ A='a2'; C='c2' }
PS> $1, $2 | Expand-Properties
A B C
- - -
a1 b1
a2 c2

In powershell, how to test in an array already contains an object with all the same properties?

I want to avoid inserting duplicates into an array in powershell. Trying to use -notcontains doesn't seem to work with a PSCUstomObject array.
Here's a code sample
$x = [PSCustomObject]#{
foo = 111
bar = 222
}
$y = [PSCustomObject]#{
foo = 111
bar = 222
}
$collection = #()
$collection += $x
if ($collection -notcontains $y){
$collection += $y
}
$collection.Count #Expecting to only get 1, getting 2
I would use Compare-Object for this.
$x = [PSCustomObject]#{
foo = 111
bar = 222
}
$y = [PSCustomObject]#{
foo = 111
bar = 222
}
$collection = [System.Collections.Arraylist]#()
[void]$collection.Add($x)
if (Compare-Object -Ref $collection -Dif $y -Property foo,bar | Where SideIndicator -eq '=>') {
[void]$collection.Add($y)
}
Explanation:
Comparing a custom object to another using comparison operators is not trivial. This solution compares the particular properties you care about (foo and bar in this case). This can be done simply with Compare-Object, which will output differences in either object by default. The SideIndicator value of => indicates the difference lies in the object passed into the -Difference parameter.
The [System.Collections.Arraylist] type is used over an array to avoid the inefficient += typically seen when growing an array. Since the .Add() method produces an output of the index being modified, the [void] cast is used to suppress that output.
You could be dynamic with the solution regarding the properties. You may not want to hard code the property names into the Compare-Object command. You can do something like the following instead.
$x = [PSCustomObject]#{
foo = 111
bar = 222
}
$y = [PSCustomObject]#{
foo = 111
bar = 222
}
$collection = [System.Collections.Arraylist]#()
[void]$collection.Add($x)
$properties = $collection[0] | Get-Member -MemberType NoteProperty |
Select-Object -Expand Name
if (Compare-Object -Ref $collection -Dif $y -Property $properties | Where SideIndicator -eq '=>') {
[void]$collection.Add($y)
}

Inner Join in PowerShell (without SQL)

How do we make Inner-Join or something a Cross-Join in PowerShell or PowerCLI?
Even though im new to PowerCLI/PowerShell , I do have a basic grasp on them, yet have practically spent 2 days trying to figure this, going through numerous documentations and blogs to no avail.
All I really want to know is if after typing my command
Get-Content File.txt
and getting:
Output1 or Table1 is
Name: Abc
Group: Bad
Policy: Great
Name: redi
Group: Good
Policy: MAD
etc. etc.
100s of these, and obviously more than just the 3 elements of Name, Group, Policy each.
Table2/Output2
Name: Abc
Limit: 10
used: 5
Name: redi
Limit: 20
used: 1
etc. etc.
100s of these.
and like 13 more of these text file tables, all with the "Name" as unique.
How can I combine it into one output at the end using Name with all the other elements?
My most obvious thought was something akin to joins, even if I had to do them 1 at a time, but even that I cant figure out how to do.
Is there anyway to do this in PowerShell itself without me having to go into Python or SQL?
If yes is there a method that is able to combine fields in spots where it's null?
If its not clear what type of result I am hoping for it will look something akin to this:
Name: Abc
Group: Bad
Policy: Great
Limit: 10
used: 5
Name: redi
Group: Good
Policy: MAD
Limit: 20
used: 1
Paweł Dyl provided you a solution
based on your two tables. However you probably need a generic solution where you don't have to specify each property by name yourself.
I would combine each table to a an array. Group the tables on the Name property using the Group-Object cmdlet. Iterate over each group and create a PsObject using the properties:
$table1 = [PSCustomObject]#{ Name = 'Abc'; Group = 'Bad'; Policy = 'Great'}, [PSCustomObject]#{ Name = 'redi'; Group = 'Good'; Policy = 'MAD'}
$table2 = [PSCustomObject]#{ Name = 'Abc'; Limit = '10'; used = '5'}, [PSCustomObject]#{ Name = 'redi'; Limit = '20'; used = '1'}
$allTables = $table1 + $table2
$allTables | group Name | Foreach {
$properties = #{}
$_.Group | Foreach {
$_.PsObject.Properties | Where Name -ne 'Name' | Foreach {
$properties += #{
"$($_.Name)" = "$($_.Value)"
}
}
}
$properties += #{Name = $_.Name}
New-Object PSObject –Property $properties
}
Output:
Group : Bad
Policy : Great
Name : Abc
Limit : 10
used : 5
Group : Good
Policy : MAD
Name : redi
Limit : 20
used : 1
You can use simple loop join as follows:
$table1 = [pscustomobject]#{Name='Abc';Group='Bad';Policy='Great'},[pscustomobject]#{Name='redi';Group='Good ';Policy='MAD'}
$table2 = [pscustomobject]#{Name='Abc';Limit=10;used=5},[pscustomobject]#{Name='redi';Limit=20;used=1}
$table1 | % {
foreach ($t2 in $table2) {
if ($_.Name -eq $t2.Name) {
[pscustomobject]#{Name=$_.Name;Group=$_.Group;Policy=$_.Policy;Limit=$t2.Limit;Used=$t2.Used}
}
}
}
Assuming uniqueness of keys you can also use faster, hashtable approach:
$hashed = $table1 | group Name -AsHashTable
$table2 | % {
$matched = $hashed[$_.Name]
if ($matched) {
[pscustomobject]#{Name=$matched.Name;Group=$matched.Group;Policy=$matched.Policy;Limit=$_.Limit;Used=$_.Used}
}
}
You can also use generic solution and wrap it in function. It matches records by their property names:
function Join-Records($tab1, $tab2){
$prop1 = $tab1 | select -First 1 | % {$_.PSObject.Properties.Name} #properties from t1
$prop2 = $tab2 | select -First 1 | % {$_.PSObject.Properties.Name} #properties from t2
$join = $prop1 | ? {$prop2 -Contains $_}
$unique1 = $prop1 | ?{ $join -notcontains $_}
$unique2 = $prop2 | ?{ $join -notcontains $_}
if ($join) {
$tab1 | % {
$t1 = $_
$tab2 | % {
$t2 = $_
foreach ($prop in $join) {
if (!$t1.$prop.Equals($t2.$prop)) { return; }
}
$result = #{}
$join | % { $result.Add($_,$t1.$_) }
$unique1 | % { $result.Add($_,$t1.$_) }
$unique2 | % { $result.Add($_,$t2.$_) }
[PSCustomObject]$result
}
}
}
}
$table1 = [pscustomobject]#{Name='Abc';Group='Bad';Policy='Great'},
[pscustomobject]#{Name='redi';Group='Good ';Policy='MAD'},
[pscustomobject]#{Name='Not joined';Group='Very bad';Policy='Great'}
$table2 = [pscustomobject]#{Name='Abc';Limit=10;used=5},
[pscustomobject]#{Name='redi';Limit=20;used=1},
[pscustomobject]#{Name='redi';Limit=20;used=2}
#name is only common property, records joined by name
Join-Records $table1 $table2
#example2
$test1 = [pscustomobject]#{A=1;B=1;C='R1'},
[pscustomobject]#{A=1;B=2;C='R2'},
[pscustomobject]#{A=2;B=2;C='R3'}
$test2 = [pscustomobject]#{A=1;B=1;D='R4'},
[pscustomobject]#{A=3;B=2;D='R5'},
[pscustomobject]#{A=4;B=2;D='R6'}
Join-Records $test1 $test2 #joined by two common columns - A and B
You can also cascade calls:
$test1 = [pscustomobject]#{A=1;B=1;C='R1'},
[pscustomobject]#{A=1;B=2;C='R2'},
[pscustomobject]#{A=2;B=2;C='R3'}
$test2 = [pscustomobject]#{A=1;B=1;D='R4'},
[pscustomobject]#{A=3;B=2;D='R5'},
[pscustomobject]#{A=4;B=2;D='R6'}
$test3 = [pscustomobject]#{B=1;E='R7'},
[pscustomobject]#{B=2;E='R8'},
[pscustomobject]#{B=3;E='R9'}
#first join by common A and B, then join result by common B
Join-Records (Join-Records $test1 $test2) $test3
So I found an Answer which was more suitable and it uses the join-Object function which was defined below:
you can access it at https://github.com/RamblingCookieMonster/PowerShell/blob/master/Join-Object.ps1
All I really had to do was Define my outputs as $A and $B and $C and so on, and just
$Join1= Join-Object -Left $A -Right $B -LeftJoinProperty Name - RightJoinProperty Name
made $Join2 then 3 so on until I got it all done
$Join2 = Join-Object -Left $Join1 -Right $C -LeftJoinProperty Name -RightJoinProperty Name
$Join3 = Join-Object -Left $Join2 -Right $D -LeftJoinProperty Name -RightJoinProperty Name
$Join4 = Join-Object -Left $Join3 -Right $E -LeftJoinProperty Name -RightJoinProperty Name
Until I got it all done
$Table1 | Join $Table2 -Using Name
$Table1 | Join $Table2 #Cross Join
See: In Powershell, what's the best way to join two tables into one?

Powershell - Prefix each line of Format-Table with String

I would like to know if there is an easy way of prefixing each line of a powershell table with a String.
For example, if I create an Array using the following code:
$Array = #()
$Object = #{}
$Object.STR_PARAM = "A"
$Object.INT_PARAM = 1
$Array += [PSCustomObject] $Object
$Object = #{}
$Object.STR_PARAM = "B"
$Object.INT_PARAM = 2
$Array += [PSCustomObject] $Object
Calling Format-Table give the following output:
$Array | Format-Table -AutoSize
STR_PARAM INT_PARAM
--------- ---------
A 1
B 2
Instead, I would like to have the following:
$Array | Format-Table-Custom -AutoSize -PrefixString " "
STR_PARAM INT_PARAM
--------- ---------
A 1
B 2
And if possible, I would also like to be able to use the Property parameter like this:
$SimpleFormat = #{Expression={$_.STR_PARAM}; Label="String Param"},
#{Expression={$_.INT_PARAM}; Label="Integer Param"};
$Array | Format-Table-Custom -Property $SimpleFormat -AutoSize -PrefixString "++"
++String Param Integer Param
++------------ -------------
++A 1
++B 2
Any help would be appreciated. Thanks.
You could just use the format expressions directly:
$f = #{Expression={"++" + $_.STR_PARAM}; Label="++String Param"},
#{Expression={$_.INT_PARAM}; Label="Integer Param"};
$Array | Format-Table $f -AutoSize
Output
++String Param Integer Param
-------------- -------------
++A 1
++B 2
Update to use expression and filter
Filter Format-Table-Custom
{
Param
(
[string]
$PrefixString,
[object]
$Property
)
end {
$rows = $input | Format-Table $property -AutoSize | Out-String
$lines = $rows.Split("`n")
foreach ($line in $lines) {
if ($line.Trim().Length -gt 0) {
$PrefixString + $line
}
}
}
}
$f = #{Expression={"--" + $_.STR_PARAM}; Label="--String Param"},
#{Expression={$_.INT_PARAM}; Label="Integer Param"};
$Array | Format-Table-Custom -Property $f -PrefixString "++"
Output
++--String Param Integer Param
++-------------- -------------
++--A 1
++--B 2

Looping through a hash, or using an array in PowerShell

I'm using this (simplified) chunk of code to extract a set of tables from SQL Server with BCP.
$OutputDirectory = 'c:\junk\'
$ServerOption = "-SServerName"
$TargetDatabase = "Content.dbo."
$ExtractTables = #(
"Page"
, "ChecklistItemCategory"
, "ChecklistItem"
)
for ($i=0; $i -le $ExtractTables.Length – 1; $i++) {
$InputFullTableName = "$TargetDatabase$($ExtractTables[$i])"
$OutputFullFileName = "$OutputDirectory$($ExtractTables[$i])"
bcp $InputFullTableName out $OutputFullFileName -T -c $ServerOption
}
It works great, but now some of the tables need to be extracted via views, and some don't. So I need a data structure something like this:
"Page" "vExtractPage"
, "ChecklistItemCategory" "ChecklistItemCategory"
, "ChecklistItem" "vExtractChecklistItem"
I was looking at hashes, but I'm not finding anything on how to loop through a hash. What would be the right thing to do here? Perhaps just use an array, but with both values, separated by space?
Or am I missing something obvious?
Shorthand is not preferred for scripts; it is less readable. The %{} operator is considered shorthand. Here's how it should be done in a script for readability and reusability:
Variable Setup
PS> $hash = #{
a = 1
b = 2
c = 3
}
PS> $hash
Name Value
---- -----
c 3
b 2
a 1
Option 1: GetEnumerator()
Note: personal preference; syntax is easier to read
The GetEnumerator() method would be done as shown:
foreach ($h in $hash.GetEnumerator()) {
Write-Host "$($h.Name): $($h.Value)"
}
Output:
c: 3
b: 2
a: 1
Option 2: Keys
The Keys method would be done as shown:
foreach ($h in $hash.Keys) {
Write-Host "${h}: $($hash.$h)"
}
Output:
c: 3
b: 2
a: 1
Additional information
Be careful sorting your hashtable...
Sort-Object may change it to an array:
PS> $hash.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Hashtable System.Object
PS> $hash = $hash.GetEnumerator() | Sort-Object Name
PS> $hash.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
This and other PowerShell looping are available on my blog.
Christian's answer works well and shows how you can loop through each hash table item using the GetEnumerator method. You can also loop through using the keys property. Here is an example how:
$hash = #{
a = 1
b = 2
c = 3
}
$hash.Keys | % { "key = $_ , value = " + $hash.Item($_) }
Output:
key = c , value = 3
key = a , value = 1
key = b , value = 2
You can also do this without a variable
#{
'foo' = 222
'bar' = 333
'baz' = 444
'qux' = 555
} | % getEnumerator | % {
$_.key
$_.value
}
I prefer this variant on the enumerator method with a pipeline, because you don't have to refer to the hash table in the foreach (tested in PowerShell 5):
$hash = #{
'a' = 3
'b' = 2
'c' = 1
}
$hash.getEnumerator() | foreach {
Write-Host ("Key = " + $_.key + " and Value = " + $_.value);
}
Output:
Key = c and Value = 1
Key = b and Value = 2
Key = a and Value = 3
Now, this has not been deliberately sorted on value, the enumerator simply returns the objects in reverse order.
But since this is a pipeline, I now can sort the objects received from the enumerator on value:
$hash.getEnumerator() | sort-object -Property value -Desc | foreach {
Write-Host ("Key = " + $_.key + " and Value = " + $_.value);
}
Output:
Key = a and Value = 3
Key = b and Value = 2
Key = c and Value = 1
Here is another quick way, just using the key as an index into the hash table to get the value:
$hash = #{
'a' = 1;
'b' = 2;
'c' = 3
};
foreach($key in $hash.keys) {
Write-Host ("Key = " + $key + " and Value = " + $hash[$key]);
}
About looping through a hash:
$Q = #{"ONE"="1";"TWO"="2";"THREE"="3"}
$Q.GETENUMERATOR() | % { $_.VALUE }
1
3
2
$Q.GETENUMERATOR() | % { $_.key }
ONE
THREE
TWO
A short traverse could be given too using the sub-expression operator $( ), which returns the result of one or more statements.
$hash = #{ a = 1; b = 2; c = 3}
forEach($y in $hash.Keys){
Write-Host "$y -> $($hash[$y])"
}
Result:
a -> 1
b -> 2
c -> 3
If you're using PowerShell v3, you can use JSON instead of a hashtable, and convert it to an object with Convert-FromJson:
#'
[
{
FileName = "Page";
ObjectName = "vExtractPage";
},
{
ObjectName = "ChecklistItemCategory";
},
{
ObjectName = "ChecklistItem";
},
]
'# |
Convert-FromJson |
ForEach-Object {
$InputFullTableName = '{0}{1}' -f $TargetDatabase,$_.ObjectName
# In strict mode, you can't reference a property that doesn't exist,
#so check if it has an explicit filename firest.
$outputFileName = $_.ObjectName
if( $_ | Get-Member FileName )
{
$outputFileName = $_.FileName
}
$OutputFullFileName = Join-Path $OutputDirectory $outputFileName
bcp $InputFullTableName out $OutputFullFileName -T -c $ServerOption
}