In powershell, how to test in an array already contains an object with all the same properties? - powershell

I want to avoid inserting duplicates into an array in powershell. Trying to use -notcontains doesn't seem to work with a PSCUstomObject array.
Here's a code sample
$x = [PSCustomObject]#{
foo = 111
bar = 222
}
$y = [PSCustomObject]#{
foo = 111
bar = 222
}
$collection = #()
$collection += $x
if ($collection -notcontains $y){
$collection += $y
}
$collection.Count #Expecting to only get 1, getting 2

I would use Compare-Object for this.
$x = [PSCustomObject]#{
foo = 111
bar = 222
}
$y = [PSCustomObject]#{
foo = 111
bar = 222
}
$collection = [System.Collections.Arraylist]#()
[void]$collection.Add($x)
if (Compare-Object -Ref $collection -Dif $y -Property foo,bar | Where SideIndicator -eq '=>') {
[void]$collection.Add($y)
}
Explanation:
Comparing a custom object to another using comparison operators is not trivial. This solution compares the particular properties you care about (foo and bar in this case). This can be done simply with Compare-Object, which will output differences in either object by default. The SideIndicator value of => indicates the difference lies in the object passed into the -Difference parameter.
The [System.Collections.Arraylist] type is used over an array to avoid the inefficient += typically seen when growing an array. Since the .Add() method produces an output of the index being modified, the [void] cast is used to suppress that output.
You could be dynamic with the solution regarding the properties. You may not want to hard code the property names into the Compare-Object command. You can do something like the following instead.
$x = [PSCustomObject]#{
foo = 111
bar = 222
}
$y = [PSCustomObject]#{
foo = 111
bar = 222
}
$collection = [System.Collections.Arraylist]#()
[void]$collection.Add($x)
$properties = $collection[0] | Get-Member -MemberType NoteProperty |
Select-Object -Expand Name
if (Compare-Object -Ref $collection -Dif $y -Property $properties | Where SideIndicator -eq '=>') {
[void]$collection.Add($y)
}

Related

Powershell - Search every element of a large array against every element of another large array

I have two large arrays. One is an array (call it Array1) of 100,000 PSCustomObjects, each of which has a property called "Token". And the other array is simply an array of strings, the size of this second array being 2500 elements.
The challenge is that EVERY element of Array1 needs to be checked against all the elements in Array2 and tagged accordingly. i.e., if the the Token value from Array1 matches any of the elements from Array2, label it as "Match found!"
Looping through would actually make it extremely slow. Is there a better way to do this?
P.S.: The items in Array1 have an ordinal number property as well, and the array is sorted in that order.
Here is the code:
$Array1 = #()
$Array2 = #()
#Sample object:
$obj = New-Object -TypeName PSCustomObject
$obj | Add-Member -MemberType NoteProperty -Name Token -Value "SOMEVALUEHERE"
$obj | Add-Member -MemberType NoteProperty -Name TokenOrdinalNum -Value 1
$Array1 += $obj # This array has 100K such objects
$Array2 = #("VAL1", "SOMEVALUEHERE", ......) #Array2 has 2500 such strings.
The output of this would need to be a new array of objects, say 'ArrayFinal', that has an additional noteproperty called 'MatchFound'.
Please help.
I would create a Hashtable for fast lookups from the values in your $Array2.
For clarity, I have renamed $Array1 and $Array2 into $objects and $tokens.
# the object array
$objects = [PsCustomObject]#{ Token = 'SOMEVALUEHERE'; TokenOrdinalNum = 1 },
[PsCustomObject]#{ Token = 'VAL1'; TokenOrdinalNum = 123 },
[PsCustomObject]#{ Token = 'SomeOtherValue'; TokenOrdinalNum = 555 } # etcetera
# the array with token keywords to check
$tokens = 'VAL1', 'SOMEVALUEHERE', 'ShouldNotFindThis' # etcetera
# create a lookup Hashtable from the array of token values for FAST lookup
# you can also use a HashSet ([System.Collections.Generic.HashSet[string]]::new())
# see https://learn.microsoft.com/en-us/dotnet/api/system.collections.generic.hashset-1
$lookup = #{}
$tokens | ForEach-Object { $lookup[$_] = $true } # it's only the Keys that matter, the value is not important
# now loop over the objects in the first array and check their 'Token' values
$ArrayFinal = foreach ($obj in $objects) {
$obj | Select-Object *, #{Name = 'MatchFound'; Expression = { $lookup.ContainsKey($obj.Token) }}
}
# output on screen
$ArrayFinal | Format-Table -AutoSize
# write to Csv ?
$ArrayFinal | Export-Csv -Path 'Path\To\MatchedObjects.csv' -NoTypeInformation
Output:
Token TokenOrdinalNum MatchFound
----- --------------- ----------
SOMEVALUEHERE 1 True
VAL1 123 True
SomeOtherValue 555 False
100kb objects isn't too big. Here's an example using compare-object. By default it checks every object against every other object (919 ms). EDIT: Ok, if I change the order of $b, it takes much longer (13 min). Sorting both lists first should work well, if most of the positions end up the same.(1.99 s with measure-command). If every item were off by 1 position it will still take a long time ($b = 1,$b).
$a = foreach ($i in 1..100kb) { [pscustomobject]#{token = get-random} }
$a = $a | sort-object token
$b = $a.token | sort-object
compare-object $a.token $b -IncludeEqual
InputObject SideIndicator
----------- -------------
1507400001 ==
120471924 ==
28523825 ==
...

Filter ArrayList on nested property

I have an PowerShell ArrayList ($displayObjects) which contains:
Name ID Tags
---- -- ----
Test1 123 {#{id=4567; name=test1;}}
Test2 345 {#{id=4567; name=test1;}, #{id=6789; name=test2}}
Test3 567 {#{id=4567; name=test1;}, #{id=6789; name=test2}, #{id=7890; name=test3}}
And another:
$filter = #('test1', 'test2')
And waht to filter the $displayObjects (Tags.name) based on the values specified in the $filter array.
So in the case above the result should contain only rows 2 and 3 (from $displayObjects).
I've strted thinking and testing with $displayObjects | Where-Object ... but cant think of a way how to loop in there. Any suggestions?
Something like this might work:
... | Where-Object {
$a = #($_.Tags.name)
($filter | Where-Object {$a -contains $_}).Count -eq $filter.Count
}
There is probably a more efficient way to do this with LINQ (like this?), but I'm not versed enough in that.
This should work. It might not be the most efficient way, though.
$displayObjects | Where-Object {
$tags = [string]$_.Tags
$returnObject = $true
$filter | foreach {
if($tags -notlike "*$_*"){
$returnObject = $false
}
}
$returnObject
}

Sorting and Comparing Arraylists

I'm pretty new to Powershell and have a problem.
It seems I can get neither the Compare-Object nor the Sort-Object funtions to work.
I got two Arraylists which I fill with 10 objects of type "Table".
#Create Arraylists
$list1 = new-object System.Collections.ArrayList
$list2 = new-object System.Collections.ArrayList
#Declare Table-Class
class Table {
[String] $name
[Int] $number
}
#Fill list1
[Int] $i = 0
while ($i -lt 10) {
$entry = new-Object Table
$entry.name = "Name";
$entry.number = $i;
$list1.Add($entry)
$i++
}
#Fill list2
[Int] $j = 10
while ($j -gt 0) {
$entry = new-Object Table
$entry.name = "name";
$entry.number = $j;
$list2.Add($entry)
$j--
}
Now I want to compare these two ArrayLists like this:
Compare-Object $list1 $list2 | ForEach-Object {$_.InputObject}
This doesn't seem to work and I think it's because I'm not really doing anything with the comparison. If someone could help me with the correct Syntax I'd be really happy.
Anyway, I think this comparison would somehow return a $false boolean.
If that were true, I'd like to sort $list2 by $entry.number.
I'm attempting this like that:
$list2 = $list2 | Sort-Object -Property { $_[1] }
But my list doesn't change at all. I already tried several different "solutions" but it seems none of these is a real "solution".
I'd be really glad if someone could help me or at least point me in the right direction.
EDIT: For future readers, here is the working syntax:
#sort list by the property number
$list2 = $list2 | sort-object -Property number
#count differences and if differences greater than 0, do stuff
[Int] $diff
$properties = $list1 | Get-Member -MemberType Property | Select-Object -ExpandProperty Name
foreach ($property in $properties) {
$diff = 0
$diff = (Compare-Object $list1 $list2 -Property "$property").count
if ($diff -ne 0) {
#do stuff
}
else {
#do something else
}
}
Your sorting does not work because -Property expects you to pass the name of an actual property. In your case the class two properties: $name and $number. You can also check what properties an object has by using $list2 | Get-Member. So in order to sort, you can use:
$list2 = $list2 | Sort-Object -Property number
Now the reason Compare-Object is not working is because it's implemented differently that one might expect. See this answer explaining how it works.
One other thing you should keep in mind: Calling $list2.Add($entry) actually returns an integer (the index of the inserted element). PowerShell returns all uncaptured output. This can cause some unexpected behavior/output if you are not careful. So you should get in the habit of writing [void] $list2.Add($entry), unless you really want to return those indexes.

Compare-Object - Separate side columns

Is it possible to display the results of a PowerShell Compare-Object in two columns showing the differences of reference vs difference objects?
For example using my current cmdline:
Compare-Object $Base $Test
Gives:
InputObject SideIndicator
987654 =>
555555 <=
123456 <=
In reality the list is rather long. For easier data reading is it possible to format the data like so:
Base Test
555555 987654
123456
So each column shows which elements exist in that object vs the other.
For bonus points it would be fantastic to have a count in the column header like so:
Base(2) Test(1)
555555 987654
123456
Possible? Sure. Feasible? Not so much. PowerShell wasn't really built for creating this kind of tabular output. What you can do is collect the differences in a hashtable as nested arrays by input file:
$ht = #{}
Compare-Object $Base $Test | ForEach-Object {
$value = $_.InputObject
switch ($_.SideIndicator) {
'=>' { $ht['Test'] += #($value) }
'<=' { $ht['Base'] += #($value) }
}
}
then transpose the hashtable:
$cnt = $ht.Values |
ForEach-Object { $_.Count } |
Sort-Object |
Select-Object -Last 1
$keys = $ht.Keys | Sort-Object
0..($cnt-1) | ForEach-Object {
$props = [ordered]#{}
foreach ($key in $keys) {
$props[$key] = $ht[$key][$_]
}
New-Object -Type PSObject -Property $props
} | Format-Table -AutoSize
To include the item count in the header name change $props[$key] to $props["$key($($ht[$key].Count))"].

Can't display PSObject

I'm trying to display some data my script generates in a PSObject, so I can then export to a CSV, but the only object that shows is whichever one I add to the array first.
$pass=#("1","2","3")
$fail=#("4")
$obj=#()
$pass | % {
$obj+=New-Object PSObject -Property #{Pass=$_}
}
$fail | % {
$obj+=New-Object PSObject -Property #{Fail=$_}
}
$obj
I've also tried this, but I get a blank line showing in the table where the value isn't in that column, which I don't want:
$pass=#("1","2","3")
$fail=#("4")
$obj=#()
$pass | % {
$obj+=New-Object PSObject -Property #{Pass=$_;Fail=""}
}
$fail | % {
$obj+=New-Object PSObject -Property #{Pass="";Fail=$_}
}
$obj
My desired result:
Pass Fail
---- ----
1 4
2
3
I am using Powershell V2.
Another answer is right - you're using objects wrong. That being said, here's a function to help you use them wrong!
Function New-BadObjectfromArray($array1,$array2,$array1name,$array2name){
if ($array1.count -ge $array2.count){$iteratorCount = $array1.count}
else {$iteratorCount = $array2.count}
$obj = #()
$iteration=0
while ($iteration -le $iteratorCount){
New-Object PSObject -Property #{
$array1name=$array1[$iteration]
$array2name=$array2[$iteration]
}
$iteration += 1
}
$obj
}
$pass=#("1","2","3")
$fail=#("4")
New-BadObjectfromArray -array1 $fail -array2 $pass -array1name "Fail" -array2name "Pass"
As you figured out yourself, PowerShell only outputs the properties of the first item in your array. Its not designed to print the ouput you are expecting in the way you are using it.
As a workaround, you could use a for loop to "build" your desired output:
$pass=#("1","2","3")
$fail=#("4")
$obj=#()
for ($i = 0; $i -lt $pass.Count; $i++)
{
if ($fail.Count -gt $i)
{
$currentFail = $fail[$i]
}
else
{
$currentFail = ""
}
$obj+=New-Object PSObject -Property #{Fail=$currentFail;Pass=$pass[$i];}
}
$obj | select Pass, Fail
Output:
Pass Fail
---- ----
1 4
2
3