Seeking balanced combination of fast, terse, and legible code to add up values from an array of objects - powershell

Given the following array of objects:
Email Domain Tally
----- ----- -----
email1#domainA.com domainA.com 4
email1#domainB.com domainB.com 1
email2#domainC.com domainC.com 6
email4#domainA.com domainA.com 1
I'd like to "group by" Domain and add up Tally as I go. The end result would like this:
Domain Tally
------ -----
domainA.com 5
domainB.com 1
domainC.com 6
I have something that works but I feel like it's overly complicated.
$AllTheAddresses = Get-AllTheAddresses
$DomainTally = #()
foreach ($Addy in $AllTheAddresses)
{
if ($DomainTally | Where-Object {$_.RecipientDomain -eq $Addy.RecipientDomain})
{
$DomainTally |
Where-Object {$_.RecipientDomain -eq $Addy.RecipientDomain} |
ForEach-Object {$_.Tally += $Addy.Tally }
}
else
{
$props = #{
RecipientDomain = $Addy.RecipientDomain
Tally = $Addy.Tally
}
$DomainTally += New-Object -TypeName PSObject -Property $props
}
}

In my example, I'm creating the addresses as hashtables, but PowerShell will let you refer to the keys by .Property similar to an object.
If you're truly just summing by the Domain, then it seems like you don't need anything more complicated than a HashTable to create your running total.
The basic summation:
$Tally = #{}
$AllTheAddresses | ForEach-Object {
$Tally[$_.Domain] += $_.Tally
}
Using this sample data...
$AllTheAddresses = #(
#{ Email = "email1#domainA.com"; Domain = "domainA.com"; Tally = 4 };
#{ Email = "email1#domainB.com"; Domain = "domainB.com"; Tally = 1 };
#{ Email = "email1#domainC.com"; Domain = "domainC.com"; Tally = 6 };
#{ Email = "email1#domainA.com"; Domain = "domainA.com"; Tally = 1 }
)
And you get this output:
PS> $tally
Name Value
---- -----
domainC.com 6
domainB.com 1
domainA.com 5

Here is a "PowerShellic" version, notice the piping and flow of the data.
You could of course write this as a one liner (I did originally before I posted the answer here). The 'better' part of this is using the Group-Object and Measure-Object cmdlets. Notice there are no conditionals, again because the example uses the pipeline.
$AllTheAddresses |
Group-Object -Property Domain |
ForEach-Object {
$_ |
Tee-Object -Variable Domain |
Select-Object -Expand Group |
Measure-Object -Sum Tally |
Select-Object -Expand Sum |
ForEach-Object {
New-Object -TypeName PSObject -Property #{
'Domain' = $Domain.Name
'Tally' = $_
}
} |
Select-Object Domain, Tally
}
A more terse version
$AllTheAddresses |
Group Domain |
% {
$_ |
Tee-Object -Variable Domain |
Select -Expand Group |
Measure -Sum Tally |
Select -Expand Sum |
% {
New-Object PSObject -Property #{
'Domain' = $Domain.Name
'Tally' = $_
}
} |
Select Domain, Tally
}

Group-Object is definitely the way to go.
In the interest of terseness:
Get-AllTheAddresses |Group-Object Domain |Select-Object #{N='Domain';E={$_.Name}},#{N='Tally';E={($_.Group.Tally |Measure-Object).Sum}}

Related

Format-Table not taking effect (Exchange - powershell)

first of all sorry if my english is not the best. but ill try to explain my issue with as much detail as i can
Im having an issue where i cant get Format-Table to effect the output i give it.
below is the part im having issues with atm.
cls
$TotalSize = $($mailboxes. #{name = ”TotalItemSize (GB)”; expression = { [math]::Round((($_.TotalItemSize.Value.ToString()).Split(“(“)[1].Split(” “)[0].Replace(“,”, ””) / 1GB), 2) } });
$UserN = $($mailboxes.DisplayName)
$itemCount = $($mailboxes.ItemCount)
$LastLogonTime = $($mailboxes.ItemCount)
$allMailboxinfo = #(
#lager dataen som skal inn i et objekt
#{Username= $UserN; ItemCount = $itemCount; LastLogonTime = $($mailboxes.ItemCount); Size = $TotalSize}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$Table = $allMailboxinfo | Format-Table | Out-String
$Table
the output of this gives me what almost looks like json syntax below each title of the table.
Username LastLogonTime ItemCount Size
-------- ------------- --------- ----
{username1, username2,username3,userna...} {$null, $null, $null, $null...} {$null, $null, $null, $null...} {$null, $null, $null, $null...}
running the commands by themselves seem to work tho. like $mailboxes.DisplayName gives the exact data i want for displayname. even in table-format.
the reason im making the table this way instead of just using select-object, is because im going to merge a few tables later. using the logic from the script below.
cls
$someData = #(
#{Name = "Bill"; email = "email#domain.com"; phone = "12345678"; id = "043546" }) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$moreData = #(
#{Name = "Bill"; company = "company 04"}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$Merge = #(
#plots the data into a new table
#{Name = $($someData.Name); e_mail = $($someData.email); phone = $($someData.phone); id = $($someData.id); merged = $($moreData.company) }) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
#formatting table
$Table = $Merge | Format-Table | Out-String
#print table
$Table
if you are wondering what im doing with this.
My goal, all in all. is a table with using the info from Exchange;
DisplayName, TotalItemSize(GB), ItemCount, LastLogonTime, E-mail adress, archive + Maxquoata, Quoata for mailbox.
You're creating a single object where each property holds an array of property values from the original array of mailbox objects.
Instead, create 1 new object per mailbox:
# construct output objects with Select-Object
$allMailBoxInfo = $mailboxes |Select #{Name='Username';Expression='DisplayName'},ItemCount,#{Name='LastLogonTime';Expression='ItemCount'},#{Name='Size';Expression={[math]::Round((($_.TotalItemSize.Value.ToString()).Split("(")[1].Split(" ")[0].Replace(",", "") / 1GB), 2) }}
# format table
$Table = $allMailBoxInfo | Format-Table | Out-String
# print table
$Table

PowerShell Compare-Object to produce change report file

I've looked at numerous examples and have made headway in producing a change report. But, I'm stuck in one area. Here's the scenario...
File 1 CSV file sample data
ID,Name,Location,Gender
1,Peter,USA,Male
2,Paul,UK,Male
3,Mary,PI,Female
File 2 CSV file sample data (No ID column)
Name,Location,Gender
Peter,USA,Female
Paul,UK,Male
Mary,USA,Female
Tom,PI,Female
Barry,CAN,Male
File 2 has changes and additions, i.e. Peter turned female, Mary moved to the US, both Tom and Barry are the new people. Change report output file contain what the changes are. Problem is, I can't figure out how to get the ID for both Peter and Mary from File 1, into my Change Report. ID is always empty Here's my code...(I hope someone can shed some light. Thanks in advance.)
$MyCSVFields = #('Name','Location','Gender')
$CompareResults = Compare-Object $RefObj $DffObj -Property $MyCSVFields -IncludeEqual
$NewOrChangedData = #()
Foreach($Row in $CompareResults)
{
if( $Row.SideIndicator -eq "=>" )
{
$TempObject = [pscustomobject][ordered] #{
ID = $Row.ID
Name = $Row.Name
Location = $Row.Location
Gender = $Row.Gender
#Sanity check "Compare Indicator" = $Row.SideIndicator
}
$NewOrChangedData += $TempObject
}
}
Thanks to Theo for providing an understanding of how to use the Where-Object. Here is the updated code that keeps it simple for beginners and still works for us.
Foreach($Row in $CompareResults)
{
if( $Row.SideIndicator -eq "=>" )
{
$myOrgID = $RefObj | Where-Object Name -eq $Row.Name
$TempObject = [pscustomobject][ordered] #{
ID = $myOrgID.ID
Name = $Row.Name
Location = $Row.Location
Gender = $Row.Gender
#Sanity check "Compare Indicator" = $Row.SideIndicator
}
$NewOrChangedData += $TempObject
}
}
I'm also alway struggling with Compare-Object, so I hope there is a better answer than this:
$RefObj = #'
ID,Name,Location,Gender
1,Peter,USA,Male
2,Paul,UK,Male
3,Mary,PI,Female
'# | ConvertFrom-Csv
$DffObj = #'
Name,Location,Gender
Peter,USA,Female
Paul,UK,Male
Mary,USA,Female
Tom,PI,Female
Barry,CAN,Male
'# | ConvertFrom-Csv
$MyCSVFields = #('Name','Location','Gender')
$CompareResults = Compare-Object $RefObj $DffObj -Property $MyCSVFields -PassThru
$NewOrChangedData = $CompareResults | Where-Object { $_.SideIndicator -eq '=>' } | ForEach-Object {
$name = $_.Name
[PsCustomObject]#{
ID = ($RefObj | Where-Object { $_.Name -eq $name }).ID
Name = $name
Location = $_.Location
Gender = $_.Gender
#Sanity check "Compare Indicator" = $_.SideIndicator
}
}
$NewOrChangedData
Result:
ID Name Location Gender
-- ---- -------- ------
1 Peter USA Female
3 Mary USA Female
Tom PI Female
Barry CAN Male

Conditional criteria in powershell group measure-object?

I have data in this shape:
externalName,day,workingHours,hoursAndMinutes
PRJF,1,11,11:00
PRJF,2,11,11:00
PRJF,3,0,0:00
PRJF,4,0,0:00
CFAW,1,11,11:00
CFAW,2,11,11:00
CFAW,3,11,11:00
CFAW,4,11,11:00
CFAW,5,0,0:00
CFAW,6,0,0:00
and so far code is
$gdata = Import-csv $filepath\$filename | Group-Object -Property Externalname;
$test = #()
$test += foreach($rostername in $gdata) {
$rostername.Group | Select -Unique externalName,
#{Name = 'AllDays';Expression = {(($rostername.Group) | measure -Property day).count}},
}
$test;
What I can't work out is how to do a conditional count of the lines where day is non-zero.
The aim is to produce two lines:
PRJF, 4, 2, 11
CFAW, 6, 4, 11
i.e. Roster name, roster length, days on, average hours worked per day on.
You need a where-object to filter for non zero workinghours
I'd use a [PSCustomObject] to generate a new table
EDIT a bit more efficient with only one Measure-Object
## Q:\Test\2018\08\06\SO_51700660.ps1
$filepath = 'Q:\Test\2018\08\06'
$filename = 'SO_S1700660.csv'
$gdata = Import-Csv (Join-Path $filepath $filename) | Group-Object -Property Externalname
$test = ForEach($Roster in $gdata) {
$WH = ($Roster.Group.Workinghours|Where-Object {$_ -ne 0}|Measure-Object -Ave -Sum)
[PSCustomObject]#{
RosterName = $Roster.Name
RosterLength = $Roster.Count
DaysOn = $WH.count
AvgHours = $WH.Average
TotalHours = $WH.Sum
}
}
$test | Format-Table
Sample output:
> .\SO_51700660.ps1
RosterName RosterLength DaysOn AvgHours TotalHours
---------- ------------ ------ -------- ----------
PRJF 4 2 11 22
CFAW 6 4 11 44

How do you export objects with a varying amount of properties?

Warning - I've asked a similar question in the past but this is slightly different.
tl;dr; I want to export objects which have a varying number of properties. eg; object 1 may have 3 IP address and 2 NICs but object 2 has 7 IP addresses and 4 NICs (but not limited to this amount - it could be N properties).
I can happily capture and build objects that contain all the information I require. If I simply output my array to the console each object is shown with all its properties. If I want to out-file or export-csv I start hitting a problem surrounding the headings.
Previously JPBlanc recommended sorting the objects based on the amount of properties - ie, the object with the most properties would come first and hence the headings for the most amount of properties would be output.
Say I have built an object of servers which has varying properties based on IP addresses and NIC cards. For example;
ServerName: Mordor
IP1: 10.0.0.1
IP2: 10.0.0.2
NIC1: VMXNET
NIC2: Broadcom
ServerName: Rivendell
IP1: 10.1.1.1
IP2: 10.1.1.2
IP3: 10.1.1.3
IP4: 10.1.1.4
NIC1: VMXNET
Initially, if you were to export-csv an array of these objects the headers would be built upon the first object (aka, you would only get ServerName, IP1, IP2, NIC1 and NIC2) meaning for the second object you would lose any subsequent IPs (eg IP3 and IP4). To correct this, before an export I sort based on the number of IP properties - tada - the first object now has the most IPs in the array and hence none of the subsequent objects IPs are lost.
The downside is when you then have a second varying property - eg NICs. Once my sort is complete based on IP we then have the headings ServerName, IP1 - IP4 and NIC1. This means the subsequent object property of NIC2 is lost.
Is there a scalable way to ensure that you aren't losing data when exporting objects like this?
Try:
$o1 = New-Object psobject -Property #{
ServerName="Mordor"
IP1="10.0.0.1"
IP2="10.0.0.2"
NIC1="VMXNET"
NIC2="Broadcom"
}
$o2 = New-Object psobject -Property #{
ServerName="Rivendell"
IP1="10.1.1.1"
IP2="10.1.1.2"
IP3="10.1.1.3"
IP4="10.1.1.4"
NIC1="VMXNET"
}
$arr = #()
$arr += $o1
$arr += $o2
#Creating output
$prop = $arr | % { Get-Member -InputObject $_ -MemberType NoteProperty | Select -ExpandProperty Name } | Select -Unique | Sort-Object
$headers = #("ServerName")
$headers += $prop -notlike "ServerName"
$arr | ft -Property $headers
Output:
ServerName IP1 IP2 IP3 IP4 NIC1 NIC2
---------- --- --- --- --- ---- ----
Mordor 10.0.0.1 10.0.0.2 VMXNET Broadcom
Rivendell 10.1.1.1 10.1.1.2 10.1.1.3 10.1.1.4 VMXNET
If you know the types(NICS, IPS..), but not the count(ex. how many NICS) you could try:
#Creating output
$headers = $arr | % { Get-Member -InputObject $_ -MemberType NoteProperty | Select -ExpandProperty Name } | Select -Unique
$ipcount = ($headers -like "IP*").Count
$niccount = ($headers -like "NIC*").Count
$format = #("ServerName")
for ($i = 1; $i -le $ipcount; $i++) { $format += "IP$i" }
for ($i = 1; $i -le $niccount; $i++) { $format += "NIC$i" }
$arr | ft -Property $format
What about getting a list of all unique property headers and then doing a select on all the objects? When you do a select on an object for a nonexistent property it will create a blank one.
$allHeaders = $arrayOfObjects | % { Get-Member -inputobject $_ -membertype noteproperty | Select -expand Name } | Select -unique
$arrayOfObjects | Select $allHeaders
Granted you are looping through ever object to get the headers, so for a very large amount of objects it may take awhile.
Here's my attempt at a solution. I'm very tired now so hopefully it makes sense. Basically I'm calculating the largest amount of NIC and IP note properties, creating a place holder object that has those amounts of properties, adding it as the first item in a CSV, and then removing it from the CSV.
# Create example objects
$o1 = New-Object psobject -Property #{
ServerName="Mordor"
IP1="10.0.0.1"
IP2="10.0.0.2"
NIC1="VMXNET"
NIC2="Broadcom"
}
$o2 = New-Object psobject -Property #{
ServerName="Rivendell"
IP1="10.1.1.1"
IP2="10.1.1.2"
IP3="10.1.1.3"
IP4="10.1.1.4"
NIC1="VMXNET"
}
# Add to an array
$servers = #($o1, $o2)
# Calculate how many IP and NIC properties there are
$IPColSize = ($servers | Select IP* | %{($_ | gm -MemberType NoteProperty).Count} | Sort-Object -Descending)[0]
$NICColSize = ($servers | Select NIC* | %{($_ | gm -MemberType NoteProperty).Count} | Sort-Object -Descending)[0]
# Build a place holder object that will contain enough properties to cover all of the objects in the array.
$cmd = '$placeholder = "" | Select ServerName, {0}, {1}' -f (#(1..$IPColSize | %{"IP$_"}) -join ", "), (#(1..$NICColSize | %{"NIC$_"}) -join ", ")
Invoke-Expression $cmd
# Convert to CSV and remove the placeholder
$csv = $placeholder,$servers | %{$_ | Select *} | ConvertTo-Csv -NoTypeInformation
$csv | Select -First 1 -Last ($csv.Count-2) | ConvertFrom-Csv | Export-Csv Solution.csv -NoTypeInformation

How to sum multiple items in an object in PowerShell?

I have:
$report.gettype().name
Object[]
echo $report
Item Average
-- -------
orange 0.294117647058824
orange -0.901960784313726
orange -0.901960784313726
grape 9.91335740072202
grape 0
pear 3.48736462093863
pear -0.0324909747292419
pear -0.0324909747292419
apple 12.1261261261261
apple -0.0045045045045045
I want to create a variable, $total, (such as a hash table) which contains the sum of the 'Average' column for each item, for example,
echo $total
orange -1.5097
grape 9.913
pear 3.423
apple 12.116
Right now I'm thinking of looping through the $report, but it's hell ugly, and I am looking for something more elegant than the following starting point (incomplete):
$tmpPrev = ""
foreach($r in $report){
$tmp = $r.item
$subtotal = 0
if($tmp <> $tmpPrev){
$subtotal += $r.average
}
How could I do this?
Cmdlets Group-Object and Measure-Object help to solve the task in a PowerShell-ish way:
Code:
# Demo input
$report = #(
New-Object psobject -Property #{ Item = 'orange'; Average = 1 }
New-Object psobject -Property #{ Item = 'orange'; Average = 2 }
New-Object psobject -Property #{ Item = 'grape'; Average = 3 }
New-Object psobject -Property #{ Item = 'grape'; Average = 4 }
)
# Process: group by 'Item' then sum 'Average' for each group
# and create output objects on the fly
$report | Group-Object Item | %{
New-Object psobject -Property #{
Item = $_.Name
Sum = ($_.Group | Measure-Object Average -Sum).Sum
}
}
Output:
Sum Item
--- ----
3 orange
7 grape
I've got a more command-line solution.
Given $report
$groupreport = $report | Group-Object -Property item -AsHashTable
is
Name Value
---- -----
grape {#{Item=grape; Average=9.91335740072202}, #{Item=grape; Average=0}}
orange {#{Item=orange; Average=0.294117647058824}, #{Item=orange; Average=-0.901960784313726...
apple {#{Item=apple; Average=12.1261261261261}, #{Item=apple; Average=-0.0045045045045045}}
pear {#{Item=pear; Average=3.48736462093863}, #{Item=pear; Average=-0.0324909747292419}, #...
then
$tab=#{}
$groupreport.keys | % {$tab += #{$_ = ($groupreport[$_] | measure-object -Property average -sum)}}
gives
PS> $tab["grape"]
Count : 2
Average :
Sum : 9,91335740072202
Maximum :
Minimum :
Property : Average
PS> $tab["grape"].sum
9,91335740072202
It seems short and usable.
Summary
$groupreport = $report | Group-Object -Property item -AsHashTable
$tab = #{}
$groupreport.keys | % {$tab += #{$_ = ($groupreport[$_] | measure-object -Property average -sum)}}
$tab.keys | % {write-host $_ `t $tab[$_].sum}
I don't know if you can get rid of looping. What about:
$report | % {$averages = #{}} {
if ($averages[$_.item]) {
$averages[$_.item] += $_.average
}
else {
$averages[$_.item] = $_.average
}
} {$averages}