PowerShell Compare-Object to produce change report file - powershell

I've looked at numerous examples and have made headway in producing a change report. But, I'm stuck in one area. Here's the scenario...
File 1 CSV file sample data
ID,Name,Location,Gender
1,Peter,USA,Male
2,Paul,UK,Male
3,Mary,PI,Female
File 2 CSV file sample data (No ID column)
Name,Location,Gender
Peter,USA,Female
Paul,UK,Male
Mary,USA,Female
Tom,PI,Female
Barry,CAN,Male
File 2 has changes and additions, i.e. Peter turned female, Mary moved to the US, both Tom and Barry are the new people. Change report output file contain what the changes are. Problem is, I can't figure out how to get the ID for both Peter and Mary from File 1, into my Change Report. ID is always empty Here's my code...(I hope someone can shed some light. Thanks in advance.)
$MyCSVFields = #('Name','Location','Gender')
$CompareResults = Compare-Object $RefObj $DffObj -Property $MyCSVFields -IncludeEqual
$NewOrChangedData = #()
Foreach($Row in $CompareResults)
{
if( $Row.SideIndicator -eq "=>" )
{
$TempObject = [pscustomobject][ordered] #{
ID = $Row.ID
Name = $Row.Name
Location = $Row.Location
Gender = $Row.Gender
#Sanity check "Compare Indicator" = $Row.SideIndicator
}
$NewOrChangedData += $TempObject
}
}
Thanks to Theo for providing an understanding of how to use the Where-Object. Here is the updated code that keeps it simple for beginners and still works for us.
Foreach($Row in $CompareResults)
{
if( $Row.SideIndicator -eq "=>" )
{
$myOrgID = $RefObj | Where-Object Name -eq $Row.Name
$TempObject = [pscustomobject][ordered] #{
ID = $myOrgID.ID
Name = $Row.Name
Location = $Row.Location
Gender = $Row.Gender
#Sanity check "Compare Indicator" = $Row.SideIndicator
}
$NewOrChangedData += $TempObject
}
}

I'm also alway struggling with Compare-Object, so I hope there is a better answer than this:
$RefObj = #'
ID,Name,Location,Gender
1,Peter,USA,Male
2,Paul,UK,Male
3,Mary,PI,Female
'# | ConvertFrom-Csv
$DffObj = #'
Name,Location,Gender
Peter,USA,Female
Paul,UK,Male
Mary,USA,Female
Tom,PI,Female
Barry,CAN,Male
'# | ConvertFrom-Csv
$MyCSVFields = #('Name','Location','Gender')
$CompareResults = Compare-Object $RefObj $DffObj -Property $MyCSVFields -PassThru
$NewOrChangedData = $CompareResults | Where-Object { $_.SideIndicator -eq '=>' } | ForEach-Object {
$name = $_.Name
[PsCustomObject]#{
ID = ($RefObj | Where-Object { $_.Name -eq $name }).ID
Name = $name
Location = $_.Location
Gender = $_.Gender
#Sanity check "Compare Indicator" = $_.SideIndicator
}
}
$NewOrChangedData
Result:
ID Name Location Gender
-- ---- -------- ------
1 Peter USA Female
3 Mary USA Female
Tom PI Female
Barry CAN Male

Related

Format-Table not taking effect (Exchange - powershell)

first of all sorry if my english is not the best. but ill try to explain my issue with as much detail as i can
Im having an issue where i cant get Format-Table to effect the output i give it.
below is the part im having issues with atm.
cls
$TotalSize = $($mailboxes. #{name = ”TotalItemSize (GB)”; expression = { [math]::Round((($_.TotalItemSize.Value.ToString()).Split(“(“)[1].Split(” “)[0].Replace(“,”, ””) / 1GB), 2) } });
$UserN = $($mailboxes.DisplayName)
$itemCount = $($mailboxes.ItemCount)
$LastLogonTime = $($mailboxes.ItemCount)
$allMailboxinfo = #(
#lager dataen som skal inn i et objekt
#{Username= $UserN; ItemCount = $itemCount; LastLogonTime = $($mailboxes.ItemCount); Size = $TotalSize}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$Table = $allMailboxinfo | Format-Table | Out-String
$Table
the output of this gives me what almost looks like json syntax below each title of the table.
Username LastLogonTime ItemCount Size
-------- ------------- --------- ----
{username1, username2,username3,userna...} {$null, $null, $null, $null...} {$null, $null, $null, $null...} {$null, $null, $null, $null...}
running the commands by themselves seem to work tho. like $mailboxes.DisplayName gives the exact data i want for displayname. even in table-format.
the reason im making the table this way instead of just using select-object, is because im going to merge a few tables later. using the logic from the script below.
cls
$someData = #(
#{Name = "Bill"; email = "email#domain.com"; phone = "12345678"; id = "043546" }) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$moreData = #(
#{Name = "Bill"; company = "company 04"}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$Merge = #(
#plots the data into a new table
#{Name = $($someData.Name); e_mail = $($someData.email); phone = $($someData.phone); id = $($someData.id); merged = $($moreData.company) }) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
#formatting table
$Table = $Merge | Format-Table | Out-String
#print table
$Table
if you are wondering what im doing with this.
My goal, all in all. is a table with using the info from Exchange;
DisplayName, TotalItemSize(GB), ItemCount, LastLogonTime, E-mail adress, archive + Maxquoata, Quoata for mailbox.
You're creating a single object where each property holds an array of property values from the original array of mailbox objects.
Instead, create 1 new object per mailbox:
# construct output objects with Select-Object
$allMailBoxInfo = $mailboxes |Select #{Name='Username';Expression='DisplayName'},ItemCount,#{Name='LastLogonTime';Expression='ItemCount'},#{Name='Size';Expression={[math]::Round((($_.TotalItemSize.Value.ToString()).Split("(")[1].Split(" ")[0].Replace(",", "") / 1GB), 2) }}
# format table
$Table = $allMailBoxInfo | Format-Table | Out-String
# print table
$Table

How to use Group-Object on this?

I am trying to get all the accounts from $f which do not match the accounts in $table4 into $accounts. But I need to also check if the occupancy number matches or not.
CSV $f:
Account_no |occupant_code
-----------|------------
12345 | 1
67890 | 2
45678 | 3
DataTable $table4
Account_no |occupant_code
-----------|------------
12345 | 1
67890 | 1
45678 | 3
Current code:
$accounts = Import-Csv $f |
select account_no, occupant_code |
where { $table4.account_no -notcontains $_.account_no }
What this needs to do is to check that occupant_code doesn't match, i.e.:
12345: account and occupant from $f and $table4 match; so it's ignored
67890: account matches $table4, but occupancy_code does not match, so it is added to $accounts.
Current result:
Desired result: 67890
I believe I need to use Group-Object, but I do not know how to use that correctly.
I tried:
Import-Csv $f |
select account_no, occupant_code |
Group-Object account_no |
Where-Object { $_.Group.occupant_code -notcontains $table4.occupant_code }
An alternative to Bill's suggestion would be to fill a hashtable with your reference data ($table4) and look up the occupant_code value for each account from $f, assuming that your account numbers are unique:
$ref = #{}
$table4 | ForEach-Object {
$ref[$_.Account_no] = $_.occupant_code
}
$accounts = Import-Csv $f |
Where-Object { $_.occupant_code -ne $ref[$_.Account_no] } |
Select-Object -Expand Account_no
Compare-Object?
csv1.csv:
Account_no,occupant_code
12345,1
67890,2
45678,3
csv2.csv:
Account_no,occupant_code
12345,1
67890,1
45678,3
PowerShell command:
Compare-Object (Import-Csv .\csv1.csv) (Import-Csv .\csv2.csv) -Property occupant_code -PassThru
Output:
Account_no occupant_code SideIndicator
---------- ------------- -------------
67890 1 =>
67890 2 <=
$f | InnerJoin $table4 {$Left.Account_no -eq $Right.Account_no -and $Left.occupant_code -ne $Right.occupant_code} #{Account_no = {$Left.$_}} | Format-Table
Result:
occupant_code Account_no
------------- ----------
{2, 1} 67890
For details see: In Powershell, what's the best way to join two tables into one?
In addition to all the other answers, you might be able to leverage the IndexOf() method on arrays
$services = get-service
$services.name.IndexOf("xbgm")
240
I am on a tablet right now and don't have a handy way to test it, but something along these lines might work for you:
$table4.account_no.IndexOf($_.account_no)
should fetch the index your account_no lives in for $table 4, so you could jam it all into one ugly pipe:
$accounts = Import-Csv $f | select account_no, occupant_code |
where { ($table4.account_no -notcontains $_.account_no) -or ($table4[$table4.account_no.IndexOf($_.account_no)].occupant_code -ne $_.occupant_code) }
An inner join or a normal loop might just be cleaner though, especially if you want to add some other stuff in. Since someone posted an innerjoin, you could try a loop like:
$accounts = new-object System.Collections.ArrayList
$testSet = $table4.account_no
foreach($myThing in Import-Csv $f)
{
if($myThing.account_no -in $testSet )
{
$i = $testSet.IndexOf($myThing.account_no)
if($table4[$i].occupant_code -eq $myThing.occupant_code) {continue}
}
$accounts.add($myThing)
}
Edit for OP, he mentioned $table4 is a data.table
There is probably a much better way to do this, as I haven't used data.table before, but this seems to work fine:
$table = New-Object system.Data.DataTable
$col1 = New-Object system.Data.DataColumn Account_no,([string])
$col2 = New-Object system.Data.DataColumn occupant_code,([int])
$table.columns.add($col1)
$table.columns.add($col2)
$row = $table.NewRow()
$row.Account_no = "12345"
$row.occupant_code = 1
$table.Rows.Add($row)
$row = $table.NewRow()
$row.Account_no = "67890"
$row.occupant_code = 1
$table.Rows.Add($row)
$row = $table.NewRow()
$row.Account_no = "45678"
$row.occupant_code = 3
$table.Rows.Add($row)
$testList = #()
$testlist += [pscustomobject]#{Account_no = "12345"; occupant_code = 1}
$testlist += [pscustomobject]#{Account_no = "67890"; occupant_code = 2}
$testlist += [pscustomobject]#{Account_no = "45678"; occupant_code = 3}
$accounts = new-object System.Collections.ArrayList
$testSet = $table.account_no
foreach($myThing in $testList)
{
if($myThing.account_no -in $testSet )
{
$i = $testSet.IndexOf($myThing.account_no)
if($table.Rows[$i].occupant_code -eq $myThing.occupant_code) {continue}
}
$accounts.add($myThing) | out-null
}
$accounts

Seeking balanced combination of fast, terse, and legible code to add up values from an array of objects

Given the following array of objects:
Email Domain Tally
----- ----- -----
email1#domainA.com domainA.com 4
email1#domainB.com domainB.com 1
email2#domainC.com domainC.com 6
email4#domainA.com domainA.com 1
I'd like to "group by" Domain and add up Tally as I go. The end result would like this:
Domain Tally
------ -----
domainA.com 5
domainB.com 1
domainC.com 6
I have something that works but I feel like it's overly complicated.
$AllTheAddresses = Get-AllTheAddresses
$DomainTally = #()
foreach ($Addy in $AllTheAddresses)
{
if ($DomainTally | Where-Object {$_.RecipientDomain -eq $Addy.RecipientDomain})
{
$DomainTally |
Where-Object {$_.RecipientDomain -eq $Addy.RecipientDomain} |
ForEach-Object {$_.Tally += $Addy.Tally }
}
else
{
$props = #{
RecipientDomain = $Addy.RecipientDomain
Tally = $Addy.Tally
}
$DomainTally += New-Object -TypeName PSObject -Property $props
}
}
In my example, I'm creating the addresses as hashtables, but PowerShell will let you refer to the keys by .Property similar to an object.
If you're truly just summing by the Domain, then it seems like you don't need anything more complicated than a HashTable to create your running total.
The basic summation:
$Tally = #{}
$AllTheAddresses | ForEach-Object {
$Tally[$_.Domain] += $_.Tally
}
Using this sample data...
$AllTheAddresses = #(
#{ Email = "email1#domainA.com"; Domain = "domainA.com"; Tally = 4 };
#{ Email = "email1#domainB.com"; Domain = "domainB.com"; Tally = 1 };
#{ Email = "email1#domainC.com"; Domain = "domainC.com"; Tally = 6 };
#{ Email = "email1#domainA.com"; Domain = "domainA.com"; Tally = 1 }
)
And you get this output:
PS> $tally
Name Value
---- -----
domainC.com 6
domainB.com 1
domainA.com 5
Here is a "PowerShellic" version, notice the piping and flow of the data.
You could of course write this as a one liner (I did originally before I posted the answer here). The 'better' part of this is using the Group-Object and Measure-Object cmdlets. Notice there are no conditionals, again because the example uses the pipeline.
$AllTheAddresses |
Group-Object -Property Domain |
ForEach-Object {
$_ |
Tee-Object -Variable Domain |
Select-Object -Expand Group |
Measure-Object -Sum Tally |
Select-Object -Expand Sum |
ForEach-Object {
New-Object -TypeName PSObject -Property #{
'Domain' = $Domain.Name
'Tally' = $_
}
} |
Select-Object Domain, Tally
}
A more terse version
$AllTheAddresses |
Group Domain |
% {
$_ |
Tee-Object -Variable Domain |
Select -Expand Group |
Measure -Sum Tally |
Select -Expand Sum |
% {
New-Object PSObject -Property #{
'Domain' = $Domain.Name
'Tally' = $_
}
} |
Select Domain, Tally
}
Group-Object is definitely the way to go.
In the interest of terseness:
Get-AllTheAddresses |Group-Object Domain |Select-Object #{N='Domain';E={$_.Name}},#{N='Tally';E={($_.Group.Tally |Measure-Object).Sum}}

compare columns in two csv files

With all of the examples out there you would think I could have found my solution. :-)
Anyway, I have two csv files; one with two columns, one with 4. I need to compare one column from each one using powershell. I thought I had it figured out but when I did a compare of my results, it comes back as false when I know it should be true. Here's what I have so far:
$newemp = Import-Csv -Path "C:\Temp\newemp.csv" -Header login_id, lastname, firstname, other | Select-Object "login_id"
$ps = Import-Csv -Path "C:\Temp\Emplid_LoginID.csv" | Select-Object "login id"
If ($newemp -eq $ps)
{
write-host "IDs match" -forgroundcolor green
}
Else
{
write-host "Not all IDs match" -backgroundcolor yellow -foregroundcolor black
}
I had to specifiy headers for the first file because it doesn't have any. What's weird is that I can call each variable to see what it holds and they end up with the same info but for some reason still comes up as false. This occurs even if there is only one row (not counting the header row).
I started to parse them as arrays but wasn't quite sure that was the right thing. What's important is that I compare row1 of the first file with with row1 of the second file. I can't just do a simple -match or -contains.
EDIT: One annoying thing is that the variables seem to hold the header row as well. When I call each one, the header is shown. But if I call both variables, I only see one header but two rows.
I just added the following check but getting the same results (False for everything):
$results = Compare-Object -ReferenceObject $newemp -DifferenceObject $ps -PassThru | ForEach-Object { $_.InputObject }
Using latkin's answer from here I think this would give you the result set you're looking for. As per latkin's comment, the property comparison is redundant for your purposes but I left it in as it's good to know. Additionally the header is specified even for the csv with headers to prevent the header row being included in the comparison.
$newemp = Import-Csv -Path "C:\Temp\_sotemp\Book1.csv" -Header loginid |
Select-Object "loginid"
$ps = Import-Csv -Path "C:\Temp\_sotemp\Book2.csv" -Header loginid |
Select-Object "loginid"
#get list of (imported) CSV properties
$props1 = $newemp | gm -MemberType NoteProperty | select -expand Name | sort
$props2 = $ps | gm -MemberType NoteProperty | select -expand Name | sort
#first check that properties match
#omit this step if you know for sure they will be
if(Compare-Object $props1 $props2){
throw "Properties are not the same! [$props1] [$props2]"
}
#pass properties list to Compare-Object
else{
Compare-Object $newemp $ps -Property $props1
}
In the second line, I see there a space "login id" and the first line doesn't have it. Could that be an issue. Try having the same name for the headers in the .csv files itself. And it works for without providing header or select statements. Below is my experiment based upon your input.
emp.csv
loginid firstname lastname
------------------------------
abc123 John patel
zxy321 Kohn smith
sdf120 Maun scott
tiy123 Dham rye
k2340 Naam mason
lk10j5 Shaan kelso
303sk Doug smith
empids.csv
loginid
-------
abc123
zxy321
sdf120
tiy123
PS C:\>$newemp = Import-csv C:\scripts\emp.csv
PS C:\>$ps = Import-CSV C:\scripts\empids.csv
PS C:\>$results = Compare-Object -ReferenceObject $newemp -DifferenceObject $ps | foreach { $_.InputObject}
Shows the difference objects that are not in $ps
loginid firstname lastname SideIndicator
------- --------- -------- -------------
k2340 Naam mason <=
lk10j5 Shaan kelso <=
303sk Doug smith <=
I am not sure if this is what you are looking for but i have used the PowerShell to do some CSV formatting for myself.
$test = Import-Csv .\Desktop\Vmtools-compare.csv
foreach ($i in $test) {
foreach ($n in $i.name) {
foreach ($m in $test) {
$check = "yes"
if ($n -eq $m.prod) {
$check = "no"
break
}
}
if ($check -ne "no") {$n}
}
}
this is how my excel csv file looks like:
prod name
1 3
2 5
3 8
4 2
5 0
and script outputs this:
8
0
so basically script takes each number under Name column and then checks it against prod column. If the number is there then it won't display else it will display that number.
I have also done it the opposite way:
$test = Import-Csv c:\test.csv
foreach ($i in $test) {
foreach ($n in $i.name) {
foreach ($m in $test) {
$check = "yes"
if ($n -eq $m.prod) {echo $n}
}
}
}
this is how my excel csv looks like:
prod name
1 3
2 5
3 8
4 2
5 0
and script outputs this:
3
5
2
so script shows the matching entries only.
You can play around with the code to look at different columns.

How to sum multiple items in an object in PowerShell?

I have:
$report.gettype().name
Object[]
echo $report
Item Average
-- -------
orange 0.294117647058824
orange -0.901960784313726
orange -0.901960784313726
grape 9.91335740072202
grape 0
pear 3.48736462093863
pear -0.0324909747292419
pear -0.0324909747292419
apple 12.1261261261261
apple -0.0045045045045045
I want to create a variable, $total, (such as a hash table) which contains the sum of the 'Average' column for each item, for example,
echo $total
orange -1.5097
grape 9.913
pear 3.423
apple 12.116
Right now I'm thinking of looping through the $report, but it's hell ugly, and I am looking for something more elegant than the following starting point (incomplete):
$tmpPrev = ""
foreach($r in $report){
$tmp = $r.item
$subtotal = 0
if($tmp <> $tmpPrev){
$subtotal += $r.average
}
How could I do this?
Cmdlets Group-Object and Measure-Object help to solve the task in a PowerShell-ish way:
Code:
# Demo input
$report = #(
New-Object psobject -Property #{ Item = 'orange'; Average = 1 }
New-Object psobject -Property #{ Item = 'orange'; Average = 2 }
New-Object psobject -Property #{ Item = 'grape'; Average = 3 }
New-Object psobject -Property #{ Item = 'grape'; Average = 4 }
)
# Process: group by 'Item' then sum 'Average' for each group
# and create output objects on the fly
$report | Group-Object Item | %{
New-Object psobject -Property #{
Item = $_.Name
Sum = ($_.Group | Measure-Object Average -Sum).Sum
}
}
Output:
Sum Item
--- ----
3 orange
7 grape
I've got a more command-line solution.
Given $report
$groupreport = $report | Group-Object -Property item -AsHashTable
is
Name Value
---- -----
grape {#{Item=grape; Average=9.91335740072202}, #{Item=grape; Average=0}}
orange {#{Item=orange; Average=0.294117647058824}, #{Item=orange; Average=-0.901960784313726...
apple {#{Item=apple; Average=12.1261261261261}, #{Item=apple; Average=-0.0045045045045045}}
pear {#{Item=pear; Average=3.48736462093863}, #{Item=pear; Average=-0.0324909747292419}, #...
then
$tab=#{}
$groupreport.keys | % {$tab += #{$_ = ($groupreport[$_] | measure-object -Property average -sum)}}
gives
PS> $tab["grape"]
Count : 2
Average :
Sum : 9,91335740072202
Maximum :
Minimum :
Property : Average
PS> $tab["grape"].sum
9,91335740072202
It seems short and usable.
Summary
$groupreport = $report | Group-Object -Property item -AsHashTable
$tab = #{}
$groupreport.keys | % {$tab += #{$_ = ($groupreport[$_] | measure-object -Property average -sum)}}
$tab.keys | % {write-host $_ `t $tab[$_].sum}
I don't know if you can get rid of looping. What about:
$report | % {$averages = #{}} {
if ($averages[$_.item]) {
$averages[$_.item] += $_.average
}
else {
$averages[$_.item] = $_.average
}
} {$averages}