I have a
List of data in csv and I need to compare it again a set of array.
And then I need to copy the whole data and then insert into new set of array.
Sample data in CSV
ID VersionLabel Created Author Modified Editor FileLeafRef Title Remarks Business_x0020_Unit DE_Department Record_x0020_Disposition_x0020_Date Note
1 0.6 02/11/2020 03:21:12 John 02/11/2020 03:21:12 John fields.csv Test1 Test111 Finance Department A 02/10/2020 23:00:00 notes
1 0.5 02/10/2020 11:16:39 John 02/10/2020 11:16:39 John fields.csv Test1 Test111 Marketing Department A old 02/10/2019 23:00:00 notes old
1 0.4 02/10/2020 10:28:54 John 02/10/2020 10:28:54 John fields.csv Test1 Test111 Sales Department A old 02/10/2019 23:00:00 notes old
1 0.3 02/10/2020 09:29:31 John 02/10/2020 09:29:31 John fields.csv Test1 Test111 Sales Department A old 02/10/2020 23:00:00 notes old
1 0.2 02/10/2020 09:28:35 John 02/10/2020 09:28:35 John fields.csv Test1 Test111 Sales Department A old 02/10/2020 23:00:00 notes old
Set of Arrary that I have
ID
VersionLabel
Created
Author
which basically i got using following code
$FinalReport = Import-Csv $OriginalReport -Delimiter ","
$Header = $FinalReport[0].psobject.properties.name
$FinalReportGrouped = $FinalReport | Group-Object {$_.ID} |Foreach-Object {
$_.Group | Sort-Object VersionLabel -Descending | Select-Object -First 1
}
The problem is, I'm not able to come out with the proper logic on how to compare the set of array.
Example, below is the code to compare it..
foreach ($row in $FinalReportGrouped)
{
$FinalReport | Where-Object {$_.ID -eq $row.ID} | Foreach-Object {
#I need to compare here, if the value inside $ is not equal $header array
if($_.value -ne $header)
{
# and then I need to copy whole content of $row into $_ variable, below code seems got problem. I can copy if using $_.Department = $row.Department, but I need to get it dynamically.
$_ = $row #this is wrong maybe..
}
}
}
Related
I have been using code from this answer to check for additions/changes to class rosters from MS Teams:
$set = [System.Collections.Generic.HashSet[string]]::new(
[string[]] (Import-CSV -Path stundent.csv).UserPrincipalName,
[System.StringComparer]::InvariantCultureIgnoreCase
)
Import-Csv ad.csv | Where-Object { $set.Add($_.UserPrincipalName) } |
Export-Csv path\to\output.csv -NoTypeInformation
Ideally, I want to be able to check if there have been removals when compared to a new file, swap the import file positions, and check for additions. If my files look like Source1 and Source2 (below), the check for removals would return Export1, and the check for additions would return Export2.
Since there will be multiple instances of students across multiple classes, I want to include TeamDesc in the filter query to make sure only the specific instance of that student with that class is returned.
Source1.csv
TeamDesc
UserPrincipalName
Name
Team 1
student1#domain.com
john smith
Team 1
student2#domain.com
nancy drew
Team 2
student3#domain.com
harvey dent
Team 3
student1#domain.com
john smith
Source2.csv
TeamDesc
UserPrincipalName
Name
Team 1
student2#domain.com
nancy drew
Team 2
student3#domain.com
harvey dent
Team 2
student4#domain.com
tim tams
Team 3
student1#domain.com
john smith
Export1.csv
TeamDesc
UserPrincipalName
Name
Team 1
student1#domain.com
john smith
Export2.csv
TeamDesc
UserPrincipalName
Name
Team 2
student4#domain.com
tim tams
Try the following, which uses Compare-Object to compare the CSV files by two column values, simply by passing the property (column) names of interest to -Property; the resulting output is split into two collections based on which input side a differing property combination is unique to, using the intrinsic .Where() method:
$removed, $added = (
Compare-Object (Import-Csv Source1.csv) (Import-Csv Source2.csv) -PassThru `
-Property TeamDesc, UserPrincipalName
).Where({ $_.SideIndicator -eq '=>' }, 'Split')
$removed |
Select-Object -ExcludeProperty SideIndicator |
Export-Csv -NoTypeInformation Export1.csv
$added |
Select-Object -ExcludeProperty SideIndicator |
Export-Csv -NoTypeInformation Export2.csv
Assuming both Csvs are stored in memory, Source1.csv is $csv1 and Source2.csv is $csv2, you already have the logic for Export2.csv using the HashSet<T>:
$set = [System.Collections.Generic.HashSet[string]]::new(
[string[]] $csv1.UserPrincipalName,
[System.StringComparer]::InvariantCultureIgnoreCase
)
$csv2 | Where-Object { $set.Add($_.UserPrincipalName) }
Outputs:
TeamDesc UserPrincipalName Name
-------- ----------------- ----
Team 2 student4#domain.com tim tams
For the first requirement, Export1.csv, the reference object would be $csv2 and instead of a HashSet<T> you could use a hash table, Group-Object -AsHashTable makes it really easy in this case:
$map = $csv2 | Group-Object UserPrincipalName -AsHashTable -AsString
# if Csv2 has unique values for `UserPrincipalName`
$csv1 | Where-Object { $map[$_.UserPrincipalName].TeamDesc -ne $_.TeamDesc }
# if Csv2 has duplicated values for `UserPrincipalName`
$csv1 | Where-Object { $_.TeamDesc -notin $map[$_.UserPrincipalName].TeamDesc }
Outputs:
TeamDesc UserPrincipalName Name
-------- ----------------- ----
Team 1 student1#domain.com john smith
Using this Join-Object script/Join-Object Module (see also: How to compare two CSV files and output the rows that are just in either of the file but not in both and In Powershell, what's the best way to join two tables into one?):
Loading your sample data:
(In your case you probably want to use Import-Csv to import your data)
Install-Script -Name Read-HtmlTable
$Csv1 = Read-HtmlTable https://stackoverflow.com/q/74452725 -Table 0 # Import-Csv .\Source1.csv
$Csv2 = Read-HtmlTable https://stackoverflow.com/q/74452725 -Table 1 # Import-Csv .\Source2.csv
Install-Module -Name JoinModule
$Csv1 |OuterJoin $Csv2 -On TeamDesc, UserPrincipalName -Name Out,In
TeamDesc UserPrincipalName OutName InName
-------- ----------------- ------- ------
Team 1 student1#domain.com john smith
Team 2 student4#domain.com tim tams
You might use the (single) result file as is. If you really want to work with two different files, you might split the results as in the nice answer from mklement0.
I am working on a scenario where my csv input file is as following:
Name
Number
Priority
John
123
Second
Rocky
345
Third
Tony
234
First
Rocky
345
Second
Matt
999
Second
Bernard
888
Third
Matt
999
First
Jacob
789
Second
Angela
777
Second
Jacob
789
First
Here through PowerShell, I am trying to get unique rows for Name, having a condition on priority column as it has values as First or Second then First should appear and vice versa if it has Second or Third then Second should appear.
Something like following
Name
Number
Priority
John
123
Second
Rocky
345
Second
Tony
234
First
Matt
999
First
Bernard
888
Third
Jacob
789
First
Angela
777
Second
I am not able to get it done, could any one throw any lead please.
Thanks
There are a number of ways you could do this. As your examples happen to be all sortable in the wanted order, I have added an extra line to the csv so also a priority 'Fourth' comes into the equasion:
input.csv
Name,Number,Priority
John,123,Second
Rocky,345,Third
Tony,234,First
Rocky,345,Second
Matt,999,Second
Bernard,888,Third
Matt,999,First
Jacob,789,Second
Angela,777,Second
Jacob,789,First
Angela,777,Fourth
Here's three methods you can use:
Method 1: create a Hashtable with the priorities and the order number for the sort
$prios = #{
First = 0
Second = 1
Third = 2
Fourth = 3
# etc
}
Import-Csv -Path 'D:\Test\input.csv' | Group-Object Name | ForEach-Object {
$_.Group | Sort-Object {$prios[$_.Priority]} | Select-Object -First 1
} | Export-Csv -Path 'D:\Test\output.csv' -NoTypeInformation
Method 2: create a string array with the priorities in the order you need
$prios = 'First','Second','Third','Fourth' # etc
Import-Csv -Path 'D:\Test\input.csv' | Group-Object Name | ForEach-Object {
$_.Group | Sort-Object {$prios.IndexOf($_.Priority)} | Select-Object -First 1
} | Export-Csv -Path 'D:\Test\output.csv' -NoTypeInformation
Method 3: create an enum with the priorities in the order you need
Enum prios {
First = 0
Second = 1
Third = 2
Fourth = 3
# etc
}
Import-Csv -Path 'D:\Test\input.csv' | Group-Object Name | ForEach-Object {
$_.Group | Sort-Object {[prios]::($_.Priority)} | Select-Object -First 1
} | Export-Csv -Path 'D:\Test\output.csv' -NoTypeInformation
Output on all cases when displayed on screen:
Name Number Priority
---- ------ --------
John 123 Second
Rocky 345 Second
Tony 234 First
Matt 999 First
Bernard 888 Third
Jacob 789 First
Angela 777 Second
I've been stuck on this for a little while however I've got an array of People and im trying to get the last person and creating a seperate column with that person only.
I've played around with #{NAME = 'NAME' Expression = {}} in Select-Object but I don't really know how to tackle it.
Current:
| Employee |
|---------------|
| John Doe |
| Jane West |
| Jordan Row |
| Paul Willson |
| Andrew Wright |
Desired Result:
| Employee | Employee2 |
|--------------|---------------|
| John Doe | |
| Jane West | |
| Jordan Row | |
| Paul Willson | Andrew Wright |
TIA!
So what I decided to do here is create 2 groups. One group contains all of the values except the last 2, and the other group contains these last 2 values
# create the sample array
$employees = #(
'John Doe'
'Jane West'
'Jordan Row'
'Paul Willson'
'Andrew Wright'
)
$employees |
# Separate objects into 2 groups: those contained in the last 2 values and those not contained in the last 2 values
Group-Object {$_ -in ($employees | Select-Object -Last 2)} |
ForEach-Object {
switch ($_) {
{$_.name -eq 'False'} { # 'False' Name of group where values are not one of the last 2
# Iterate through all the values and assign them to Employee property. Leave Employee2 property blank
$_.group | ForEach-Object {
[PSCustomObject]#{
Employee = $_
Employee2 = ''
}
}
}
{$_.name -eq 'True'} { # 'True' Name of group where values are those of the last 2
# Create an object that assigns the values to Employee and Employee2
[PSCustomObject]#{
Employee = $_.group[0]
Employee2 = $_.group[1]
}
}
}
}
Output
Employee Employee2
-------- ---------
John Doe
Jane West
Jordan Row
Paul Willson Andrew Wright
Edit
Here is another way you can do it
$employees[0..($employees.Count-3)] | ForEach-Object {
[PSCustomObject]#{
Employee = $_
Employee2 = ''
}
}
[PSCustomObject]#{
Employee = $employees[-2]
Employee2 = $employees[-1]
}
I put this question on here before, but I missed an important detail which causes huge issue. There will be duplicate account numbers. So I'm doing it by current_read_date now to avoid duplicates in account number. To ensure values being added to $accounts are new from the CSV.
I am trying to get all the accounts from $f which do not match the accounts in $table4 into $accounts. But I need to also check if the current_read_date matches or not.
CSV into Array $f:
Account_no |occupant_code|current_read_date
-----------|-------------|-----------------
12345 | 1 | 7/17/2017 15:32:00 AM
67890 | 2 | 7/17/2017 12:00:00 AM
45678 | 3 | 7/17/2017 12:00:00 AM
DataTable $table4
Account_no |occupant_code|current_read_date
-----------|-------------|-----------------
12345 | 1 | 7/17/2017 12:00:00 AM
12345 | 1 | 7/17/2017 15:32:00 AM
67890 | 1 | 7/17/2017 13:00:00 AM
67890 | 1 | 7/17/2017 22:00:00 AM
45678 | 3 | 7/17/2017 12:00:00 AM
Desired result:
$accounts =
67890 | 2 | 7/17/2017 12:00:00 AM
Current code:
$accounts = Import-Csv $f |
select account_no, current_read_date |
where { $table4.account_no -notcontains $_.account_no }
What this needs to do is to check that current_read_date doesn't match, i.e.:
12345: account and date from $f and $table4 match; so it's ignored
67890: account matches $table4, but the current_read_date does not match, so it is a new value, thus it is added to $accounts.
I believe I need to use Group-Object, but I do not know how to use that correctly.
I tried:
Import-Csv $f |
select account_no, occupant_code |
Group-Object account_no |
Where-Object { $_.Group.current_read_date -notcontains $table4.current_read_date }
This is the previous question:
How to use Group-Object on this?
All the answers here failed because I forgot to provide the information that account_no is not unique; there will be frequent duplicates.
All assistance would be greatly appreciated, I've been stuck on this for awhile.
I've also tried this
$testList = #()
$testList = Import-Csv $f | select account_no, occupant_code, current_read_date, current_reading
$accounts = new-object System.Collections.ArrayList
$testSet = $table4
foreach($myThing in $testList)
{
if($myThing.account_no -in $testSet.account_no )
{
foreach($ts in $testSet)
{
if ($myThing.account_no -match $ts.account_no -and $myThing.occupant_code -match $ts.occupant_code)
{
$ts.account_no
$ts.occupant_code
}
else {
$accounts.add($myThing) | out-null
write-host $mything
}
}
}
This fails because it goes through each number, therefore, 12345 will be checked against 67890, and will added 12345 to the $accounts list, even though it already exists, because I cannot compare each individually at a time with table4.
Thanks
$accounts = $f | Where {
$Record = $_
$AccNo = $table4 | Where {$_.Account_no -eq $Record.Account_no}
!($AccNo | Where {$_.current_read_date -eq $Record.current_read_date})
}
$accounts | Format-Table
Result:
Account_no occupant_code current_read_date
---------- ------------- -----------------
67890 2 7/17/2017 12:00:00 AM
Build a reference list from the records in $table4
$ref = $table4 | ForEach-Object {
$_.Account_no, $_.occupant_code, $_.current_read_date -join ','
}
then filter the records from $f by that reference list:
$accounts = Import-Csv $f | Where-Object {
$ref -notcontains ($_.Account_no, $_.occupant_code, $_.current_read_date -join ',')
} | Select-Object -Expand Account_no -Unique
I need to combine a slew of Excel spreadsheets. I used PowerSHell to convert them to CSVs and now need to merge them, but not as you typically would. The merge doesn't use a join. If I have 3 files with 100 rows each, my new file should have 300 rows. So, this is more if a UNION than a JOIN to use database terms.
Some of the columns do have the same name. Some don't. If they have the same name, a new column shouldn't be created. Is there a way to do this without manually having to list out all the columns as properties?
Example (with only 2 files)
File1:
Name Address
Bob 123 Main
File2:
Name City
Bob LA
Tom Boston
Results
Name Address City
Bob 123 Main
Bob LA
Tom Boston
At the end of the day this might not be sorted right. The trick here is to read the header of each file and collect it as a string array and remove and of the duplicates.
This code assumes all the files are in the same location. If not you will need to account for that.
$files = Get-ChildItem -Path 'C:\temp\csv\' -Filter '*.csv' | Select-Object -ExpandProperty FullName
# Gather the headers for all the files.
$headers = $files | ForEach-Object{
(Get-Content $_ -Head 1).Split(",") | ForEach-Object{$_.Trim()}
} | Sort-Object -Unique
# Loop again now and read in the csv files as objects
$files | ForEach-Object{
Import-Csv $_
} | Select-Object $headers
The output would look like this:
Address City Name
------- ---- ----
123 Main Bob
LA Bob
Boston Tom