Outputting sorted sections of sorted data to a variable? - powershell

I have the following PowerShell code:
[xml]$xml = Get-Content uccxResourceList_forReference.xml
$xml.resources.resource |
Select-Object firstname, lastname, extension,
#{Name="Team"; Expression={($_.team.name)}} |
Sort Team |
Format-Table
which produces a table like this:
firstName lastName extension Team
--------- -------- --------- ----------------
Homer Simpson 1000 SafetyInspectors
Frank Grimes 1001 SafetyInspectors
Lenford Leonard 1002 SafetyInspectors
Carlton Carlson 1003 SafetyInspectors
Montgomery Burns 2000 Executives
Waylon Smithers 2001 Executives
What I would like to do is output each team into its own file. So not just a simple | Out-File teamlist.txt at the end, but I would like to output a text file containing all of the "SafetyInspectors" and another with all of the "Executives".
I know I could get this done with a subsequent foreach loop but I feel it could also be done in the pipeline and I just don't know how to do it.

I'd prefer to output to a csv file (which easily is imported again) , so:
[xml]$xml = Get-Content uccxResourceList_forReference.xml
$xml.resources.resource |
Select-Object firstname, lastname, extension,
#{Name="Team"; Expression={($_.team.name)}} |
Group-Object Team | ForEach-Object {
$_.Group | Export-Csv ("{0}.csv" -f $_.Name) -NoTypeInformation
}
Should return something like this:
> gc .\Executives.csv
"firstName","lastName","extension","Team"
"Montgomery","Burns","2000","Executives"
"Waylon","Smithers","2001","Executives"
> gc .\SafetyInspectors.csv
"firstName","lastName","extension","Team"
"Homer","Simpson","1000","SafetyInspectors"
"Frank","Grimes","1001","SafetyInspectors"
"Lenford","Leonard","1002","SafetyInspectors"
"Carlton","Carlson","1003","SafetyInspectors"

Related

how can I correct my reconciliation of .csv files to remove dupes/nulls

I have been using code from this answer to check for additions/changes to class rosters from MS Teams:
$set = [System.Collections.Generic.HashSet[string]]::new(
[string[]] (Import-CSV -Path stundent.csv).UserPrincipalName,
[System.StringComparer]::InvariantCultureIgnoreCase
)
Import-Csv ad.csv | Where-Object { $set.Add($_.UserPrincipalName) } |
Export-Csv path\to\output.csv -NoTypeInformation
Ideally, I want to be able to check if there have been removals when compared to a new file, swap the import file positions, and check for additions. If my files look like Source1 and Source2 (below), the check for removals would return Export1, and the check for additions would return Export2.
Since there will be multiple instances of students across multiple classes, I want to include TeamDesc in the filter query to make sure only the specific instance of that student with that class is returned.
Source1.csv
TeamDesc
UserPrincipalName
Name
Team 1
student1#domain.com
john smith
Team 1
student2#domain.com
nancy drew
Team 2
student3#domain.com
harvey dent
Team 3
student1#domain.com
john smith
Source2.csv
TeamDesc
UserPrincipalName
Name
Team 1
student2#domain.com
nancy drew
Team 2
student3#domain.com
harvey dent
Team 2
student4#domain.com
tim tams
Team 3
student1#domain.com
john smith
Export1.csv
TeamDesc
UserPrincipalName
Name
Team 1
student1#domain.com
john smith
Export2.csv
TeamDesc
UserPrincipalName
Name
Team 2
student4#domain.com
tim tams
Try the following, which uses Compare-Object to compare the CSV files by two column values, simply by passing the property (column) names of interest to -Property; the resulting output is split into two collections based on which input side a differing property combination is unique to, using the intrinsic .Where() method:
$removed, $added = (
Compare-Object (Import-Csv Source1.csv) (Import-Csv Source2.csv) -PassThru `
-Property TeamDesc, UserPrincipalName
).Where({ $_.SideIndicator -eq '=>' }, 'Split')
$removed |
Select-Object -ExcludeProperty SideIndicator |
Export-Csv -NoTypeInformation Export1.csv
$added |
Select-Object -ExcludeProperty SideIndicator |
Export-Csv -NoTypeInformation Export2.csv
Assuming both Csvs are stored in memory, Source1.csv is $csv1 and Source2.csv is $csv2, you already have the logic for Export2.csv using the HashSet<T>:
$set = [System.Collections.Generic.HashSet[string]]::new(
[string[]] $csv1.UserPrincipalName,
[System.StringComparer]::InvariantCultureIgnoreCase
)
$csv2 | Where-Object { $set.Add($_.UserPrincipalName) }
Outputs:
TeamDesc UserPrincipalName Name
-------- ----------------- ----
Team 2 student4#domain.com tim tams
For the first requirement, Export1.csv, the reference object would be $csv2 and instead of a HashSet<T> you could use a hash table, Group-Object -AsHashTable makes it really easy in this case:
$map = $csv2 | Group-Object UserPrincipalName -AsHashTable -AsString
# if Csv2 has unique values for `UserPrincipalName`
$csv1 | Where-Object { $map[$_.UserPrincipalName].TeamDesc -ne $_.TeamDesc }
# if Csv2 has duplicated values for `UserPrincipalName`
$csv1 | Where-Object { $_.TeamDesc -notin $map[$_.UserPrincipalName].TeamDesc }
Outputs:
TeamDesc UserPrincipalName Name
-------- ----------------- ----
Team 1 student1#domain.com john smith
Using this Join-Object script/Join-Object Module (see also: How to compare two CSV files and output the rows that are just in either of the file but not in both and In Powershell, what's the best way to join two tables into one?):
Loading your sample data:
(In your case you probably want to use Import-Csv to import your data)
Install-Script -Name Read-HtmlTable
$Csv1 = Read-HtmlTable https://stackoverflow.com/q/74452725 -Table 0 # Import-Csv .\Source1.csv
$Csv2 = Read-HtmlTable https://stackoverflow.com/q/74452725 -Table 1 # Import-Csv .\Source2.csv
Install-Module -Name JoinModule
$Csv1 |OuterJoin $Csv2 -On TeamDesc, UserPrincipalName -Name Out,In
TeamDesc UserPrincipalName OutName InName
-------- ----------------- ------- ------
Team 1 student1#domain.com john smith
Team 2 student4#domain.com tim tams
You might use the (single) result file as is. If you really want to work with two different files, you might split the results as in the nice answer from mklement0.

Check if a value exists in one csv file and add static text to another csv field with Powershell

I need to check if the column value from the first csv file, exists in any of the three other csv files and return a value into a new csv file, in order of precedence.So if the username field from allStaff.csv exists in the list of usernames in the sessionVPNct.csv file, put the static text into the final csv file as 'VPN'. If it does not exist, check the next csv file: sessionCRXct.csv then put the static text 'CRX', if not check the last csv file: sessionTMSct.csv then put the static text: TM if not the put the static text 'none' into the final csv file.
I have four csv files as below:
1. allStaff.csv
2. VPN.csv
3. CRX.csv
4. TMS.csv
I have imported the csv files into variables as below:
$allUsers = Import-Csv -Path "C:\allStaff.csv"
$vpn = Import-Csv -Path "C:\VPN.csv" | Select-Object -ExpandProperty UserName
$crx = Import-Csv -Path "C:\CRX.csv" | Select-Object -ExpandProperty UserName
$tms = Import-Csv -Path "C:\TMS.csv" | Select-Object -ExpandProperty UserName
The $allUsers variable displays the following:
Firstname LastName Username Position Area
--------- -------- -------- -------- ----
Joe Bloggs jbloggs Gardener Maintenance
Jim Smith jsmith Accountant Finance
Bob Seger bseger HR Advisor Human Resources
Adam Boson aboson Customer Support IT
Adele bree abree Payroll Finance
The $vpn variable displays the following:
Username
--------
jbloggs
jsmith
The $crx variable displays the following:
Username
--------
jbloggs
jsmith
bseger
The $tms variable displays the following:
Username
--------
jbloggs
jsmith
bseger
aboson
Then I have the following line to start the result csv file
$result = $allUsers | Select-Object *,ConnectionMethod
Not quite sure how to do the final query, which I believe should be an if else loop to go through all rows in the $result variable, and check the other csv if the username field exists, then return the static text.
$result | Export-Csv -Path "C:\allStaffConnections.csv"
This is how I need the final allStaffConnections.csv file to be displayed.
Firstname LastName Username Position Area ConnectionMethod
--------- -------- -------- -------- ---- --------------
Joe Bloggs jbloggs Gardener Maintenance VPN
Jim Smith jsmith Accountant Finance VPN
Bob Seger bseger HR Advisor Human Resources CRX
Adam Boson aboson Customer Support IT TMS
Adele bree abree Payroll Finance none
Am I on the right track with the below code?
$allUsers = Import-Csv -Path "C:\allStaff.csv"
$vpn = Import-Csv -Path "C:\VPN.csv" | Select-Object -ExpandProperty UserName
$crx = Import-Csv -Path "C:\CRX.csv" | Select-Object -ExpandProperty UserName
$tms = Import-Csv -Path "C:\TMS.csv" | Select-Object -ExpandProperty UserName
$vpnText = 'VPN'
$crxText = 'CRX'
$txsText = 'TMS'
$noneText = 'none'
$allUsersExtended = $allUsers | Select-Object *,ConnectionMethod
$results = $allUsersExtended.ForEach(
{
if($vpn -Contains $PSItem.UserName) {
# add $vpnText to ConnectionMethod column for that row in the $result
$PSItem.ConnectionMethod = $vpnText
}elseif($crx -Contains $PSItem.UserName) {
# add $crxText to ConnectionMethod column for that row in the $result
$PSItem.ConnectionMethod = $crxText
}elseif($tms -Contains $PSItem.UserName) {
# add $txsText to ConnectionMethod column for that row in the $result
$PSItem.ConnectionMethod = $tmsText
}else {
# add $noneText to ConnectionMethod column for that row in the $result
$PSItem.ConnectionMethod = $noteText
}
})
$results | Export-Csv -Path "C:\allStaffConnections.csv" -NoTypeInformation
This gives me an empty allStaffConnections.csv file.
I have run the code line by line and can get as far as:
$allUsersExtended = $allUsers | Select-Object *,ConnectionMethod
Which gives me the extra column "ConnectionMethod", but after running the loop, it gives me an empty allStaffConnections.csv file.
here is one way to do the job. [grin] it presumes that you only want to 1st connection type found. if you want all of them [for instance, JBloggs has all 3 types listed], you will need to concatenate them.
what it does ...
fakes reading in the CSV files
when ready to use real data, comment out or remove the entire #region/#endregion section and use Get-Content.
iterates thru the main collection
uses a switch to test for membership in each connection type list
this breaks out of the switch when it finds a match since it presumes you only want the 1st match. if you want all of them, then you will need to accumulate them instead of breaking out of the switch block.
sets the $ConnectionType as appropriate
builds a PSCO with all the wanted props
this likely could be shortened by using Select-Object, a wildcard property selector, and a calculated property.
sends it out to the $Results collection
shows it on screen
saves it to a CSV file
the code ...
#region >>> fake reading in CSV files
# in real life, use Import-CSV
$AllUsers = #'
FirstName, LastName, UserName, Position, Area
Joe, Bloggs, jbloggs, Gardener, Maintenance
Jim, Smith, jsmith, Accountant, Finance
Bob, Seger, bseger, HR Advisor, Human Resources
Adam, Boson, aboson, Customer Support, IT
Adele, bree, abree, Payroll, Finance
'# | ConvertFrom-Csv
$Vpn = #'
UserName
jbloggs
jsmith
'# | ConvertFrom-Csv
$Crx = #'
UserName
jbloggs
jsmith
bseger
'# | ConvertFrom-Csv
$Tms = #'
UserName
jbloggs
jsmith
bseger
aboson
'# | ConvertFrom-Csv
#endregion >>> fake reading in CSV files
$Results = foreach ($AU_Item in $AllUsers)
{
# this presumes you want only the 1st connection type found
# if you want all of them, then you will need to concatenate them
switch ($AU_Item.UserName)
{
{$_ -in $Vpn.UserName}
{
$ConnectionType = 'VPN'
break
}
{$_ -in $Crx.UserName}
{
$ConnectionType = 'CRX'
break
}
{$_ -in $Tms.UserName}
{
$ConnectionType = 'TMS'
break
}
default
{
$ConnectionType = 'None'
}
}
[PSCustomObject]#{
FirstName = $AU_Item.FirstName
LastName = $AU_Item.LastName
UserName = $AU_Item.UserName
Position = $AU_Item.Position
Area = $AU_Item.Area
ConnectionTYpe = $ConnectionType
}
}
# on screen
$Results
# send to CSV
$Results |
Export-Csv -LiteralPath "$env:TEMP\brokencrow_-_UserConnectionType.csv" -NoTypeInformation
truncated on screen output ...
FirstName : Joe
LastName : Bloggs
UserName : jbloggs
Position : Gardener
Area : Maintenance
ConnectionTYpe : VPN
[*...snip...*]
FirstName : Adele
LastName : bree
UserName : abree
Position : Payroll
Area : Finance
ConnectionTYpe : None
the CSV file content from brokencrow_-_UserConnectionType.csv ...
"FirstName","LastName","UserName","Position","Area","ConnectionTYpe"
"Joe","Bloggs","jbloggs","Gardener","Maintenance","VPN"
"Jim","Smith","jsmith","Accountant","Finance","VPN"
"Bob","Seger","bseger","HR Advisor","Human Resources","CRX"
"Adam","Boson","aboson","Customer Support","IT","TMS"
"Adele","bree","abree","Payroll","Finance","None"

How to get a numbered count of different items in powershell

Im using the following powershell query to get a list of disks:
Get-NcDisk | Select-Object -Property model | Sort-Object -Property Model -Descending | foreach {$_.model}
It outputs like below:
X316_SMKRE06TA07
X316_HARIH06TA07
X316_HARIH06TA07
X316_HARIH06TA07
How can I get it to output a numbered count of each type of disk like below:
1 X316_SMKRE06TA07
3 X316_HARIH06TA07
Group-Object will do this for you..
I can't use Get-NcDisk but it may just be:
Get-NcDisk | Select-Object -ExpandProperty model | Group-Object
Example output using a string array:
"X316_SMKRE06TA07","X316_HARIH06TA07","X316_HARIH06TA07","X316_HARIH06TA07" | Group-Object
Count Name Group
----- ---- -----
1 X316_SMKRE06TA07 {X316_SMKRE06TA07}
3 X316_HARIH06TA07 {X316_HARIH06TA07, X316_HARIH06TA07, X316_HARIH06TA07}

Compare-Object and include properties not being compared in output

I'm comparing two CSV files that come from different sources (different column/property names) using the Compare-Object cmdlet. How can I include properties that are in either CSV file in the output without including them in the comparison?
Example CSV data
users1.csv
e-mail-address,name,side
luke#sw.com,Luke,light
users2.csv
e-mail-address,hiredate,hobbies
lando#sw.com,5/2/17,Sabacc
The following gives me a column with the e-mail address and side indicator, but how can I get $Users1.name and $Users2.hiredate without using them in the comparison?
$Users1 = Import-Csv users1.csv
$Users2 = Import-Csv users2.csv
Compare-Object $Users1 $Users2 -Property "E-mail-Address"
I'd like output similar to:
e-mail-address | SideIndicator | name | hiredate
---------------|---------------|------|----------
luke#sw.com | <= | Luke |
lando#sw.com | => | | 5/2/17
Add the PassThru parameter to have Compare-Object return all the properties, then use Select-Object to grab the name and hiredate properties:
Compare-Object $users1 $users2 -Property e-mail-address -PassThru|Select-Object e-mail-address,SideIndicator,name,hiredate

Merge csv's - no join

I need to combine a slew of Excel spreadsheets. I used PowerSHell to convert them to CSVs and now need to merge them, but not as you typically would. The merge doesn't use a join. If I have 3 files with 100 rows each, my new file should have 300 rows. So, this is more if a UNION than a JOIN to use database terms.
Some of the columns do have the same name. Some don't. If they have the same name, a new column shouldn't be created. Is there a way to do this without manually having to list out all the columns as properties?
Example (with only 2 files)
File1:
Name Address
Bob 123 Main
File2:
Name City
Bob LA
Tom Boston
Results
Name Address City
Bob 123 Main
Bob LA
Tom Boston
At the end of the day this might not be sorted right. The trick here is to read the header of each file and collect it as a string array and remove and of the duplicates.
This code assumes all the files are in the same location. If not you will need to account for that.
$files = Get-ChildItem -Path 'C:\temp\csv\' -Filter '*.csv' | Select-Object -ExpandProperty FullName
# Gather the headers for all the files.
$headers = $files | ForEach-Object{
(Get-Content $_ -Head 1).Split(",") | ForEach-Object{$_.Trim()}
} | Sort-Object -Unique
# Loop again now and read in the csv files as objects
$files | ForEach-Object{
Import-Csv $_
} | Select-Object $headers
The output would look like this:
Address City Name
------- ---- ----
123 Main Bob
LA Bob
Boston Tom