PowerShell. Group-object usage in one file - powershell

I am trying to combine several rows into one, provided that the key cell is the same. And write data from all lines with the same key to the final line.
Example Pic
**Before**
ID | Name | DateTime | Duration | Call_Type |
1234509 | Mike | 2020-01-02T01:22:33 | | Start_Call |
1234509 | | 2020-01-02T01:32:33 | 600 | End_call |
AFTER
ID | Name | DateTime | Duration | Start_Call | End_call |
1234509 | Mike | 2020-01-02T01:22:33 | 600 |2020-01-02T01:22:33 | 2020-01-02T01:32:33 |
Before
ID;Name;DateTime;Duration;Call_Type
1234509;Mike;2020-01-02T01:22:33;;Start_Call
1234509;;2020-01-02T01:32:33;600;End_call
After
ID;Name;Duration;Start_Call;End_call
1234509;Mike;600;2020-01-02T01:22:33;2020-01-02T01:32:33
How to use here
$csv | Group-Object ID
and get the data as in the picture?

After grouping by ID with Group-Object, you can iterate each group and create a new System.Management.Automation.PSCustomObject with the properties you want to export in your output CSV file.
For ID we simply use the grouping key. Name and Duration we choose the first object that doesn't have a $null or empty version of that property using System.String.IsNullOrEmpty(). For Start_Call and End_Call we choose the object that has those values for the Call_Type property.
The filtering is done by Where-Object. To get the first and expanded versions of the properties, we also use -First and -ExpandProperty from Select-Object.
$csv = Import-Csv -Path .\data.csv -Delimiter ";"
$groups = $csv | Group-Object -Property ID
& {
foreach ($group in $groups)
{
[PSCustomObject]#{
ID = $group.Name
Name = $group.Group | Where-Object {-not [string]::IsNullOrEmpty($_.Name)} | Select-Object -First 1 -ExpandProperty Name
Duration = $group.Group | Where-Object {-not [string]::IsNullOrEmpty($_.Duration)} | Select-Object -First 1 -ExpandProperty Duration
Start_Call = $group.Group | Where-Object {$_.Call_Type -eq "Start_Call"} | Select-Object -First 1 -ExpandProperty DateTime
End_Call = $group.Group | Where-Object {$_.Call_Type -eq "End_Call"} | Select-Object -First 1 -ExpandProperty DateTime
}
}
} | Export-Csv -Path .\output.csv -Delimiter ";" -NoTypeInformation
output.csv
"ID";"Name";"Duration";"Start_Call";"End_Call"
"1234509";"Mike";"600";"2020-01-02T01:22:33";"2020-01-02T01:32:33"
If you want to remove quotes from the CSV file, you can use the -UseQuotes switch from Export-Csv. However, yhis does require PowerShell 7. If your using a lower PowerShell version, you can use some of the recommendations from How to remove all quotations mark in the csv file using powershell script?.

Related

export csv rows where duplicate values found in column

I have a csv file where I am trying to export rows into another csv file only where the values in the id column have duplicates.
I have the following csv file...
"id","blablah"
"valOne","valTwo"
"valOne","asdfdsa"
"valThree","valFour"
"valFive","valSix"
"valFive","qwreweq"
"valSeven","valEight"
I need the output csv file to look like the following...
"valOne","valTwo"
"valOne","asdfdsa"
"valFive","valSix"
"valFive","qwreweq"
Here is the code I have so far:
$inputCsv = Import-CSV './test.csv' -delimiter ","
#$output = #()
$inputCsv | Group-Object -prop id, blablah | Where-Object {$_.id -gt 1} |
Select-Object
##{n='id';e={$_.Group[0].id}},
##{n='blablah';e={$_.Group[0].blablah}}
#Export-Csv 'C:\scripts\powershell\output.csv' -NoTypeInformation
#Write-Host $output
#$output | Export-Csv 'C:\scripts\powershell\output.csv' -NoTypeInformation
I've searched multiple how-to's but can't seem to find the write syntax. Can anyone help with this?
Just group on the ID property and if there is more than 1 count in the group then expand those and export.
$inputCsv = Import-CSV './test.csv' -delimiter ","
$inputCsv |
Group-Object -Property ID |
Where-Object count -gt 1 |
Select-Object -ExpandProperty group |
Export-Csv output.csv -NoTypeInformation
output.csv will contain
"id","blablah"
"valOne","valTwo"
"valOne","asdfdsa"
"valFive","valSix"
"valFive","qwreweq"

How cut off all value in column under -Delimiter -Header 'xxx'

My important part of code (this works):
$getFile = Get-Content $file |
ConvertFrom-Csv -Delimiter ";" -Header "Printer","Model","MAC","IP_adresa","ID_terminalu" |
Sort-Object -Property MAC, IP_adresa |
Group-Object MAC |
ForEach {$_.Group | Select-Object -First 1} |
ConvertTo-Csv -NoTypeInformation |
% {$_.Replace('"','')} |
Select-Object -SkipLast 1
My header in the CSV is: "Printer", "Model", "MAC", "IP_adresa", "ID_terminalu".
Problem is that in some specific rows the field "MAC" contains two MAC adresses separated by ,, for example:
MAC
00-1B-A9-85-88-1A,00-22-58-5E-7D-31
After using the script I get output from these specific rows like this:
Printer Model MAC IP_adresa ID_terminalu
TCZ176100 Brother_MFC 00-1B-A9-85-88-1A 00-22-58-5E-7D-31 192.168.x.x
In this specific row the IP adress is shifted by one position and MAC is here 2x and it's bad.
I need to cut off all rows only for MAC header on 17 characters == length one MAC adress. I try really many different ways but NONE WORKED. I don't know how I get values under specific header (when headers have 5 attributes) and edit one specific from them.
I tried for example this (doesn't work):
$getFile = (Get-Content $file | ConvertFrom-Csv -Delimiter ";" -Header 'Tiskarna','Model','MAC','IP_adresa','ID_terminalu') |
select Tiskarna, Model, #{Name="MAC".Substring(0,16);Expression={$_."MAC"}}, IP_adresa, ID_terminalu |
Sort-Object -Property MAC, IP_adresa |
Group-Object MAC |
ForEach {$_.Group | Select-Object -First 1} |
ConvertTo-Csv -NoTypeInformation |
% {$_.Replace('"','')} |
Select-Object -SkipLast 1
Assuming you want only the first of the MAC addresses you should do something like this:
$headers = 'Printer', 'Model', 'MAC', 'IP_adresa', 'ID_terminalu'
$csv = Import-Csv -Delimiter ';' -Header $headers
foreach ($row in $csv) {
if ($row.MAC.Contains(',')) {
$row.MAC = $row.MAC.Split(',')[0]
}
}
$csv | Sort-Object MAC, IP_adresa | ...

Parse existing CSV file and strip certain columns

I have a CSV file which contains, let's say 50 different columns with 1000 rows. I don't need all of this information though and would like to now parse through it and remove the columns for which I do not want.
I want to keep every row (users) so I shouldn't be removing those wholesale, however, I have about a dozen or so columns for which I need to remove the data. How can I do this?
Ex.
User1 | Name | Age | Location | Gender | HairColor
keep | keep | remove | remove | keep | remove
If you want the columns removed completely:
$CSV = Import-Csv $Path | Select-Object -Property User1, Name, Gender
$CSV | Export-Csv $NewPath -NoTypeInformation
Or this, if it's easier:
$CSV = Import-Csv $Path | Select-Object -Property * -ExcludeProperty Age, Location, HairColor
$CSV | Export-Csv $NewPath -NoTypeInformation
If you want the columns to remain but be empty:
$CSV = Import-Csv $Path | Select-Object -Property User1, Name, #{n='Age';e={}}, #{n='Location';e={}}, Gender, #{n='HairColor';e={}}
$CSV | Export-Csv $NewPath -NoTypeInformation

Possible to combine .csv where-object filters?

I'm trying to filter a .csv file based on a location column. The column has various location entries and I only need information from the rows that contain certain locations, that information then gets exported out to a separate .csv file. I can get it to work by searching the .csv file multiple times with each location filter, but I haven't had any luck when trying to combine it into 1 search.
What I have now is:
$csv = Import-Csv "${filepath}\temp1.csv"
$csv | Where-Object location -like "co*" | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -NoTypeInformation
$csv | Where-Object location -like "cc*" | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -Append -NoTypeInformation
$csv | Where-Object location -like "dc*" | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -Append -NoTypeInformation
$csv | Where-Object location -like "mf*" | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -Append -NoTypeInformation
What I'd like to have is something like below. I don't get any errors with it, but all I get is a blank .csv file:
$locations = "co*","cc*","dc*","mf*"
$csv = Import-Csv "${filepath}\temp1.csv"
$csv | Where-Object location -like $locations | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -NoTypeInformation
I've been lurking here for a while and I'm usually able to frankenstein a script together from what I find, but I can't seem to find anything on this. Thanks for your help.
You can replace multiple -like tests with a single -match test using an alternating regex:
$csv = Import-Csv "${filepath}\temp1.csv"
$csv | Where-Object {$_.location -match '^(co|cc|dc|mf)'} |
select EmployeeNumber |
Export-Csv "${filepath}\disablelist.csv" -NoTypeInformation
You can build that regex from a string array:
$locations = 'co','cc','dc','mf'
$LocationMatch = '^({0})' -f ($locations -join '|')
$csv = Import-Csv "${filepath}\temp1.csv"
$csv | Where-Object { $_.location -match $LocationMatch } |
select EmployeeNumber |
Export-Csv "${filepath}\disablelist.csv" -NoTypeInformation

Powershell: How to access and manipulate the "Count" generated from a .CSV

Right now, I have the following code:
import-csv $import | ? {$_.'Summary' -like '*apple*'} | Group-Object {$_.'Value1'} | Sort-Object -Property Name| Format-Table Count,Name -AutoSize|Out-File $export
import-csv $import | ? {$_.'Summary' -like '*orange*'} | Group-Object {$_.'Value1'} | Sort-Object -Property Name| Format-Table Count,Name -AutoSize|Out-File $export -append
import-csv $import | ? {$_.'Summary' -like '*banana*'} | Group-Object {$_.'Value1'} | Sort-Object -Property Name| Format-Table Count,Name -AutoSize|Out-File $export -append
My output looks something like this:
Count Name
----- ----
4 Juice
2 Pie
6 Tart
Count Name
----- ----
36 Juice
1 Pie
Count Name
----- ----
46 Tart
I want to take the values output as the Count from my table, and add them together in a variable. How would I go about doing this?
Use the Measure-Object on the content of the files and store in a variable
$a = (Get-Content $import | Measure-Object -line )
EDIT
$a = 0
import-csv $import | ? {$_.'Summary' -like '*apple*'} | Group-Object {$_.'Value1'} | Sort-Object -Property Name| Foreach {$a += $_.Count} | Format-Table Count,Name -AutoSize|Out-File $export