My important part of code (this works):
$getFile = Get-Content $file |
ConvertFrom-Csv -Delimiter ";" -Header "Printer","Model","MAC","IP_adresa","ID_terminalu" |
Sort-Object -Property MAC, IP_adresa |
Group-Object MAC |
ForEach {$_.Group | Select-Object -First 1} |
ConvertTo-Csv -NoTypeInformation |
% {$_.Replace('"','')} |
Select-Object -SkipLast 1
My header in the CSV is: "Printer", "Model", "MAC", "IP_adresa", "ID_terminalu".
Problem is that in some specific rows the field "MAC" contains two MAC adresses separated by ,, for example:
MAC
00-1B-A9-85-88-1A,00-22-58-5E-7D-31
After using the script I get output from these specific rows like this:
Printer Model MAC IP_adresa ID_terminalu
TCZ176100 Brother_MFC 00-1B-A9-85-88-1A 00-22-58-5E-7D-31 192.168.x.x
In this specific row the IP adress is shifted by one position and MAC is here 2x and it's bad.
I need to cut off all rows only for MAC header on 17 characters == length one MAC adress. I try really many different ways but NONE WORKED. I don't know how I get values under specific header (when headers have 5 attributes) and edit one specific from them.
I tried for example this (doesn't work):
$getFile = (Get-Content $file | ConvertFrom-Csv -Delimiter ";" -Header 'Tiskarna','Model','MAC','IP_adresa','ID_terminalu') |
select Tiskarna, Model, #{Name="MAC".Substring(0,16);Expression={$_."MAC"}}, IP_adresa, ID_terminalu |
Sort-Object -Property MAC, IP_adresa |
Group-Object MAC |
ForEach {$_.Group | Select-Object -First 1} |
ConvertTo-Csv -NoTypeInformation |
% {$_.Replace('"','')} |
Select-Object -SkipLast 1
Assuming you want only the first of the MAC addresses you should do something like this:
$headers = 'Printer', 'Model', 'MAC', 'IP_adresa', 'ID_terminalu'
$csv = Import-Csv -Delimiter ';' -Header $headers
foreach ($row in $csv) {
if ($row.MAC.Contains(',')) {
$row.MAC = $row.MAC.Split(',')[0]
}
}
$csv | Sort-Object MAC, IP_adresa | ...
Related
I have a csv file where I am trying to export rows into another csv file only where the values in the id column have duplicates.
I have the following csv file...
"id","blablah"
"valOne","valTwo"
"valOne","asdfdsa"
"valThree","valFour"
"valFive","valSix"
"valFive","qwreweq"
"valSeven","valEight"
I need the output csv file to look like the following...
"valOne","valTwo"
"valOne","asdfdsa"
"valFive","valSix"
"valFive","qwreweq"
Here is the code I have so far:
$inputCsv = Import-CSV './test.csv' -delimiter ","
#$output = #()
$inputCsv | Group-Object -prop id, blablah | Where-Object {$_.id -gt 1} |
Select-Object
##{n='id';e={$_.Group[0].id}},
##{n='blablah';e={$_.Group[0].blablah}}
#Export-Csv 'C:\scripts\powershell\output.csv' -NoTypeInformation
#Write-Host $output
#$output | Export-Csv 'C:\scripts\powershell\output.csv' -NoTypeInformation
I've searched multiple how-to's but can't seem to find the write syntax. Can anyone help with this?
Just group on the ID property and if there is more than 1 count in the group then expand those and export.
$inputCsv = Import-CSV './test.csv' -delimiter ","
$inputCsv |
Group-Object -Property ID |
Where-Object count -gt 1 |
Select-Object -ExpandProperty group |
Export-Csv output.csv -NoTypeInformation
output.csv will contain
"id","blablah"
"valOne","valTwo"
"valOne","asdfdsa"
"valFive","valSix"
"valFive","qwreweq"
I am trying to combine several rows into one, provided that the key cell is the same. And write data from all lines with the same key to the final line.
Example Pic
**Before**
ID | Name | DateTime | Duration | Call_Type |
1234509 | Mike | 2020-01-02T01:22:33 | | Start_Call |
1234509 | | 2020-01-02T01:32:33 | 600 | End_call |
AFTER
ID | Name | DateTime | Duration | Start_Call | End_call |
1234509 | Mike | 2020-01-02T01:22:33 | 600 |2020-01-02T01:22:33 | 2020-01-02T01:32:33 |
Before
ID;Name;DateTime;Duration;Call_Type
1234509;Mike;2020-01-02T01:22:33;;Start_Call
1234509;;2020-01-02T01:32:33;600;End_call
After
ID;Name;Duration;Start_Call;End_call
1234509;Mike;600;2020-01-02T01:22:33;2020-01-02T01:32:33
How to use here
$csv | Group-Object ID
and get the data as in the picture?
After grouping by ID with Group-Object, you can iterate each group and create a new System.Management.Automation.PSCustomObject with the properties you want to export in your output CSV file.
For ID we simply use the grouping key. Name and Duration we choose the first object that doesn't have a $null or empty version of that property using System.String.IsNullOrEmpty(). For Start_Call and End_Call we choose the object that has those values for the Call_Type property.
The filtering is done by Where-Object. To get the first and expanded versions of the properties, we also use -First and -ExpandProperty from Select-Object.
$csv = Import-Csv -Path .\data.csv -Delimiter ";"
$groups = $csv | Group-Object -Property ID
& {
foreach ($group in $groups)
{
[PSCustomObject]#{
ID = $group.Name
Name = $group.Group | Where-Object {-not [string]::IsNullOrEmpty($_.Name)} | Select-Object -First 1 -ExpandProperty Name
Duration = $group.Group | Where-Object {-not [string]::IsNullOrEmpty($_.Duration)} | Select-Object -First 1 -ExpandProperty Duration
Start_Call = $group.Group | Where-Object {$_.Call_Type -eq "Start_Call"} | Select-Object -First 1 -ExpandProperty DateTime
End_Call = $group.Group | Where-Object {$_.Call_Type -eq "End_Call"} | Select-Object -First 1 -ExpandProperty DateTime
}
}
} | Export-Csv -Path .\output.csv -Delimiter ";" -NoTypeInformation
output.csv
"ID";"Name";"Duration";"Start_Call";"End_Call"
"1234509";"Mike";"600";"2020-01-02T01:22:33";"2020-01-02T01:32:33"
If you want to remove quotes from the CSV file, you can use the -UseQuotes switch from Export-Csv. However, yhis does require PowerShell 7. If your using a lower PowerShell version, you can use some of the recommendations from How to remove all quotations mark in the csv file using powershell script?.
I'm trying to do script for finding non-unique files.
The script should take one .csv file with data: name of files, LastWriteTime and Length. Then I try to make another .csv based on that one, which will contain only those objects whose combination of Name+Length+LastWriteTime is NON-unique.
I tried following script which uses $csvfile containing files list:
$csvdata = Import-Csv -Path $csvfile -Delimiter '|'
$csvdata |
Group-Object -Property Name, LastWriteTime, Length |
Where-Object -FilterScript { $_.Count -gt 1 } |
Select-Object -ExpandProperty Group -Unique |
Export-Csv $csvfile2 -Delimiter '|' -NoTypeInformation -Encoding Unicode
$csvfile was created by:
{
Get-ChildItem -Path $mainFolderPath -Recurse -File |
Sort-Object $sortMode |
Select-Object Name, LastWriteTime, Length, Directory |
Export-Csv $csvfile -Delimiter '|' -NoTypeInformation -Encoding Unicode
}
(Get-Content $csvfile) |
ForEach-Object { $_ -replace '"' } |
Out-File $csvfile -Encoding Unicode
But somehow in another $csvfile2 there is only the one (first) non-unique record. Does anyone have an idea how to improve it so it can list all non-unique records?
You need to use -Property * -Unique to get a list of unique objects. However, you cannot use -Property and -ExpandProperty at the same time here, because you want the latter parameter to apply to the input objects ($_) and the former parameter to apply to an already expanded property of those input objects ($_.Group).
Expand the property Group first, then select the unique objects:
... |
Select-Object -ExpandProperty Group |
Select-Object -Property * -Unique |
...
I have a data file of about 10-15 columns from which I want to extract specific columns. Some of the columns I know the exact column header and others I only know that the first two letters will always be "FC".
How do I select only the columns where I know the column header and those that start with "FC"?
Starting with just the "FC" columns, I have tried like this:
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t"
$FCcols = $myCSV[0].psobject.Properties | foreach { $_.Name } | Where {$_ -match "FC"}
$myCSV | select $FCcols
But I just get an error:
Select-Object : Cannot convert System.Management.Automation.PSObject to one of
the following types {System.String, System.Management.Automation.ScriptBlock}.
At line:3 char:16
+ $myCSV | select <<<< $FCcols
+ CategoryInfo : InvalidArgument: (:) [Select-Object], NotSupport
edException
+ FullyQualifiedErrorId : DictionaryKeyUnknownType,Microsoft.PowerShell.Co
mmands.SelectObjectCommand
Then, if I try:
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t"
$FCcols = [System.Collections.ArrayList]#()
$myCSV[0].psobject.Properties | foreach { $_.Name } | Where {$_ -match "FC"} | %{$FCcols.Add($_)}
$myCSV | select $FCcols
I get the output I want except that it is in "column header : value" format, like this:
FC1839 : 0
FC1842 : 1
FC1843 : 6
FC1844 : 12
FC1845 : 4
FC1839 : 0
FC1842 : 0
FC1843 : 19
FC1844 : 22
FC1845 : 14
I am probably just missing something simple, but how do I get to the point that I am able to select these matching columns and then output them to another .txt file (without the header : value format)?
First things first: Mathias R. Jessen's helpful tip not only solves your problem, but significantly simplifies the approach (and also works in PSv2):
$myCSV | Select-Object FC*
The (implied) -Property parameter supports wildcard expressions, so FC* matches all property (column names) that start with FC.
As for the output format you're seeing: Because you're selecting 5 properties, PowerShell defaults to implicit Format-List formatting, with each property name-value pair on its own line.
To fix this display problem, pipe to Format-Table explicitly (which is what PowerShell would do implicitly if you had selected 4 or fewer properties):
$myCSV | Select-Object FC* | Format-Table
To re-export the results to a CSV (TSV) file:
Import-Csv mydata.txt -Delimiter "`t" | Select-Object FC* |
Export-Csv myresults.txt -Encoding Utf8 -Delimiter "`t" -NoTypeInformation
To do so without a header line:
Import-Csv mydata.txt -Delimiter "`t" | Select-Object FC* |
ConvertTo-Csv -Delimiter "`t" -NoTypeInformation | Select-Object -Skip 1 |
Set-Content myresults.txt -Encoding Utf8
As for your specific symptom:
The problem occurs only in PSv2, and it smells like a bug to me.
The workaround is make your column-name array a strongly typed string array ([string[]]):
[string[]] $FCcols = $myCSV[0].psobject.Properties | % { $_.Name } | ? { $_ -match '^FC' }
Note that, for brevity, I've used built-in alias % in lieu of ForEach-Object and ? in lieu of Where-Object.
Also note that the regex passed to -match was changed to ^FC to ensure that only columns that start with FC are matched.
Your code works as-is in PSv3+, but can be simplified:
$FCcols = $myCSV[0].psobject.Properties.Name -match "^FC"
Note how .Name is applied directly to .psobject.Properties, which in v3+ causes the .Name member to be invoked on each item of the collection, a feature called member-access enumeration.
I would use Get-Member to get your columns, something like this:
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t"
$myCSV | select ($myCSV | gm -MemberType NoteProperty | ? {$_.Name -match 'FC'}).Name
Mathias's helpful comment is best way to go for selecting; simple and elegant - dind't know it was an option.
$myCSV | Select *FC*,ColumnIKnowTheNameOf
I believe you need to add Export-Csv to answer your last question. Here's another approach I'd already worked on that makes use of Get-Member and NoteProperty if you need to interrogate csv/similar objects in future.
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t"
# you can get the headings by using Get-Member and Selecting NoteProperty members.
$FCcols = $myCSV |
Get-Member |
Where-Object {$_.MemberType -eq "NoteProperty" -and $_.Name -match "FC"} |
Select-Object -ExpandProperty Name
# you add names to this array.
$FCcols += "ColumnIKnowTheNameOf"
$myCSV | Select-Object $FCcols
# to get a tab-delimited file similar to the one you imported, use Export-Csv
$myCSV | Export-csv "myresults.txt" -Delimiter "`t" -NoTypeInformation
I finally came up with a "quick and dirty" solution which I'm disappointed to not have figured out earlier.
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t" | select FC*
for ($i = 0; $i -lt $myCSV.count; $i++){
$writeline = ($myCSV[$i] | %{$_.PSObject.Properties | %{$_.Value}}) -join "`t"
ac "myresults.txt" $writeline -Encoding utf8}
The first line gives me the columns I want, then the for loop gets the value properties of each column and joins them as tabulated lines, finally each line is appended to a text file.
This may not be the pedagogically correct way to achieve the result, but it works so far.
Thanks to everyone for their input!
I'm trying to filter a .csv file based on a location column. The column has various location entries and I only need information from the rows that contain certain locations, that information then gets exported out to a separate .csv file. I can get it to work by searching the .csv file multiple times with each location filter, but I haven't had any luck when trying to combine it into 1 search.
What I have now is:
$csv = Import-Csv "${filepath}\temp1.csv"
$csv | Where-Object location -like "co*" | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -NoTypeInformation
$csv | Where-Object location -like "cc*" | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -Append -NoTypeInformation
$csv | Where-Object location -like "dc*" | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -Append -NoTypeInformation
$csv | Where-Object location -like "mf*" | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -Append -NoTypeInformation
What I'd like to have is something like below. I don't get any errors with it, but all I get is a blank .csv file:
$locations = "co*","cc*","dc*","mf*"
$csv = Import-Csv "${filepath}\temp1.csv"
$csv | Where-Object location -like $locations | select EmployeeNumber | Export-Csv "${filepath}\disablelist.csv" -NoTypeInformation
I've been lurking here for a while and I'm usually able to frankenstein a script together from what I find, but I can't seem to find anything on this. Thanks for your help.
You can replace multiple -like tests with a single -match test using an alternating regex:
$csv = Import-Csv "${filepath}\temp1.csv"
$csv | Where-Object {$_.location -match '^(co|cc|dc|mf)'} |
select EmployeeNumber |
Export-Csv "${filepath}\disablelist.csv" -NoTypeInformation
You can build that regex from a string array:
$locations = 'co','cc','dc','mf'
$LocationMatch = '^({0})' -f ($locations -join '|')
$csv = Import-Csv "${filepath}\temp1.csv"
$csv | Where-Object { $_.location -match $LocationMatch } |
select EmployeeNumber |
Export-Csv "${filepath}\disablelist.csv" -NoTypeInformation