I have got a source CSV file (without a header, all columns delimited by a comma) which I am trying split out into separate CSV files based upon the value in the first column and using that column value as the output file name.
Input file:
S00000009,2016,M04 01/07/2016,0.00,0.00,0.00,0.00,0.00,0.00,750.00,0.00,0.00
S00000009,2016,M05 01/08/2016,0.00,0.00,0.00,0.00,0.00,0.00,600.00,0.00,0.00
S00000009,2016,M06 01/09/2016,0.00,0.00,0.00,0.00,0.00,0.00,600.00,0.00,0.00
S00000010,2015,W28 05/10/2015,2275.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
S00000010,2015,W41 04/01/2016,0.00,0.00,0.00,0.00,0.00,0.00,568.75,0.00,0.00
S00000010,2015,W42 11/01/2016,0.00,0.00,0.00,0.00,0.00,0.00,568.75,0.00,0.00
S00000012,2015,W10 01/06/2015,0.00,0.00,0.00,0.00,0.00,0.00,650.00,0.00,0.00
S00000012,2015,W11 08/06/2015,0.00,0.00,0.00,0.00,0.00,0.00,650.00,0.00,0.00
S00000012,2015,W12 15/06/2015,0.00,0.00,0.00,0.00,0.00,0.00,650.00,0.00,0.00
My PowerShell script looks like this:
Import-Csv INPUT_FILE.csv -Header service_id,year,period,cash_exp,cash_inc,cash_def,act_exp,act_inc,act_def,comm_exp,comm_inc,comm_def |
Group-Object -Property "service_id" |
Foreach-Object {
$path = $_.Name + ".csv";
$_.group | Export-Csv -Path $path -NoTypeInformation
}
Output files:
S00000009.csv:
"service_id","year","period","cash_exp","cash_inc","cash_def","act_exp","act_inc","act_def","comm_exp","comm_inc","comm_def"
"S00000009","2016","M04 01/07/2016","0.00","0.00","0.00","0.00","0.00","0.00","750.00","0.00","0.00"
"S00000009","2016","M05 01/08/2016","0.00","0.00","0.00","0.00","0.00","0.00","600.00","0.00","0.00"
"S00000009","2016","M06 01/09/2016","0.00","0.00","0.00","0.00","0.00","0.00","600.00","0.00","0.00"
S00000010.csv:
"service_id","year","period","cash_exp","cash_inc","cash_def","act_exp","act_inc","act_def","comm_exp","comm_inc","comm_def"
"S00000010","2015","W28 05/10/2015","2275.00","0.00","0.00","0.00","0.00","0.00","0.00","0.00","0.00"
"S00000010","2015","W41 04/01/2016","0.00","0.00","0.00","0.00","0.00","0.00","568.75","0.00","0.00"
"S00000010","2015","W42 11/01/2016","0.00","0.00","0.00","0.00","0.00","0.00","568.75","0.00","0.00"
It is generating the new files using the header value in column 1 (service_id).
There are 2 problems.
The output CSV file contains a header row which I don't need.
The columns are enclosed with double quotes which I don't need.
First of all the .csv file needs headers and the quote marks as a csv file structure. But if you don't want them then you can go on with a text file or...
$temp = Import-Csv INPUT_FILE.csv -Header service_id,year,period,cash_exp,cash_inc,cash_def,act_exp,act_inc,act_def,comm_exp,comm_inc,comm_def | Group-Object -Property "service_id" |
Foreach-Object {
$path=$_.name+".csv"
$temp0 = $_.group | ConvertTo-Csv -NoTypeInformation | Select-Object -Skip 1
$temp1 = $temp0.replace("""","")
$temp1 > $path
}
But this output is not a "real" csv file.
Hope that helps.
For your particular scenario you could probably use a simpler approach. Read the input file as a plain text file, group the lines by splitting off the first field, then write the groups to output files named after the groups:
Get-Content 'INPUT_FILE.csv' |
Group-Object { $_.Split(',')[0] } |
ForEach-Object { $_.Group | Set-Content ($_.Name + '.csv') }
Another solution,
using no named headers but simply numbers (as they aren't wanted in output anyway)
avoiding unneccessary temporary files.
removing only field delimiting double quotes.
Import-Csv INPUT_FILE.csv -Header (1..12) |
Group-Object -Property "1" | Foreach-Object {
($_.Group | ConvertTo-Csv -NoType | Select-Object -Skip 1).Trim('"') -replace '","',',' |
Set-Content -Path ("{0}.csv" -f $_.Name)
}
I want to read a CSV file and output a CSV file with only one (1) field. I have tried to create a concise example.
PS C:\src\powershell> Get-Content .\t.csv
field1,field2,field3
1,2,3
4,55,6
7,888,9
PS C:\src\powershell> Import-Csv -Path .\t.csv | `
>> ForEach-Object {
>> $_.field2 `
>> } | `
>> Export-Csv -Path .\x.csv -NoTypeInformation
>>
The problem is that the Length of field2 is written to the exported CSV file. I want the field header to be "field2" and the values to be the value from the original CSV file. Also, I only want quotes where they are required; not everywhere.
I have read Export-CSV exports length but not name and Export to CSV only returning string length. But these do not seem to address producing an actual CSV file with a header and one field value.
PS C:\src\powershell> get-content .\x.csv
"Length"
"1"
"2"
"3"
CSV object uses note properties in each row to store its fields so we'll need to filter each row object and leave just the field(s) we want using Select-Object cmdlet (alias: select), which processes the entire CSV object at once:
Import-Csv 1.csv | select field2 | Export-Csv 2.csv -NoTypeInformation
Note, there's no need to escape the end of line if it ends with |, {, (, or ,.
It's possible to specify several fields: select field2, field3.
To strip unneeded doublequotes, general multi-field case:
Import-Csv 1.csv |
select field2 |
%{
$_.PSObject.Properties | %{ $_.value = $_.value -replace '"', [char]1 }
$_
} |
ConvertTo-Csv -NoTypeInformation |
%{ $_ -replace '"(\S*?)"', '$1' -replace '\x01', '""' } |
Out-File 2.csv -Encoding ascii
Simplified one-field case:
Import-Csv 1.csv |
select field2 |
%{
$_.field2 = $_.field2 -replace '"', [char]1
$_
} |
ConvertTo-Csv -NoTypeInformation |
%{ $_ -replace '"(\S*?)"', '$1' -replace '\x01', '""' } |
Out-File 2.csv -Encoding ascii
A tricky case of embedded quotes inside a field was solved by temporary replacing them with a control character code 01 (there are just a few that can be used in a typical non-broken text file: 09/tab, 0A/line feed, 0D/carriage return).
As per WOxxOm's response, Select-Object is best way to select only field from an input and pipe to output.
Regarding the quote marks, this is a known (and frustrating) issue with PowerShell. Specifying , as the delimiter did not help.
I have gotten round it by using ConvertTo-Csv and Foreach-Object replacements. THe replacements will need to be more complex if your data contains quote marks.
Import-Csv .\1.csv |
Select-Object field2 |
ConvertTo-Csv -NoTypeInformation |
ForEach-Object {$_ -replace '"',''} |
Out-File .\2.csv
I am trying to append a CSV file. Here are the lines I am using. I wasn't able to find an append option for export-csv unfortunately. Any ideas would be helpful to get this to work.
Get-ADGroupMember "Domain Admins" | select name, samaccountname | Export-Csv c:\bin\DomainAdmins.csv
$admins = Import-Csv C:\bin\DomainAdmins.csv
foreach ($i in $admins) {Get-ADUser $i.samaccountname -properties * | select name, lastlogondate | Export-Csv c:\bin\dalogon.csv}
The documentation suggests that there is an -append flag. The example given ends with
| export-csv –append –path \\Archive01\Scripts\Scripts.csv
Have you tried that? It works fine for me. I'm on version 3, if that matters.
-Append was introduced with PowerShell v3, it's not available in PowerShell v2 and earlier. You can work around it like this, though:
... |
ConvertTo-Csv -NoTypeInformation |
Select-Object -Skip 1 |
Out-File -Append "c:\bin\dalogon.csv"
I ran into this issue also a few days ago. There are a really two solutions I know in Powershell 2. The first would be to save all the data in an array and then use the export-csv commandlet. That did not work for me unless I rewrote my script. I needed to create a CSV and append line by line to build the file. So, I solved it with out-file -append and changing the encoding to ascii.
I basically created a string with my data in it and then piped it to out-file. Here is an example:
$myCSV = "C:\_PSScripts\data\myCSV.csv"
$firstOutputLine = "Column-1,Column-3,Column-3"
$firstOutputLine | out-file $myCSV -Encoding ascii -Force -Append
It's good! but it's only 1 column in CSV file! how to put to many columns as the code below:
Import-Module -Name 'C:\Program Files\Quest Software\Management Shell for AD\Quest.ActiveRoles.ArsPowerShellSnapIn.dll'
$group="HO-Internet","Internet1","Internet2","Internet3"
$group |ForEach-Object {Echo "--------------------Group Name $_ ----------------"; Get-QADGroupMember $_ | Select-Object Email,LogonName,ParentContainer,LastLogon,AccountIsDisabled |ConvertTo-Csv -NoTypeInformation `
| select -Skip 1 `
| Out-File -Append "D:\test.csv"}