I'm exporting a report to a CSV in Powershell and I've been asked to provide three rows above the header row with title information, date of report, etc.
If I export to CSV, i.e. $reportdata | export-csv -notypeinformation fooreport.csv, and then manually add the Title rows in Notepad, i.e.,
"Title"
"Date of Report - XX-XX-XXXX"
"***Important disclaimer about report***"
"Column1","Column2","Column3", etc
"Data1","Data2","Data3", etc
the CSV Report opens fine in Excel and the header and data rows are handled appropriately.
However, when I attempt to do the following in PS,
#Create Report Header
$date = get-date -format "dd MMM yyyy"
$title = #"
"Mailbox Send As, Full Permission and Send on Behalf Report"
"$date"
"***** Important Disclaimer - Only Explicit or Non-Inherited Permissions Are Displayed in this Report *****"
"#
$reportpath = "D:\dev\report.csv"
$rep | export-csv -notypeinformation $reportpath
$temp = gc $reportpath
echo $title > $reportpath
$temp >> $reportpath
the CSV doesn't open properly in Excel and is improperly delimited (i.e., quotes and commas are visible in the cells), even though both files look identical in Notepad.
Is there a way to do what I'm trying to do from Powershell without messing up the integrity of the CSV file? I'm wondering if I'm somehow removing some critical piece of information that signifies a header row in CSV files - since both files are identical, I'm wondering if I"m maybe missing a hidden or special character?
You should not need to output to file with export-csv just to read the file back in to make a change. Lets output the data all at once without reading the back in again.
$rep = $rep | ConvertTo-Csv -NoTypeInformation
$title | Set-Content -Path $reportpath
$rep | Add-Content -Path $reportpath
Not what I intended since I think you could get this in one line but you dont need to read the file back this way. Mostly because it was bugging me I got a one-liner that I was looking for.
$reportpath = "D:\dev\report.csv"
Set-Content -Path c:\temp\export.csv -Value "$title`r`n$(($rep | ConvertTo-Csv -NoTypeInformation) -Join "`r`n")"
ConvertTo-Csv creates a string array. We join all the strings with a newline "`r`n". Then we attach the title to that data and split it up with another newline.
Related
$Headers = "Username"
Import-CSV -Path "$WorkingFolder\Test.csv" | Select $Headers | Export-CSV -Path "$WorkingFolder\Master.txt" -Force -NoTypeInformation
I am exporting a CSV File as a .txt file and only want to select the data from the Username column. It succesfully outputs as a .txt file and select all the items listed as Usernames. but contains "Information" and the header in the .txt file. I just want the raw data and want to remove the quotation marks and the header. I understand already that I can just skip the first row for exporting if need be but was going to see if there was another way round this.
I guess then what you want is
(Import-Csv -Path "$WorkingFolder\Test.csv").UserName |
Set-Content -Path "$WorkingFolder\Master.txt"
P.S. Personally, I would rather use Join-Path -Path $WorkingFolder -ChildPath 'test.csv' to make sure you get the path separator correct
This is my first post here - my apologies in advance if I didn't follow a certain etiquette for posting. I'm a newbie to powershell, but I'm hoping someone can help me figure something out.
I'm using the following powershell script to tell me the total count of rows in a CSV file, minus the header. This generated into a text file.
$x = (Get-Content -Path "C:\mysql\out_data\18*.csv" | Measure-Object -Line).Lines
$logfile = "C:\temp\MyLog.txt"
$files = get-childitem "C:\mysql\out_data\18*.csv"
foreach($file in $files)
{
$x--
"File: $($file.name) Count: $x" | out-file $logfile -Append
}
I am doing this for 10 individual files. But there is just ONE file that keeps adding exactly 807 more rows to the actual count. For example, for the code above, the actual row count (minus the header) in the file is 25,083. But my script above generates 25,890 as the count. I've tried running this for different iterations of the same type of file (same data, different days), but it keeps adding exactly 807 to the row count.
Even when running only (Get-Content -Path "C:\mysql\out_data\18*.csv" | Measure-Object -Line).Lines, I still see the wrong record count in the powershell window.
I'm suspicious that there may be a problem specifically with the csv file itself? I'm coming to that conclusion since 9 out of 10 files generate the correct row count. Thank you advance for your time.
To measure the items in a csv you should use Import-Csv rather than Get-Content. This way you don't have to worry about headers or empty lines.
(Import-Csv -Path $csvfile | Measure-Object).Count
It's definitely possible there's a problem with that csv file. Also, note that if the csv has cells that include linebreaks that will confuse Get-Content so also try Import-CSV
I'd start with this
$PathToQuestionableFile = "c:\somefile.csv"
$TestContents = Get-Content -Path $PathToQuestionableFile
Write-Host "`n-------`nUsing Get-Content:"
$TestContents.count
$TestContents[0..10]
$TestCsv = Import-CSV -Path $PathToQuestionableFile
Write-Host "`n-------`nUsing Import-CSV:"
$TestCsv.count
$TestCsv[0..10] | Format-Table
That will let you see what Get-Content is pulling so you can narrow down where the problem is.
If it is in the file itself and using Import-CSV doesn't fix it I'd try using Notepad++ to check both the encoding and the line endings
encoding is a drop down menu, compare to the other csv files
line endings can be seen with (View > Show Symbol > Show All Characters). They should be consistent across the file, and should be one of these
CR (typically if it came from a mac)
LF (typically if it came from *nix or the internet)
CRLF (typically if it came from windows)
I have multiple .txt files which have data as per below format in single line
MACADD||TEST|Street1|CITY||Taiwan, Republic of China|||10000000|||FIRE||Taiwan||||||||12 days||30 Days|DDTE||812148709231890||124-Taiwan|DENE|||LDE
I am trying to save these .txt files as .csv files in such a way that the entire data string appears in 1st cell of the .csv file.
I can achieve this by quoting the entire data string in " " but the problem is when I again process the .csv files to generate XML files then those inverted commas become a problem. For example, the 1st data will appear as "MACADD and last one as LDE" which is not accepted by the system.
Use Get-Content to import it and out-csv to export it, because you don't really need csv formatting:
Single File:
Get-Content C:\temp\test.txt | Out-File C:\temp\test.csv
Multiple Files in one Folder:
$files = Get-ChildItem "C:\temp" | where { ! $_.PSIsContainer }
foreach ($file in $files){
$newFileName = ($file.Fullname) -Replace ".txt",".csv"
Get-Content $file.FullName | Out-File $newFileName
}
Please test it, let me know if it worked and if it did, please mark the post as the answer.
I have a tab delimited txt file and i need to switch first and second column names (without switching columns data). In other words I need to rename A(Id) to B(ExternalId) and B(ExternalId) to A(Id). Other columns in the file (other data) should stay unchanged. I'm very new in PowerShell, please advice. As I understand I need to use import/export csv cmdlet.
I tryed this, but it's not working the right way...
Import-Csv 'C:\original_users.txt' |
Select-Object Id, #{Name="ExternalId";Expression={$_."Id"}}; Select-Object ExternalId, #{Name="Id";Expression={$_."ExternalId"}} |
Export-Csv 'C:\changed_users.txt'
The Import-CSV and Export-CSV cmdlets have their strengths but this might not be one of them. The latter cmdlet would introduce quoting that might not be in your original file and that might not be desired.
Either way why not just do some text manipulation on the first line! Lets read in the file and and output the first lined, edited, and the remainder of the file. This sample uses a new location but you could easily write it back to the same file.
# Get the full file into a variable
$fullFile = Get-Content "c:\temp\mockdata.csv"
# Parse the first line into a column array
$columns = $fullFile[0].Split("`t")
# Rebuild the header by switching the columns order as desired.
$newHeader = ($columns[1],$columns[0] + ($columns | Select-Object -Skip 2)) -join "`t"
# Write the header back to file then the rest of the data.
$outputPath = "C:\somepath.txt"
$newHeader | Set-Content $outputPath
$fullFile | Select-Object -Skip 1 | Add-Content $outputPath
This also preserves the presence of other columns and their data.
I have a .csv file and I want to import it into powershell then iterate through the file changing certain values. I then want the output to append to the original .csv file, so that the values have been updated.
My issue is that the .csv file has headers which aren't unique, and can't be changed as then it won't work in another program. Originally I defined my own headers in the powershell to get around this but then the output file has these new headers when it needs to have the old ones.
I have also tried ConvertFrom-Csv which means I can no longer access the columns I need to, so lots of runtime errors.
What would be ideal is to be able to use the defined column headers and then convert back to the original column headers. My current code is below:
$csvfile = Import-Csv C:\test.csv| Where-Object {$_.'3' -eq $classID} | ConvertFrom-Csv
foreach($record in $csvfile){
*do something*}
$csvfile | Export-Csv -path C:\test.csv -NoTypeInformation -Append
I've searched the web now for some hours and tried everything I've come across, to no avail.
Thanks in advance.
This is a somewhat hackish implementation but should work.
Remove all the headers as a single line and save it somewhere
Parse the new result-set (with the headers removed)
Add the line at the top when you are finished
A CSV is a comma delimited file, you don't have to treat it like structured data. Feel free to splice and dice as you want.
Since you know beforehand how many columns are in the input CSV file, you can import without the header and process internally. Example:
$columns = 78
Import-Csv "inputfile.csv" -Header (0..$($columns - 1)) | Select-Object -Skip 1 | ForEach-Object {
$row = $_
$outputObject = New-Object PSObject
0..$($columns- 1) | ForEach-Object {
$outputObject | Add-Member NoteProperty "Col$_" $row.$_
}
$outputObject
} | Export-Csv "outputfile.csv" -NoTypeInformation
This example generates new PSObjects and then outputs a new CSV file with generic column names (Col0, Col1, etc.).