How to fix broken lines records of a file using PowerShell? - powershell

Question

In my csv file I'm getting data in incorrect format for a few rows, sometimes a line is broken into two lines as shown in below table. For EmpId 2, line is broken into two lines. How can I find such records and merge them into one line in correct format to fix the issue for such records using PowerShell. Expected output is shown in below table.
Input File data:
EmpId,EmpName,EmpLocation
1,"Jack","Austin"
2,"Pet
er","NYC"
3,"Raj","Delhi"
Expected Output:
EmpId,EmpName,EmpLocation
1,"Jack","Austin"
2,"Peter","NYC"
3,"Raj","Delhi"

Answer 1

My instinct was to do something similar to Karthick's answer, however I first took a look at the output of Import-Csv. Surprisingly it puts the line break in the individual property where it was found like:
Import-Csv C:\temp\Broken.csv | fl
EmpId : 1
EmpName : Jack
EmpLocation : Austin
EmpId : 2
EmpName : Pet
er
EmpLocation : NYC
EmpId : 3
EmpName : Raj
EmpLocation : Delhi
Notice "peter" is broken across 2 lines.
So I saw some potential to bring the objects in and modify the underlying property values instead of trying to fix up the string data. I cooked up the below:
$CSVData = Import-Csv C:\temp\Broken.csv
$CSVData |
ForEach-Object{
ForEach( $Property in $_.PSObject.Properties.Name )
{
$_.($Property) = $_.($Property) -replace "(`r|`n)"
}
}
$CSVData
# If you want to re-export:
$CSVData | Export-Csv -Path c:\temp\Fixed.csv -NoTypeInformation
This code should work regardless of which field has the line break. Give it a shot and let me know. Thanks!

Answer 2

You can try the below. This worked for me. I assumed the first line is the header.
$filepath = "D:\file.csv"
[string[]]$data = Get-Content $filepath
$data_Final = New-Object System.Collections.ArrayList
for($i = $j = 0; $i -lt $data.Count; $(if($i -eq $j){$i++}else{$i=$j+1}), ($j=$i)) {
While ( ($data[$i] -split ",").Count -ne 3 ) {
$j = $j+1
# Concatenate the target line ($i) with successive line(s) ($j) until the elements Count to 3
$data[$i] = $data[$i] + $data[$j]
}
$data_Final.Add($data[$i]) | Out-Null
}
$inputData = $data_Final | ConvertFrom-Csv
# Or, if you want to fix the csv uncomment the below
# $data_Final | ConvertFrom-Csv | Export-Csv $filepath -NoTypeInformation