Merge CSV files in subfolders - powershell

Question

I have found a script which does everything that I need, but it's only useful if you run it in a single folder. What I'd like is:
Script is located in c:/temp/. Upon running the script, it would go into each subfolder and execute. Each subfolder would then have a separate Final.csv.
Somebody mentioned just add -Recurse, but it doesn't complete the job as described. With -Recurse added, it goes into each subfolder and creates a Final.csv final in the root dir (C:/temp/) instead of creating a Final.csv in each subfolder.
$getFirstLine = $true
get-childItem *.csv | foreach {
$filePath = $_
$lines = Get-Content $filePath
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 2}
}
$getFirstLine = $false
Add-Content Final.csv $linesToWrite
}

Answer 1

If you are certain the csv files combined this way will leave you a valid 'Final.csv', you need to use Group-Object in order to create a combined file in each of the directories where the csv files to combine are found.
Suppose you have a folder with subfolders 'Folder1' and 'Folder2', both having csv files in them like these:
first.csv
Lorem,Ipsum,Dolor,Sic,Amet
data1-1,data1-2,data1-3,data1-4,data1-5
data2-1,data2-2,data2-3,data2-4,data2-5
second.csv
Lorem,Ipsum,Dolor,Sic,Amet
something,blah,whatever,very important,here's more..
Then this should do it for you:
$targetFileName = 'Final.csv'
# loop over the CSV files, but exclude the Final.csv file
# Group the files by their DirectoryNames
Get-ChildItem -Path 'D:\Test' -Filter '*.csv' -File -Recurse -Exclude $targetFileName | Group-Object DirectoryName | ForEach-Object {
# reset the $getFirstLine variable for each group
$getFirstLine = $true
# create the target path for the combined csv inside this folder.
# ($_.Name is the name of the group, which is the Directory name of the files inside the group)
$target = Join-Path -Path $_.Name -ChildPath $targetFileName
foreach ($file in $_.Group) {
if ($getFirstLine) {
# copy the first CSV as a whole
Get-Content -Path $file.FullName | Set-Content -Path $target
$getFirstLine = $false
}
else {
# add the content of the next file(s) without the header line
Get-Content -Path $file.FullName | Select-Object -Skip 1 | Add-Content -Path $target
}
}
}
The end result is that each subfolder will have a new 'Final.csv' file containing
Lorem,Ipsum,Dolor,Sic,Amet
data1-1,data1-2,data1-3,data1-4,data1-5
data2-1,data2-2,data2-3,data2-4,data2-5
something,blah,whatever,very important,here's more..
Of course I'm just showing an example for one of the subfolders.. Other subfolders will contain different 'Final.csv' content