Choosing columns for merged files in PowerShell - powershell

Question

I am using get-childitem to recurse through directories ( skipping some at the top level ) open a series of csv files, append the filename to the end each line of data
and combine the data into one.
$mergedData= Get-ChildItem $path -Exclude yesterday,"OHCC Extract",output |
Get-ChildItem -recurse -Filter *csv |
Where-Object { $_.CreationTime -gt (Get-Date).AddDays(-1) } |
% {
$file = $_.Name
$fn = $_.FullName
## capture the header line
$FirstLine = Get-Content $fn -TotalCount 1
## add the column header for filename
$header = $FirstLine + ",Filename"
## get the contents of the files without the first line
Get-Content $fn | SELECT -Skip 1 | %{ "$_,$file" }
}
Now each file had 5 columns , ID, First Name , Last Name , Phone , Address. The column names are surrounded by double quotes ( "ID", "First Name" ) .
The request is now to skip everything but the ID and the Last Name column. So I tried ( starting with just ID, will add First Name later)
Get-Content $fn | SELECT -Skip 1 -Property ID | %{ "$_,$file" }
I get #{ID=} in the resulting file.
Then I tried
Get-Content $fn | SELECT -Skip 1 | %{ $_.ID }
which yield blanks and then
Import-Csv -Path $fn -Delimiter ',' | SELECT ID
Which gives #{ID=73aec2fe-6cb3-492e-a157-25e355ed9691}
At this point I am just flailing because I obviously don't know how to handle objects in PS.
I have PowerShell 5.1.19041.1682 on windows 10.
Thanks

Answer 1

I was asked for sample data , so here it is. There are 35 files across multiple subdirectories
Input FileA
column1
ID
Column3
East
12
apple
west
5
pear
Input FileB
column1
ID
Column3
East
15
kiwi
Output
Column1
column3
Filename
East
kiwi
FileB
East
apple
FileA
west
pear
FileB
But I did figure it out myself . Working code
$path = "<directory where the files are located> "
$pathout = "<path to outputted file>"
$out = "$pathout\csv_merged_$(get-date -f MMddyyyy).csv"
$mergedData= Get-ChildItem $path -Exclude yesterday,output | Get-ChildItem -recurse -Filter *csv | Where-Object { $_.CreationTime -gt (Get-Date).AddDays(-1) } | % {
$file = $_.Name
$fn = $_.FullName
write-host $fn , $_.CreationTime
## get the contents of the files ,exclude columns and add columns
$Data = Import-Csv -Path $fn -Delimiter ',' | SELECT *, #{Name = 'Filename'; Expression = {$file}} -ExcludeProperty ID
# get the headers
$header= $Data | ConvertTo-Csv -NoTypeInformation | Select-Object -First 1
write-host $header
## convert the object and remove the column headers for each file
$Data | ConvertTo-Csv -NoTypeInformation | Select-Object -Skip 1
write-host
write-host '-----------------------'
}
# Prefix the header before the compiled data
$header, $mergedData | Set-Content -Encoding utf8 $out
The missing piece was the ConvertTo_Csv which expanded the object.