Powershell - Insert column in between specific columns in csv file - powershell

Question

I have 2 csv files
First file:
firstName,secondName
1234,Value1
2345,Value1
3456,Value1
4567,Value3
7645,Value3
Second file:
firstName,fileSplitter,Csv2ColumnOne,Csv2ColumnTwo,Csv2ColumnThree
1234,,1234,abc,Value1
1234,,1234,asd,Value1
3456,,3456,qwe,Value1
4567,,4567,mnb,Value1
I want to insert column secondName in the second file in between columns firstName and fileSplitter.
The result should look like this:
firstName,secondName,fileSplitter,Csv2ColumnOne,Csv2ColumnTwo,Csv2ColumnThree
1234,Value1,,1234,abc,Value1
1234,Value1,,1234,asd,Value1
3456,Value1,,3456,qwe,Value1
4567,Value3,,4567,mnb,Value1
I'm trying the following code:
Function InsertColumnInBetweenColumns
{
Param ($FirstFileFirstColumnTitle, $firstFile, [string]$1stColumnName, [string]$2ndColumnName, [string]$columnMergedFileBeforeInput)
Write-Host "Creating hash table with columns values `"$1stColumnName`" `"$2ndColumnName`" From $OimFileWithMatches"
$hashFirstFileTwoColumns = #{}
Import-Csv $firstFile | ForEach-Object {$hashFirstFileTwoColumns[$_.$1stColumnName] = $_.$2ndColumnName}
Write-Host "Complete."
Write-Host "Appending Merge file with column `"$2ndColumnName`" from file $secondCsvFileWithLocalPath"
Import-Csv $outputCsvFileWithLocalPath | Select-Object $columnMergedFileBeforeInput, #{n=$2ndColumnName; e={
if ($hashFirstFileTwoColumns.ContainsKey($_.$FirstFileFirstColumnTitle)) {
$hashFirstFileTwoColumns[$_.$FirstFileFirstColumnTitle]
} Else {
'Not Found'
}}}, * | Export-Csv "$outputCsvFileWithLocalPath-temp" -NoType -Force
Move-Item "$outputCsvFileWithLocalPath-temp" $outputCsvFileWithLocalPath -Force
Write-Host "Complete."
Write-Host ""
}
This function will be called in a for loop for each column found in the first file (can contain an indefinite number). For testing, I am only using 2 columns from the first file.
I'm getting an error output resulting the following:
Select : Property cannot be processed because property "firstName" already exists.
At C:\Scripts\Tests\Compare2CsvFilesOutput1WithMatchesOnly.ps1:490 char:43
+ Import-Csv $outputCsvFileWithLocalPath | Select $columnMergedFileBeforeInput, # ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (#{firstName=L...ntName=asdfas}:PSObject) [Select-Object], PSArgume
ntException
+ FullyQualifiedErrorId : AlreadyExistingUserSpecifiedPropertyNoExpand,Microsoft.PowerShell.Commands.SelectObjectC
ommand
I know the issue is where it says Select-Object $columnMergedFileBeforeInput,.
How can I get the loop statement to insert the column in between the before column (name is specified), and append the rest using *?
Update
Just an fyi, changing this line Select-Object $columnMergedFileBeforeInput, #{n=$2ndColumnName..... to this line Select-Object #{n=$2ndColumnName..... works, it just attaches the columns out of order. That is why I'm trying to insert the column in between. Maybe if i do it this way but insert the columns in backwards using the for loop, this would work...

Answer 1

Not sure if this is the most efficient way to do it, but it should do the trick. It just adds the property to the record from file2, then reorders the output so secondName is the second column. You can output results to csv where required too (ConvertTo-Csv).
$file1 = Import-Csv -Path file1.csv
$file2 = Import-Csv -Path file2.csv
$results = #()
ForEach ($record In $file2) {
Add-Member -InputObject $record -MemberType NoteProperty -Name secondName -Value $($file1 | ? { $_.firstName -eq $record.firstName } | Select -ExpandProperty secondName)
$results += $record
}
$results | Select-Object -Property firstName,secondName,fileSplitter,Csv2ColumnOne,Csv2ColumnTwo,Csv2ColumnThree

Answer 2

I've created the following function. What it does is find the match (in this case "firstname") and adds the matching columnname to the new array afther the columnname on which the match is made (little difficult to explain in my poor English).
function Add-ColumnAfterMatchingColumn{
[CmdletBinding()]
param(
[string]$MainFile,
[string]$MatchingFile,
[string]$MatchColumnName,
[string]$MatchingColumnName
)
# Import data from two files
$file1 = Import-Csv -Path $MainFile
$file2 = Import-Csv -Path $MatchingFile
# Find column names and order them
$columnnames = $file2 | gm | where {$_.MemberType -like "NoteProperty"} | Select Name | %{$_.Name}
[array]::Reverse($columnnames)
# Find $MatchColumnName index and put the $MatchingColumnName after it
$MatchColumnNameIndex = [array]::IndexOf($columnnames, $MatchColumnName)
if($MatchColumnNameIndex -eq -1){
$MatchColumnNameIndex = 0
}
$columnnames = $columnnames[0..$MatchColumnNameIndex] + $MatchingColumnName + $columnnames[($MatchColumnNameIndex+1)..($columnnames.Length -1)]
$returnObject = #()
foreach ($item in $file2){
# Find corresponding value MatchingColumnName in $file1 and add it to the current item
$item | Add-Member -Name "$MatchingColumnName" -Value ($file1 | ?{$_."$($MatchColumnName)" -eq $item."$($MatchColumnName)"})."$MatchingColumnName" -MemberType NoteProperty
# Add current item to the returnObject array, in the correct order
$newItem = New-Object psobject
foreach ($columnname in [string[]]$columnnames){
$newItem | Add-Member -Name $columnname -Value $item."$columnname" -MemberType NoteProperty
}
$returnObject += $newItem
}
return $returnObject
}
When you run this function you will have the following output:
Add-ColumnAfterMatchingColumn -MainFile C:\Temp\file1.csv -MatchingFile C:\Temp\file2.csv -MatchColumnName "firstname" -MatchingColumnName "secondname" | ft
firstName secondname fileSplitter Csv2ColumnTwo Csv2ColumnThree Csv2ColumnOne
--------- ---------- ------------ ------------- --------------- -------------
1234 Value1 abc Value1 1234
1234 Value1 asd Value1 1234
3456 Value1 qwe Value1 3456
4567 Value3 mnb Value1 4567