Removing items from one CSV based on items in another CSV file - powershell

Question

I have a script that will generate a CSV file. The purpose of the script is to verify if a certain file is missing. For example, let's say I have the following files:
1.jpg
2.jpg
3.jpg
4.jpg
1.gif
3.gif
2.txt
3.txt
Once the script is run, it will generate a report so I can visually see what file is missing. The report looks like:
JPG Files GIF Files TXT Files
1.jpg 1.gif
2.jpg 2.txt
3.jpg 3.gif 3.txt
So you can see, I'm missing 1.txt and 2.gif.
Here's where my problem comes in....
I now have a SECOND CSV file that has a list of files that MUST be kept in the FIRST CSV. Anything that is NOT in the SECOND CSV file must now be removed from my FIRST CSV. For example:
My FIRST CSV contains:
1.jpg
2.jpg
3.jpg
1.gif
3.gif
2.txt
3.txt
The SECOND CSV says that the following files need to remain:
1.jpg
3.jpg
1.gif
2.txt
Therefore, anything that does not appear in the SECOND CSV file, needs to be removed from the FIRST CSV while retaining the same format, meaning that if 1.jpg is missing (it is still listed in the SECOND CSV but does not exist in the C:\JPG folder) it must show a blank space in the FIRST CSV.
I hope this make sense. Please ask me if you have any questions or need clarification.
Below is the portion of code from my script that generates the FIRST CSV:
# Get dirs
$dirJPG = "C:\JPG"
$dirGIF = "C:\GIF"
$dirTXT = "C:\TXT"
$files = #()
$files += Get-ChildItem -Path $dirBGR -Filter "*.jpg"
$files += Get-ChildItem -Path $dirMI -Filter "*.gif"
$files += Get-ChildItem -Path $dirW3F -Filter "*.txt"
# Write a datetime stamped CSV file
$datetime = Get-Date -Format "MM_dd_yyyy_hhmm"
$files | Sort-Object -Property { $_.Name } | Group-Object -Property {
[System.IO.Path]::GetFileNameWithoutExtension($_.Name) } | % {
New-Object psobject -Property #{
"JPG" Files" = $_.Group | ? { $_.Extension -eq ".jpg" } | % { $_.Name }
"GIF Files" = $_.Group | ? { $_.Extension -eq ".gif" } | % { $_.Name }
"TXT Files" = $_.Group | ? { $_.Extension -eq ".txt" } | % { $_.Name }
} } | Export-Csv -Path "$datetime.csv" -NoTypeInformation
Thanks in advance for your assistance! :D

Answer 1

It is possible to use arrays, but it will probably be more efficient to use hashtables. You can check iterate (foreach) through the first CSV items and check if files are in CSV1 and not in CSV2:
# Get the files by directory for each file type
function Get-FilesByType() {
param ([hashtable]$filters)
$result = #{}
foreach ($filter in $filters.Keys) {
$path = $filters[$filter]
Get-ChildItem -Path $path -Filter $filter | % {
$result.Add($_.Name, $_)
}
}
return $result
}
# Assume CSV1 hashtable already exists and is loaded
# Get the hashtable of files for CSV2
$csv2 = Get-FilesByType #{"*.jpg"="C:\JPG"; "*.gif"="C:\GIF"; "*.txt"="C:\TXT" }
# Remove items from CSV1 that do not exist in CSV2
# NOTE: You cannot remove items from the hashtable while
# iterating through the collection, so use a copy of the
# keys to iterate.
$keys = #()
$keys += $csv1.Keys
$keys | % {
if ( ! $csv2.ContainsKey($_) ) {
Write-Host "Removing $_"
$csv1.Remove($_)
}
}
# Write a datetime stamped CSV file
$datetime = Get-Date -Format "MM_dd_yyyy_hhmm"
$csv1.Values | Sort-Object -Property { $_.Name } | Group-Object -Property {
[System.IO.Path]::GetFileNameWithoutExtension($_.Name)
} | % {
New-Object psobject -Property #{
"JPG Files" = $_.Group | ? { $_.Extension -eq ".jpg" } | % { $_.Name }
"GIF Files" = $_.Group | ? { $_.Extension -eq ".gif" } | % { $_.Name }
"TXT Files" = $_.Group | ? { $_.Extension -eq ".txt" } | % { $_.Name }
}
} | Export-Csv -Path "$datetime.csv" -NoTypeInformation

Answer 2

Don't use an array - use the Hashtable like Ryan said. The array is not a good choice when you want to remove elements from it.

Answer 3

Found what my problem was...I was calling the files that needed to be removed and then removing them. I simply needed to add a not condition:
$keys = #()
$keys += $currentFiles.Keys
$keys | % {
if (! $filesToKeep.ContainsKey($_)) {
Write-Host "Removing $_"
$currentFiles.Remove($_)
}
}