Perform action after multiple files are created - powershell

Question

I'm new to Powershell scripting, and I'm struggling with how to identify when multiple files have been created in order to initiate a process.
Ultimately, I need to wait until a handful of files have been created (this usually occurs within a finite period of time the same time each day). These files are created at separate times. Once all files have been created and are at their final location, I need to perform a separate action with these files. The trouble I'm having is:
Identifying when all files are available
Identifying how to initiate a separate process once these files are available
If necessary,unregistering the events (I plan to run this script each morning...I don't know how long these events are registered)
I've toyed with using the IO.FileSystemWatcher with some success in order to monitor when any individual directory has this file. While this logs appropriately, I don't know how to consolidate the collection of these files. Possibly a flag? Not sure how to implement. I've also considered using Test-Path as a way of checking to see if these files exist -- but a drawback with this is that I'd need to periodically run the script (or Sleep) for a pre-defined period of time.
Does anyone have experience with doing something like this? Or possibly provide guidance?
What I've tried (with respect to IO.FileSystemWatcher) using test data:
$i=0
$paths = "C:\","Z:\"
$fileName='Test'+$Datestr.Trim()+'.txt'
foreach ($path in $paths)
{
$fsw = New-Object IO.FileSystemWatcher $path, $fileName -Property #{IncludeSubdirectories = $tru;NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'}
Register-ObjectEvent $fsw Created -SourceIdentifier "$i+fileCreated" -Action {
$name = $Event.SourceEventArgs.Name
$changeType = $Event.SourceEventArgs.ChangeType
$fpath = $Event.SourceEventArgs.FullPath
$timeStamp = $Event.TimeGenerated
Write-Host "The folder "$Event.SourceEventArgs.FullPath" was $changeType at $timeStamp" -fore green
Out-File -FilePath Z:\log.txt -Append -InputObject "The folder $fpath was $changeType at $timeStamp"
}

Answer 1

I am guessing you do not have control over the process(es) that create the files, or else you would use the completion of that job to trigger the "post processing" you need to do. I have used IO.FileSystemWatcher on a loop/timer like you described, and then I Group-Object on the file names to get a distinct list of the files that have changed. I was using this to monitor for small files (about 100 files at ~100KB each) that did not change frequently, and it still generated a large number of events every time the files changed. If you want to take action/start a process every time a change is made, then IO.FileSystemWatcher is your friend.
If the files are larger/take longer to generate, and because you only care once they are all done (not when they are created/modified) you may be better off skipping the filewatcher and just check if all of the files are there. IE: the process(s) to generate the files usually finishes by 6am. So at 6am you run a script that checks to see if all the files exist. (and I would also check the file size/last modified dttm to help you ensure that the process which generates the file is done writing to it.) You still may want to build a loop into this, especially if you want the process to start as soon as the files are done.
$filesIWatch = #()
$filesIWatch += New-Object -TypeName psobject -Property #{"FullName"="C:\TestFile1.txt"
"AvgSize"=100}
$filesIWatch += New-Object -TypeName psobject -Property #{"FullName"="C:\TestFile2.txt"
"AvgSize"=100}
[bool]$filesDone = $true
foreach($file in $filesIWatch){
#Check if the file is there. Also, maybe check the file size/modified dttm to help be sure that some other process is still writing to the file.
if(-not (Test-Path ($file.FullName))){
$filesDone = $false
}
}
if($filesDone){
#do your processing
}
else{
#throw error/handle them not being done
}