powershell file system event watcher service - powershell

I have a file transformation solution built upon powershell with filesystemwatcher and powershell noexit. The problem I have is that the events don't fire after a while (I have no idea when it stops, I just get a call from the production), and I'm not sure why that is.
The question is basically if/how I can check health of existing active eventwatchers. I tried first by adding control if event object existed so that I could simply execute the powershell script a few times a day, but soon figured out it was powershell session bound and I was not able to fetch it from a new powershell session.
This is the basic structure of the solution.
(Anonymized, so functions etc renamed a bit and the workload logics is stripped out and on the places where it says '...' it is not syntax error)
powershell.exe -NoExit -ExecutionPolicy Bypass -inputformat none -File C:\|...|.ps1 WINDOWSTARTUP="hidden" WAITFORINPUT="NO" DOS="YES"
function Test-FileLock {
param ([parameter(Mandatory=$true)][string]$Path)
$oFile = New-Object System.IO.FileInfo $Path
if ((Test-Path -Path $Path) -eq $false)
{
return $false
}
try
{
$oStream = $oFile.Open([System.IO.FileMode]::Open, [System.IO.FileAccess]::ReadWrite, [System.IO.FileShare]::None)
if ($oStream)
{
$oStream.Close()
}
$false
}
catch
{
# file is locked by a process.
return $true
}
}
function ConvertType1($sourceFile, $custno, $destFile)
{
$result = "-1"
try {
$OFS = "`r`n"
$data = ""
$data += "..."
$data += $OFS + "`tType=""..."""
# |...|
$data | Out-File $destFile
$result = "Done"
Write-Host $result
}
catch
{
$ErrorMessage = $_.Exception.Message
$FailedItem = $_.Exception.ItemName
$result = "ConvertType1: " + $ErrorMessage + " - " + $FailedItem
Write-Host $result
}
finally
{
}
return $result
}
function ConvertType2($sourceFile, $custno, $destFile)
{
$result = "-1"
try {
$OFS = "`r`n"
$data = ""
$data += "..."
$data += $OFS + "`tType=""..."""
# |...|
$data | Out-File $destFile
$result = "Done"
Write-Host $result
}
catch
{
$ErrorMessage = $_.Exception.Message
$FailedItem = $_.Exception.ItemName
$result = "ConvertType2: " + $ErrorMessage + " - " + $FailedItem
Write-Host $result
}
finally
{
}
return $result
}
#$devTest = $true
$devTest = $false
if ($devTest)
{
$logfile = "....LOG"
$monitorFolder = '\\..._transformation\Development\Test\In' # Enter the root path you want to monitor.
$destinationFolder = '\\..._transformation\Development\Test\Out' # Enter the root path you want to monitor.
$filter = '*.*' # You can enter a wildcard filter here.
}
else
{
$logfile = ".....LOG"
$monitorFolder = '\\..._transformation\In' # Enter the root path you want to monitor.
$destinationFolder = '\\..._transformation\Out' # Enter the root path you want to monitor.
$filter = '*.*' # You can enter a wildcard filter here.
}
$fsw = New-Object IO.FileSystemWatcher $monitorFolder, $filter -Property #{IncludeSubdirectories = $false;NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'}
#Check if eventsubcriber already exists, if it does just terminate script. This way we can schedule this script continously to make sure we always have the watcher in place
#Added note: This does not work because the events are per session and we won't be able to see events from previous sessions.
if($devTest)
{
if(Get-EventSubscriber | Where-Object -Property SourceIdentifier -eq "devTest_...FileCreated") {
Exit
}
Register-ObjectEvent $fsw Created -SourceIdentifier devTest_...FileCreated -Action {
$sourceFilename = $Event.SourceEventArgs.Name
$sourcePathFilename = $Event.SourceEventArgs.FullPath
$sourcePath = split-path $sourcePathFilename
$changeType = $Event.SourceEventArgs.ChangeType
$timeStamp = $Event.TimeGenerated
$destinationFile = (join-path $destinationFolder ([system.io.fileinfo]$sourceFilename).BaseName) + ".wli"
Write-Host "$logTime - The file '$sourcePathFilename' was $changeType at $timeStamp" -fore green
try {
$logTime = (Get-Date -format (Get-culture).DateTimeFormat.UniversalSortableDateTimePattern)
Out-File -FilePath $logfile -Append -InputObject "$logTime - The file '$sourceFilename' was $changeType at $timeStamp"
do {
sleep -seconds 1
$fileLockTest = Test-FileLock $sourcePathFilename
} while ($fileLockTest)
switch ($sourceFilename.ToLower())
{
{($_ -like "*hire*") -or ($_ -like "*change*")} { $result = ConvertType1 $sourcePathFilename 123456 $destinationFile }
{($_ -like "*termination*") -or ($_ -like "*...*")} { $result = ConvertType2 $sourcePathFilename 123456 $destinationFile }
default {$result = "Error. File name is not recognized. File name must |...|"}
}
Write-Host "$logTime - Transformation of file '$sourceFilename': $result"
Out-File -FilePath $logfile -Append -InputObject "$logTime - Transformation of file '$sourceFilename': $result"
Move-Item $sourcePathFilename (join-path $sourcePath "Backup") -force
}
Catch
{
$logTime = (Get-Date -format (Get-culture).DateTimeFormat.UniversalSortableDateTimePattern)
$ErrorMessage = $_.Exception.Message
$FailedItem = $_.Exception.ItemName
Move-Item $sourcePathFilename (join-path $sourcePath "Error") -force
Write-Host "$logTime - The file '$sourceFilename' encountered an error during transformation. The error message was $ErrorMessage" -fore red
Out-File -FilePath $logfile -Append -InputObject "$logTime - The file '$sourceFilename' encountered an error during transformation. The error message was $ErrorMessage"
}
}
}
else
{
if(Get-EventSubscriber | Where-Object -Property SourceIdentifier -eq "...FileCreated") {
Exit
}
Register-ObjectEvent $fsw Created -SourceIdentifier ...FileCreated -Action {
$sourceFilename = $Event.SourceEventArgs.Name
$sourcePathFilename = $Event.SourceEventArgs.FullPath
$sourcePath = split-path $sourcePathFilename
$changeType = $Event.SourceEventArgs.ChangeType
$timeStamp = $Event.TimeGenerated
$destinationFile = (join-path $destinationFolder ([system.io.fileinfo]$sourceFilename).BaseName) + ".wli"
Write-Host "$logTime - The file '$sourcePathFilename' was $changeType at $timeStamp" -fore green
try {
$logTime = (Get-Date -format (Get-culture).DateTimeFormat.UniversalSortableDateTimePattern)
Out-File -FilePath $logfile -Append -InputObject "$logTime - The file '$sourceFilename' was $changeType at $timeStamp"
do {
sleep -seconds 1
$fileLockTest = Test-FileLock $sourcePathFilename
} while ($fileLockTest)
switch ($sourceFilename.ToLower())
{
{($_ -like "*hire*") -or ($_ -like "*change*")} { $result = ConvertType1 $sourcePathFilename 123456 $destinationFile }
{($_ -like "*termination*") -or ($_ -like "*...*")} { $result = ConvertType2 $sourcePathFilename 123456 $destinationFile }
default {$result = "Error. File name is not recognized. File name must include information about type of file |...|"}
}
Write-Host "$logTime - Transformation of file '$sourceFilename': $result"
Out-File -FilePath $logfile -Append -InputObject "$logTime - Transformation of file '$sourceFilename': $result"
Move-Item $sourcePathFilename (join-path $sourcePath "Backup") -force
}
Catch
{
$logTime = (Get-Date -format (Get-culture).DateTimeFormat.UniversalSortableDateTimePattern)
$ErrorMessage = $_.Exception.Message
$FailedItem = $_.Exception.ItemName
Move-Item $sourcePathFilename (join-path $sourcePath "Error") -force
Write-Host "$logTime - The file '$sourceFilename' encountered an error during transformation. The error message was $ErrorMessage" -fore red
Out-File -FilePath $logfile -Append -InputObject "$logTime - The file '$sourceFilename' encountered an error during transformation. The error message was $ErrorMessage"
}
}
}

Related

Check if file exist on watched folder

I want to have a checked folder, but before that I need to check that if a certain file exist within that folder. If it does I need to make some actions, if not make another.
And I need to check if that file exist everytime. So its a running process. I cant just execute once.
Thanks in advance
So far I have this:
$root= [Environment]::GetFolderPath("Desktop")
$foldertmp = "$root\UNB\TMP\"
$filter = 'FACT_TEMPORAL.TXT' # <-- set this according to your requirements
$PDFDestination = "$root\UNB\NO_PROCESADO\"
#C:\Users\JuanMa\Desktop\UNB\TMP
Write-Host "EL ROOT ES $root" -fore white
$fsw = New-Object IO.FileSystemWatcher $foldertmp, $filter -Property #{
IncludeSubdirectories = $true # <-- set this according to your requirements
NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
}
$onCreated = Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action {
$path = $Event.SourceEventArgs.FullPath
$name = $Event.SourceEventArgs.Name
$changeType = $Event.SourceEventArgs.ChangeType
$timeStamp = $Event.TimeGenerated
Write-Host "The file '$name' at '$path'was $changeType at $timeStamp" -fore white
comprobar
limpia
printpdf
renombra
borrar
}
function comprobar()
{
if(!(Test-Path 'C:\Users\JuanMa\Desktop\UNB\TMP\FACT_TEMPORAL.TXT')){
Write-Host "No hay archivo temporal. Proceso correcto" -fore white
Get-Printer -Name "UNBILLING" | select Name, PrinterStatus, JobCount, DriverName
}
if((Test-Path 'C:\Users\JuanMa\Desktop\UNB\TMP\FACT_TEMPORAL.TXT')){
Write-Host "Archivo existe. Error" -fore white
Get-Printer -Name "UNBILLING" | select Name, PrinterStatus, JobCount, DriverName
limpia
renombra_noprocesado
borrar
}
}
function limpia()
{
$regex = [System.Text.RegularExpressions.Regex]
write-host "FUNCION LIMPIA ARCHIVO TXT" -fore white
$a = get-content $root\UNB\TMP\FACT_TEMPORAL.txt |
Foreach-Object { $regex::Replace($_, '^[.! pÿ>i#]{1,}', '') } |
Foreach-Object { $regex::Replace($_, '^0 {2,}', '') } |
Foreach-Object { $regex::Replace($_, '>', '') } |
Foreach-Object { ($_.TrimEnd() )} | Where-Object {$_ -ne ""} |
set-content $root\UNB\FINAL_TEXTO\FACT_FINAL.txt
}
function printpdf()
{
write-host "FUNCION IMPRIMIR A PDF" -fore white
start-process -filepath "$root\UNB\FINAL_TEXTO\FACT_FINAL.txt" -verb print
}
function borrar()
{
write-host "FUNCION BORRAR" -fore white
#Start-Sleep -s 2
Remove-Item $root\UNB\TMP\FACT_TEMPORAL.txt
}
function renombra()
{
write-host "FUNCION RENOMBRAR ARCHIVO" -fore white
Get-Item $root\UNB\FINAL_TEXTO\FACT_FINAL.txt | Rename-Item -NewName {("print-"+'{0:yyyyMMddhhmmss}{1}' -f (Get-Date),".txt")}
}
function renombra_noprocesado()
{
write-host "FUNCION RENOMBRAR ARCHIVO NO PROCESADO" -fore white
Move-Item $root\UNB\FINAL_TEXTO\FACT_FINAL.txt $PDFDestination
Get-Item $root\UNB\NO_PROCESADO\FACT_FINAL.txt | Rename-Item -NewName {("print-"+'{0:yyyyMMddhhmmss}{1}' -f (Get-Date),".txt")}
}

Powershell Script not working after inclusion of Log Function

Please find below my powershell script that is used to move files from one folder to another :
$folder = 'C:\test'
$filter = '*.*'
$directory_source = $directory_source = 'C:\test1\*' # <-- set this according to your requirements
$directory_target_cop = 'C:\Folder1'
$directory_target_prop = 'C:\Folder2'
$directory_target_dom = 'C:\Folder3'
$LogTime = Get-Date -Format "MM-dd-yyyy"
$dtNow = Get-Date -Format "MM-dd-yyyy_hh-mm-ss"
$script:loggingpathpreference = 'C:\PSLogs\'+$LogTime+".log"
Set-PSBreakpoint -Variable Now -Mode Read -Action {Set-Variable Now (get-date -uformat '%Y\%m\%d %H:%M:%S') -Option ReadOnly, AllScope -Scope Global -Force -ErrorAction SilentlyContinue} -ErrorAction SilentlyContinue
$fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property #{
IncludeSubdirectories = $true # <-- set this according to your requirements
NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
}
Function Write-Log {
[cmdletbinding()]
Param(
[Parameter(Position = 0, ValueFromPipeline = $true)]
[ValidateNotNull()]
$Message,
[switch]$Warning
)
Process{
if($Message -isnot [string]){
$Message = $Message | Out-String
}
Write-Verbose $message
Add-Content -Path $script:LoggingPathPreference -Value "`n$($dtNow) : $($Message)"
if($Warning){
Write-Warning $Message
}
}
}
$onCreated = Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action {
$path = $Event.SourceEventArgs.FullPath
$name = $Event.SourceEventArgs.Name
$changeType = $Event.SourceEventArgs.ChangeType
$timeStamp = $Event.TimeGenerated
Write-Log "The file '$name' was $changeType at $timeStamp"
try{
$files = $(ls $directory_source)
foreach ($file in $files) {
if ( $(gc $file | Select-String "PROP" -Quiet) ) {
mv -Path $file.Fullname -Destination $directory_target_prop
} elseif ( $(gc $file | Select-String "COP" -Quiet) ) {
mv -Path $file.Fullname -Destination $directory_target_cop
} elseif ( $(gc $file | Select-String "DOM" -Quiet) ) {
mv -Path $file.Fullname -Destination $directory_target_dom
}
}
}catch{
$ErrorMessage = $_.Exception.Message
Write-Log $ErrorMessage
}
}
Now if I remove my Write-Log function then the above script moves the files from test folder to respective three folders, but when I do include the Write-Log function, the above code is not able to move the files.
The above code is doing the logging but not moving the files.
What am I doing wrong here?
Looking forward to your solutions and thanks in advance.

Monitor files with FileSystemWatcher

I'm trying to create a PowerShell program to monitor when a file in a certain path is modified, and only if it now contains the string "loaded successfully", to Write-Host. Eventually an email component will be added if the string is not found in a certain amount of time.
$folder = 'C:\Users\afila\Desktop\TEST' # Enter the root path you want to monitor.
$files = Get-ChildItem $folder
$filter = '*.txt' # You can enter a wildcard filter here.
# In the following line, you can change 'IncludeSubdirectories to $true if required.
$fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property #{
IncludeSubdirectories = $false;
NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
}
# Function to search folder contents for a string
function Search-Folder {
$SearchTerm = "Loaded successfully"
foreach ( $file in $files) {
$text = Get-Content "$folder\$file"
if ($text | where {$text -like "*$SearchTerm*" }) {
Write-Host "The file '$file succeeded" -fore white
Out-File -FilePath C:\Users\afila\Desktop\SucceededLog.txt -Append -InputObject "The file '$file' completed at $timeStamp"
}
}
}
# Here, all the Changed event is registerd.
Register-ObjectEvent $fsw Changed -SourceIdentifier FileChanged -Action {
$name = $Event.SourceEventArgs.Name
$changeType = $Event.SourceEventArgs.ChangeType
$timeStamp = $Event.TimeGenerated
$searchTerm = "Loaded successfully"
foreach ($file in $files) {
$text = Get-Content "$folder\$file"
if ($text | where {$text -like "*$searchTerm*"}) {
Write-Host "The file '$name' was $changeType at $timeStamp" -fore magenta
Out-File -FilePath C:\Users\afila\Desktop\TestLogItems\TestLog.txt -Append -InputObject "The file '$name' was $changeType at $timeStamp"
}
}
}
# To stop the monitoring, run the following commands:
Unregister-Event FileChanged
The FileSystemWatcher already gives you the full path to the file via the property $Event.SourceEventArgs.Path, so you don't need to loop over a list of files.
Register-ObjectEvent $fsw Changed -SourceIdentifier FileChanged -Action {
$path = $Event.SourceEventArgs.Path
$changeType = $Event.SourceEventArgs.ChangeType
$timeStamp = $Event.TimeGenerated
$searchTerm = 'Loaded successfully'
if ((Get-Content $path) -like "*$searchTerm*"}) {
Write-Host "The file '$name' was $changeType at $timeStamp"
}
}

"A device attached to the system is not functioning " error when using robocopy to move folder in sharepoint

We are using Sharepoint 2010. There is a requirement to archive folders/Subfolders from a given folder to Archive folder which is part of document library. So we used Robocopy from powershell environment. But we're getting error "A device attached to the system is not functioning". ERROR 31 (0x0000001F) Changing File Attributes
Can any one help me what I have to do fix the above error.
Here is my code
$sourceDir = "\\labserver\sites\XXX\YYY Accounts Documents"
$targetDir = "\\labserver\sites\XXXX\Archived YYY Accounts Documents"
$FilePath = "C:\InactiveAccount_SharepointFolder.xlsx"
$sourceDir1= "\\labserver\sites\xxx\yyy Accounts Documents"
$SheetName = "Sheet1"
# Create an Object Excel.Application using Com interface
$objExcel = New-Object -ComObject Excel.Application
# Disable the 'visible' property so the document won't open in excel
$objExcel.Visible = $false
# Open the Excel file and save it in $WorkBook
$WorkBook = $objExcel.Workbooks.Open($FilePath)
$strSheetName = "sheet1"
# Load the WorkSheet 'BuildSpecs'
if ($strSheetName -eq "")
{
$worksheet = $WorkBook.sheets.item(1)
}
else
{
$worksheet = $WorkBook.sheets.item($strSheetName)
}
try {
$intRowMax = ($worksheet.UsedRange.Rows).count
for($intRow = 2 ; $intRow -le $intRowMax ; $intRow++)
{
$isFodlerExists = $worksheet.cells.item($intRow,3).value2
if($isFodlerExists -match "Yes")
{
$accountName = $worksheet.cells.item($intRow,1).value2
#[system.io.directory]::CreateDirectory($tempSource + $accountName)
#write-host $accountName
$sFolder = $sourceDir + "\" + $accountName
$sFolder1 = $sourceDir1 + "\" + $accountName
write-host $sFolder
$tFolder = $targetDir
Add-Content -Path $ErrorLog -Value $sFolder
if((Test-path -Path $sFolder1) -eq $TRUE)
{
try {
#Copy-Item -Path $sFolder -Destination $tFolder -Recurse -force
robocopy $sFolder $tFolder /e /r:2 /log:$ErrorLog /tee
}
Catch {
Add-Content -Path $ErrorLog -Value "Exception write"
$DateTime = (Get-Date).ToShortDateString() + " " + (Get-Date).ToShortTimeString()
$Target = $_.TargetObject
$e = $_
Add-Content -Path $ErrorLog -Value "$DateTime - $e $Target"
#Write-Host "$e $Target"
$ErrorActionPreference = "Continue"
}
if($?)
{
add-content -path $ErrorLog -value ($?)
}
else
{
add-content -path $ErrorLog -value ($?)
}
}
}
#write-host $accountName
}
}
Catch {
Add-Content -Path $ErrorLog -Value "Exception write"
$DateTime = (Get-Date).ToShortDateString() + " " + (Get-Date).ToShortTimeString()
$Target = $_.TargetObject
$e = $_
Add-Content -Path $ErrorLog -force -Value "$DateTime - $e $Target"
#Write-Host "$e $Target"
$ErrorActionPreference = "Continue"
}
$WorkBook.close()
$objExcel.Quit()
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($objExcel)
Can any one help me what I have to do fix the above error.

Filewatcher - catching multiple events issue

I have a script that watches for files and triggers events on things like change. It works except there are duplicate events triggered for the same file (previous Stackflow question) which I need to handle. I am writing the filename to a logfile and only re-logging it if it is not in there.
The problem is I think the events are being fired synchronously and I am getting 2 entries in the log for 1 file change even though I am checking for dups. I am not sure how to work around this.
Here is the code:
Unregister-Event FW*
$source = 'S:\FIS-BIC Reporting\Report Output Files\IT\'
$filter = '*.*'
$MasterFile = "H:\PS\EmailNotifications.csv"
$LogFile = "H:\EmailNotify.log"
$FilesToCheck = import-csv -path $Masterfile
if (! (Test-Path $Logfile))
{
$Output = "Date|Filename|FilenameSpec|Email"
$Output | Out-File $LogFile
}
function Get-CharRight($str,$chars)
{
return $str.Substring($str.Length – $chars, $chars)
}
function ProcessFile()
{
param ([string]$filename)
Write-Host "file: $filename"
$ext= (Get-CharRight $filename 3)
if ($ext -ne "tmp")
{
foreach($FileToCheck in $FilesToCheck)
{
if($filename -like $FileToCheck.FilenameSpec)
{
Write-Host "Match"
#check log to see if an email was already sent
$Exists = import-csv -path $Logfile -delimiter "|" | where-object {$_.Filename -eq $filename}
Write-Host ($Exists.Count -eq $null)
if ($Exists.Count -eq $null)
{
Write-Host "Not in logfile"
$Date = Get-Date -format G
$Output = $Date + "|" + $filename + "|" + $FileToCheck.FilenameSpec + "|" + $FileToCheck.Email
$Output | Out-File $LogFile -append
}
else
{write-host "duplicate"}
}
}
}
}
$fsw = New-Object IO.FileSystemWatcher $source, $filter -Property #{IncludeSubdirectories = $true; NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite, Attributes'}
Register-ObjectEvent $fsw Changed -SourceIdentifier FWFileChanged -Action {
ProcessFile $Event.SourceEventArgs.FullPath
}
Register-ObjectEvent $fsw Created -SourceIdentifier FWFileCreated -Action {
ProcessFile $Event.SourceEventArgs.FullPath
}
while($true) {
start-sleep -s 2
}
`