I have a FileSystemWatcher program in PowerShell that is supposed to run on a server as long as the server is on. The program is started when the server starts. The FSW is supposed to run a program each time a new file is added to the folder it is watching, which is on a network drive. But for some reason, it doesn't execute the "action" program after some time. After a restart, the program works fine, running the "action" each time a new file arrives. I have not been able to find a clear pattern - it seems to stop responding, sometimes after a day, other times after just one "firing" of the action program.
I suspect this is because I am watching a file on a network drive, which according to other threads on stackoverflow, is unreliable and might need resetting: FileSystemWatcher stops catching events
The provided link sovles it in C#, so I wonder if a similar resetting could be written in Powershell?
Here is the program I am trying to run. I am working on a try catch for the fsw, but from what I have gathered so far, the problem must likely be solved by resetting if the network connection is interrupted.
$ErrorActionPreference = "Stop"
$action = "someprogram.ps1"
function log($string, $color, $logfile)
{
if ($Color -eq $null) {$color = 'white'}
write-host $string -foregroundcolor $color
$string | out-file -Filepath $logfile -append
}
$folder = "somefolder"
$filter = '*.csv'
$fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property
#{
IncludeSubdirectories = $false
NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
}
$onCreated = Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action
{
$global:FilePath = $Event.SourceEventArgs.FullPath
$time = get-date
log -String "File $FilePath detected at $time" -logfile $logfile
& $action
}
I am thinking a solution like this would be preferred to running a C# code:
while (Test-Path -Path $folder)
{
Start-Sleep -Milliseconds 100 #to reduce cpu load
##FileSystemWatcher code here
}
##Somehow restart file watcher if connection is lost
However, I dont want the script to run again every second or so. So I am thinking I might have to have another script running in parralell that checks if the folder path exists, and upon a disconnect, run a
Unregister-Event -SourceIdentifier FileCreated
and then restart the script. Then again, what happens if the connection to the folder is broken for one millisecond, while my script is sleeping? In that case, the test-path will not notice anything, and the script will fail as the filewatcher will no longer be able to detect a new file
Related
in the source (parent) folder, there are certain contents (subfolders, files), changes will happen in these contents time to time. my idea is to monitor this parent folder with the help of robocopy and look for changes every 2 minutes. If changes occurs, copy changes to the destination folder. Below is my command in robocopy:
robocopy C:\Users\username\Desktop\test_robocopy\source /E C:\Users\username\Desktop\test_robocopy\destination /mot:2
At first it detects changes automatically and the command runs and files are copied from source to destination. The problem is that, it is not automated afterwards, it needs key interrupt for example "hit enter" in the terminal and then it runs through. What am I missing above in the robocopy command to make it automated so that no other intervention needed.
This isn't exactly what you expect, but I think it accomplishes what you want.
I have based my script on the one found here.
This is a powershell script, so you need to save it as .ps1 and run it.
You can check the original answer for more details.
Basically what it does is to watch for file changes and trigger robocopy whenever this change happens.
In your case, I think you basically need to raise the wait time
$block = {
function Do-Something
{
param ($message, $event)
# function to call when event is raised
# do a robocopy or whatever
foreach ($folder in $watchedFolder) {
robocopy $folder ("o:\" + $folder.Replace("C:\Users\User\", "")) /MIR /e #here I'm using a string replace so I keep only the
#name of the folder to be mirrored, not the whole path - I'm sure there's a way to do this without the string replacement but
#I still didn't have time to find out how.
}
}
$watchedFolder = ("C:\Users\User\Desktop\a", "C:\Users\User\Desktop\b") #here I have created an array instead of only one folder.
#This wai I can simply add all my folders here and when my "foreach" is fired above, it will run for all my folders
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = $watchedFolder
Register-ObjectEvent -InputObject $watcher -EventName Created -SourceIdentifier File.Created -Action { Do-Something "Created" $event }
Register-ObjectEvent -InputObject $watcher -EventName Deleted -SourceIdentifier File.Deleted -Action { Do-Something "Deleted" $event }
Register-ObjectEvent -InputObject $watcher -EventName Changed -SourceIdentifier File.Changed -Action { Do-Something "Changed" $event }
Register-ObjectEvent -InputObject $watcher -EventName Renamed -SourceIdentifier File.Renamed -Action { Do-Something "Renamed" $event }
}
$encodedBlock = [Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes($block))
Start-Process PowerShell.exe -argumentlist '-WindowStyle Hidden', '-NoExit', '-EncodedCommand', $encodedBlock
while ($true) {sleep 5} #here the script sleeps 5 miliseconds and repeats all again - That way, I can listen to file changes and copy
#the changes instantly. In your case, you can use sleep -Seconds 120
How can I watch a log file and each time a line is added to it, I want to reformat that line and broadcast it to a webhook using Invoke-RestMethod (basically using discord as a log file) then possibly also output some/same info to the console.
This is a similar question to others asked like it. I've been trying to do this for 2 weeks. In the following post they answered most of this question:
Watch file for changes and run command with powershell
However, it does not show how to take the added line to the file and perform actions on it all while it is still waiting for the log file to be updated again with the next entry. Log files can be updated fast in succession so not sure if these methods will keep up with the change if another change comes to the log file while the program is performing said action on the detected change.
So far, I was trying with something like this in a loop (this is not the entire code):
get-content '<file-name>' -tail 1 -wait
But It looks like this may miss lines in succession as while it's processing the additional actions, more lines may come in. So it looks like I may need to do this in a much more complicated way (which is okay). Just trying to figure out which direction to go.
Any suggestions or direction to go are much appreciated. Even a link if that's what it takes.
What you are doing is only allowed to show a file in real-time on the screen. You cannot mess with the output doing that.
The command you are using is not for interactive use cases.
You can monitor for file updates without doing what you are doing, by using a SystemFileWatcher, which allows for monitor for file actions, that you can then take action on.
'PowerShell filesystemwatcher monitor file'
https://duckduckgo.com/?q=%27PowerShell+filesystemwatcher+monitor+file%27&t=h_&ia=web
For example from one of the hits from the above link.
https://powershell.one/tricks/filesystem/filesystemwatcher
Monitoring Folders for File Changes
With a FileSystemWatcher, you can monitor folders for file changes and
respond immediately when changes are detected. This way, you can
create “drop” folders and respond to log file changes.
Specifically, as per your use case:
Advanced Mode (asynchonous)
If you expect changes to happen in rapid succession or even
simultaneously, you can use the FileSystemWatcher in asynchronous
mode: the FileSystemWatcher now works in the background and no longer
blocks PowerShell. Instead, whenever a change occurs, an event is
raised. So with this approach, you get a queue and won’t miss any
change.
On the back side, this approach has two challenges:
Handling Events: since PowerShell is single-threaded by nature, it is
not trivial to respond to events, and even more cumbersome to debug
event handler code.
Keeping PowerShell running: ironically, because the FileSystemWatcher
now no longer blocks PowerShell, this leads to another problem. You
need to keep PowerShell waiting for events but you cannot use
Start-Sleep or and endless loop because as long as PowerShell is busy
and it is considered busy even if it sleeps - no events can be handled.
Implementation
The script below does the exact same thing as the synchronous version
from above, only it is event-based and won’t miss any events anymore:
# find the path to the desktop folder:
$desktop = [Environment]::GetFolderPath('Desktop')
# specify the path to the folder you want to monitor:
$Path = $desktop
# specify which files you want to monitor
$FileFilter = '*'
# specify whether you want to monitor subfolders as well:
$IncludeSubfolders = $true
# specify the file or folder properties you want to monitor:
$AttributeFilter = [IO.NotifyFilters]::FileName, [IO.NotifyFilters]::LastWrite
try
{
$watcher = New-Object -TypeName System.IO.FileSystemWatcher -Property #{
Path = $Path
Filter = $FileFilter
IncludeSubdirectories = $IncludeSubfolders
NotifyFilter = $AttributeFilter
}
# define the code that should execute when a change occurs:
$action = {
# the code is receiving this to work with:
# change type information:
$details = $event.SourceEventArgs
$Name = $details.Name
$FullPath = $details.FullPath
$OldFullPath = $details.OldFullPath
$OldName = $details.OldName
# type of change:
$ChangeType = $details.ChangeType
# when the change occured:
$Timestamp = $event.TimeGenerated
# save information to a global variable for testing purposes
# so you can examine it later
# MAKE SURE YOU REMOVE THIS IN PRODUCTION!
$global:all = $details
# now you can define some action to take based on the
# details about the change event:
# let's compose a message:
$text = "{0} was {1} at {2}" -f $FullPath, $ChangeType, $Timestamp
Write-Host ""
Write-Host $text -ForegroundColor DarkYellow
# you can also execute code based on change type here:
switch ($ChangeType)
{
'Changed' { "CHANGE" }
'Created' { "CREATED"}
'Deleted' { "DELETED"
# to illustrate that ALL changes are picked up even if
# handling an event takes a lot of time, we artifically
# extend the time the handler needs whenever a file is deleted
Write-Host "Deletion Handler Start" -ForegroundColor Gray
Start-Sleep -Seconds 4
Write-Host "Deletion Handler End" -ForegroundColor Gray
}
'Renamed' {
# this executes only when a file was renamed
$text = "File {0} was renamed to {1}" -f $OldName, $Name
Write-Host $text -ForegroundColor Yellow
}
# any unhandled change types surface here:
default { Write-Host $_ -ForegroundColor Red -BackgroundColor White }
}
}
# subscribe your event handler to all event types that are
# important to you. Do this as a scriptblock so all returned
# event handlers can be easily stored in $handlers:
$handlers = . {
Register-ObjectEvent -InputObject $watcher -EventName Changed -Action $action
Register-ObjectEvent -InputObject $watcher -EventName Created -Action $action
Register-ObjectEvent -InputObject $watcher -EventName Deleted -Action $action
Register-ObjectEvent -InputObject $watcher -EventName Renamed -Action $action
}
# monitoring starts now:
$watcher.EnableRaisingEvents = $true
Write-Host "Watching for changes to $Path"
# since the FileSystemWatcher is no longer blocking PowerShell
# we need a way to pause PowerShell while being responsive to
# incoming events. Use an endless loop to keep PowerShell busy:
do
{
# Wait-Event waits for a second and stays responsive to events
# Start-Sleep in contrast would NOT work and ignore incoming events
Wait-Event -Timeout 1
# write a dot to indicate we are still monitoring:
Write-Host "." -NoNewline
} while ($true)
}
finally
{
# this gets executed when user presses CTRL+C:
# stop monitoring
$watcher.EnableRaisingEvents = $false
# remove the event handlers
$handlers | ForEach-Object {
Unregister-Event -SourceIdentifier $_.Name
}
# event handlers are technically implemented as a special kind
# of background job, so remove the jobs now:
$handlers | Remove-Job
# properly dispose the FileSystemWatcher:
$watcher.Dispose()
Write-Warning "Event Handler disabled, monitoring ends."
}
So, with the above, you tweak it to look for updates/modifications, then use
$CaptureLine = Get-Content -Path 'UNCToTheLogFile' | Select-Object -Last 1
Or
$CaptureLine = Get-Content -Path 'D:\temp\book1.csv' -Tail 1
And do what you want from that.
That's probably a simple question for someone who knows Powershell. I have already tried to implement the suggestions of several answers to similar questions, such as these and these. I have a filewatcher that monitors a folder and writes a log file in case of changes. This works well.
What I didn't manage to do is to execute an additional Python script. I am grateful for any advice.
$folder = 'F:\myPath\myFolder' # Enter the root path you want to monitor.
$filter = '*.*' # You can enter a wildcard filter here.
# In the following line, you can change 'IncludeSubdirectories to $true if required.
$fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property #{IncludeSubdirectories = $false;NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'}
# Registered events:
Register-ObjectEvent $fsw Renamed -SourceIdentifier FileRenamed -Action {
$OldName = $Event.SourceEventArgs.OldName
$name = $Event.SourceEventArgs.Name
$changeType = $Event.SourceEventArgs.ChangeType
$timeStamp = $Event.TimeGenerated
Write-Host "The file '$name' was $changeType at $timeStamp" -fore white
Out-File -FilePath F:\myPath\myLogFile.txt -Append -InputObject "The file $name was $changeType at $timeStamp"
python F:\myPath\myPythonScript.py}
How can I achieve this with Powershell ISE?
You'll have much more luck using the Powershell console than you will the ISE. If you have Python added to your PATH, then it's as simple as python F:\myPath\myPythonScript.py
If the Python script returns any values, in order to pass them back to Powershell, you need to print them to the screen and they will be stored as an array in the order they're returned.
I would like to run a PowerShell script on a Windows 2012 Server in the background at startup that does the following:
Watches a folder and all subfolders (not a scheduled task; needs to be "live")
Invokes a antivirus scan (if anyone is familiar with Symantec, I would like to utilize DoScan.exe)
Copy files to a destination folder if they are "clean"; quarantine those that are not
Log all activity
I have found bits and pieces of code and built a skeleton for this, but I am running into various issues. Firstly, PowerShell script quits after running for less than 10 seconds. Runs continuously in the ISE, however. Secondly, the -WindowStyle Hidden does not function as I hoped. I do not want the user to directly know the script is running. Thirdly (this may be more of a Symantec thing), my "invoke scan" routine seems to register as 1 big scan, instead of an individual file scan.
Any guidance is appreciated. Please explain why/why not this will work, and if I should use a different language, as well as if I should break this into more than one script.
$incomingfile = "E:\"
$filter = "*.*"
$watcher = New-Object IO.FileSystemWatcher $incomingfile, $filter -Property #{
IncludeSubdirectories = $true
NotifyFilter = [IO.NotifyFilters] 'FileName, LastWrite'
}
$onCreated = Register-ObjectEvent $watcher Created -SourceIdentifier FileCreated -Action {
$path = $Event.SourceEventArgs.FullPath
$name = $Event.SourceEventArgs.Name
$changeType = $Even.SourceEventArgs.ChangeType
$timeStamp = $Event.TimeGenerated
foreach ($incomingfile in $watcher) {
& "C:\Program Files (x86)\Symantec\Symantec Endpoint Protection\DoScan.exe" /ScanFile $path
}
}
Several things:
There's a t missing in one of the statements of your watcher action:
$changeType = $Even.SourceEventArgs.ChangeType
# ^^
Shouldn't have an effect, though, since you're not using that variable anyway.
The watcher only remains active while its parent PowerShell process is running.
Your foreach loop is wrong. The watcher fires for each created file, so remove the loop and put the command directly in the action scriptblock.
$onCreated = Register-ObjectEvent ... -Action {
$path = $Event.SourceEventArgs.FullPath
& "C:\Pro...can.exe" /ScanFile $path
}
Everything you want the script to do SEP can already do by itself. There's no need at all to re-implement the wheel with a FileSystemWatcher.
I have remote server, where will be uploaded one file per day. I don't know when the file will be uploaded. I need to COPY this file to another server for processing and I need to do this just once per file (once a day). When the file is uploaded on remote server, I need to copy it within a hour, so I have to run this script at least once per hour. I'm using this script:
# Get yesterday date
$date = (Get-Date).Adddays(-1) | Get-Date -Format yyyyMMdd
$check = ""
$check = Get-Content c:\checkiftransfered.txt
# Test if file checkiftransfered.txt contains True or False. If it contains True, file for this day was already copyied
if ($check -ne "True") {
#Test if file exists - it has specific name and yesterday date
if(Test-Path \\remoteserver\folder\abc_$date.xls) {
Copy-Item \\remoteserver\folder\abc_$date.xls \\remoteserver2\folder\abc_$date.xls
# Write down information that file was already copyied
$check = "True" | Out-File c:\checkiftransfered.txt
} else { Write-Host "File has not been uploaded."}
} else { Write-Host "File has been copyied."}
# + I will need another script that will delete the checkiftransfered.txt at 0:00
It will work fine, I think, but I'm looking for more elegant solution - the best way how to solve it. Thank you
In PowerShell V3, Test-Path has a handy -NewerThan and -OlderThan parameters so you could simplify to this:
$yesterday = (Get-Date).AddDays(-1)
$date = $yesterday | Get-Date -Format yyyyMMdd
$path = "\\remoteserver\folder\abc_$date.xls"
if (Test-Path $path -NewerThan $yesterday)
{
Copy-Item $path \\remoteserver2\folder\abc_$date.xls -Verbose
(Get-Item $path).LastWriteTime = $yesterday
}
This eliminates the need to track copy status in a separate by using the LastWriteTime. One note about using -NewerThan and -OlderThan - don't use them together. It doesn't work as expected.
And lest we forget about some great native tools, here's a solution using robocopy:
robocopy $srcdir $destdir /maxage:1 /mot:60
The /mot:n option will cause robocopy to continuously monitor the source dir - every 60 minutes as specified above.
There is a much, much easier and more reliable way. You can use the FileSystemWatcher class.
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = 'C:\Uploads'
$watcher.IncludeSubdirectories = $true
$watcher.EnableRaisingEvents = $true
$created = Register-ObjectEvent $watcher "Created" -Action {
Sleep (30*60)
Copy-Item $($eventArgs.FullPath) '\\remoteserver2\folder\'
}
So lets take a look at what we doing here, we create a new watcher and tell it to watch C:\Uploads when a new file is uploaded there the file system sends a notification through the framework to our program, which in turn fires the created event. When that happens, we tell our program to sleep to for 30 minutes to allow the upload to finish (that may be to long depending on the size of the upload) then we call Copy-Item on the event arguments which contains a full path to our new file.
By the way you would need to paste this in a powershell window and leave it open on the server, alternatively you could use the ISE and leave that open. Either way it is way more reliable that what you currently have.