I am trying to create a PowerShell script which will act as a watcher to a log file and point out the changes in log file.
Somehow I am not able to achieve the pointing out of changes, I tried the following code, here I tried to fetch changes in last 15 minutes but it did not worked:
$File = "C:\Program Files (x86)\Apache Software Foundation\Apache2.2\logs\error.log"
$Action = 'awk ' $0 >= from' from=$(date -u -d -15min "+####<%d-%b-%Y %H:%M:%S o'clock GMT>") error.log'
$global:FileChanged = $false
function Wait-FileChange {
param(
[string]$File,
[string]$Action
)
$FilePath = Split-Path $File -Parent
$FileName = Split-Path $File -Leaf
$ScriptBlock = [scriptblock]::Create($Action)
$Watcher = New-Object IO.FileSystemWatcher $FilePath, $FileName -Property #{
IncludeSubdirectories = $false
EnableRaisingEvents = $true
}
$onChange = Register-ObjectEvent $Watcher Changed -Action {$global:FileChanged = $true}
while ($global:FileChanged -eq $false){
Start-Sleep -Milliseconds 100
}
& $ScriptBlock
Unregister-Event -SubscriptionId $onChange.Id
}
Wait-FileChange -File $File -Action $Action
Later I changed the $Action part with the following code which will fetch last 5 lines and I got it working fine
$Action = 'Get-Content "C:\Program Files (x86)\Apache Software Foundation\Apache2.2\logs\error.log" | Select-Object -last 5 '
But my target is to watch the change and print the changes which are made.
Can anyone help me out regarding this, any help is valuable.
But my target is to watch the change and print the changes which are
made.
Are you aware that the Get-Content cmdlet has a -Wait switch:
Get-Content $File -Wait
Related
I have some code that checks a target file, waits for a change, and I want it to only move the most recent files based on their LastWriteTime Value. However, every time I change a file within the target directory nothing is copying over and I am having the copy-item directory change to "C:\Users\run". I
it recognizes that there are files to copy and even states their filename when throwing the error. What can I do in this situation to make sure my copy-item command is copying from my target directory?
Code for Reference:
$File = "C:\Users\run\Desktop\Target"
$destinationFolder = "c:\users\run\desktop\dest"
$maxDays = "-1"
$maxMins = "20"
$date = Get-Date
Write-Host "Waiting For File To Change in Job Cloud..."
$Action = '
dateChecker
Write-Host "Moving Files From Job Cloud To Server Shares... Please Do Not Disrupt This Service"
write-host "files copied to job cloud..."
exit
'
$global:FileChanged = $false
function dateChecker {
Foreach($File in (Get-ChildItem -Path $File)){
if($File.LastWriteTime -lt ($date).AddMinutes($maxMins)){
Write-Host "Moving Files From Job Cloud To Server Shares... Please Do Not Disrupt This Service"
Copy-Item -Path $File -Destination $destinationFolder -recurs #-ErrorAction #silentlyContinue
}
}
}
while($true) {
function Wait-FileChange {
param(
[string]$File,
[string]$Action
)
$FilePath = Split-Path $File -Parent
$FileName = Split-Path $File -Leaf
$ScriptBlock = [scriptblock]::Create($Action)
$Watcher = New-Object IO.FileSystemWatcher $FilePath, $FileName -Property #{
IncludeSubdirectories = $false
EnableRaisingEvents = $true
}
$onChange = Register-ObjectEvent $Watcher Changed -Action {$global:FileChanged = $true}
while ($global:FileChanged -eq $false){
Start-Sleep -Milliseconds 100
}
& $ScriptBlock
Unregister-Event -SubscriptionId $onChange.Id
}
Wait-FileChange -File $File -Action $Action
}
PowerShell is not switching directories - although I can certainly see why you'd think that based on the behavior. The explanation is closer than you might think though:
The -Path parameter takes a [string] argument.
$File is not a string - it's a [FileInfo] object - and PowerShell therefore converts it to a string before passing it to Copy-Item -Path. Unfortunately, this results in the name of the file (not the full path) being passed as the argument, and Copy-Item therefore has to resolve the full path, and does so relative to the current working directory.
You can fix this by passing the full path explicitly to Copy-Item -LiteralPath:
Copy-Item -LiteralPath $File.FullName ... |...
or you can let the pipeline parameter binder do it for you by piping the $File object to Copy-Item:
$File |Copy-Item ... |...
Why -LiteralPath instead of -Path? -Path accepts wildcard patterns like filenameprefix[0-9] and tries to resolve it to a file on disk, meaning if you have to operate on files with [ or ] in the name, it'll result in some unexpected behavior :)
Please find below script, what it does is find *.txt files and rename with either filename.host1.txt or filename.host2.txt.
What my requirement is it should create in sequence mode and not in random (Currently it is creating Host1 or host2 as random basis and not in sequence mode), If first file it renames as file1.host1.txt, then second file should be file2.host2.txt,and third file will be file3.host3.txt, fourth will be again starts with file4.host1.txt,file5.host2.txt,file4.host3.txt,.
Basically from i want to add "host1" to "host3" whenever new file detects in folder and create files in sequential mode from 1 to 3 only. Host1,Host2,Host3 i used three virtual machine to execute sperete script.
Here is my script..
### SET FOLDER TO WATCH + FILES TO WATCH + SUBFOLDERS YES/NO
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = "C:\test"
$watcher.Filter = "*.txt"
$watcher.IncludeSubdirectories = $false
$watcher.EnableRaisingEvents = $true
### DEFINE ACTIONS AFTER AN EVENT IS DETECTED
$action = { $path = $Event.SourceEventArgs.FullPath
$filename = [io.path]::GetFileNameWithoutExtension($path)
###$changeType = $Event.SourceEventArgs.ChangeType
###$logline = "$(Get-Date), $changeType, $path, $filename"
###Add-content "D:\log.txt" -value $logline
###$proc=start-process "D:\source\$filename.bat" -Wait -NoNewWindow
$script:counter++
$hostname=""
if(($script:counter % 2) -eq 0){
$hostname="host1"
} Else {
$hostname="host2"
}
Rename-Item -Path "C:\test\$filename.txt" -NewName "$filename.$hostname.txt"
}
### DECIDE WHICH EVENTS SHOULD BE WATCHED
Register-ObjectEvent $watcher "Created" -Action $action
### Register-ObjectEvent $watcher "Changed" -Action $action
### Register-ObjectEvent $watcher "Deleted" -Action $action
### Register-ObjectEvent $watcher "Renamed" -Action $action
while ($true) {sleep 5}
You can Register a new ObjectEvent after renaming a file, so you would prepare for the next file that is coming. If a new file get created while its renaming a file. You will miss a file to rename. Chances of this happeningen is close to zero.
### SET FOLDER TO WATCH + FILES TO WATCH + SUBFOLDERS YES/NO
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = "C:\test"
$watcher.Filter = "*.txt"
$watcher.IncludeSubdirectories = $false
$watcher.EnableRaisingEvents = $true
### DEFINE ACTIONS AFTER AN EVENT IS DETECTED
$action = { $path = $Event.SourceEventArgs.FullPath
$filename = [io.path]::GetFileNameWithoutExtension($path)
###$changeType = $Event.SourceEventArgs.ChangeType
###$logline = "$(Get-Date), $changeType, $path, $filename"
###Add-content "D:\log.txt" -value $logline
###$proc=start-process "D:\source\$filename.bat" -Wait -NoNewWindow
$script:counter++
$hostname=""
if(($script:counter % 2) -eq 0){
$hostname="host1"
} Else {
$hostname="host2"
}
Rename-Item -Path "C:\test\$filename.txt" -NewName "$filename.$hostname.txt"
# ADD this line
Register-ObjectEvent $watcher "Created" -Action $action
}
### DECIDE WHICH EVENTS SHOULD BE WATCHED
Register-ObjectEvent $watcher "Created" -Action $action
OR
You could do it this way, it will run slower and proccess multiple files at the same time. It will look at the creation date to determine which file was created first. Note that if you move a file here the creationdate of the file is not updated and it will use the original creation date of the file.
code:
$counter = 1
while($True){
# get all the files in C:\ test, filter out the ones that are already renamed and sort them on creation time
# if a file is moved into this directory you will not have the creationtime will be the time that it was originally created and not the moved time
Get-ChildItem -Path c:\test | ? {$_.FullName -notlike "*.host*.txt"} | Sort-Object -Property CreationTime | % {
#determine the suffix
if($counter % 3 -eq 0){
$hostSuffix = 3
}elseif($counter % 3 -eq 2){
$hostSuffix = 2
}elseif($counter % 3 -eq 1){
$hostSuffix = 1
}
# renaming file
$filename = "$($_.BaseName).host$($hostSuffix).txt"
write-host "#$($counter): renaming: $($_.FullName) to: $filename"
Rename-Item $_.FullName -NewName $filename
#count
$counter++
}
sleep(4)
}
I am trying to write something like a CSharp watch task in PowerShell. So, what I want to happen is when a CSharp file changes in a certain directory, it tries to find the csproj file and initiates a build.
Here's what I have currently
function Watch-CSharp-Files() {
set-location "D:\path\to\csharp\files\"
$originalPath = Get-Location
Write-host "Welcome to The Watcher. It keeps track of changing files in this solution directory (including subdirectories) and triggers a build when something changes..."
$existingEvents = get-eventsubscriber
foreach ($item in $existingEvents) {
Unregister-event -SubscriptionId $item.SubscriptionId
write-host "Unsubscribed existing event:" $item.Action.Name
}
$folder = get-location
$filter = '*.*'
$watcher = New-Object IO.FileSystemWatcher $folder, $filter -Property #{IncludeSubdirectories = $true;NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'}
Register-ObjectEvent $watcher Changed -SourceIdentifier FileChanged -Action {
$path = $Event.SourceEventArgs.FullPath
if ($path -match "(\.cs~|.cs$)") {
write-host "file changed: $path"
Invoke-Expression -Command "Find-And-Build-Project '$path'"
}
}
}
function Find-And-Build-Project([string]$path) {
write-host "BUILD PROJECT REPORTING FOR DUTY"
$pathParts = "$path".Split("\\")
$end = $pathParts.Count - 2 # skip the file name to the parent directory
$testPath = $pathParts[0..$end] -join "\"
write-host "testing path $testPath"
$csproj = Get-ChildItem -path $testPath *.csproj
For ($i = 0; $i -le 10; $i++) {
$newEnd = $end - $i
$newPath = $pathParts[0..$newEnd] -join "\"
$csproj = Get-ChildItem -path $newPath *.csproj
write-host "$i. trying: $newPath, csproj: $csproj"
if ($csproj) {
write-host "found on $i, at $newPath, $csproj"
break
}
}
write-host "Ready: $newPath\$csproj"
$msbuild = "C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe"
write-host "trying to MSBUILD"
& $msbuild ("$newPath\$csproj", "/target:Build", "/p:configuration=debug", "/verbosity:n")
}
Watch-CSharp-Files
What I have found is that within the function Find-And-Build-Project, the & $msbuild doesn't get invoked. But, I don't understand why.
Any ideas?
Ok, this helped me: https://stackoverflow.com/a/37724701/1326235.
This script targets the saved .cs file (or .cs~tmp[0-9].cs as Visual Studio seems to create), then searches back up the directory tree to find a .csproj file and builds it.
I published the module to the PowerShell Gallery, it's called CSharp-Watch
Hi I have installed the PowerShellPack and I am using the FileSystem watcher module,but the problem when I safe the file as a script and execute it.
The problem is that if you execute the script it runs and the monitors the folder for changes but once the script stops (gets to the end of execution) the folder is no longer monitored.
I have tried to place everything in a do while loop but that does not seem to work.
PowerShellPack Install
Import-Module -Name FileSystem
$TempCopyFolder = "c:\test"
$PatchStorage = "c:\testpatch"
Start-FileSystemWatcher -File $TempCopyFolder -Do {
$SearchPath = $File
$PatchesPath = $PatchStorage
$NewFolderFullPath = "$($eventArgs.FullPath)"
$NewFolderName = "$($eventArgs.Name)"
$PathToCheck = "$PatchesPath\$NewFolderName"
#Check if it is a filde or folder
switch ($ObjectType)
{{((Test-Path $NewFolderFullPath -PathType Container) -eq $true)}{$ObjectType = 1;break}
{((Test-Path $NewFolderFullPath -PathType Leaf) -eq $true)}{$ObjectType = 2;break}}
# Its a folder so lets check if we have a folder in the $PatchesPath already
IF($ObjectType -eq 1){
IF(!(Test-Path -LiteralPath $PathToCheck -EA 0))
{
sleep -Seconds 3
#Make a new directory where we store the patches
New-item -Path $PatchesPath -Name $NewFolderName -ItemType directory
#Make a folde in the folder for TC1
$TcFolder=$NewFolderName + '_1'
$NewPatchesPath = "$PatchesPath\$NewFolderName"
New-item -path $NewPatchesPath -Name $TcFolder -ItemType directory
$CopySrc = $NewFolderFullPath
$CopyDes = "$NewPatchesPath\$TcFolder"
}
# There is a folder there so lets get the next number
Else{
$HighNumber = Get-ChildItem -Path $PathToCheck | select -Last 1
#Core_SpanishLoginAttemptsConfiguration_Patch_03
$NewNumber = [int](Select-String -InputObject $HighNumber.Name -Pattern "(\d\d|\d)" | % { $_.Matches } | % { $_.Value } )+1
$TcFolder= $NewFolderName + '_' + $NewNumber
$NewPatchesPath = "$PatchesPath\$NewFolderName"
$CopySrc = $NewFolderFullPath
$CopyDes = "$NewPatchesPath\$TcFolder"
}
#Lets copy the files to their new home now that we know where every thing goes
$robocopy = "robocopy.exe"
$arguments = '''' + $CopySrc + '''' +' '+ ''''+ $CopyDes + '''' + '/E'
Invoke-Expression -Command "$robocopy $arguments"
Do {sleep -Seconds 1;$p = Get-Process "robo*" -ErrorAction SilentlyContinue}
While($p -ne $null)
#Now lets check every thing copyed
$RefObj = Get-ChildItem -LiteralPath $NewFolderFullPath -Recurse
$DifObj = Get-ChildItem -LiteralPath $CopyDes -Recurse
IF(Compare-Object -ReferenceObject $RefObj -DifferenceObject $DifObj)
{write-host "Fail"}
Else{# Now lets delete the source
Remove-Item -LiteralPath $CopySrc -Force -Recurse
}
}}
You don't need add-on modules or WMI for this. Just set of the FileSystemWatcher yourself and register an event. Sure, it's a bit more code, but at least you know what's going on. :)
$watcher = new-object System.IO.FileSystemWatcher
$watcher.Path = 'c:\logs'
$watcher.Filter = '*.log' # whatever you need
$watcher.IncludeSubDirectories = $true # if needed
$watcher.EnableRaisingEvents = $true
Register-ObjectEvent $watcher -EventName Changed -SourceIdentifier 'Watcher' -Action { param($sender, $eventArgs)
<process event here>
}
When done:
Unregister-Event -SourceIdentifier 'Watcher'
This is something you probably need: Monitoring file creation using WMI and PowerEvents module
PowerEvents Module isn't mandatory if you know how to create Permanent Events in WMI. For more information on that, check my eBook on WQL via PowerShell: http://www.ravichaganti.com/blog/?page_id=2134
Below is a script that monitors a directory and its subfolders for deposited files. Every 10 minutes or so, I look for new files and then match them against a database table that tell me where they need to be moved to - then it copies the files to a local archive, moves them to the locations they need to be moved to, and inserts a record into another database table with the file's attributes and where it came and went. If there is no match in the database - or there is an script error - it sends me an email.
However, since files are getting deposited to the directory constantly, it's possible that a file is still being written when the script executes. As a result, I get the error The process cannot access the file because it is being used by another process. emailed to me all the time. In addition, because I'm not dealing with the error up front; it goes through the loop and a false entry is inserted into my log table in the database with incorrect file attributes. When the file finally frees up, it gets inserted again.
I'm looking for a way to identify files that have processes attached to them; and skipping them when the script executes - but several days of web searches and some testing hasn't yielded an answer yet.
## CLEAR ERROR LOG
$error.clear()
Write-Host "***File Transfer Script***"
## PARAMETERS
$source_path = "D:\Files\In\"
$xferfail_path = "D:\Files\XferFailed\"
$archive_path = "D:\Files\XferArchive\"
$email_from = "SQLMail <SQLMail#bar.com>"
$email_recip = [STRING]"foo#bar.com"
$smtp_server = "email.bar.com"
$secpasswd = ConvertTo-SecureString "Pa$$w0rd" -AsPlainText -Force
$smtp_cred = New-Object System.Management.Automation.PSCredential ("BAR\SQLAdmin", $secpasswd)
## SQL LOG FUNCTION
function Run-SQL ([string]$filename, [string]$filepath, [int]$filesize, [int]$rowcount, [string]$xferpath)
{
$date = get-date -format G
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server=SQLSERVER;Database=DATABASE;Uid=SQLAdmin;Pwd=Pa$$w0rd;"
$SqlConnection.Open()
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = "INSERT INTO DATABASE..Table VALUES ('$date','$filename','$filepath',$filesize,$rowcount,'$xferpath',0)"
$SqlCmd.Connection = $SqlConnection
$SqlCmd.ExecuteNonQuery()
$SqlConnection.Close()
}
## DETERMINE IF THERE ARE ANY FILES TO PROCESS
$file_count = Get-ChildItem -path $source_path |? {$_.PSIsContainer} `
| Get-ChildItem -path {$_.FullName} -Recurse | Where {$_.psIsContainer -eq $false} | Where {$_.Fullname -notlike "D:\Files\In\MCI\*"} `
| Measure-Object | Select Count
If ($file_count.Count -gt 0)
{
Write-Host $file_count.Count "File(s) Found - Processing."
Start-Sleep -s 5
## CREATE LIST OF DIRECTORIES
$dirs = Get-ChildItem -path $source_path -Recurse | Where {$_.psIsContainer -eq $true} | Where {$_.Fullname -ne "D:\Files\In\MCI"} `
| Where {$_.Fullname -notlike "D:\Files\In\MCI\*"}
## CREATE LIST OF FILES IN ALL DIRECTORIES
$files = ForEach ($item in $dirs)
{
Get-ChildItem -path $item.FullName | Where {$_.psIsContainer -eq $false} | Sort-Object -Property lastWriteTime -Descending
}
## START LOOPING THROUGH FILE LIST
ForEach ($item in $files)
{
## QUERY DATABASE FOR FILENAME MATCH, AND RETURN TRANSFER DIRECTORY
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server=SQLSERVER;Database=DATABASE;Uid=SQLAdmin;Pwd=Pa$$w0rd;"
$SqlConnection.Open()
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = "SELECT F.DirTransfer FROM DATABASE..Files F WHERE '$item.Name.Trim()' LIKE F.FileName"
$SqlCmd.Connection = $SqlConnection
$DirTransfer = $SqlCmd.ExecuteScalar()
$SqlConnection.Close()
If ($DirTransfer) # if there is a match
{
Write-Host $item.FullName"`t->`t"$DirTransfer
$filename = $item.Name
$filepath = $item.FullName
$filesize = $item.Length
If (!($filesize))
{
$filesize = 0
}
$rowcount = (Get-Content -Path $item.FullName).Length
If (!($rowcount))
{
$rowcount = 0
}
$xferpath = $DirTransfer
Run-SQL -filename "$filename" -filepath "$filepath" -filesize "$filesize" -rowcount "$rowcount" -xferpath "$DirTransfer"
Copy-Item -path $item.FullName -destination $DirTransfer -force -erroraction "silentlycontinue"
Move-Item -path $item.FullName -destination $archive_path -force -erroraction "silentlycontinue"
#Write-Host "$filename $filepath $filesize $rowcount $xferpath"
}
Else # if there is no match
{
Write-Host $item.FullName "does not have a mapping"
Move-Item -path $item.FullName -destination $xferfail_path -force
$filename = $item.FullName
$email_body = "$filename `r`n`r`n does not have a file transfer mapping setup"
Send-MailMessage -To $email_recip `
-From $email_from `
-SmtpServer $smtp_server `
-Subject "File Transfer Error - $item" `
-Body $email_body `
-Priority "High" `
-Credential $smtp_cred
}
}
}
## IF NO FILES, THEN CLOSE
Else
{
Write-Host "No File(s) Found - Aborting."
Start-Sleep -s 5
}
## SEND EMAIL NOTIFICATION IF SCRIPT ERROR
If ($error.count -gt 0)
{
$email_body = "$error"
Send-MailMessage -To $email_recip `
-From $email_from `
-SmtpServer $smtp_server `
-Subject "File Transfer Error - Script" `
-Body $email_body `
-Priority "High" `
-Credential $smtp_cred
}
You can use the SysInternals handles.exe to find the open handles on a file. The exe can be downloaded from http://live.sysinternals.com/.
$targetfile = "C:\Users\me\Downloads\The-DSC-Book.docx"
$result = Invoke-Expression "C:\Users\me\Downloads\handle.exe $targetfile" | Select-String ([System.IO.Path]::GetFileNameWithoutExtension($targetfile))
$result
Outputs:
WINWORD.EXE pid: 3744 type: File 1A0: C:\Users\me\Downloads\The-DSC-Book.docx
Alternatively, you can check for errors either via try/catch or by looking at the $error collection after the Move-Item attempt then handle the condition appropriately.
$error.Clear()
Move-Item -path $item.FullName -destination $xferfail_path -force -ea 0
if($error.Count -eq 0) {
# do something useful
}
else {
# do something that doesn't involve spamming oneself
}
To expand on Arluin's answer. It fails if there's spaces in either the handle.exe or the $targetfile.
This will work for spaces in both and also formats the result to give you the Program Name.exe
$targetfile = "W:\Apps Folder\File.json"
$result = & "W:\Apps (Portable)\handle.exe" "$targetfile" | Select-String ([System.IO.Path]::GetFileNameWithoutExtension($targetfile))
$result = $result -replace '\s+pid\:.+'
$result
# PS> FreeCommander.exe
One way to avoid file locks caused by running the script on a timer is to use an event driven approach using a file system watcher. It has the ability to execute code when an event such as a new file is created in the folder you are monitoring.
To run code when the file is finished copying you would need to listen for the changed event. There is a slight issue with this event in that it fires once when the file begins copying and again when it is finished. I got an idea to work around this chicken/egg problem after checking out the module Mike linked to in the comments. I've updated the code below so that it will only fire off code when file has fully been written.
To try, change $folderToMonitor to the folder you want to monitor and add some code to process the file.
$processFile = {
try {
$filePath = $event.sourceEventArgs.FullPath
[IO.File]::OpenRead($filePath).Close()
#A Way to prevent false positive for really small files.
if (-not ($newFiles -contains $filePath)) {
$newFiles += $filePath
#Process $filePath here...
}
} catch {
#File is still being created, we wait till next event.
}
}
$folderToMonitor = 'C:\Folder_To_Monitor'
$watcher = New-Object System.IO.FileSystemWatcher -Property #{
Path = $folderToMonitor
Filter = $null
IncludeSubdirectories = $true
EnableRaisingEvents = $true
NotifyFilter = [System.IO.NotifyFilters]'FileName,LastWrite'
}
$script:newFiles = #()
Register-ObjectEvent $watcher -EventName Changed -Action $processFile > $null