How to parse and delete archived event logs in Powershell - powershell

Question

I'm trying to parse archived Security logs to track down an issue with changing permissions. This script greps through .evtx files that are +10 days old. It currently outputs what I want, but when it goes to clean up the old logs (About 50GB/daily, uncompressed, each of which are archived into their own daily folder via another script that runs at midnight) it begins complaining that the logs are in use and cannot be deleted. The process that seems to be in use when I try to delete the files through Explorer is alternately DHCP Client or Event Viewer, stopping both of these services works, but clearly I can't run without eventvwr. DHCP client is used for networking niceness but is not needed.
The only thing that touches the .evtx files is this script, they're not backed up, they're not monitored by anything else, they're not automatically parsed by the Event Log service, they're just stored on disk waiting.
The script originally deleted things as it went, but then since that failed all the deletions were moved to the end, then to the KillLogWithFire() function. Even the timer doesn't seem to help. I've also tried moving the files to a Processed subfolder, but that does't work for the same reason.
I assume that there's some way to release any handles that this script opens on any files, but attempting to .close() or .dispose() of the EventLog variable in the loop doesn't work.
$XPath = #'
*[System[Provider/#Name='Microsoft-Windows-Security-Auditing']]
and
*[System/EventID=4670]
'#
$DeletableLogs = #()
$logfile = "L:\PermChanges.txt"
$AdminUsers = ("List","of","Admin","Users")
$today = Get-Date
$marker = "
-------------
$today
-------------
"
write-output $marker >> $logfile
Function KillLogWithFire($log){
Try {
remove-item $log
}
Catch [writeerror]{
$Timer += 1
sleep $timer
write-output "Killing log $log in $timer seconds"
KillLogWithFire($log)
}
}
Function LogPermissionChange($PermChanges){
ForEach($PermChange in $PermChanges){
$Change = #{}
$Change.ChangedBy = $PermChange.properties[1].value.tostring()
#Filter out normal non-admin users
if ($AdminUsers -notcontains $Change.ChangedBy){continue}
$Change.FileChanged = $PermChange.properties[6].value.tostring()
#Ignore temporary files
if ($Change.FileChanged.EndsWith(".tmp")){continue}
elseif ($Change.FileChanged.EndsWith(".partial")){continue}
$Change.MadeOn = $PermChange.TimeCreated.tostring()
$Change.OriginalPermissions = $PermChange.properties[8].value.tostring()
$Change.NewPermissions = $PermChange.properties[9].value.tostring()
write-output "{" >> $logfile
write-output ("Changed By : "+ $Change.ChangedBy) >> $logfile
write-output ("File Changed : "+ $Change.FileChanged) >> $logfile
write-output ("Change Made : "+ $Change.MadeOn) >> $logfile
write-output ("Original Permissions :
"+ $Change.OriginalPermissions) >> $logfile
write-output ("New Permissions :
"+ $Change.NewPermissions) >> $logfile
"}
" >> $logfile
}
}
GCI -include Archive-Security*.evtx -path L:\Security\$Today.AddDays(-10) -recurse | ForEach-Object{
Try{
$PermChanges = Get-WinEvent -Path $_ -FilterXPath $XPath -ErrorAction Stop
}
Catch [Exception]{
if ($_.Exception -match "No events were found that match the specified selection criteria."){
}
else {
Throw $_
}
}
LogPermissionChange($PermChanges)
$PermChanges = $Null
$DeletableLogs += $_
}
foreach ($log in $DeletableLogs){
$Timer = 0
Try{
remove-item $log
}
Catch [IOException]{
KillLogWithFire($log)
}
}
UPDATE
Rather than editing the original code as I've been told not to do, I wanted to post the full code that's now in use as a separate answer. The Initial part, which parses the logs and is run every 30 minutes is mostly the same as above:
$XPath = #'
*[System[Provider/#Name='Microsoft-Windows-Security-Auditing']]
and
*[System/EventID=4670]
'#
$DeletableLogs = #()
$logfile = "L:\PermChanges.txt"
$DeleteList = "L:\DeletableLogs.txt"
$AdminUsers = ("List","Of","Admins")
$today = Get-Date
$marker = "
-------------
$today
-------------
"
write-output $marker >> $logfile
Function LogPermissionChange($PermChanges){
ForEach($PermChange in $PermChanges){
$Change = #{}
$Change.ChangedBy = $PermChange.properties[1].value.tostring()
#Filter out normal non-admin users
if ($AdminUsers -notcontains $Change.ChangedBy){continue}
$Change.FileChanged = $PermChange.properties[6].value.tostring()
#Ignore temporary files
if ($Change.FileChanged.EndsWith(".tmp")){continue}
elseif ($Change.FileChanged.EndsWith(".partial")){continue}
$Change.MadeOn = $PermChange.TimeCreated.tostring()
$Change.OriginalPermissions = $PermChange.properties[8].value.tostring()
$Change.NewPermissions = $PermChange.properties[9].value.tostring()
write-output "{" >> $logfile
write-output ("Changed By : "+ $Change.ChangedBy) >> $logfile
write-output ("File Changed : "+ $Change.FileChanged) >> $logfile
write-output ("Change Made : "+ $Change.MadeOn) >> $logfile
write-output ("Original Permissions :
"+ $Change.OriginalPermissions) >> $logfile
write-output ("New Permissions :
"+ $Change.NewPermissions) >> $logfile
"}
" >> $logfile
}
}
GCI -include Archive-Security*.evtx -path L:\Security\ -recurse | ForEach-Object{
Try{
$PermChanges = Get-WinEvent -Path $_ -FilterXPath $XPath -ErrorAction Stop
}
Catch [Exception]{
if ($_.Exception -match "No events were found that match the specified selection criteria."){
}
else {
Throw $_
}
}
LogPermissionChange($PermChanges)
$PermChanges = $Null
$DeletableLogs += $_
}
foreach ($log in $DeletableLogs){
write-output $log.FullName >> $DeleteList
}
The second portion does the deletion, including the helper function above graciously provided by TheMadTechnician. The code still loops as the straight delete is faster than the function, but not always successful even ages after the files have not been touched.:
# Log Cleanup script. Works around open log issues caused by PS parsing of
# saved logs in EventLogParser.ps1
$DeleteList = "L:\DeletableLogs.txt"
$DeletableLogs = get-content $DeleteList
Function Close-LockedFile{
Param(
[Parameter(Mandatory=$true,ValueFromPipeline=$true)][String[]]$Filename
)
Begin{
$HandleApp = 'C:\sysinternals\Handle.exe'
If(!(Test-Path $HandleApp)){Write-Host "Handle.exe not found at $HandleApp`nPlease download it from www.sysinternals.com and save it in the afore mentioned location.";break}
}
Process{
$HandleOut = Invoke-Expression ($HandleApp+' '+$Filename)
$Locks = $HandleOut |?{$_ -match "(.+?)\s+pid: (\d+?)\s+type: File\s+(\w+?): (.+)\s*$"}|%{
[PSCustomObject]#{
'AppName' = $Matches[1]
'PID' = $Matches[2]
'FileHandle' = $Matches[3]
'FilePath' = $Matches[4]
}
}
ForEach($Lock in $Locks){
Invoke-Expression ($HandleApp + " -p " + $Lock.PID + " -c " + $Lock.FileHandle + " -y") | Out-Null
If ( ! $LastexitCode ) { "Successfully closed " + $Lock.AppName + "'s lock on " + $Lock.FilePath}
}
}
}
Function KillLogWithFire($log){
Try {
Close-LockedFile $Log -
}
Catch [System.IO.IOException]{
$Timer += 1
sleep $timer
write-host "Killing $Log in $Timer seconds with fire."
KillLogWithFire($Log)
}
}
foreach ($log in $DeletableLogs){
Try {
remove-item $log -ErrorAction Stop
}
Catch [System.IO.IOException]{
$Timer = 0
KillLogWithFire($Log)
}
}
remove-item $DeleteList

Answer 1

One solution would be to get HANDLE.EXE and use it to close any open handles. Here's a function that I use roughly based off of this script. It uses handle.exe, finds what has a file locked, and then closes handles locking that file open.
Function Close-LockedFile{
Param(
[Parameter(Mandatory=$true,ValueFromPipeline=$true)][String[]]$Filename
)
Begin{
$HandleApp = 'C:\sysinternals\Handle.exe'
If(!(Test-Path $HandleApp)){Write-Host "Handle.exe not found at $HandleApp`nPlease download it from www.sysinternals.com and save it in the afore mentioned location.";break}
}
Process{
$HandleOut = Invoke-Expression ($HandleApp+' '+$Filename)
$Locks = $HandleOut |?{$_ -match "(.+?)\s+pid: (\d+?)\s+type: File\s+(\w+?): (.+)\s*$"}|%{
[PSCustomObject]#{
'AppName' = $Matches[1]
'PID' = $Matches[2]
'FileHandle' = $Matches[3]
'FilePath' = $Matches[4]
}
}
ForEach($Lock in $Locks){
Invoke-Expression ($HandleApp + " -p " + $Lock.PID + " -c " + $Lock.FileHandle + " -y") | Out-Null
If ( ! $LastexitCode ) { "Successfully closed " + $Lock.AppName + "'s lock on " + $Lock.FilePath}
}
}
}
I have handle.exe saved in C:\Sysinternals, you may want to adjust the path in the function, or save the executable there.

Answer 2

I was having a very similar problem and after lots of searching found this article. Whilst handle.exe worked when I first tried I did note the -c carries a warning "Closing handles can cause application or system instability"
I am also using get-winevent and it seems to (sometimes) lock the .evtx file being processed. I have written a loop to wait 5 secs an retry. Sometimes it takes up to 2 minutes or the file to be released, I have had one run overnight and it had hundreds of retries.
When I used handle the first time it worked perfectly. I then implemented it into the script and later found it to be looping an "unexplained error". I ended up having to reboot the server to get things working again so removed the handle.exe from the script and back to waiting for the file to be closed.
I can reliably release the file by stopping the script and closing down the powershell ise. As soon as the ISE is closed the file can be deleted without a problem.
Unfortunately I need this script to keep running and not be held up by the file remaining open. I am surprised that have to resort to sysinternals to release the file and that powershell does not offer an easy way to close the file.

Answer 3

I had the same issue as GTEM where closing the handles would cause corruption when processing hundreds of event log files. Eventually Get-WinEvent would not work properly. It would either freeze or give me the same "unexplained error".
So I opened a premier case with MS. They lead me to the actual variable I was storing the Get-WinEvent events in was what was locking the file. I guess it doesn't actually unlock the file if you are still using that variable. So to resolve this I added some code to my script after I transferred the variable to a new variable. You can see the code I added in the 3rd region listed below.
#***************************************************************************
#region *** Get the log entries.
# clear the log entry for each pass
$LogEntry = #()
# Get the vent from the log file and export it to the logentry variable and output to the screen
Get-WinEvent -Path $NewPath -FilterXPath $XPathFilter -ErrorAction SilentlyContinue | Tee-Object -Variable LogEntry
#endregion *** End get the log entries
#***************************************************************************
#***************************************************************************
#region *** This is where I copy it to the new variable for later output.
# if there are any log entries
if ($LogEntry.Count -gt 0) {
# Add the log entries to the log file
$LogEntries += $LogEntry
} # if there are any log entries
#endregion *** End were I copy to the new variable.
#***************************************************************************
#***************************************************************************
#region *** This is where I added code to allow me to remove the file lock.
# Remove the variable to release the evtx file lock
Remove-Variable -Name LogEntry
# Garbage collect to remove any additional memory tied to the file lock.
[GC]::Collect()
# sleep for 1 seconds
Sleep -Seconds 1
#endregion **** Code to remove the file lock.
#***************************************************************************
After this was done, I no longer have to use Handle.exe to close the file anymore.