Log and read only last 10 minutes - powershell

Question

I am begginer in Powershell and I have a problem with script.
I have a log and I need to send an email notification with an error. I would like to plan a task (TASK SCHEDULE) that will run my script regularly every ten minutes. This script verifies the last lines written in the last ten minutes. If the word ERROR is found, it will send an e-mail with this line where is the word ERROR.
My log:
2022-02-08 12:04:35,152 [105] ERROR RSeC.NET.RedundantHttpClient - No server found
2022-02-08 14:28:51,317 [4] DEBUG RSeC.NET.RSeC - Logging initialised
2022-02-08 14:28:53,835 [4] DEBUG RSeC.NET.JsonParser - Response binary data decoded. Size=424132
2022-02-08 14:29:20,494 [105] DEBUG RSeC.NET.RSeC - Logging initialised
2022-02-08 15:38:35,152 [105] ERROR RSeC.NET.RedundantHttpClient - No server found
2022-03-08 15:28:51,317 [4] DEBUG RSeC.NET.RSeC - Logging initialised
2022-03-08 15:28:53,835 [4] DEBUG RSeC.NET.JsonParser - Response binary data decoded. Size=424132
2022-03-08 15:39:20,494 [105] DEBUG RSeC.NET.RSeC - Logging initialised
2022-03-08 15:39:35,152 [105] ERROR RSeC.NET.RedundantHttpClient - No server found
My script :-(
$file = "C:\Soubory\esel.log"
$date = (Get-Date).AddDays(-50).Date
$cont = Get-Content -Path $file | Select-String -Pattern $date | Select-String "ERROR" | Measure-Object -line
Foreach-Object {
if ($cont -match "ERROR")
{
$kontent = Get-Content -Path $file | Select-String -Pattern $date | Select-String "ERROR" | Measure-Object -line
Write-Host $cont
}
else
{
#NOTING
}
}
Thank You for help
GILD

Answer 1

I think Lee_Dailey gave you the answer in his comment.
Simply figure out the worst-case scenario to read the bottom number of lines of the file, where you can be certain the last ten minutes are in there.
Then do:
$maxLines = 20 # just a guess here, but you can narrow the number of lines to read by trial and error
$lastTenMinutes = (Get-Date).AddMinutes(-10)
$errorLines = Get-Content -Path 'C:\Soubory\esel.log' -Tail $maxLines |
Where-Object { [datetime]($_ -split ',')[0] -gt $lastTenMinutes -and $_ -match 'ERROR' }
# test if there were error lines found
if (#($errorLines).Count) {
# send your email alert.
# If this email is in HTML format, use: $errorLines -join '<br>'
# if the email is plain text, join with newlines: $errorLines -join [environment]::NewLine
# for demo just output to console
Write-Host ("Errors found:`r`n{0}" -f ($errorLines -join [environment]::NewLine))
}
else {
Write-Host "No error lines found" -ForegroundColor Green
}
On my Dutch locale, [datetime]($_ -split ',')[0] parses the date correctly, but on your machine you may have to use [datetime]::ParseExact(($_ -split ',')[0], 'yyyy-MM-dd HH:mm:ss', $null)

Answer 2

It looks like a csv without the header, and I can compare the first column like it's a date and time, so:
import-csv log -header time,message |
where { (get-date).AddMinutes(-10) -lt $_.time -and
$_.message -match 'error' }
time message
---- -------
2022-03-09 10:11:35 152 [105] ERROR RSeC.NET.RedundantHttpClient - No server found
I would use the windows event log for easier filtering.

Answer 3

As I said in the comment on your question, "figure out what is the largest number of entries you could ever expect in 10 minutes". When you have that number, change the "20" in line "$MinLinesToGet = 20" in the code below to that value. Also, change the line "$MaxLengthOfALine = 100" so that it has the length of the longest line you expect to see.
If for some reason you need more than 10 minutes, change the value in the line "$MinutesOld = 10".
The code:
Uses ReadBytesFromFileEnd to read $ByteCount bytes from the end of $FilePath.
Uses ReadLinesFromFileEnd to read $MinLineCount of lines, each are expected to have less than $MaxLineLength, from the end of $FilePath. In reality, it should read several lines more than we want - which is a good thing.
Uses "[System.Text.Encoding]::UTF8.GetString" to convert the bytes to a string. And then uses Split to make an array of strings that is saved in $Lines.
The UTF8 before GetString is a type of encoding, and is the most likely file encoding the log file is in, so you shouldn't have to change anything. But, if there are problems, find the line "<## >" and remove the space when testing. The code will then give the raw text lines that are being returned by ReadLinesFromFileEnd. At that point, you can try the other encodings listed in the comment line above the GetString statement. Most encoding have a character size of 1, but in some unlikely, or absurd case where your log file has a larger character size, then change the line "$CharSize = 1" as needed.
You didn't provide your regex you were using, so I built my own and used the switch statement to loop through all the lines that are returned by ReadLinesFromFileEnd. I made the assumption that comma after the date WAS NOT part of a comma delimited line, but still part of the time stamp, and the numbers just afterwards is the milliseconds. I don't know what the numbers in the brackets are, so I just gave it the name "Code".
Those lines older than 10 minutes are used to build a PSObject that you can use in later code.
The last 3 lines are an example using the returned log entries in a Write-Host statement.
This worked well in my testing, but never know till the code is actually tried in the real world.
function ReadBytesFromFileEnd{
param (
[Parameter(Mandatory = $true, Position = 0)]
[string]$FilePath,
[Parameter(Mandatory = $true, Position = 1)]
[int]$ByteCount
)
$fs = [IO.File]::OpenRead($FilePath) # Open file
if($ByteCount -gt $fs.Length) {$ByteCount = $fs.Length} # Prevent reading more bytes than exist
$null = $fs.Seek(-$ByteCount, [System.IO.SeekOrigin]::End) # Position for reading
$Return = new-object Byte[] $ByteCount # Define the buffer $Return
$fs.Read($Return, 0, $ByteCount) | Out-Null # Fill the buffer
$fs.Close() # Close the file
return $Return
}
function ReadLinesFromFileEnd{
param (
[Parameter(Mandatory = $true, Position = 0)]
[string]$FilePath,
[Parameter(Mandatory = $true, Position = 1)]
[int]$MinLineCount,
[Parameter(Mandatory = $true, Position = 2)]
[int]$MaxLineLength
)
$CharSize = 1
[int]$ByteCount = 1.25 * ($MinLineCount + 1) * $MaxLineLength * $CharSize
# Encoding Options: ASCII, BigEndianUnicode, Default, Unicode, UTF32, UTF7,UTF8
$null, $Return = [System.Text.Encoding]::UTF8.GetString((ReadBytesFromFileEnd $FilePath $ByteCount)).Split(#("`n","`r"),[System.StringSplitOptions]::RemoveEmptyEntries)
return $Return
}
$MinLinesToGet = 20
$MaxLengthOfALine = 100
$MinutesOld = 10
$Lines = ReadLinesFromFileEnd 'C:\Soubory\esel.log' $MinLinesToGet $MaxLengthOfALine
<## >
$Lines
exit
#>
$Now = Get-Date
$RecentLogs = switch -Regex ($Lines) {
'^(?<DateTime>\d{4}-\d\d-\d\d\s+\d\d:\d\d:\d\d,\d+)\s*\[(?<Code>\d+)\]\s*(?<Description>.*)$' {
$DateTime = [DateTime]::ParseExact($Matches.DateTime, 'yyyy-MM-dd HH:mm:ss,fff', $null)
if ($Now.Subtract($DateTime).TotalMinutes -lt $MinutesOld) {
New-Object PSObject -Property #{
DateTime = $DateTime
Code = $Matches.Code
Description = $Matches.Description
}
}
continue
}
Default {
continue
}
}
$RecentLogs | ForEach-Object {
Write-Host "$($_.DateTime) [$($_.Code)] $($_.Description)"
}
EDIT:
You may want to check for lines that are not caught by the Regex. If so, place the following lines between the Default { statement and the coninue statement.
Write-Color "Regex failed to match: " -ForegroundColor Red -NoNewLine
Write-Color "$_" -ForegroundColor Yellow