Display entire error message from logfile using powershell - powershell

One of the batch scripts creates a logfile and it can have error messages like as below
Msg 2714, Level 16, State 1:
Server 'ABC', Procedure 'proc_test1', Line 189:
There is already an object named 'table_test' in the database.
Msg 207, Level 16, State 4:
Server 'ABC', Procedure 'proc_test2', Line 197:
Invalid column name 'employee'.
Msg 207, Level 16, State 4:
Server 'ABC', Procedure 'proc_test2', Line 197:
Invalid column name 'address'.
Using a powershell script to read and report errors to a mail from the log file. And i'm using this, which excludes certain warning messages too.
Select-String -Path 'C:\Users\BatchLog_20200911.txt' -Pattern "Msg","Error" | Select-String -Pattern "SQLState = S1000, NativeError = 0" -notmatch | select-object -Property Line,LineNumber
Output is as below
Line LineNumber
---- ----------
Msg 2714, Level 16, State 1: 2791
Msg 207, Level 16, State 4: 2794
Msg 207, Level 16, State 4: 2797
CT-LIBRARY error: 2828
ct_results(): network packet layer: internal net library error: Net-Library operation terminated due to disconnect 2829
"Errors encountered during execution. Exited with status: 596" 4168
So, it just prints line which match the query. I want to print the next lines which has actual error message description. Any advise here. Thanks.

Use the -Context parameter of the Select-String cmdlet:
Select-String -Path 'C:\Users\BatchLog_20200911.txt' "Msg","Error" -Context 0,1 |
ForEach-Object {
if ($_.Context.PostContext[0] -notmatch 'SQLState = S1000, NativeError = 0') {
[pscustomobject] #{
Line = $_.Context.PostContext[0]
LineNumber = $_.LineNumber
}
}
}
-Context 0,1 captures 0 lines before and 1 line after along with the actually matching line.
In the resulting Microsoft.PowerShell.Commands.MatchInfo instance, .Context.PostContext[0] provides access to the first post-context (after) line.
With your sample input, the above yields:
Line LineNumber
---- ----------
Server 'ABC', Procedure 'proc_test1', Line 189: 1
Server 'ABC', Procedure 'proc_test2', Line 197: 4
Server 'ABC', Procedure 'proc_test2', Line 197: 7

Related

Joining value from Context parameter to Line property from Select-String output

I'm trying to compile log errors from multiple files in a single directory. The error messages are included over the span of two lines. I would like to concatenate both lines into a single line/object and then export all errors into a a neat csv.
I'm attempting to accomplish this with the Select-String utility, and the -Context parameter. Prior to piping the results through the Select-Object utility, everything's Kosher. However, Once I pipe the results through Select-Object or Export-CSV, the -Context line is lost.
$trigger = 'ERROR'
$folderPath = 'C:\Users\test\Desktop\testpath'
$logFiles = gci -Path $folderPath -Filter *.txt -File
$logFiles | Select-String -Pattern $trigger -CaseSensitive -SimpleMatch -Context 0,1 | Select-Object LineNumber, Line, Filename |
Export-Csv -Path .\$(Get-Date -Format yyyymmddhhmmss).csv -Encoding UTF8 -NoTypeInformation
Omitting the Select-Object and Export-Csv Cmdlets yields the desired, raw, results with the friendly right angle bracket '>' (ASCII 62). The raw results can even be exported via the Out-File Cmdlet, no problem.
However, what I would like to do, is combine the Pattern line with the Context line, creating a single object, which would eventually be output as a csv for further analysis.
I would like apologize if this question seems trivial. I've scoured resources trying to figure this out and unfortunately haven't been able to. Thanks in advance!
Pipe select-string through fl * to see what the properties are.
$a = ls log | select-string error -context 0,1
$a | fl *
IgnoreCase : True
LineNumber : 2
Line : error
Filename : log
Path : /Users/js/log
Pattern : error
Context : Microsoft.PowerShell.Commands.MatchInfoContext
Matches : {0}
$a.context
PreContext PostContext DisplayPreContext DisplayPostContext
---------- ----------- ----------------- ------------------
{} {after } {} {after }
This worked for me:
ls log | select-string error -context 0,1 | select linenumber, line,
#{n='PostContext'; e={$_.context.postcontext}}, filename
LineNumber Line PostContext Filename
---------- ---- ----------- --------
2 error after log

Powershell to match the current line and next line then out-file

I'm trying to extract the data whereby:
line 1 = Report ID + Line 2 = "Machine no" + Line 3 = OFFLINE
Then Out-File to a new file.
Sample data
Report ID page1
Machine no 1234
OTHERS
12
offline
12
OTHERS
23
offline
37
OTHERS
89
offline
65
The result I'm looking for look something like the below after processing:
Report ID page 4
Machine no 1234
offline
12
offline
37
offline
65
You can use the Select-String cmdlet with the -Context Parameter to search through a file and then select how many lines of contextual info you want to get back from your search.
For instance, if we take your input file and store it in a variable called $input like so:
$inputFile= Get-Content C:\Path\To\YourFile.txt
$inputFile| Select-string 'machine no'
>Machine no 1234
We can then find matches for the phrase 'offline' with this:
$inputFile| Select-String offline -Context 0,1
This states that I want you to search for the word 'offline' and give me zero lines proceeding it, and one line after it, giving us this output:
> offline
12
> offline
37
> offline
65
We can put this all together to build this and generate a new output file that would look like this.
$out= ''
$out += $inputFile| Select-string 'machine no'
$out += "`n"
$out += $inputFile| Select-String offline -Context 0,1 | ForEach-Object {$_.Context.DisplayPostContext}
#Resulting file would look this, just the name of the machine and then the number of each offline...uh, whatever it is.
Machine no 1234
12 37 65
If I were you, I'd adapt this flow to make PowerShell objects and properties instead, like this:
$Report = [pscustomobject]#{ID='';MachineNo='';OfflineMembers=#()}
$Report.ID = ($inputFile | select-string page -Context 0 ).ToString().Split('page')[-1]
$Report.MachineNo = ($inputFile | Select-string 'machine no').ToString().Trim().Split()[-1]
$Report.OfflineMembers = $inputFile | Select-String offline -Context 0,1 | ForEach-Object {
[pscustomobject]#{Value='Offline';ID=$_.Context.DisplayPostContext.Trim()}
}
>$Report
ID MachineNo OfflineMembers
-- --------- --------------
1 1234 {#{Value=Offline; ID=12}, #{Value=Offline; ID=37}, #{Value=Offline; ID=65}}
$Report.OfflineMembers
Value ID
----- --
Offline 12
Offline 37
Offline 65

Select-string multiple variable patterns

I am trying to extract error lines from a log file which are defined by two things. The log file line looks like this:
2018-05-22 06:25:35.309 +0200 (Production,S8320,DKMdczmpOXVJtYCSosPS6SfK8kGTSN1E,WwObvwqUw-0AAEnc-XsAAAPR) catalina-exec-12 : ERROR com.tableausoftware.api.webclient.remoting.RemoteCallHandler - Exception raised by call target: User 2027 does not have permissions to view comments for view 13086. (errorCode=1)
com.tableausoftware.domain.exceptions.PermissionDeniedException: User 2027 does not have permissions to view comments for view 13086. (errorCode=1)
The error is described in two lines, so I need to filter the error and the current hour and then copy it into a file.
This code does does copy all the errors, but not only from the current hour.
$hodina = (Get-Date -UFormat "%H").ToString()
$hodina = " " + $hodina +":"
$err = ": ERROR"
$errors = Select-String -Path "D:\..\file.log" -Pattern $hodina, $err -Context 0, 1
echo ($errors).Line >> Errors_file.txt
So I was wondering, how to put multiple variables into -Pattern, or if there is another solution to this problem.
Here is how to get all of the matching lines:
Get-Content "file.log" |
Select-String -Pattern "^(?:(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})).* : ERROR (?:(.*))$" |
ForEach-Object {
[PsCustomObject]#{
TimeStamp=(Get-Date $_.Matches.Groups[1].Value)
LineNumber=$_.LineNumber
Error=$_.Matches.Groups[2].Value
}
}
This will give you output like this:
TimeStamp LineNumber Error
--------- ---------- -----
22/05/2018 06:25:35 1 com.tableausoftware.api.webclient.remoting.RemoteCallHandler - Exception raised...
22/05/2018 06:25:35 4 com.tableausoftware.api.webclient.remoting.RemoteCallHandler - Exception raised...
22/05/2018 06:25:35 8 com.tableausoftware.api.webclient.remoting.RemoteCallHandler - Exception raised...
22/05/2018 06:25:35 10 com.tableausoftware.api.webclient.remoting.RemoteCallHandler - Exception raised...
If you only want the items where the hour of the timestamp matches the current hour, modify the code like this:
Get-Content "file.log" |
Select-String -Pattern "^(?:(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})).* : ERROR (?:(.*))$" |
ForEach-Object {
[PsCustomObject]#{
TimeStamp=(Get-Date $_.Matches.Groups[1].Value)
LineNumber=$_.LineNumber
Error=$_.Matches.Groups[2].Value
}
} | Where-Object {$_.TimeStamp.Hour -eq (Get-Date).Hour}
You can then send the output to file, or better (if you plan to manipulate them later in PowerShell), CSV (Export-Csv) or CliXml (Export-CliXml)

Change json values

Is it possible to change the value in a json file.
Little bit of background ground I'm able to pull ticket data from Freshdesk using their api. The status of the ticket comes down as a number. So for example a "open" ticket would have a value of 2. I want it to actually say open instead of two. Any thoughts?
If you are saving it as a file you could just do a find and replace
(Get-Content c:\temp\file.json).replace('status: 1', 'status: \'Open\'') | Set-Content c:\temp\file.json
and repeat for the different possible values of status
Or
$a = Get-Content 'D:\temp\file.json' -raw | ConvertFrom-Json
$a.update | % {if($_.status -eq 1){$_.status='Open'}}
$a | ConvertTo-Json | set-content 'D:\temp\file.json'
and repeat line 2 for each value of status
You could maintain a map and check it against it every time.
For example
const getType = t => ({
2: "Open",
3: "Pending",
4: "Resolved",
5: "Closed"
})[t]
;

How to search a text file for string and perform a lookup using results and the contents of a CSV file?

I've been using a PowerShell script that reads a file and extracts error codes. It quick, simple and does the job that I want it to but I've now been asked to share it with a wide audience so I need to make it a bit more robust.
The problem I've got is that I need to take the output from my script and use it to lookup against a CSV file so that I get a user friendly table at the end that lists:
A count of the how many time each error occurred (in descending order)
The Error code
The corresponding error message that it displays to the end user
This is the line format in the source file, there's normally upwards on 2000 lines
17-12-2016,10:17:44:487{String=ERROR->(12345678)<inData:{device=printer, formName=blah.frm, eject=FALSE, operation=readForm}><outData:{fields=0, CODE=, field1=}> <outError:{CODE=Error103102, extendedErrorCode=-1, VARS=0}>}
This is my current script:
$WS = Read-Host "Enter computer name"
$date = Read-host "Enter Date"
# Search pattern for select-string (date always at the beginning of the line and the error code somewhere further in)
$Pattern = $date+".*<outError:{CODE="
# This find the lines in the files that contain the search pattern
$lines = select-string -path "\\$WS\c$\folder\folder\file.dat" -pattern $Pattern
# This is the specific Error code pattern that I'm looking for in each line
$regex = [regex] 'Error\d{1,6}'
$Var = #()
# Loops through each line and extracts Error code
foreach ($line in $lines) { $a = $line -match $regex
# Adds each match to variable
$Var += $matches.Values
}
# Groups and sorts results in to easy to read format
$Var | group | Sort-Object -Property count -descending
And this is the result it gives me:
Count Name Group
----- ---- -----
24 Error106013 {Error106013, Error106013, Error106013, Error106013...}
14 Error106109 {Error106109, Error106109, Error106109, Error106109...}
12 Error203002 {Error203002, Error203002, Error203002, Error203002...}
The CSV that I need to lookup against is as simple as it gets, with just 2 values per line in the format:
Code,Error message
What I need to get to is something like this:
Count Name Error Message
----- ---- -----
24 Error106013 Error:blah
14 Error106109 Error:blah,blah
12 Error203002 Error:blah,blah,balh
Google has failed me so I'm hoping that there is someone out there that can at the least point me in the right direction.
Not tested but it should work with a simple calculated property - just replace the last line with:
$errorMap = Import-Csv 'your_errorcode.csv'
$Var | Group-Object | Sort-Object -Property count -descending |
Select-Object Count, Name, #{l='Error Message'; e={($errorMap | Where-Object Code -eq $_.Name)."Error message"}}
Note: You also have to replace path to your CSV.