.txt Log File Data Extraction Output to CSV with REGEX - powershell

Question

I have asked this question before to which LotPings came up with a perfect result. When speaking to the user this relates to I only got half the information in the first place!
Knowing now exactly what is required I will explain the scenario again...
Things to be bear in mind:
Terminal will always be A followed by 3 digits i.e. A123
User ID is at the top of the log file and only appears once, will always start with 89 and be six digits long. the line will always start SELECTED FOR OPERATOR 89XXXX
There are two Date patterns in the file (one is the date of search the other DOB) each needs extracting to separate columns. Not all records have a DOB and some only have the year.
Enquirer doesn't always begin with a 'C' and needs the whole proceeding line.
The search result always has 'Enquiry' and then extraction after that.
Here is the log file
L TRANSACTIONS LOGGED FROM 01/05/2018 0001 TO 31/05/2018 2359
SELECTED FOR OPERATOR 891234
START TERMINAL USER ENQUIRER TERMINAL IP
========================================================================================================================
01/05/18 1603 A555 CART87565 46573 RBCO NPC SERVICES GW/10/0043
SEARCH ENQUIRY RECORD NO : S48456/06P CHAPTER CODE =
RECORD DISPLAYED : S48853/98D
PRINT REQUESTED : SINGLE RECORD
========================================================================================================================
03/05/18 1107 A555 CERT16574 BTD/54/1786 16475
REF ENQUIRY DHF ID : 58/94710W CHAPTER CODE =
RECORD DISPLAYED : S585988/84H
========================================================================================================================
24/05/18 1015 A555 CERT15473 19625 CBRS DDS SERVICES NM/18/0199
IMAGE ENQUIRY NAME : TREVOR SMITH CHAPTER CODE =
DATE OF BIRTH : / /1957
========================================================================================================================
24/05/18 1025 A555 CERT15473 15325 CBRS DDS SERVICES NM/12/0999
REF ENQUIRY DDS ID : 04/102578R CHAPTER CODE =
========================================================================================================================
Here is an example of the log file and what needs to be extracted and under what header.
To a CSV looking like this
The PowerShell Script LotPings has done works perfectly, I just need User ID to be extracted from the top line, to account for not all records having DOB and there being more than one type of enquiry i.e. Ref Enquiry, Search Enquiry, Image Enquiry.
$FileIn = '.\SO_51209341_data.txt'
$TodayCsv = '.\SO_51209341_data.csv'
$RE1 = [RegEx]'(?m)(?<Date>\d{2}\/\d{2}\/\d{2}) (?<Time>\d{4}) +(?<Terminal>A\d{3}) +(?<User>C[A-Z0-9]+) +(?<Enquirer>.*)$'
$RE2 = [RegEx]'\s+SEARCH REF\s+NAME : (?<Enquiry>.+?) (PAGE|CHAPTER) CODE ='
$RE3 = [RegEx]'\s+DATE OF BIRTH : (?<DOB>[0-9 /]+?/\d{4})'
$Sections = (Get-Content $FileIn -Raw) -split "={30,}`r?`n" -ne ''
$Csv = ForEach($Section in $Sections){
$Row= #{} | Select-Object Date, Time, Terminal, User, Enquirer, Enquiry, DOB
$Cnt = 0
if ($Section -match $RE1) {
++$Cnt
$Row.Date = $Matches.Date
$Row.Time = $Matches.Time
$Row.Terminal = $Matches.Terminal
$Row.User = $Matches.User
$Row.Enquirer = $Matches.Enquirer.Trim()
}
if ($Section -match $RE2) {
++$Cnt
$Row.Enquiry = $Matches.Enquiry
}
if ($Section -match $RE3){
++$Cnt
$Row.DOB = $Matches.DOB
}
if ($Cnt -eq 3) {$Row}
}
$csv | Format-Table
$csv | Export-Csv $Todaycsv -NoTypeInformation

Answer 1

With such precise data the first answer could have been:
## Q:\Test\2018\07\12\SO_51311417.ps1
$FileIn = '.\SO_51311417_data.txt'
$TodayCsv = '.\SO_51311417_data.csv'
$RE0 = [RegEx]'SELECTED FOR OPERATOR\s+(?<UserID>\d{6})'
$RE1 = [RegEx]'(?m)(?<Date>\d{2}\/\d{2}\/\d{2}) (?<Time>\d{4}) +(?<Terminal>A\d{3}) +(?<Enquirer>.*)$'
$RE2 = [RegEx]'\s+(SEARCH|REF|IMAGE) ENQUIRY\s+(?<SearchResult>.+?)\s+(PAGE|CHAPTER) CODE'
$RE3 = [RegEx]'\s+DATE OF BIRTH : (?<DOB>[0-9 /]+?/\d{4})'
$Sections = (Get-Content $FileIn -Raw) -split "={30,}`r?`n" -ne ''
$UserID = "n/a"
$Csv = ForEach($Section in $Sections){
If ($Section -match $RE0){
$UserID = $Matches.UserID
} Else {
$Row= #{} | Select-Object Date,Time,Terminal,UserID,Enquirer,SearchResult,DOB
$Cnt = 0
If ($Section -match $RE1){
$Row.Date = $Matches.Date
$Row.Time = $Matches.Time
$Row.Terminal = $Matches.Terminal
$Row.Enquirer = $Matches.Enquirer.Trim()
$Row.UserID = $UserID
}
If ($Section -match $RE2){
$Row.SearchResult = $Matches.SearchResult
}
If ($Section -match $RE3){
$Row.DOB = $Matches.DOB
}
$Row
}
}
$csv | Format-Table
$csv | Export-Csv $Todaycsv -NoTypeInformation
Sample output
Date Time Terminal UserID Enquirer SearchResult DOB
---- ---- -------- ------ -------- ------------ ---
01/05/18 1603 A555 891234 CART87565 46573 RBCO NPC SERVICES GW/10/0043 RECORD NO : S48456/06P
03/05/18 1107 A555 891234 CERT16574 BTD/54/1786 16475 DHF ID : 58/94710W
24/05/18 1015 A555 891234 CERT15473 19625 CBRS DDS SERVICES NM/18/0199 NAME : TREVOR SMITH / /1957
24/05/18 1025 A555 891234 CERT15473 15325 CBRS DDS SERVICES NM/12/0999 DDS ID : 04/102578R