Restructure text file using Powershell - powershell

Question

I have an text file that is structured in the format below, I need to extract the Hostname, Stratum=x and Offset=y values into a structured format e.g. CSV. I intend to use the output to write windows event logs if the values meet certain thresholds. My thinking is that creating objects (e.g. the hostnames) and adding the stratum and offset values as members would allow me to achieve this, but my PowerShell skills fail me here..
___________________________________________________________________________
02/04/2020 08:11:00 : Started [TEST] Command Scrape
Text I don't care about
___________________________________________________________________________
Hostname_1 (192.168.1.254):
assID=0 status=0544 leap_none, sync_local_proto, 4 events, event_peer/strat_chg,
version="ntpd 4.2.2p1#1.1570-o Tue May 19 13:57:55 UTC 2009 (1)",
processor="x86_64", system="Linux/2.6.18-164.el5", leap=00, stratum=4,
precision=-10, rootdelay=0.000, rootdispersion=11.974, peer=59475,
refid=LOCAL(0),
reftime=d495c32c.0e71eaf2 Mon, Jan 7 2013 13:57:00.056, poll=10,
clock=d495c32c.cebd43bd Mon, Jan 7 2013 13:57:00.807, state=4,
offset=0.123, frequency=0.000, jitter=0.977, noise=0.977,
stability=0.000, tai=0
___________________________________________________________________________
Hostname_2 (10.10.1.1):
assID=0 status=0544 leap_none, sync_local_proto, 4 events, event_peer/strat_chg,
version="ntpd 4.2.2p1#1.1570-o Tue May 19 13:57:55 UTC 2009 (1)",
processor="x86_64", system="Linux/2.6.18-164.el5", leap=00, stratum=4,
precision=-10, rootdelay=0.000, rootdispersion=11.974, peer=59475,
refid=LOCAL(0),
reftime=d495c32c.0e71eaf2 Mon, Jan 7 2013 13:57:00.056, poll=10,
clock=d495c32c.cebd43bd Mon, Jan 7 2013 13:57:00.807, state=4,
offset=2.456, frequency=0.000, jitter=0.977, noise=0.977,
stability=0.000, tai=0
___________________________________________________________________________
Hostname_3 (10.10.1.2):
...
I found I can create the CSV if I manually reformat the data into keypairs (as below), using ConvertFrom-StringData and outputting to CSV;
(Get-Content 'file.txt' -Raw) -split '####' |
ForEach-Object {
$results = Convertform-StringData - StringData ($PSitem -replace '\n-\s+')
New-Object PSObject -Property $results | Select-Object Hostname, Stratum, Offset
} | Export-Csv 'file.csv' - NoTypeInformation
Hostname=Hostname_1
stratum=3
offset=-10.345
####
Hostname=Hostname_2
stratum=4
offset=-8.345
Becomes the following CSV:
"Hostname","Stratum","offset"
"Hostname_1","3","-10.345"
"Hostname_2","4","-8.345"

Answer 1

You could do this with the code below.
In your example, the text blocks are separated by a series of underscores. If in real life that is different, change the -split '_{2,}' accordingly.
$regex = '(?s)^\s*([\w_\d]+)\s+\(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\):.*stratum=(\d+).*offset=([\d.]+)'
$result = (Get-Content 'D:\file.txt' -Raw) -split '_{2,}' | Where-Object {$_ -match $regex} | ForEach-Object {
[PsCustomObject]#{
'Hostname' = $matches[1]
'Stratum' = $matches[2]
'Offset' = $matches[3]
}
}
# output to console
$result
#output to csv file
$result | Export-Csv -Path 'D:\file.csv' -NoTypeInformation
Output on screen:
Hostname Stratum Offset
-------- ------- ------
Hostname_1 4 0.123
Hostname_2 4 2.456
Output as CSV:
"Hostname","Stratum","Offset"
"Hostname_1","4","0.123"
"Hostname_2","4","2.456"
Regex details:
^ Assert position at the beginning of the string
\s Match a single character that is a “whitespace character” (spaces, tabs, line breaks, etc.)
* Between zero and unlimited times, as many times as possible, giving back as needed (greedy)
( Match the regular expression below and capture its match into backreference number 1
[\w_\d] Match a single character present in the list below
A word character (letters, digits, etc.)
The character “_”
A single digit 0..9
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
)
\s Match a single character that is a “whitespace character” (spaces, tabs, line breaks, etc.)
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
\( Match the character “(” literally
\d Match a single digit 0..9
{1,3} Between one and 3 times, as many times as possible, giving back as needed (greedy)
\. Match the character “.” literally
\d Match a single digit 0..9
{1,3} Between one and 3 times, as many times as possible, giving back as needed (greedy)
\. Match the character “.” literally
\d Match a single digit 0..9
{1,3} Between one and 3 times, as many times as possible, giving back as needed (greedy)
\. Match the character “.” literally
\d Match a single digit 0..9
{1,3} Between one and 3 times, as many times as possible, giving back as needed (greedy)
\) Match the character “)” literally
: Match the character “:” literally
. Match any single character
* Between zero and unlimited times, as many times as possible, giving back as needed (greedy)
stratum= Match the characters “stratum=” literally
( Match the regular expression below and capture its match into backreference number 2
\d Match a single digit 0..9
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
)
. Match any single character
* Between zero and unlimited times, as many times as possible, giving back as needed (greedy)
offset= Match the characters “offset=” literally
( Match the regular expression below and capture its match into backreference number 3
[\d.] Match a single character present in the list below
A single digit 0..9
The character “.”
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
)

Answer 2

Fantastic, thank you so much. I've made the following adjustments to accommodate non-word characters in the hostname and offset, and exclude the trailing comma on the offset values.
$regex = '(?s)^\s*([\w_\d]+)\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):.*stratum=(\d+).*offset=([\d.]+)'
is now:
(?s)^\s*([\w|\W|\d]+)\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):.*stratum=(\d+).*offset=([\W|\d.][^,]