How to convert powershell array to table - powershell

I found a similar post regarding the problem in the link below.
How to fetch first column from given powershell array?
I am not able to directly convert it to a table as some fields are missing and do operations.
Customer ID Client Name Computer Name Computer Brand Duration Connection Time Lang
123 first last 127.0.0.1 lenovo 10:00 8/18/2019 6:00 PM Eng
1 lastname 127.0.0.2 apple 2:30:00 8/18/2019 1:00 AM Chn
86 user3 127.0.0.1 dell 8/18/2019 2:00 PM
21 user4 127.0.0.4 apple 30:00 8/17/2019 1:00 PM Eng
I want to first filter with a specific user who is connected for more than 30 minutes and then list its id.
Update
The result should be
1
21
because they are connected for 30min and over.

If the data you show is indeed the output of a Fixed-Width file, you need to try and get the widths for each field in order to parse it. A handicap here is that the original header names contain a space character and we need to replace that by an underscore.
For that, you can use the below function:
function ConvertFrom-FixedWith {
[CmdletBinding()]
Param(
[Parameter(Mandatory = $true, Position = 0)]
[string[]]$Content
)
$splitter = '§¤¶' # some unlikely string: Alt-21, [char]164, Alt-20
$needQuotes = '^\s+|[",]|\s+$' # quote the fields if needed
function _FWClean ([string]$field) {
# internal helper function to clean a field value with regards to quoted fields
$field = $_.Trim() -replace '(?<!\\)\\"|""', '§DQUOTE¶'
if ($field -match '^"(.*)"$') { $field = $matches[1] }
if ($field -match $needQuotes) { $field = '"{0}"' -f $field }
return $field -replace '§DQUOTE¶', '""'
}
# try and calculate the field widths using the first header line
# this only works if none of the header names have spaces in them
# and where the headers are separated by at least one space character.
Write-Verbose "Calculating column widths using first row"
$row = ($Content[0] -replace '\s+', ' ').Trim()
$fields = #($row -split ' ' ) # | ForEach-Object { _FWClean $_ })
$ColumnBreaks = for ($i = 1; $i -lt $fields.Length; $i++) {
$Content[0].IndexOf($fields[$i])
}
$ColumnBreaks = $ColumnBreaks | Sort-Object -Descending
Write-Verbose "Splitting fields and generating output"
$Content | ForEach-Object {
if ($null -ne $_ -and $_ -match '\S') {
$line = $_
# make sure lines that are too short get padded on the right
if ($line.Length -le $ColumnBreaks[0]) { $line = $line.PadRight(($ColumnBreaks[0] + 1), ' ') }
# add the splitter string on every column break point
$ColumnBreaks | ForEach-Object {
$line = $line.Insert($_, $splitter)
}
# split on the splitter string, trim, and dedupe possible quotes
# then join using the delimiter character
#($line -split $splitter | ForEach-Object { _FWClean $_ }) -join ','
}
} | ConvertFrom-Csv # the result is an array of PSCustomObjects
}
With that function in place, parsing the text can be done like so:
$text = #"
Customer_ID Client_Name Computer_Name Computer_Brand Duration Connection_Time Lang
123 first last 127.0.0.1 lenovo 10:00 8/18/2019 6:00 PM Eng
1 lastname 127.0.0.2 apple 2:30:00 8/18/2019 1:00 AM Chn
86 user3 127.0.0.1 dell 8/18/2019 2:00 PM
21 user4 127.0.0.4 apple 30:00 8/17/2019 1:00 PM Eng
"# -split '\r?\n'
# replace the single space characters in the header names by underscore
$text[0] = $text[0] -replace '(\w+) (\w+)', '$1_$2'
# the 'ConvertFrom-FixedWith' function takes a string array as input
$table = ConvertFrom-FixedWith -Content $text
#output on screen
$table | Format-Table -AutoSize
# export to CSV file
$table | Export-Csv -Path 'D:\test.csv' -NoTypeInformation
Output (on screen)
Customer ID Client Name Computer Name Computer Brand Duration Connection Time Lang
----------- ----------- ------------- -------------- -------- --------------- ----
123 first last 127.0.0.1 lenovo 10:00 8/18/2019 6:00 PM Eng
1 lastname 127.0.0.2 apple 2:30:00 8/18/2019 1:00 AM Chn
86 user3 127.0.0.1 dell 8/18/2019 2:00 PM
21 user4 127.0.0.4 apple 30:00 8/17/2019 1:00 PM Eng
If your input $text is already a string array storing all the ines as we see them in your question, then leave out the -split '\r?\n'
Having parsed the input to a table of PsCustomObjects, you can get the customers that are connected for 30 minutes or more with the help of another small helper function:
function Get-DurationInMinutes ([string]$Duration) {
$h, $m, $s = (('0:{0}' -f $Duration) -split ':' | Select-Object -Last 3)
return [int]$h * 60 + [int]$m
}
($table | Where-Object { (Get-DurationInMinutes $_.Duration) -ge 30 }).Customer_ID
This will output
1
21
Update
Now that we finally know the data is from a TAB delimited CSV file, you don't need the ConvertFrom-FixedWith function.
Simply import the data using if it comes from a file
$table = Import-Csv -Path 'D:\customers.csv' -Delimiter "`t"
Or, if it comes from the output of another command as string or string array:
$table = $original_output | ConvertFrom-Csv -Delimiter "`t"
Then, use the Get-DurationInMinutes helper function just like above to get the Customer ID's that are connected for more than 30 minutes:
function Get-DurationInMinutes ([string]$Duration) {
$h, $m, $s = (('0:{0}' -f $Duration) -split ':' | Select-Object -Last 3)
return [int]$h * 60 + [int]$m
}
($table | Where-Object { (Get-DurationInMinutes $_.Duration) -ge 30 }).'Customer ID'

Uhh. I'm surprised there's not a canonical way to do this. Based on https://www.reddit.com/r/PowerShell/comments/211ewa/how_to_convert_fixedwidth_to_pipedelimited_or/.
# 0 19 38 59 81 97 120 123
# Customer ID Client Name Computer Name Computer Brand Duration Connection Time Lang
# 123 first last 127.0.0.1 lenovo 10:00 8/18/2019 6:00 PM Eng
# 1 lastname 127.0.0.2 apple 2:30:00 8/18/2019 1:00 AM Chn
# 86 user3 127.0.0.1 dell 8/18/2019 2:00 PM
# 21 user4 127.0.0.4 apple 30:00 8/17/2019 1:00 PM Eng
$cols = 0,19,38,59,81,97,120,123 # fake extra column at the end, assumes all rows are that wide
$firstline = get-content columns.txt | select -first 1
$headers = for ($i = 0; $i -lt $cols.count - 1; $i++) {
$firstline.substring($cols[$i], $cols[$i+1]-$cols[$i]).trim()
}
# string Substring(int startIndex, int length)
$lines = Get-Content columns.txt | select -skip 1
$lines | ForEach {
$hash = [ordered]#{}
for ($i = 0; $i -lt $headers.length; $i++) {
$hash += #{$headers[$i] = $_.substring($cols[$i], $cols[$i+1]-$cols[$i]).trim()}
}
[pscustomobject]$hash
}
Output:
PS /Users/js/foo> ./columns | ft
Customer ID Client Name Computer Name Computer Brand Duration Connection Time Lan
----------- ----------- ------------- -------------- -------- --------------- ---
123 first last 127.0.0.1 lenovo 10:00 8/18/2019 6:00 PM Eng
1 lastname 127.0.0.2 apple 2:30:00 8/18/2019 1:00 AM Chn
86 user3 127.0.0.1 dell 8/18/2019 2:00 PM
21 user4 127.0.0.4 apple 30:00 8/17/2019 1:00 PM Eng

I think you have a couple of requirements here. I'm going to describe one way to do it using a generic 'for loop' and regular expression - something you can play with and tweak to your needs. There are better ways of doing this (Powershell shortcuts), but based on the way you asked I'm going to assume that understanding is your goal, so this code should serve well if you have a background in any programming language. Hope this helps!
# Here is your data formatted in an array. Missing values are just empty fields.
# You could have fewer or more fields, but I've broken up your data into nine fields
# (0-8 when counting elements in an array)
# Customer ID, FName, LName, ComputerHostName, Brand, Duration, ConnectionDate, ConnectionTime, Lang
$myarray = #(
('123', 'firstname', 'lastname', '127.0.0.1', 'lenovo', '10:00', '8/18/2019', '6:00 PM', 'Eng'),
('1', 'lastnam', '', '127.0.0.2', 'apple', '2:30:00', '8/18/2019', '1:00 AM', 'Chn'),
('86', 'user3', '', '127.0.0.1', 'dell', '04:33', '8/18/2019', '2:00 PM', ''),
('21', 'user4', '', '127.0.0.4', 'apple', '30:00', '8/17/2019', '1:00 PM', 'Eng')
)
# This is a generic for loop that prints the ComputerHostName, which is the 4th column.
# The 4th column is column #3 if counting from zero (0,1,2,3)
# I'm using a regular expression to match duration above 30 minutes with the '-match' operator
for ( $i = 0; $i -lt $myarray.Length; $i++ ) {
if ( $myarray[$i][5] -match "[3-5][0-9]:[0-9][0-9]$" ){
"$($myarray[$i][5]) - $($myarray[$i][3])"
}
}
Printed Result:
2:30:00 - 127.0.0.2
30:00 - 127.0.0.4

Related

updating column value for exported csv powershell

I have following code which is working correctly.
Although I now need to modify the output in one specific column, so I can sort by this column correctly.
Here is my code:
$inputFile = "C:\Data\expPasswords\expPasswords.csv"
$outputFile = "C:\Data\expPasswords\expPasswordsUp.csv"
$result = Import-Csv $inputFile |
Select-Object #{ Name = 'Account'; Expression = { $_.Account } },
#{ Name = 'Days until Expiry'; Expression = { $_.'time until password expires' } },
#{ Name = 'Email address'; Expression = { $_.'email address' } }
# output on screen
$result | Sort-Object -Property 'Days until Expiry' | Format-Table -AutoSize
# output to csv
$result | Sort-Object -Property 'Days until Expiry' | Export-Csv -Path $outputFile -NoTypeInformation
I need to sort by the 'Days until Expiry' column. Although makes it hard when the output is as below:
0 minutes
0 minutes
1 day and 19 hours
1 day and 2 hours
1 day and 20 hours
1 day and 23 hours
13 hours
2 days
20 hours
Basically, what I would like to do is:
- If less than 1 day, make the value: Today
- Remove the hours and minutes blocks.
- So if it is 13 hours, make the value: Today
- If the value is 1 day and 1 hours and 35 minutes, make the value: 1 day
Any assistance will be greatly appreciated. ;-)
Its a shame you should spend time to make some sense out of this rather foolish output, but of course it can be done.
Basically, all you want to do is find out if the string starts with a number followed by the word 'day' or 'days' and cut off all the rest. If this is not the case, the returned value should be 'Today'.
The easiest way to do that I think is by using switch -Regex.
Try
$inputFile = "C:\Data\expPasswords\expPasswords.csv"
$outputFile = "C:\Data\expPasswords\expPasswordsUp.csv"
$result = Import-Csv $inputFile | ForEach-Object {
$daysLeft = switch -Regex ($_.'time until password expires') {
'^(\d+ days?)' { $matches[1] }
default { 'Today' }
}
[PsCustomObject]#{
'Account' = $_.Account
'Days until Expiry' = $daysLeft
'Email address' = $_.'email address'
}
} | Sort-Object -Property 'Days until Expiry'
# output on screen
$result | Format-Table -AutoSize
# output to csv
$result | Export-Csv -Path $outputFile -NoTypeInformation
Regex details:
^ Assert position at the beginning of the string
\d Match a single character that is a “digit” (any decimal number in any Unicode script)
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
\ day Match the character string “ day” literally (case sensitive)
s Match the character “s” literally (case sensitive)
? Between zero and one times, as many times as possible, giving back as needed (greedy)
Seeing your comment, I would suggest adding a real DateTime object to sort on.
Something like this:
$today = (Get-Date).Date
$result = Import-Csv 'D:\test.csv' | ForEach-Object {
$expiryString = $_.'time until password expires'
$expiryDate = $today
if ($expiryString -match '(\d+)\s*day') { $expiryDate = $expiryDate.AddDays([int]$matches[1]) }
if ($expiryString -match '(\d+)\s*hour') { $expiryDate = $expiryDate.AddHours([int]$matches[1]) }
if ($expiryString -match '(\d+)\s*minute') { $expiryDate = $expiryDate.AddMinutes([int]$matches[1]) }
if ($expiryString -match '(\d+)\s*second') { $expiryDate = $expiryDate.AddSeconds([int]$matches[1]) }
$daysLeft = if ($expiryDate.Date -eq $today) { 'Today' } else { ($expiryDate - $today).Days}
[PsCustomObject]#{
'Account' = $_.Account
'Email address' = $_.'email address'
'Days until Expiry' = $daysLeft
'Expiration Date' = $expiryDate
}
} | Sort-Object -Property 'Expiration Date'
# output on screen
$result
Output:
Account Email address Days until Expiry Expiration Date
------- ------------- ----------------- ---------------
User1 user1#yourcompany.com Today 6-4-2020 0:00:00
User6 user6#yourcompany.com Today 6-4-2020 0:03:00
User8 user8#yourcompany.com Today 6-4-2020 13:00:00
User4 user4#yourcompany.com Today 6-4-2020 20:00:00
User9 user9#yourcompany.com 1 7-4-2020 2:00:00
User2 user2#yourcompany.com 1 7-4-2020 19:00:00
User5 user5#yourcompany.com 1 7-4-2020 20:00:00
User7 user7#yourcompany.com 1 7-4-2020 23:00:00
User3 user3#yourcompany.com 2 8-4-2020 0:00:00
If you don't want that new property 'Expiration Date' in your output, simply filter it away with:
$result | Select-Object * -ExcludeProperty 'Expiration Date'
I think the following might be of help (you will need to edit some of it, off course):
$Timings = #("0 minutes","0 minutes","1 day and 19 hours","1 day and 2 hours","1 day and 20 hours","1 day and 23 hours","13 hours","2 days","20 hours")
foreach ($Timing in $Timings) {
$Output = $null
if ($Timing -like "* minutes") {$Output = 0}
elseif ($Timing -like "* Day and * hours") {$Output = [int](($Timing).Split(' day')[0])}
elseif ($Timing -like "* hours") {$Output = 0}
else {$Output = [int](($Timing).Split(' day')[0]) }
switch ($Output) {
0 {$Result = "Today"}
1 {$Result = "Tomorrow"}
default {$Result = "Over $Output Days"}
}
Write-Output "$timing ==> $Result"
}
The constrains you defined will likely make it more confusing. I would just convert it to a [TimeSpan] structure which makes it easy to sort:
$Result = ConvertFrom-Csv #'
"Account","Days until Expiry", "Email address"
"Account1","0 minutes", "Name1#gmail.com"
"Account2","1 day and 19 hours","Name2#gmail.com"
"Account3","2 days", "Name3#gmail.com"
"Account4","20 hours", "Name4#gmail.com"
"Account5","1 day and 20 hours","Name5#gmail.com"
"Account6","3 minutes", "Name6#gmail.com"
"Account7","1 day and 23 hours","Name7#gmail.com"
"Account8","13 hours", "Name8#gmail.com"
"Account9","1 day and 2 hours", "Name9#gmail.com"
'#
Function ConvertTo-TimeSpan([String]$String) {
$Days = If ($String -Match '\d+(?=\s*day)') {$Matches[0]} Else {0}
$Hours = If ($String -Match '\d+(?=\s*hour)') {$Matches[0]} Else {0}
$Minutes = If ($String -Match '\d+(?=\s*minute)') {$Matches[0]} Else {0}
$Seconds = If ($String -Match '\d+(?=\s*second)') {$Matches[0]} Else {0}
New-TimeSpan -Days $Days -Hours $Hours -Minutes $Minutes -Seconds $Seconds
}
$Result | Sort #{e = {ConvertTo-TimeSpan $_.'Days until Expiry'}}
Result:
Account Days until Expiry Email address
------- ----------------- -------------
Account1 0 minutes Name1#gmail.com
Account6 3 minutes Name6#gmail.com
Account8 13 hours Name8#gmail.com
Account4 20 hours Name4#gmail.com
Account9 1 day and 2 hours Name9#gmail.com
Account2 1 day and 19 hours Name2#gmail.com
Account5 1 day and 20 hours Name5#gmail.com
Account7 1 day and 23 hours Name7#gmail.com
Account3 2 days Name3#gmail.com

How to convert text files in a folder into a CSV file

I am trying to write a script to take a bunch of text files in a folder (which are all in the same format) and output them to a csv file. Each file has the same "header" information. I have been able to get information in a more easily usable format (removing the first and last lines, which aren't needed), but am having some trouble after that.
Here is the beginning of the text file, though there will be more than just these 7 lines, there will be a total of 36 lines per file:
TYPE VOID
DOB 20200131
DATE 20200131
TIME 21:19:42
TERMINAL 3
ORGTERM 2
EMPLOYEE 1234 John Doe
And here is what I have so far, though I know that it doesn't work:
$currentdir = '.\'
$results = #()
$outputfilename = 'data.csv'
foreach ($req in Get-ChildItem($currentdir)) {
(Get-Content $req)[1..((Get-Content $req).count - 2)] |
ForEach-Object {
$header = $_[0] -split '`t'
$data = $_[1] -split '`t'
$results = $header, $data
}
}
The final product would look something like this:
A B C D E F G
1 TYPE DOB DATE TIME TERMINAL ORGTERM EMPLOYEE
2 VOID 20200131 20200131 21:19:42 3 2 1234 John Doe
3 AUTHORIZE 20200131 20200131 23:29:22 2 4678 Jane Doe
Full sample of VOID file:
BEGIN
TYPE VOID
DOB 20200131
DATE 20200131
TIME 21:19:42
TERMINAL 3
ORGTERM 2
EMPLOYEE 1234 Jane Doe
TABLE TBL 101
CHECK 20030
PAYMENT 20029
AUTHAMT 20.68
BATCHAMT 20.68
CARDTYPE MASTERCARD
CARDMASK XXXXXXXXXXXXXXXXX
{XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX}
EXP 0423
REF 482
STANDALONE YES
PINDEX 1
APPROVEAMT 20.68
LOGTIME 21:07:01
FOHFEATS 10000000000000000000000000000000
TERMCAPS 00000000000000000000000000000000
FOHVERSION 15.1.34.2.97
ACTIONCODE 000
LASTSEND 1580585993
ORIGDATE 20200131
ORIGTIME 21:02:11
ORIGTYPE AUTHORIZE
ORIGREF 482
ORGREFTIME 21:02:11
TENDER_NUM 12
CRCY 840
VPD Sequence #: 107
REVID 2
REVNAME 712 Bar
END
Sample AUTHORIZE file:
BEGIN
TYPE AUTHORIZE
DOB 20200131
DATE 20200131
TIME 23:29:22
TERMINAL 2
EMPLOYEE 1234 Jane Doe
TABLE Table 121
CHECK 20045
PAYMENT 20038
AUTHAMT 72.42
BATCHAMT 72.42
CARDTYPE VISA
CARDMASK XXXXXXXXXXXXXXXX
{XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX}
EXP 0124
REF 485900
STANDALONE YES
PINDEX 1
LOGTIME 23:29:22
FOHFEATS 10000000000000000000000000000000
TERMCAPS 00000000000000000000000000000000
FOHVERSION 15.1.34.2.97
LASTSEND 1580586235
TENDER_NUM 13
CRCY 840
REVID 1
REVNAME 712 Restaurant
COMMERROR TRUE
END
Sample adjust file:
BEGIN
TYPE ADJUST
DOB 20200131
DATE 20200131
TIME 22:18:27
TERMINAL 8
ORGTERM 8
EMPLOYEE 789 Judy Garland
TABLE BAR GUEST
CHECK 80161
PAYMENT 80036
BATCHAMT 30.43
BATCHTIP 6
CARDTYPE MASTERCARD
CARDMASK XXXXXXXXXXXX8699
{XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX}
EXP 0323
REF 1504602
STANDALONE YES
PINDEX 1
LOGTIME 22:18:27
FOHFEATS 10000000000000000000000000000000
TERMCAPS 00000000000000000000000000000000
FOHVERSION 15.1.34.2.97
LASTSEND 1580638928
TENDER_NUM 12
CRCY 840
REVID 4
REVNAME 712 Second Bar
END
here's one way to merge those text files into a CSV. it presumes the files are in a specific dir and can be loaded by matching the names OR by simply grabbing all the files.
what it does ...
sets the source dir
sets the file filter
grabs all the matching files
iterates thru the file list
loads each file into a $Var
uses the way that PoSh handles a collection on the LEFT side of a match
that gives you the matching item, not the usual [bool].
builds a PSCustomObject
it does that by matching the line with the target word, getting the 1st item in the returned array, replaces the unwanted part of the line with nothing, and finally assigns that value to the desired property.
this is rather inefficient, but i can't think of a better way. [blush]
sends the PSCO out to the $Results collection
shows what is in $Results on the screen
exports $Results to a CSV file
here's the code ...
$SourceDir = $env:TEMP
$Filter = 'harlan_*.txt'
$FileList = Get-ChildItem -LiteralPath $SourceDir -Filter $Filter -File
$Results = foreach ($FL_Item in $FileList)
{
$Lines = Get-Content -LiteralPath $FL_Item.FullName
[PSCustomObject]#{
Type = ($Lines -match '^type')[0] -replace '^type\s{1,}'
DOB = ($Lines -match '^dob')[0] -replace '^dob\s{1,}'
Date = ($Lines -match '^date')[0] -replace '^date\s{1,}'
Time = ($Lines -match '^time')[0] -replace '^time\s{1,}'
Terminal = ($Lines -match '^terminal')[0] -replace '^terminal\s{1,}'
OrgTerm = ($Lines -match '^orgterm')[0] -replace '^orgterm\s{1,}'
Employee = ($Lines -match '^employee')[0] -replace '^employee\s{1,}'
}
}
# show on screen
$Results
# save to CSV
$Results |
Export-Csv -LiteralPath "$SourceDir\Harlan_-_MergedFiles.csv" -NoTypeInformation
display on screen ...
Type : ADJUST
DOB : 20200131
Date : 20200131
Time : 22:18:27
Terminal : 8
OrgTerm : 8
Employee : 789 Judy Garland
Type : AUTHORIZE
DOB : 20200131
Date : 20200131
Time : 23:29:22
Terminal : 2
OrgTerm :
Employee : 1234 Jane Doe
Type : VOID
DOB : 20200131
Date : 20200131
Time : 21:19:42
Terminal : 3
OrgTerm : 2
Employee : 1234 Jane Doe
content of the csv file ...
"Type","DOB","Date","Time","Terminal","OrgTerm","Employee"
"ADJUST","20200131","20200131","22:18:27","8","8","789 Judy Garland"
"AUTHORIZE","20200131","20200131","23:29:22","2","","1234 Jane Doe"
"VOID","20200131","20200131","21:19:42","3","2","1234 Jane Doe"
To capture all fields in the files without hardcoding the headers and combine them into a CSV file, the below code should do it.
Snag is that there is one line in each file that does not have a 'Header', it is just a string {XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX}.
I'm guessing that should be the Card Number, so I'm manually inserting the header CARDNUMBER there. If this is something else, please change that in the code.
$files = Get-ChildItem -Path 'D:\Test' -File
$result = foreach($file in $files) {
$obj = [PsCustomObject]#{}
Get-Content -Path $file.FullName | Where-Object { $_ -notmatch '^(BEGIN|END)$' } | ForEach-Object {
# There is a line without 'header' name. Is this the card number?
if ($_ -like '{*}') {
$name = 'CARDNUMBER' # <-- add your own preferred header name here
$value = $_
}
else {
$name,$value = $_ -split '\s+', 2
}
$obj | Add-Member -MemberType NoteProperty -Name $name -Value $value
}
# output the object for this file to be colected in the $result variable
$obj
}
# output on screen
$result
#output to CSV file
$result | Export-Csv -Path 'D:\output.csv' -NoTypeInformation
You need to set the paths for Get-ChildItem and Export-CSV to match your own situation of course
If I'm reading this correctly you have some files each has a single record of data delimited between the aptly positions words "BEGIN" & "END" You want each file to be translated into a single CSV file?
I think I've cooked up something worth while. Though I'm sure it's not perfect.
$Select = 'TYPE','DOB','DATE','TIME','TERMINAL','ORGTERM','EMPLOYEE'
ForEach( $InputFile in (Get-ChildItem $CurrentDirectory) )
{
$OutputFile = $InputFile.BaseName + '.csv'
$Table = Get-Content $InputFile
$TempHash = [Ordered]#{}
ForEach( $Column in $Table )
{
If( $Column -notmatch '(^BEGIN$|^END$)' )
{
$TempArr = $Column.Split( ' ', 2, [System.StringSplitOptions]::RemoveEmptyEntries ) | ForEach{$_.Trim()}
If( $Select -contains $TempArr[0] )
{
$TempHash.Add($TempArr[0], $TempArr[1] )
}
}
}
#Now $TempHash should have enough to create the object and export to CSV
[PSCustomObject]$TempHash | Export-Csv -Path $OutputFile -NoTypeInformation
}
A few points:
I'm ignoring the lines BEGIN & END
I'm manipulating each line thereafter into an array, which for the
most part should be 2 elements.
If the first element [0] is in the collection of fileds your looking
for I'll add as a key/value pair to the hash. Otherwise do nothing.
After processing the lines Convert the object to a PSCustomObject and
export to a CSV file.
I only tested it on a single file I created from your question. I wrapped up the outer loop just as pseudo code.
This works, but the output looks a little choppy, like numbers being strings and such. That said, as a rev one I think we've got something to work with.
If misread your comment, and you want a single output CSV file the adjustment is just to declare the filename before the loop and use the append param on the Export-CSV cmdlet. See below, though I didn't test it any further:
$OutputFile = 'YourOutput.csv'
$Select = 'TYPE','DOB','DATE','TIME','TERMINAL','ORGTERM','EMPLOYEE'
ForEach( $InputFile in (Get-ChildItem $CurrentDirectory) )
{
$Table = Get-Content $InputFile
$TempHash = [Ordered]#{}
ForEach( $Column in $Table )
{
If( $Column -notmatch '(^BEGIN$|^END$)' )
{
$TempArr = $Column.Split( ' ', 2, [System.StringSplitOptions]::RemoveEmptyEntries ) | ForEach{$_.Trim()}
If( $Select -contains $TempArr[0] )
{
$TempHash.Add($TempArr[0], $TempArr[1] )
}
}
}
#Now $TempHash should have enough to create the object and export to CSV
[PSCustomObject]$TempHash | Export-Csv -Path $OutputFile -NoTypeInformation -Append
}
Sorry about the variable names, that could obviously use a refactor...
Let me know what you think.

.txt Log File Data Extraction Output to CSV with REGEX

I have asked this question before to which LotPings came up with a perfect result. When speaking to the user this relates to I only got half the information in the first place!
Knowing now exactly what is required I will explain the scenario again...
Things to be bear in mind:
Terminal will always be A followed by 3 digits i.e. A123
User ID is at the top of the log file and only appears once, will always start with 89 and be six digits long. the line will always start SELECTED FOR OPERATOR 89XXXX
There are two Date patterns in the file (one is the date of search the other DOB) each needs extracting to separate columns. Not all records have a DOB and some only have the year.
Enquirer doesn't always begin with a 'C' and needs the whole proceeding line.
The search result always has 'Enquiry' and then extraction after that.
Here is the log file
L TRANSACTIONS LOGGED FROM 01/05/2018 0001 TO 31/05/2018 2359
SELECTED FOR OPERATOR 891234
START TERMINAL USER ENQUIRER TERMINAL IP
========================================================================================================================
01/05/18 1603 A555 CART87565 46573 RBCO NPC SERVICES GW/10/0043
SEARCH ENQUIRY RECORD NO : S48456/06P CHAPTER CODE =
RECORD DISPLAYED : S48853/98D
PRINT REQUESTED : SINGLE RECORD
========================================================================================================================
03/05/18 1107 A555 CERT16574 BTD/54/1786 16475
REF ENQUIRY DHF ID : 58/94710W CHAPTER CODE =
RECORD DISPLAYED : S585988/84H
========================================================================================================================
24/05/18 1015 A555 CERT15473 19625 CBRS DDS SERVICES NM/18/0199
IMAGE ENQUIRY NAME : TREVOR SMITH CHAPTER CODE =
DATE OF BIRTH : / /1957
========================================================================================================================
24/05/18 1025 A555 CERT15473 15325 CBRS DDS SERVICES NM/12/0999
REF ENQUIRY DDS ID : 04/102578R CHAPTER CODE =
========================================================================================================================
Here is an example of the log file and what needs to be extracted and under what header.
To a CSV looking like this
The PowerShell Script LotPings has done works perfectly, I just need User ID to be extracted from the top line, to account for not all records having DOB and there being more than one type of enquiry i.e. Ref Enquiry, Search Enquiry, Image Enquiry.
$FileIn = '.\SO_51209341_data.txt'
$TodayCsv = '.\SO_51209341_data.csv'
$RE1 = [RegEx]'(?m)(?<Date>\d{2}\/\d{2}\/\d{2}) (?<Time>\d{4}) +(?<Terminal>A\d{3}) +(?<User>C[A-Z0-9]+) +(?<Enquirer>.*)$'
$RE2 = [RegEx]'\s+SEARCH REF\s+NAME : (?<Enquiry>.+?) (PAGE|CHAPTER) CODE ='
$RE3 = [RegEx]'\s+DATE OF BIRTH : (?<DOB>[0-9 /]+?/\d{4})'
$Sections = (Get-Content $FileIn -Raw) -split "={30,}`r?`n" -ne ''
$Csv = ForEach($Section in $Sections){
$Row= #{} | Select-Object Date, Time, Terminal, User, Enquirer, Enquiry, DOB
$Cnt = 0
if ($Section -match $RE1) {
++$Cnt
$Row.Date = $Matches.Date
$Row.Time = $Matches.Time
$Row.Terminal = $Matches.Terminal
$Row.User = $Matches.User
$Row.Enquirer = $Matches.Enquirer.Trim()
}
if ($Section -match $RE2) {
++$Cnt
$Row.Enquiry = $Matches.Enquiry
}
if ($Section -match $RE3){
++$Cnt
$Row.DOB = $Matches.DOB
}
if ($Cnt -eq 3) {$Row}
}
$csv | Format-Table
$csv | Export-Csv $Todaycsv -NoTypeInformation
With such precise data the first answer could have been:
## Q:\Test\2018\07\12\SO_51311417.ps1
$FileIn = '.\SO_51311417_data.txt'
$TodayCsv = '.\SO_51311417_data.csv'
$RE0 = [RegEx]'SELECTED FOR OPERATOR\s+(?<UserID>\d{6})'
$RE1 = [RegEx]'(?m)(?<Date>\d{2}\/\d{2}\/\d{2}) (?<Time>\d{4}) +(?<Terminal>A\d{3}) +(?<Enquirer>.*)$'
$RE2 = [RegEx]'\s+(SEARCH|REF|IMAGE) ENQUIRY\s+(?<SearchResult>.+?)\s+(PAGE|CHAPTER) CODE'
$RE3 = [RegEx]'\s+DATE OF BIRTH : (?<DOB>[0-9 /]+?/\d{4})'
$Sections = (Get-Content $FileIn -Raw) -split "={30,}`r?`n" -ne ''
$UserID = "n/a"
$Csv = ForEach($Section in $Sections){
If ($Section -match $RE0){
$UserID = $Matches.UserID
} Else {
$Row= #{} | Select-Object Date,Time,Terminal,UserID,Enquirer,SearchResult,DOB
$Cnt = 0
If ($Section -match $RE1){
$Row.Date = $Matches.Date
$Row.Time = $Matches.Time
$Row.Terminal = $Matches.Terminal
$Row.Enquirer = $Matches.Enquirer.Trim()
$Row.UserID = $UserID
}
If ($Section -match $RE2){
$Row.SearchResult = $Matches.SearchResult
}
If ($Section -match $RE3){
$Row.DOB = $Matches.DOB
}
$Row
}
}
$csv | Format-Table
$csv | Export-Csv $Todaycsv -NoTypeInformation
Sample output
Date Time Terminal UserID Enquirer SearchResult DOB
---- ---- -------- ------ -------- ------------ ---
01/05/18 1603 A555 891234 CART87565 46573 RBCO NPC SERVICES GW/10/0043 RECORD NO : S48456/06P
03/05/18 1107 A555 891234 CERT16574 BTD/54/1786 16475 DHF ID : 58/94710W
24/05/18 1015 A555 891234 CERT15473 19625 CBRS DDS SERVICES NM/18/0199 NAME : TREVOR SMITH / /1957
24/05/18 1025 A555 891234 CERT15473 15325 CBRS DDS SERVICES NM/12/0999 DDS ID : 04/102578R

Powershell Sort and transpose data based on particular column

I am new to powershell. I have a data which is in columnar format, which needs to be converted into rows by sorting it numerically.
Input data:
Fruit month amount
1apple jan 10
3blueberry jan 20
4mango jan 30
2banana jan 50
1apple feb 20
3blueberry feb 50
4mango feb 80
2banana feb 95
Desired Output:
Fruit JanAmount FebAmount
1apple 10 20
2banana 50 95
3blueberry 20 50
4mango 30 80
Can anyone pls help me with this?
As long as there are no whitespace in the fruit-names then you can read the file as CSV with whitespace as delimiter. Then merge them using Group-Object and Add-Member to dynamically add x months. Ex:
Import-Csv -Path $InSort -Delimiter " " |
#Group all records per fruit
Group-Object Fruit |
#Sort by fruitname
Sort-Object Name |
#Process each group (fruit) to merge the rows
ForEach-Object {
#Create object (row) per fruit
$obj = New-Object -TypeName psobject -Property #{
Fruit = $_.Name
}
#For each record (month), add amount column
$_.Group | ForEach-Object {
#Turn month-value into TitleCase (first letter uppercase)
$month = (Get-Culture).TextInfo.ToTitleCase($_.Month)
#Add amount-column per record (month)
Add-Member -InputObject $obj -MemberType NoteProperty -Name "$($Month)Amount" -Value $_.Amount
}
#Output new objects
$obj
} | Export-CSV -Path newfile.csv -NoTypeInformation -Delimiter " "
Output:
Fruit JanAmount FebAmount
----- --------- ---------
1apple 10 20
2banana 50 95
3blueberry 20 50
4mango 30 80
if you delimiter into your file is space, try this:
import-csv "C:\Temp\yourfile.txt" -Delimiter ' ' |
group fruit |
select Name, #{N="JanAmount";E={($_.Group | where month -eq 'jan').amount}} , #{N="FebAmount";E={($_.Group | where month -eq 'feb').amount}}

Easier way to parse 'query user' in PowerShell (or quser)

I currently have the following query in PowerShell:
query user /server:$server
Which returns output:
USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
svc_chthost 2 Disc 1:05 8/16/2016 12:01 PM
myusername rdp-tcp 3 Active . 8/29/2016 11:29 AM
Currently, I'm using #(query user /server:$server).Count - 1 as a value to represent the number of users logged on (it's not pretty, I know). However now I would like to obtain information such as USERNAME, ID, and LOGON TIME to use in other parts of my script.
My question is surrounding an easier way to parse the information above, or maybe a better solution to my problem all together: Counting and gathering information related to logged on users.
I've found other solutions that seem to work better, but I'm sure there's got to be a simpler way to accomplish this task:
$ComputerName | Foreach-object {
$Computer = $_
try
{
$processinfo = #(Get-WmiObject -class win32_process -ComputerName $Computer -EA "Stop")
if ($processinfo)
{
$processinfo | Foreach-Object {$_.GetOwner().User} |
Where-Object {$_ -ne "NETWORK SERVICE" -and $_ -ne "LOCAL SERVICE" -and $_ -ne "SYSTEM"} |
Sort-Object -Unique |
ForEach-Object { New-Object psobject -Property #{Computer=$Computer;LoggedOn=$_} } |
Select-Object Computer,LoggedOn
}#If
}
catch
{
}
Old question, but it seems a workable solution:
(query user) -split "\n" -replace '\s\s+', ';' | convertfrom-csv -Delimiter ';'
This chunks the output into lines, as the answer above does, but then replaces more than one white space character (\s\s+) with a semi-colon, and then converts that output from csv using the semi-colon as a delimiter.
The reason for more than one white space is that the column headers have spaces in them (idle time, logon time), so with just one space it would try to interpret that as multiple columns. From the output of the command, it looks as if they always preserve at least 2 spaces between items anyway, and the logon time column also has spaces in the field.
Awesome references in the comments, and still open to more answers for this question as it should have an easier solution!
foreach ($s in $servers) #For Each Server
{
foreach($ServerLine in #(query user /server:$s) -split "\n") #Each Server Line
{
#USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
$Parsed_Server = $ServerLine -split '\s+'
$Parsed_Server[1] #USERNAME
$Parsed_Server[2] #SESSIONNAME
$Parsed_Server[3] #ID
$Parsed_Server[4] #STATE
$Parsed_Server[5] #IDLE TIME
$Parsed_Server[6] #LOGON TIME
}
}
This solution solves the problem for now, kind of sloppy.
For more in-depth solutions with more functionalities, check the comments on the original question :)
Function Get-QueryUser(){
Param([switch]$Json) # ALLOWS YOU TO RETURN A JSON OBJECT
$HT = #()
$Lines = #(query user).foreach({$(($_) -replace('\s{2,}',','))}) # REPLACES ALL OCCURENCES OF 2 OR MORE SPACES IN A ROW WITH A SINGLE COMMA
$header=$($Lines[0].split(',').trim()) # EXTRACTS THE FIRST ROW FOR ITS HEADER LINE
for($i=1;$i -lt $($Lines.Count);$i++){ # NOTE $i=1 TO SKIP THE HEADER LINE
$Res = "" | Select-Object $header # CREATES AN EMPTY PSCUSTOMOBJECT WITH PRE DEFINED FIELDS
$Line = $($Lines[$i].split(',')).foreach({ $_.trim().trim('>') }) # SPLITS AND THEN TRIMS ANOMALIES
if($Line.count -eq 5) { $Line = #($Line[0],"$($null)",$Line[1],$Line[2],$Line[3],$Line[4] ) } # ACCOUNTS FOR DISCONNECTED SCENARIO
for($x=0;$x -lt $($Line.count);$x++){
$Res.$($header[$x]) = $Line[$x] # DYNAMICALLY ADDS DATA TO $Res
}
$HT += $Res # APPENDS THE LINE OF DATA AS PSCUSTOMOBJECT TO AN ARRAY
Remove-Variable Res # DESTROYS THE LINE OF DATA BY REMOVING THE VARIABLE
}
if($Json) {
$JsonObj = [pscustomobject]#{ $($env:COMPUTERNAME)=$HT } | convertto-json # CREATES ROOT ELEMENT OF COMPUTERNAME AND ADDS THE COMPLETED ARRAY
Return $JsonObj
} else {
Return $HT
}
}
Get-QueryUser
or
Get-QueryUser -Json
For gathering information.
based on https://ss64.com/nt/query-user.html
$result = &quser
$result -replace '\s{2,}', ',' | ConvertFrom-Csv
My own column based take. I'm not sure how much the ID column can extend to the left. Not sure how wide the end is. This is turning out to be tricky. Maybe this way is better: Convert fixed width txt file to CSV / set-content or out-file -append?
# q.ps1
# USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
# js1111 rdp-tcp#20 136 Active . 6/20/2020 4:26 PM
# jx111 175 Disc . 6/23/2020 1:26 PM
# sm1111 rdp-tcp#126 17 Active . 6/23/2020 1:13 PM
#
# di111111 rdp-tcp#64 189 Active 33 7/1/2020 9:50 AM
# kp111 rdp-tcp#45 253 Active 1:07 7/1/2020 9:43 AM
#
#0, 1-22, 23-40, 41-45, 46-53, 54-64, 65-80/82
$q = quser 2>$null | select -skip 1
$q | foreach {
$result = $_ -match '.(.{22})(.{18})(.{5})(.{8})(.{11})(.{16,18})'
[pscustomobject] #{
USERNAME = $matches[1].trim()
SESSIONNAME = $matches[2].trim()
ID = [int]$matches[3].trim()
STATE = $matches[4].trim()
IdleTime = $matches[5].trim()
LogonTime = [datetime]$matches[6].trim()
}
if (! $matches) {$_}
}
Invoke-command example. This is good if you're using Guacamole.
$c = get-credential
icm comp1,comp2,comp3 q.ps1 -cr $c | ft
USERNAME SESSIONNAME ID STATE IdleTime LogonTime PSComputerName RunspaceId
-------- ----------- -- ----- -------- --------- -------------- ----------
js1 136 Disc . 6/20/2020 4:26:00 PM comp1 a8e670cd-4f31-4fd0-8cab-8aa11ee75a73
js2 137 Disc . 6/20/2020 4:26:00 PM comp2 a8e670cd-4f31-4fd0-8cab-8aa11ee75a74
js3 138 Disc . 6/20/2020 4:26:00 PM comp3 a8e670cd-4f31-4fd0-8cab-8aa11ee75a75
Here's another version. The number in the ID column can be at least 1 column before the header. I figure out where the line ends on every line. The Sessionname ends in 3 dots if it's too long, and at least 2 spaces are between each column. The column headers always start at the same place.
ID can be 4 digits. Tricky.
USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
rwo rdp-sxs22010... 342 Active 48 2/8/2022 1:41 PM
ym326 rdp-sxs22062... 1012 Active 9 9/27/2022 3:42 PM
cw7 rdp-tcp#4 4 Active 11:16 9/26/2022 7:58 AM
# q2.ps1
$first = 1
quser 2>$null | ForEach-Object {
if ($first -eq 1) {
$userPos = $_.IndexOf("USERNAME")
$sessionPos = $_.IndexOf("SESSIONNAME") # max length 15
$idPos = $_.IndexOf("ID") - 4 # id is right justified
# $idPos = $_.IndexOf("SESSIONNAME") + 15
$statePos = $_.IndexOf("STATE") # max length 6
$idlePos = $_.IndexOf("IDLE TIME") - 2 # right justified too
$logonPos = $_.IndexOf("LOGON TIME")
$first = 0
}
else {
$user = $_.substring($userPos,$sessionPos-$userPos).Trim()
$session = $_.substring($sessionPos,$idPos-$sessionPos).Trim()
$id = [int]$_.substring($idPos,$statePos-$idPos).Trim()
$state = $_.substring($statePos,$idlePos-$statePos).Trim()
$idle = $_.substring($idlePos,$logonPos-$idlePos).Trim()
$logon = [datetime]$_.substring($logonPos,$_.length-$logonPos).Trim()
[pscustomobject]#{User = $user; Session = $session; ID = $id;
State = $state; Idle = $idle; Logon = $logon}
}
}
Output:
User Session ID State Idle Logon
---- ------- -- ----- ---- -----
rwo rdp-sxs22010... 342 Active 48 2/8/2022 1:41:00 PM
Edited: Looks like someone have already created a script that actually works pretty well: https://gallery.technet.microsoft.com/scriptcenter/Get-LoggedOnUser-Gathers-7cbe93ea
Cant believe after so many years there is still no native PowerShell for this.
I've touched up what Tyler Dickson has done and ensure the result comes back as PSCustomObject
$Servers = #("10.x.x.x", "10.y.y.y")
$Result = #()
foreach ($Server in $Servers) {
$Lines = #(query user /server:$s) -split "\n"
foreach($Line in $Lines) #Each Server Line
{
if ($Line -match "USERNAME\s+SESSIONNAME\s+ID\s+STATE\s+IDLE TIME\s+LOGON TIME") {
continue # If is the header then skip to next item in array
}
$Parsed_Server = $Line -split '\s+'
$Result += [PSCustomObject]#{
SERVER = $Server
USERNAME = $Parsed_Server[1]
SESSIONNAME = $Parsed_Server[2]
ID = $Parsed_Server[3]
STATE = $Parsed_Server[4]
IDLE_TIME = $Parsed_Server[5]
LOGON_TIME = $Parsed_Server[6]
}
}
}
$Result | Format-Table
Example output:
SERVER USERNAME SESSIONNAME ID STATE IDLE_TIME LOGON_TIME
------ -------- ----------- -- ----- --------- ----------
10.x.x.x user01 rdp-tcp#13 6 Active . 28/06/2020
10.x.x.x user02 rdp-tcp#35 11 Active 59 29/06/2020
10.y.y.y user03 rdp-tcp#38 12 Active . 29/06/2020
10.y.y.y user04 rdp-tcp#43 14 Active 5 29/06/2020
Unfortunately, no one that proposes solutions with replace method didn't notice that it will be a data collision if SESSIONNAME will empty (it will be when user disc)
So you will have SESSIONNAME contain ID, ID contain STATE etc.
It's not good.
So I`ve fixed it by -replace 'rdp-tcp#\d{1,3}' and propose to you solution with headers.
$Header = "UserName", "ID", "State", "Idle", "Logon", "Time"
$Result = $(quser) -replace 'rdp-tcp#\d{1,3}' -replace "^[\s>]", "" -replace "\s+", "," | ConvertFrom-Csv -Header $Header
Now you can access to any object $Result.Username, $Result.Idle
Was looking for the easy solution to the query user problem that also addresses the issue when SessionName is blank. Ended up combining bits and pieces from the above and came up with this. This isn't perfect, but it does seem to work better than most.
$q = (query user) -split "\n" -replace '\s{18}\s+', " blank "
$qasobject = $q -split "\n" -replace '\s\s+', "," | convertfrom-csv
The First pass with -split will replace any chunk of 18 or more spaces with " blank ", NOTE; there are 2 spaces before and after blank.
The second pass with -split will replace anything with 2 or more spaces with a ",", then pass that through convertfrom-csv to make it an object.
If you want a quick solution and don't need all information, you can also do this:
$a = Get-CimInstance -ClassName Win32_UserProfile -ComputerName "Server-1" | where {$_.Loaded -and $_.LocalPath.split('\')[1] -eq "Users" -and $_.Special -eq $false}
$a | ft -a #{N='Name';E={$_.LocalPath.split('\')[2]}},LastUseTime,Loaded
I Further appended the above code to properly format and also consider the Disconnected users
$HaSH = #()
foreach($ServerLine in #(query user) -split "\n") {
$Report = "" | Select-Object UserName, Session, ID, State, IdleTime, LogonTime
$Parsed_Server = $ServerLine -split '\s+'
if($Parsed_Server -like "USERNAME*") {
Continue
}
$Report.UserName = $Parsed_Server[1]
$Report.Session = $Parsed_Server[2]
$Report.ID = $Parsed_Server[3]
$Report.State = $Parsed_Server[4]
$Report.IdleTime = $Parsed_Server[5]
$Report.LogonTime = $Parsed_Server[6]+" " +$Parsed_Server[7]+" "+$Parsed_Server[8]
if($Parsed_Server[3] -eq "Disc") {
$Report.Session = "None"
$Report.ID = $Parsed_Server[2]
$Report.State = $Parsed_Server[3]
$Report.IdleTime = $Parsed_Server[4]
$Report.LogonTime = $Parsed_Server[5]+" " +$Parsed_Server[6]+" "+$Parsed_Server[7]
}
if($Parsed_Server -like ">*") {
$Parsed_Server=$Parsed_Server.Replace(">","")
$Report.UserName = $Parsed_Server[0]
$Report.Session = $Parsed_Server[1]
$Report.ID = $Parsed_Server[2]
$Report.State = $Parsed_Server[3]
$Report.IdleTime = $Parsed_Server[4]
$Report.LogonTime = $Parsed_Server[5]+" " +$Parsed_Server[6]+" "+$Parsed_Server[7]
}
$HaSH+=$Report
}
$result = (&quser) -replace '\s{2,}', ',' | ConvertFrom-Csv | Select -ExpandProperty USERNAME
$loggedinuser = $result.Trim(">")