Powershell parsing date from log - powershell

I'm trying to parse a log file to extract the date from the log file entry. I am able to find the line, but my parsing appears to not be converting the date.
$SBError = "DbConnection"
$SBWebPath = "E:\Temp\server.log"
$result = Get-Content $SBWebPath | Select-String $SBError -casesensitive | Select -last 1 | Out-String
$result | Select-String '####<(\S+ \S+, \S+ \S+ \S+) \S+>' | ForEach-Object {
$DBdateTime = $_.Matches[0].Groups[1].Value -as [DateTime]
}
Write-Output $result
Write-Output $DBdateTime
server.log file contents:
####<Dec 9, 2018 2:59:08,082 AM EST> <Info> <HTTP> Data flowing fine.
####<Dec 9, 2018 2:59:08,085 AM EST> <Warning> <HTTP> framework.db.DbConnection.
Output from script:
####<Dec 9, 2018 2:59:08,085 AM EST> <Warning> <HTTP> framework.db.DbConnection.
Variable from $DBdateTime is not populating, I suspect due to incorrect or invalid parsing and/or inclusion of milliseconds in the log date.
I don't care about the milliseconds, only the Month day year hour minutes seconds and the AM/PM tag. Also don't care about EST but need to keep in mind that when server changes to EDT this value would exist in place of EST.
Any assistance is appreciated.

You're getting an empty value for $DBDateTime because PowerShell doesn't recognize the value of your captured group as a valid DateTime, so the string cannot be cast to that type. You need to actually parse it.
$culture = [Globalization.CultureInfo]::InvariantCulture
$pattern = 'MMM d, yyyy h:mm:ss,fff tt'
$DBDateTime = [DateTime]::ParseExact($_.Matches[0].Groups[1].Value, $pattern, $culture)
See here and here for valid format strings.

Related

How to Convert datetime formats using powershell

I Have a variable stored with this value
PS C:\Users\> $Time
Monday, November 30, 2020 8:55:01 AM
Sunday, October 18, 2020 11:10:01 PM
Sunday, November 8, 2020 10:40:34 PM
Sunday, November 8, 2020 11:47:37 PM
Sunday, November 8, 2020 10:59:57 PM
Tuesday, December 1, 2020 3:15:42 AM
Monday, November 30, 2020 7:00:32 PM
Monday, November 30, 2020 12:19:06 AM
Monday, November 30, 2020 7:01:34 PM
Tuesday, December 1, 2020 1:12:10 AM
Tuesday, December 1, 2020 2:37:18 AM
Sunday, November 1, 2020 7:39:34 PM
Sunday, September 27, 2020 11:48:38 PM
I want the time formats of the variable $time to change to "yyyy-mm-dd hh:mm:ss" so that ALL of the list is displayed
PS C:\Users\> $Time
2020-11-30 08:55:01
2020-10-18 11:10:01
2020-11-08 10:40:34
2020-11-08 11:47:37
2020-11-08 10:59:57
2020-12-01 03:15:42
2020-11-30 07:00:32
2020-11-30 12:19:06
2020-11-30 07:01:34
2020-12-01 01:12:10
2020-12-01 02:37:18
2020-11-01 07:39:34
2020-09-27 11:48:38
Please help me creating a code for the same
thanks
This is a pretty common question that gets asked here on StackOverflow, however, it seems most of the answers are directed towards converting a variable which stores a single date, to a formatted string.
Whereas you have an array of dates you want to convert.
I'm going to make the assumption that you have an Array of DateTime values, and not an Array of String.
For starters, there's TONS of blogs and articles about this, not to mention the documentation.
https://devblogs.microsoft.com/scripting/formatting-date-strings-with-powershell/
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/get-date
Depending on how you need to use this data there is a million different ways to do this.
Primarily, you need to learn how to perform actions against an array of objects. Using things like ForEach, ForEach-Object, Select-Object, etc. Once you learn how to use those, then the problem just becomes "how do you format a date to a string", which is all over the place on here and the rest of the internet.
Here's some examples:
# Use this to generate sample data:
$Time = 10000,9000,8000,7000,6000,5000,4000,3000,2000,1000 |
ForEach-Object { (Get-Date).AddMinutes(-$_) }
## Various solutions:
$Time | ForEach-Object { $_.ToString('yyyy-MM-dd HH:mm:ss') }
$Time | ForEach-Object { $_ | Get-Date -Format 'yyyy-MM-dd HH:mm:ss' }
$Time | ForEach-Object { $_ | Get-Date -f 'yyyy-MM-dd HH:mm:ss' }
$Time | ForEach-Object { Get-Date $_ -f 'yyyy-MM-dd HH:mm:ss' }
$Time | Select-Object #{N='TimeString'; E={$_.ToString('yyyy-MM-dd HH:mm:ss')}}
foreach ($tv in $time) { $tv.ToSTring('yyyy-MM-dd HH:mm:ss') }
$Time.ForEach({$_.ToString('yyyy-MM-dd HH:mm:ss')})
# Other methods submitted in comments, thanks #iRon
$Time | ForEach-Object ToString('yyyy-MM-dd HH:mm:ss')
$Time.ForEach('ToString', 'yyyy-MM-dd HH:mm:ss')
Note that this is case sensitive.
MM - Means the two digit month value
mm - Means the two digit day value

Powershell - change file Date Created and Date Modified based on Filename

I have lots of file scanned PDF documents that have the file named with an included date. For example:
FileA_2017-10-15.pdf
FileB_2016-04-08.pdf
FileC_2018-01-30.pdf
some files also are formatted with an underscore at the end as well such as...
FileD_2018-01-30_1.pdf
FileE_2018-01-30_2.pdf
there are even a few that have two underscores before the date such as...
FileF_Example_2018-01-30_1.pdf
FileG_Example_2018-01-30_2.pdf
Unfortunately, the date they were scanned in is different than the actual date of the document. So the "Date Created" and "Date Modified" attributes are different than what is shown in the file name.
I would like a script that I could run to change the "Date Created" and "Date Modified" to match that of the filename.
I attempted this using someone else's script but I don't know enough about PowerShell to make it actually work. Note that I do not want to change the name of the file, only the time stamp.
$Directory = "C:\TestFolder"
$DateFormat = "yyyy-MM-dd"
foreach ($file in (Get-ChildItem $Directory)) {
$date_from_file=GetFileName::[datetime])
$file.CreationTime = $date_from_file
$file.LastAccessTime = $date_from_file
$file.LastWriteTime = $date_from_file
Write-Host ($file.Name + " - " + $date_from_file)
}
The code above can be scraped if something else has already been written since what I have doesn't work.
Edit
Wondering if it would also be possible to add to the script so that it could include files in sub-folders as well. Maybe it could be scripted in a way that would only consider the files in a folder if the Date Modified on the folder is today. I would like to run this on a parent folder that could potentially have many sub-folders and if those folders don't have a "Date Modified" of today, then it should skip the files in that folder. I was thinking that could speed up the process. Open to thoughts and thanks for the help!
You are quite near, you need
split the date part from filename and convert it to a [datetime]
I use a RegEx with a capture group anchored at the end $ of the BaseName
## Q:\Test\2019\05\19\SO_56211626.ps1
$Directory = "C:\TestFolder"
foreach ($file in (Get-ChildItem -Path $Directory -Filter *.pdf)){
if($File.BaseName -match '_(\d{4}-\d{2}-\d{2})(_\d)?$'){
$date_from_file= (Get-Date $Matches[1])
$file.CreationTime = $date_from_file
$file.LastAccessTime = $date_from_file
$file.LastWriteTime = $date_from_file
$file | Select-Object Name,CreationTime,LastAccessTime,LastWriteTime
}
}
Sample output:
> Q:\Test\2019\05\19\SO_56211626.ps1
Name CreationTime LastAccessTime LastWriteTime
---- ------------ -------------- -------------
FileA_2017-10-15.pdf 2017-10-15 00:00:00 2017-10-15 00:00:00 2017-10-15 00:00:00
FileB_2016-04-08.pdf 2016-04-08 00:00:00 2016-04-08 00:00:00 2016-04-08 00:00:00
FileC_2018-01-30.pdf 2018-01-30 00:00:00 2018-01-30 00:00:00 2018-01-30 00:00:00
An English locale (en-US) produces:
Name CreationTime LastAccessTime LastWriteTime
---- ------------ -------------- -------------
FileA_2017-10-15.pdf 10/15/2017 12:00:00 AM 10/15/2017 12:00:00 AM 10/15/2017 12:00:00 AM
FileB_2016-04-08.pdf 4/8/2016 12:00:00 AM 4/8/2016 12:00:00 AM 4/8/2016 12:00:00 AM
FileC_2018-01-30.pdf 1/30/2018 12:00:00 AM 1/30/2018 12:00:00 AM 1/30/2018 12:00:00 AM
[
edit - since the OP is getting very strange errors with my suggested fix - errors that i cannot reproduce with the sample data - i've changed this answer to the full suggested code.
edit 2 - added new file name variants and code to deal with them.
edit 3 - changed from splitting to a regex match since the sample data has changed yet again. [*sigh ...*]
]
you are not actually creating the datetime object that you need. the $date_from_file= line doesn't actually do anything other than create red error msgs ... [grin]
replace this line ...
$date_from_file=GetFileName::[datetime])
... with this line ...
$date_from_file = [datetime]::ParseExact($File.BaseName.Split('_')[-1], $DateFormat, $Null)
... and your $date_from_file variable will contain a proper [datetime] object that will work in your assignments.
i would likely change the sequence of those assignments to put the $file.LastAccessTime = $date_from_file LAST so that it doesn't get changed by the next line.
also, that value will change any time that the file is accessed, so it may not be worth changing. [grin]
here is the full script along with what it does -
what it does ...
sets the location & the date format to use
creates a set of test files from the OPs sample file names
gets the files from the source
converts the .BaseName into a [datetime] object
assigns the .CreationTime, .LastWriteTime, & .LastAccessTime values to the datetime from the file name
displays the changed values
here is the code ...
$Directory = $env:TEMP
$DateFormat = "yyyy-MM-dd"
# create some test files
$TestFileList = #(
'FileA_2017-10-15.pdf'
'FileB_2016-04-08.pdf'
'FileC_2018-01-30.pdf'
'FileD_2019-09-09_1.pdf'
'FileE_2015-05-05_2.pdf'
)
foreach ($TFL_Item in $TestFileList)
{
$Null = New-Item -Path $Directory -Name $TFL_Item -ItemType File -Force
}
$FileList = Get-ChildItem -LiteralPath $Directory -Filter '*.pdf' -File
foreach ($FL_Item in $FileList) {
# removed split, added regex match to work with ever-growing list of variant file names
$Null = $FL_Item.BaseName -match '_(?<DateString>\d{4}-\d{2}-\d{2})'
$DateString = $Matches.DateString
$date_from_file = [datetime]::ParseExact($DateString, $DateFormat, $Null)
$FL_Item.CreationTime = $date_from_file
$FL_Item.LastWriteTime = $date_from_file
$FL_Item.LastAccessTime = $date_from_file
# show the resulting datetime info
'=' * 20
$CurrentFileInfo = Get-Item -LiteralPath $FL_Item.FullName
$CurrentFileInfo.FullName
$CurrentFileInfo.CreationTime
$CurrentFileInfo.LastWriteTime
$CurrentFileInfo.LastAccessTime
}
screen output ...
====================
C:\Temp\FileA_2017-10-15.pdf
2017 October 15, Sunday 12:00:00 AM
2017 October 15, Sunday 12:00:00 AM
2017 October 15, Sunday 12:00:00 AM
====================
C:\Temp\FileB_2016-04-08.pdf
2016 April 08, Friday 12:00:00 AM
2016 April 08, Friday 12:00:00 AM
2016 April 08, Friday 12:00:00 AM
====================
C:\Temp\FileC_2018-01-30.pdf
2018 January 30, Tuesday 12:00:00 AM
2018 January 30, Tuesday 12:00:00 AM
2018 January 30, Tuesday 12:00:00 AM
====================
C:\Temp\FileD_2019-09-09_1.pdf
2019 September 09, Monday 12:00:00 AM
2019 September 09, Monday 12:00:00 AM
2019 September 09, Monday 12:00:00 AM
====================
C:\Temp\FileE_2015-05-05_2.pdf
2015 May 05, Tuesday 12:00:00 AM
2015 May 05, Tuesday 12:00:00 AM
2015 May 05, Tuesday 12:00:00 AM
i checked the files directly in explorer & they match the displayed values.
Thanks. I was stuck without this thread. I ended up with a variation that matched any filename with a correctly formatted date, thus:
# Call like:
# powershell -NoLogo -ExecutionPolicy Unrestricted -Sta -NonInteractive -WindowStyle Normal -File ".\Rename_files_selected_folders_ModifyDateStamps.ps1" -Folder "T:\files"
# 1. capture a commandline parameter 1 as a mandatory "Folder string" with a default value
param ( [Parameter(Mandatory=$true)] [string]$Folder = "T:\HDTV\autoTVS-mpg\Converted" )
[console]::BufferWidth = 512
$DateFormat = "yyyy-MM-dd"
write-output "Processing Folder: ",$Folder
# 2. Iterate the files
$FileList = Get-ChildItem -Recurse $Folder -Include '*.mp4','*.bprj','*.ts' -File
foreach ($FL_Item in $FileList) {
$ixxx = $FL_Item.BaseName -match '(?<DateString>\d{4}-\d{2}-\d{2})'
if($ixxx){
#write-output $FL_Item.FullName
$DateString = $Matches.DateString
$date_from_file = [datetime]::ParseExact($DateString, $DateFormat, $Null)
$FL_Item.CreationTime = $date_from_file
$FL_Item.LastWriteTime = $date_from_file
$FL_Item | Select-Object FullName,CreationTime,LastWriteTime
}
}
# https://stackoverflow.com/questions/56211626/powershell-change-file-date-created-and-date-modified-based-on-filename

Split a logfile

I have a log file with this:
Wed Oct 17 05:39:27 2018 : Resource = 'test04' cstep= 'titi04' time =18.751s
Wed Oct 17 05:40:31 2018 : Resource = 'test05' cstep= 'titi05' time =58.407s
Wed Oct 17 05:41:31 2018 : Resource = 'test06' cstep= 'titi06' time =3.400s
Wed Oct 17 05:42:31 2018 : Resource = 'test07' cstep= 'titi07' time =4.402s
I want split and want only the values greater than 5:
18.751
58.407
My script is in PowerShell and collects all values, not just values greater than 5:
$list = Get-Content "C:\Users\Desktop\slow_trans\log_file.txt"
$results = foreach ($line in $list) {
$line.Split('=')[3].Trim().TrimEnd('s')
}
$results
Results are
18.751
58.407
3.400
4.402
I want only
3.400
4.402
Changing the requirements on the fly is normally a no go,
so you don't deserve it.
Also the wording Superior 5 reminds me at a previous question from another user account..
Nevertheless here a script with a single pipe and datetime conversion.
## Q:\Test\2018\11\06\SO_53170145.ps1
Get-Content .\logfile.txt |
Where-Object {$_ -match '^(.*?) : .*time =([0-9\.]+)s'}|
Select-Object #{n='DT';e={([datetime]::ParseExact($Matches[1],'ddd MMM dd HH:mm:ss yyyy',[cultureinfo]::InvariantCulture).ToString('yyyy-MM-dd HH:mm:ss'))}},
#{n='val';e={[double]$Matches[2]}} |
Where-Object val -le 5
Sample output (decimal comma due to my German locale)
DT val
-- ---
2018-10-17 05:41:31 3,4
2018-10-17 05:42:31 4,402
the following casts the selected string as double and then returns only those which are less than 5
$results = Foreach ($line in $list) {
$val = [double]$line.Split('=')[3].Trim().TrimEnd('s')
if($val -lt 5) {
$val
}
}
Select-String is one option:
(Select-String -Path "TargetLog.txt" -Pattern ".*(?<time>\d+\.\d+)s").Matches |
ForEach-Object {
if([double]$_.Groups['time'].Value -lt 5.0) {$_.Value}
}
This will output the entire matching line:
Wed Oct 17 05:41:31 2018 : Resource = 'test06' cstep= 'titi06' time =3.400s
Wed Oct 17 05:42:31 2018 : Resource = 'test07' cstep= 'titi07' time =4.402s
If you only want the number from each line, change the if block to this:
{$_.Groups['time'].Value}

String manipulation while parsing CSV

The third column of a CSV file is as like this Every Monday 12am-2am of a CSV file.
ACB;Third Week;Every Monday 12am-2am
XYZ;Third Week;Every Tuesday 8pm-10pm
I would like to capture only 12am and 8pm and so on until the end of the csv file and write the details to a separate CSV file the complete output, like below.
ACB;12am
XYZ;8pm
Use calculated properties extract a modified part of the data. The idea is to use Select-Object to select and modify the data like so,
select code, #{name="day";expression={ if($_.day -match "\d{1,2}[ap]m") {$Matches[0]} }}
So what's going on? The select (shorthand for Select-Object) can use a script block as an expression for processing. The argument is a hash table that contains manipulated property's name as a data source and expression that does stuff to it. The part if($_.day -match "\d{1,2}[ap]m") {$Matches[0]} checks if the day column value matches a regex for 1 or 2 digits followed by am or pm.
As for more complete an example,
$data = #'
ACB;Third Week;Every Monday 12am-2am
XYZ;Third Week;Every Tuesday 8pm-10pm
'#
$c = ConvertFrom-Csv -InputObject $data -Delimiter ";" -Header #('code','week','day')
$c | select code, #{name="day";expression={ if($_.day -match "\d{1,2}[ap]m") {$Matches[0]} }}
# Output
code day
---- ---
ACB 12am
XYZ 8pm

PowerShell: Comparing dates

I am querying a data source for dates. Depending on the item I am searching for, it may have more than date associated with it.
get-date ($Output | Select-Object -ExpandProperty "Date")
An example of the output looks like:
Monday, April 08, 2013 12:00:00 AM
Friday, April 08, 2011 12:00:00 AM
I would like to compare these dates and return which one is set further out into the future.
As Get-Date returns a DateTime object you are able to compare them directly. An example:
(get-date 2010-01-02) -lt (get-date 2010-01-01)
will return false.
I wanted to show how powerful it can be aside from just checking "-lt".
Example: I used it to calculate time differences take from Windows event view Application log:
Get the difference between the two date times:
PS> $Obj = ((get-date "10/22/2020 12:51:1") - (get-date "10/22/2020 12:20:1 "))
Object created:
PS> $Obj
Days : 0
Hours : 0
Minutes : 31
Seconds : 0
Milliseconds : 0
Ticks : 18600000000
TotalDays : 0.0215277777777778
TotalHours : 0.516666666666667
TotalMinutes : 31
TotalSeconds : 1860
TotalMilliseconds : 1860000
Access an item directly:
PS> $Obj.Minutes
31
Late but more complete answer in point of getting the most advanced date from $Output
## Q:\test\2011\02\SO_5097125.ps1
## simulate object input with a here string
$Output = #"
"Date"
"Monday, April 08, 2013 12:00:00 AM"
"Friday, April 08, 2011 12:00:00 AM"
"# -split '\r?\n' | ConvertFrom-Csv
## use Get-Date and calculated property in a pipeline
$Output | Select-Object #{n='Date';e={Get-Date $_.Date}} |
Sort-Object Date | Select-Object -Last 1 -Expand Date
## use Get-Date in a ForEach-Object
$Output.Date | ForEach-Object{Get-Date $_} |
Sort-Object | Select-Object -Last 1
## use [datetime]::ParseExact
## the following will only work if your locale is English for day, month day abbrev.
$Output.Date | ForEach-Object{
[datetime]::ParseExact($_,'dddd, MMMM dd, yyyy hh:mm:ss tt',$Null)
} | Sort-Object | Select-Object -Last 1
## for non English locales
$Output.Date | ForEach-Object{
[datetime]::ParseExact($_,'dddd, MMMM dd, yyyy hh:mm:ss tt',[cultureinfo]::InvariantCulture)
} | Sort-Object | Select-Object -Last 1
## in case the day month abbreviations are in other languages, here German
## simulate object input with a here string
$Output = #"
"Date"
"Montag, April 08, 2013 00:00:00"
"Freidag, April 08, 2011 00:00:00"
"# -split '\r?\n' | ConvertFrom-Csv
$CIDE = New-Object System.Globalization.CultureInfo("de-DE")
$Output.Date | ForEach-Object{
[datetime]::ParseExact($_,'dddd, MMMM dd, yyyy HH:mm:ss',$CIDE)
} | Sort-Object | Select-Object -Last 1
Considering you want to include time also, I have included sample. I am putting datetime in the ISO8601, so it works in locale agnostic manner.
Monday, April 08, 2013 12:00:00 AM
Friday, April 08, 2011 12:00:00 AM
(Get-date "2013-04-08T00:00:00") -lt (Get-Date "2011-04-08T00:00:00")
False