Novice at PS here, but I have a log file where each entry is written on a separate line with this date format: 2013-04-29 08:55:09,261
I am trying to use PowerShell to delete all lines older than 30 days. I have been trying to plug get-date -format "yyyy-MM-dd hh" output with some sort of greater than code, but at this point I’m just guessing. Also been trying forfiles with a batch file but would like to stick with PS if I can.
Any help would be much appreciated.
Assuming a file with the contents:
2013-04-29 08:55:09,261 line1
2013-01-29 08:55:09,261 line2
2013-03-31 08:55:09,261 line3
For me your dates are in the international standard date notation so you can just use :
$a = [datetime]"2013-04-29 08:55:09"
Then $a will be a date, so nothing to do with the culture.
You can just write the following to filter all lines from a date (here "2013-03-31")
get-content "C:\temp\date.txt" | where { [datetime]($_.split(','))[0] -ge "2013-03-31"}
I just split the line on the coma, take the first part and convert it to a date before compararing it.
For your 30 days (get-date).date give the date without hours and (get-date).date.adddays(-30) give the date 30 days before today.
get-content C:\temp\date.txt | where { [datetime]($_.split(','))[0] -ge (get-date).date.adddays(-30)}
You can pipe the result in a new file | set-content "C:\temp\newdate.txt"
Try this:
gc .\logfile.txt | %{if ([datetime][regex]::match($_, '^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}').Value -gt (Get-Date).AddDays(-30)) {$_}} > purgedlogfile.txt
Breakdown and explanation of the components:
.\logfile.txt is the path to the log file
^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3} is a regular expression that matches your log file's timestamp format at the beginning of each line
[regex]::match($_, '^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}').Value returns the timestamp matched by the regular expression
Preceding it with the [datetime] typecast operator converts it to a DateTime object (in order to compare it to another DateTime)
(Get-Date) returns a DateTime object representing the current date and time, and .AddDays(-30) subtracts 30 days from it, to return a DateTime object representing 30 days ago
For each line in the logfile, the if block prints the line if the DateTime representing that line's timestamp is greater than the DateTime representing 30 days ago (i.e., the timestamp is more recent than 30 days), otherwise it ignores the line
Assuming a file with the contents:
2013-04-29 08:55:09,261 line1
2013-01-29 08:55:09,261 line2
2013-03-31 08:55:09,261 line3
You could get your desired output with:
$culture = [System.Globalization.CultureInfo]::InvariantCulture
$format = 'yyyy-MM-dd HH:mm:ss,fff'
Get-Content .\test.txt | ? { [datetime]::ParseExact(([string]$_).Substring(0,23), $format, $culture) -ge (Get-Date).AddDays(-30) }
Adi Inbar's answer is good, but if you'd prefer to avoid regular expressions, here's another way:
Get-Content e:\pathto\logfile.txt | Where-Object { ( (Get-Date) - (Get-Date $_.Substring(0,10)) ).Days -le 30} | Add-Content e:\pathto\NewLog.txt
Related
I have txt file with date value, line by line
I try to compare them to today date in powershell but its not working
$DateTimeNow = (Get-Date).ToString('dd/MM/yyyy')
$data2 = get-content "output.txt"
$z= #()
foreach($line2 in $data2)
{
if($line2 -match $DateTimeNow){
write-host "same date"
}
}
the compare with "match" not work, I have try -eq and = but nothing better.
Have you any idea what I am doing wrong ?
The input dates all use 2-digit notation for the year (20 for 2020), but your string representing today's date uses 4-digits. Change to the appropriate format and it will work:
$DateTimeNow = Get-Date -Format 'dd/MM/yy'
I would like to get date from the log file text.
Text in log file.
Error code. 200105. Simple text and so on -------------> it should get date as 2020 Jan 05
Error code. 2000207. Simple text and so on -------------> it should get date as 2020 Feb 07
I try this but it doesnt work.
Get-Date "200105" -format "y-m-d" but it doesnt work.
I also try "200105" | Date but still same issue
This does work [datetime]::ParseExact("120105", "y.m.d", $null) but how do I get just the date but ignore all of the other text
If you want a shorter version you can do that by piping the output as follows
$date = [datetime]::ParseExact($text2, "y.M.d", $null)
$date | Get-Date -Format dd-MMMM-yyyy
Or
$date.ToString("yyyy MMMM dd")
Your second example 2000207 is invalid because of the extra 0 in there.
I would use the TryParseExact method here to see if what you have got is actually a parsable datetime string.
$logLine = 'Error code. 200105. Simple text and so on'
if ($logLine -match '^Error code\s*\.?\s*(\d{6})') {
$date = Get-Date # any valid DateTime object will do
if ([datetime]::TryParseExact($Matches[1], 'yyMMdd', [cultureinfo]::InvariantCulture, 0, [ref]$date)) {
# do something with the date found. For demo, just output in the console
"Found a date: $date"
}
}
You are probably reading the log file line-by-line, something like:
Get-Content -Path 'TheLogFile' | ForEach-Object {
if ($_ -match '^Error code\s*\.?\s*(\d{6})') {
$date = Get-Date # any valid DateTime object will do
if ([datetime]::TryParseExact($Matches[1], 'yyMMdd', [cultureinfo]::InvariantCulture, 0, [ref]$date)) {
# do something with the date found. For demo, just output in the console
"Found a date: $date"
}
}
}
According to the documentation, Get-Date converts a string to a date if it recognises the date format from the locale settings.
For instance, in UK it recognises Get-Date "2020/03/21" but not Get-Date "20200321"
The format string is only used for formatting the current date.
This works: the number of characters in the format string represents the size of the input (it matches the number of digits in the day and year - it is more complicated for months) and M represents months (m represents minutes).
PS /home/alistair> [datetime]::ParseExact("200321","yyMMdd",$null)
Saturday, 21 March 2020 00:00:00
I have numerous .txt files that are output from a handle.exe run of several days. I need to reorganize the data to get it into a relational database. The first thing I need to do is get the dates re-formatted.
Each file has in excess of 800 dates, disbursed unevenly throughout the file. The dates are formatted:
June 29, 2016 12:05:45 PM and I need 06-29-16 12:05:45.
I'm just working on a single file for now, to get things dialed in. I've tried to replace the dates in situ (using an array for the original dates) with Get-Date and got nowhere. Then I tried -replace and that didn't work.
I've spent 3 or 4 days on this and I think I've broken my head. I've tried so many permutations of stuff that I don't know even where I am anymore.
The last thing I tried was below. An attempt to use a hashtable, with the old date and new date in the table.
##To set "|" as separator for arrays
$OFS = '|'
##To get original dates into array
$a = #(sls .\hp.txt -pattern '(june 29|june 30|july 1|july 2|july 3|july 4)' | select -ExpandProperty line)
##To get dates with corrected format into array
$b = #($a | foreach {$_ | Get-Date -Format "MM-dd-yy hh:mm:ss"})
##To get old and new dates into hash table
$dates = #{$a = $b}
##To bring in content from file
$file = (Get-Content C:\hp.txt)
##To replace "NAME" with "VALUE" from hash table into file
foreach ($d in $dates) {
$file = $file -replace $d.Name, $d.Value
}
##To save corrected file with new file name
Set-Content -Path C:\hpnew.txt -Value $file
The $a array contains (in small part):
June 29, 2016 12:04:51 PM
June 29, 2016 12:05:58 PM
June 29, 2016 12:07:00 PM
[NOTE: LOTS MORE DATES HERE]
June 30, 2016 12:01:17 AM
June 30, 2016 12:02:19 AM
June 30, 2016 12:04:22 AM
[NOTE:CONTINUING TO END]
The $b array contains:
06-29-16 12:04:51
06-29-16 12:05:58
06-29-16 12:07:00
[NOTE: LOTS MORE DATES ]
06-30-16 12:01:17
06-30-16 12:02:19
06-30-16 12:04:22
[NOTE: CONTINUING TO END]
There is probably a MUCH simpler, more elegant solution. But any help/direction would be great.
Use a regular expression to extract the date strings from your text, then pass the matches to a callback function where you parse them to actual DateTime values and format those according to your requirements:
$re = '((?:january|february|...|december) \d{1,2}, \d{4} \d{1,2}:\d{2}:\d{2} [ap]m)'
$input_fmt = 'MMMM d, yyyy h:mm:ss tt'
$output_fmt = 'MM-dd-yy HH:mm:ss'
$culture = [Globalization.CultureInfo]::InvariantCulture
$options = [Text.RegularExpressions.RegexOptions]::IgnoreCase
$callback = {
[DateTime]::ParseExact($args[0].Groups[1].Value, $input_fmt, $culture).ToString($output_fmt)
}
$txt = Get-Content '.\hp.txt' -Raw
[regex]::Replace($txt, $re, $callback, $options) | Set-Content '.\hpnew.txt'
I have a log file that I have to scan every 5 minutes for specific keywords - specifically "Order Failed" then capture the lines around that. I have that part programmed with no issues. The rest of the log entries don't matter. I'm able to use the command:
[string] $readFile = get-content c:\test\friday.log | select-String -pattern '(order.*failed)' -context 1,7
which outputs:
Message: Order submission failed.
Timestamp: 4/1/2016 4:05:09 PM
Severity: Error
Message: Modifier not authorized for parent item
Message: Order submission failed.
Timestamp: 4/1/2016 4:18:15 PM
Severity: Error
Message: Modifier not authorized for parent item
Which is exactly what I want. My problem is trying to read through the above output and store the "Timestamp" into a variable that I can manipulate.
The first challenge is the "Timestamp" time is written in UTC time and we are located in the Pacific Time Zone.
The second challenge is I need to compare the "Timestamp" time to the current time and store that as an integer. We only want to report errors that are within 5 minutes of the current time.
My current code only grabs the first "Timestamp" entry since I hard-coded it:
[string] $readFile = get-content c:\test\friday.log | select-String -pattern '(order.*failed)' -context 1,7
$fileArray = $readFile.Split(“`n”)
[dateTime] $TrimTime = $fileArray[3].trim("Timestamp: ")
[dateTime] $currentTime = get-date
[int32] $getMinutes = (new-timespan -start $TrimTime -end $currentTime).minutes
I don't know how to loop through the output of the Get-content - to check for all of the Timestamps - we only want to report errors that are within 5 minutes of the current time.
Don't cast your matches to a string right away.
$readFile = get-content c:\test\friday.log | select-String -pattern '(order.*failed)' -context 1,7
If you leave the MatchInfo objects intact for the time being you can extract the timestamps from the aftercontexts like this:
$readFile | ForEach-Object {
$_.Context.AfterContext | Where-Object {
$_ -match 'timestamp: (\d{1,2}/\d{1,2}/\d{4} \d{1,2}:\d{2}:\d{2} [ap]m)'
} | ForEach-Object {
$timestring = $matches[1]
}
}
Use the ParseExact() method to convert the matched substring to a DateTime value:
$fmt = 'M\/d\/yyyy h:mm:ss ttK'
$culture = [Globalization.CultureInfo]::InvariantCulture
$timestamp = [DateTime]::ParseExact("${timestring}Z", $fmt, $culture)
The Z that is appended to $timestring indicates the UTC timezone and the trailing K in the format string makes the method recognize it as such. The result is automatically converted to your local time. If you need the time in UTC: append .ToUniversalTime() to the method call.
I want yesterday's date as "MMdd", like "0418" today's date is "0419".
I tried this:
$d = (Get-Date).AddDays(-1) | select -Property Month,Day
$e = $d.Day
$d = $d.Month
$total = "$d" + "$e"
Write-Host $total
But the output was "418".
As you can see, this is a three-letter string, but I want a four-letter one. Because I want to search files they have that format and created a day before.
The date object has a method ToString, which allows you to specify the format:
$d = (Get-Date).AddDays(-1);
$d.ToString("MMdd");
I am calling this on April the 20th, and the output is
0419
See this link for a description of the available date format strings.