I have a (plain text) log file with dates that appear in most (but not all) lines of text. The date does not always appear in the same position on each line. Though not all lines have a date, there is always a date in any given set of 10 lines. A sample of the log:
03/05/2019 Event A occurred
03/05/2019 Event B occurred
Event B Details: Details Details
03/07/2019 Event C occurred
Logging completed on 03/08/2019
03/08/2019 Event D occurred
I need to get the date of last item logged. I don't need the content of the line, just the date that's in the line.
#Get the log file. Tail it because it's huge. The regex pattern matches date format dd/mm/yyyy.
$log = (Get-Content -tail 10 C:\mylog.log | Select-String -Pattern "\d{2}/\d{2}\/d{4}"
#Get the last item in the $log variable. Convert it to a string.
$string = $log[-1].toString()
#Split the string. Match each element to a date format.
#When an element matches a date format, assign its value to a DateTime variable.
foreach ($s in $string.split() ){
if ($s -match "\d{2}/\d{2}\/d{4}"){
[DateTime]$date = $s
}
}
"The last entry in the log was made on $date"
Both parts of this code (finding the last line with a date, and pulling the date from the line) seem really kludgy. Is there a better way to do this?
You may do the following:
[datetime]((Get-Content mylog.log -tail 10) |
Select-String '\d{2}/\d{2}/\d{4}')[-1].Matches.Value
Select-String returns a collection MatchInfo objects. .Matches.Value of each object contains the matched string. Using the [-1] index, we can grab the last object.
Note: Select-String has a -Path parameter that can read a file. But given you don't want to read the entire file, Get-Content -Tail is used.
Related
I have seen this post:
Add-Content - append to specific line
But I cannot add these lines because "Array index is out of range".
What my script is doing:
Find the line
Loop through the array that contains the data i want to add
$file[$line]+=$data
$line+=1
Write to file
Should I create a new file content and then add each line of the original file to it?
IF so, do you know how to do that and how to stop and add my data in between?
Here is the part of my code where I try to add:
$f=Get-Content $path
$ct=$begin+1 #$begin is the line where I want to place content under
foreach($add in $add_to_yaml)
{
$f[$ct]+=$add
$ct+=1
}
$f | Out-File -FilePath $file
Let's break down your script and try to analyze what's going on:
$f = Get-Content $path
Get-Content, by default, reads text files and spits out 1 string per individual line in the file. If the file found at $path has 10 lines, the resulting value stored in $f will be an array of 10 string values.
Worth noting is that array indices in PowerShell (and .NET in general) are zero-based - to get the 10th line from the file, we'd reference index 9 in the array ($f[9]).
That means that if you want to concatenate stuff to the end of (or "under") line 10, you need to specify index 9. For this reason, you'll want to change the following line:
$ct = $begin + 1 #$begin is the line where i want to place content under
to
$ct = $begin
Now that we have the correct starting offset, let's look at the loop:
foreach($add in $add_to_yaml)
{
$f[$ct] += $add
$ct += 1
}
Assuming $add_to_yaml contains multiple strings, the loop body will execute more than once. Let's take a look at the first statement:
$f[$ct] += $add
We know that $f[$ct] resolves to a string - and strings have the += operator overloaded to mean "string concatenation". That means that the string value stored in $f[$ct] will be modified (eg. the string will become longer), but the array $f itself does not change its size - it still contains the same number of strings, just one of them is a little longer.
Which brings us to the crux of your issue, this line right here:
$ct += 1
By incrementing the index counter, you effectively "skip" to the next string for every value in $add_to_yaml - so if the number of elements you want to add exceeds the number of lines after $begin, you naturally reach a point "beyond the bounds" of the array before you're finished.
Instead of incrementing $ct, make sure you concatenate your new string values with a newline sequence:
$f[$ct] = $f[$ct],$add -join [Environment]::Newline
Putting it all back together, you end up with something like this (notice we can discard $ct completely, since its value is constant an equal to $begin anyway):
$f = Get-Content $path
foreach($add in $add_to_yaml)
{
$f[$begin] = $f[$begin],$add -join [Environment]::Newline
}
But wait a minute - all the strings in $add_to_yaml are simply going to be joined by newlines - we can do that in a single -join operation and get rid of the loop too!
$f = Get-Content $path
$f[$begin] = #($f[$begin];$add_to_yaml) -join [Environment]::Newline
$f | Out-File -FilePath $file
Much simpler :)
I have a specific date "2021/11/28", i want the list of files from example filenames(below) whose file name is greater than 2021/11/28. remember not the creation time of the file names.
"test_20211122_aba.*"
"abc_20211129_efg.*"
"hij_20211112_lmn.*"
"opq_20211130_rst.*"
I'm expecting to get
"abc_20211129_efg.*"
"opq_20211130_rst.*"
Really appreciate your help.
You don't strictly need to parse your strings into dates ([datetime] instances): Because the date strings embedded in your file names are in a format where their lexical sorting is equivalent to chronological sorting, you can compare the string representations directly:
# Simulate output from a Get-ChildItem call.
$files = [System.IO.FileInfo[]] (
"test_20211122_aba1.txt",
"abc_20211129_efg2.txt",
"hij_20211112_lmn3.txt",
"hij_20211112_lmn4.txt",
"opq_20211130_rst5.txt"
)
# Filter the array of files.
$resultFiles =
$files | Where-Object {
$_.Name -match '(?:^|.*\D)(\d{8})(?:\D.*|$)' -and
$Matches[1] -gt ('2021/11/28"' -replace '/')
}
# Print the names of the filtered files.
$resultFiles.Name
$_.Name -match '(?:^|.*\D)(\d{8})(?:\D.*|$)' looks for (the last) run of exactly 8 digits in each file name via a capture group ((...)), reflected in the automatic $Matches variable's entry with index 1 ($Matches[1]) afterwards, if found.
'2021/11/28"' -replace '/' removes all / characters from the input string, to make the format of the date strings the same. For brevity, the solution above performs this replacement in each loop operation. In practice, you would perform it once, before the loop, and assign the result to a variable for use inside the loop.
I have the line
Select-String -Path ".\*.txt" -Pattern "6,16" -Context 20 | Select-Object -First 1
that would return 20 lines of context looking for a pattern of "6,16".
I need to look for the next line containing the string "ID number:" after the line of "6,16", read what is the text right next to "ID number:", find if this exact text exists in another "export.txt" file located in the same folder (so in ".\export.txt"), and see if it contains "6,16" on the same line as the one containing the text in question.
I know it may seem confusing, but what I mean is for example:
example.txt:5218: ID number:0002743284
shows whether this is true:
export.txt:9783: 0002743284 *some text on the same line for example* 6,16
If I understand the question correctly, you're looking for something like:
Select-String -List -Path *.txt -Pattern '\b6,16\b' -Context 0, 20 |
ForEach-Object {
if ($_.Context.PostContext -join "`n" -match '\bID number:(\d+)') {
Select-String -List -LiteralPath export.txt -Pattern "$($Matches[1]).+$($_.Pattern)"
}
}
Select-String's -List switch limits the matching to one match per input file; -Context 0,20 also includes the 20 lines following the matching one in the output (but none (0) before).
Note that I've placed \b, a word-boundary assertion at either end of the search pattern, 6,16, to rule out accidental false positives such as 96,169.
$_.Context.PostContext contains the array of lines following the matching line (which itself is stored in $_.Line):
-join "`n" joins them into a multi-line string, so as to ensure that the subsequent -match operation reports the captured results in the automatic $Matches variable, notably reporting the ID number of interest in $Matches[1], the text captured by the first (and only) capture group ((\d+)).
The captured ID is then used in combination with the original search pattern to form a regex that looks for both on the same line, and is passed to a second Select-String call that searches through export.txt
Note: An object representing the matching line, if any, is output by default; to return just $true or $false, replace -List with -Quiet.
There's a lot wrong with what you're expecting and the code you've tried so let's break it down and get to the solution. Kudos for attempting this on your own. First, here's the solution, read below this code for an explanation of what you were doing wrong and how to arrive at the code I've written:
# Get matching lines plus the following line from the example.txt seed file
$seedMatches = Select-String -Path .\example.txt -Pattern "6,\s*16" -Context 0, 2
# Obtain the ID number from the line following each match
$idNumbers = foreach( $match in $seedMatches ) {
$postMatchFields = $match.Context.PostContext -split ":\s*"
# Note: .IndexOf(object) is case-sensitive when looking for strings
# Returns -1 if not found
$idFieldIndex = $postMatchFields.IndexOf("ID number")
# Return the "ID number" to `$idNumbers` if "ID number" is found in $postMatchFields
if( $idFieldIndex -gt -1 ) {
$postMatchFields[$idFieldIndex + 1]
}
}
# Match lines in export.txt where both the $id and "6,16" appear
$exportMatches = foreach( $id in $idNumbers ) {
Select-String -Path .\export.txt -Pattern "^(?=.*\b$id\b)(?=.*\b6,\s*16\b).*$"
}
mklement0's answer essentially condenses this into less code, but I wanted to break this down fully.
First, Select-String -Path ".\*.txt" will look in all .txt files in the current directory. You'll want to narrow that down to a specific naming pattern you're looking for in the seed file (the file we want to find the ID to look for in the other files). For this example, I'll use example.txt and export.txt for the paths which you've used elsewhere in your question, without using globbing to match on filenames.
Next, -Context gives context of the surrounding lines from the match. You only care about the next line match so 0, 1 should suffice for -Context (0 lines before, 1 line after the match).
Finally, I've added \s* to the -Pattern to match on whitespace, should the 16 ever be padded from the ,. So now we have our Select-String command ready to go:
$seedMatches = Select-String -Path .\example.txt -Pattern "6,\s*16" -Context 0, 2
Next, we will need to loop over the matching results from the seed file. You can use foreach or ForEach-Object, but I'll use foreach in the example below.
For each $match in $seedMatches we'll need to get the $idNumbers from the lines following each match. When $match is ToString()'d, it will spit out the matched line and any surrounding context lines. Since we only have one line following the match for our context, we can grab $match.Context.PostContext for this.
Now we can get the $idNumber. We can split example.txt:5218: ID number:0002743284 into an array of strings by using the -split operator to split the string on the :\s* pattern (\s* matches on any or no whitespace). Once we have this, we can get the index of "ID Number" and get the value of the field immediately following it. Now we have our $idNumbers. I'll also add some protection below to ensure the ID numbers field is actually found before continuing.
$idNumbers = foreach( $match in $seedMatches ) {
$postMatchFields = $match.Context.PostContext -split ":\s*"
# Note: .IndexOf(object) is case-sensitive when looking for strings
# Returns -1 if not found
$idFieldIndex = $postMatchFields.IndexOf("ID number")
# Return the "ID number" to `$idNumbers` if "ID number" is found in $postMatchFields
if( $idFieldIndex -gt -1 ) {
$postMatchFields[$idFieldIndex + 1]
}
}
Now that we have $idNumbers, we can look in export.txt for this ID number "6,\s*16" on the same line, once again using Select-String. This time, I'll put the code first since it's nothing new, then explain the regex a bit:
$exportMatches = foreach( $id in $idNumbers ) {
Select-String -Path .\export.txt -Pattern "^(?=.*\b$id\b)(?=.*\b6,\s*16\b).*$"
}
$exportMatches will now contain the lines which contain both the target ID number and the 6,16 value on the same line. Note that order wasn't specified so the expression uses positive lookaheads to find both the $id and 6,16 values regardless of their order in the string. I won't break down the exact expression but if you plug ^(?=.*\b0123456789\b)(?=.*\b6,\s*16\b).*$ into https://regexr.com it will break down and explain the regex pattern in detail.
The full code is above in at the top of this answer.
Just beginning with Powershell. I have a text file that contains the string "CloseYear/2019" and looking for a way to increment the "2019" to "2020". Any advice would be appreciated. Thank you.
If the question is how to update text within a file, you can do the following, which will replace specified text with more specified text. The file (t.txt) is read with Get-Content, the targeted text is updated with the String class Replace method, and the file is rewritten using Set-Content.
(Get-Content t.txt).Replace('CloseYear/2019','CloseYear/2020') | Set-Content t.txt
Additional Considerations:
General incrementing would require a object type that supports incrementing. You can isolate the numeric data using -split, increment it, and create a new, joined string. This solution assumes working with 32-bit integers but can be updated to other numeric types.
$str = 'CloseYear/2019'
-join ($str -split "(\d+)" | Foreach-Object {
if ($_ -as [int]) {
[int]$_ + 1
}
else {
$_
}
})
Putting it all together, the following would result in incrementing all complete numbers (123 as opposed to 1 and 2 and 3 individually) in a text file. Again, this can be tailored to target more specific numbers.
$contents = Get-Content t.txt -Raw # Raw to prevent an array output
-join ($contents -split "(\d+)" | Foreach-Object {
if ($_ -as [int]) {
[int]$_ + 1
}
else {
$_
}
}) | Set-Content t.txt
Explanation:
-split uses regex matching to split on the matched result resulting in an array. By default, -split removes the matched text. Creating a capture group using (), ensures the matched text displays as is and is not removed. \d+ is a regex mechanism matching a digit (\d) one or more (+) successive times.
Using the -as operator, we can test that each item in the split array can be cast to [int]. If successful, the if statement will evaluate to true, the text will be cast to [int], and the integer will be incremented by 1. If the -as operator is not successful, the pipeline object will remain as a string and just be output.
The -join operator just joins the resulting array (from the Foreach-Object) into a single string.
AdminOfThings' answer is very detailed and the correct answer.
I wanted to provide another answer for options.
Depending on what your end goal is, you might need to convert the date to a datetime object for future use.
Example:
$yearString = 'CloseYear/2019'
#convert to datetime
[datetime]$dateConvert = [datetime]::new((($yearString -split "/")[-1]),1,1)
#add year
$yearAdded = $dateConvert.AddYears(1)
#if you want to display "CloseYear" with the new date and write-host
$out = "CloseYear/{0}" -f $yearAdded.Year
Write-Host $out
This approach would allow you to use $dateConvert and $yearAdded as a datetime allowing you to accurately manipulate dates and cultures, for example.
I have a log file that I have to scan every 5 minutes for specific keywords - specifically "Order Failed" then capture the lines around that. I have that part programmed with no issues. The rest of the log entries don't matter. I'm able to use the command:
[string] $readFile = get-content c:\test\friday.log | select-String -pattern '(order.*failed)' -context 1,7
which outputs:
Message: Order submission failed.
Timestamp: 4/1/2016 4:05:09 PM
Severity: Error
Message: Modifier not authorized for parent item
Message: Order submission failed.
Timestamp: 4/1/2016 4:18:15 PM
Severity: Error
Message: Modifier not authorized for parent item
Which is exactly what I want. My problem is trying to read through the above output and store the "Timestamp" into a variable that I can manipulate.
The first challenge is the "Timestamp" time is written in UTC time and we are located in the Pacific Time Zone.
The second challenge is I need to compare the "Timestamp" time to the current time and store that as an integer. We only want to report errors that are within 5 minutes of the current time.
My current code only grabs the first "Timestamp" entry since I hard-coded it:
[string] $readFile = get-content c:\test\friday.log | select-String -pattern '(order.*failed)' -context 1,7
$fileArray = $readFile.Split(“`n”)
[dateTime] $TrimTime = $fileArray[3].trim("Timestamp: ")
[dateTime] $currentTime = get-date
[int32] $getMinutes = (new-timespan -start $TrimTime -end $currentTime).minutes
I don't know how to loop through the output of the Get-content - to check for all of the Timestamps - we only want to report errors that are within 5 minutes of the current time.
Don't cast your matches to a string right away.
$readFile = get-content c:\test\friday.log | select-String -pattern '(order.*failed)' -context 1,7
If you leave the MatchInfo objects intact for the time being you can extract the timestamps from the aftercontexts like this:
$readFile | ForEach-Object {
$_.Context.AfterContext | Where-Object {
$_ -match 'timestamp: (\d{1,2}/\d{1,2}/\d{4} \d{1,2}:\d{2}:\d{2} [ap]m)'
} | ForEach-Object {
$timestring = $matches[1]
}
}
Use the ParseExact() method to convert the matched substring to a DateTime value:
$fmt = 'M\/d\/yyyy h:mm:ss ttK'
$culture = [Globalization.CultureInfo]::InvariantCulture
$timestamp = [DateTime]::ParseExact("${timestring}Z", $fmt, $culture)
The Z that is appended to $timestring indicates the UTC timezone and the trailing K in the format string makes the method recognize it as such. The result is automatically converted to your local time. If you need the time in UTC: append .ToUniversalTime() to the method call.