This self-answered question addresses the scenario originally described in Increment version number in file:
A version number embedded in a text file is to be incremented.
Sample text-file content:
nuspec{
id = XXX;
version: 0.0.30;
title: XXX;
For instance, I want embedded version number 0.0.30 updated to 0.0.31.
The line of interest can be assumed to match the following regex: ^\s+version: (.+);$
Note hat the intent is not to replace the version number with a fixed new version, but to increment the existing version.
Ideally, the increment logic would handle version strings representing either [version] (System.Version) or [semver] (System.Management.Automation.SemanticVersion) instances, ranging from 2 - 4 components; e.g.:
1.0
1.0.2
1.0.2.3 - [version] format (up to 4 numeric components)
1.0.2-preview2 - [semver] format (up to 3 numeric components), optionally with a --separated preview label
1.0.2-preview2+001 - ditto, additionally with a +-separated build label
In PowerShell [Core] (v6.1+), a concise solution is possible:
$file = 'somefile.txt'
(Get-Content -Raw $file) -replace '(?m)(?<=^\s+version: ).+(?=;$)', {
# Increment the *last numeric* component of the version number.
# See below for how to target other components.
$_.Value -replace '(?<=\.)\d+(?=$|-)', { 1 + $_.Value }
} | Set-Content $file
Note:
* In PowerShell [Core] 6+, BOM-less UTF-8 is the default encoding; use -Encoding with Set-Content if you need a different encoding.
* By using -Raw, the command reads the entire file into memory first, which enables writing back to that same file in the same pipeline; however, there is a slight risk of data loss if writing back to the input file gets interrupted.
* -replace invariably replaces all substrings that match the regex.
* Inline regex option (?m) ensures that ^ and $ match the start and end of individual lines, which is necessary due to Get-Content -Raw reading the entire file as a single, multi-line string.
Note:
For simplicity, text-based manipulation of the version string is performed, but you could also cast $_.Value to [version] or [semver] (PowerShell [Core] v6+ only) and work with that.
The advantage of the text-based manipulation is the concise ability to retain all other components of the input version string as-is, without adding previously unspecified ones.
The above relies on the -replace operator's ability to perform regex-based string substitutions fully dynamically, via a script block ({ ... }) - as explained in this answer.
The regexes use look-around assertions ((?<=...) and (?=...)) so as to ensure that only the part of the input to be modified is matched.
Only the (?<=^\s+version: ) and (?=;$) look-arounds are specific to the sample file format; adjust these parts as needed to match the version number in your file format.
The above increment's the input version's last numeric component.
To target the various version-number components, use the following inner regex instead:
Increment the major number (e.g., 2.0.9 -> 3.0.9):
'2.0.9' -replace '\d+(?=\..+)', { 1 + [int] $_.Value }
The minor number:
'2.0.9' -replace '(?<=^\d+\.)\d+(?=.*)', { 1 + [int] $_.Value }
The patch / build number (3rd component; 2.0.9 -> 2.0.10):
'2.0.9' -replace '(?<=^\d+\.\d+\.)\d+(?=.*)', { 1 + [int] $_.Value }
The last / revision number, as above, whatever it is, even if followed by a pre-release label (e.g.,; 2.0.9.10 -> 2.0.9.11 or 7.0.0-preview2 -> 7.0.1-preview2):
'2.0.9.10' -replace '(?<=\.)\d+(?=$|-)', { 1 + [int] $_.Value }
Note: If the targeted component doesn't exist, the original version is returned as-is.
In Windows PowerShell, where -replace doesn't support script-block-based substitutions, you can use the switch statement with the -File and -Regex options instead:
$file = 'someFile.txt'
$updatedFileContent =
switch -regex -file $file { # Loop over all lines in the file.
'^\s+version: (.+);$' { # line with version number
# Extract the old version number...
$oldVersion = $Matches[1]
# ... and update it, by incrementing the last component in this
# example.
$components = $oldVersion -split '\.'
$components[-1] = 1 + $components[-1]
$newVersion = $components -join '.'
# Replace the old version with the new version in the line
# and output the modified line.
$_.Replace($oldVersion, $newVersion)
}
default { # All other lines.
# Pass them through.
$_
}
}
# Save back to file. Use -Encoding as needed.
$updatedFileContent | Set-Content $file
Related
I have a text file with a large number of log messages.
I want to extract the messages between two string patterns. I want the extracted message to appear as it is in the text file.
I tried the following methods. It works, but doesn't support Get-Content's -Wait and -Tail options. Also, the extracted results are displayed in one line, but not like the text file. Inputs are welcome :-)
Sample Code
function GetTextBetweenTwoStrings($startPattern, $endPattern, $filePath){
# Get content from the input file
$fileContent = Get-Content $filePath
# Regular expression (Regex) of the given start and end patterns
$pattern = "$startPattern(.*?)$endPattern"
# Perform the Regex opperation
$result = [regex]::Match($fileContent,$pattern).Value
# Finally return the result to the caller
return $result
}
# Clear the screen
Clear-Host
$input = "THE-LOG-FILE.log"
$startPattern = 'START-OF-PATTERN'
$endPattern = 'END-OF-PATTERN'
# Call the function
GetTextBetweenTwoStrings -startPattern $startPattern -endPattern $endPattern -filePath $input
Improved script based on Theo's answer.
The following points need to be improved:
The beginning and end of the output is somehow trimmed despite I adjusted the buffer size in the script.
How to wrap each matched result into START and END string?
Still I could not figure out how to use the -Wait and -Tail options
Updated Script
# Clear the screen
Clear-Host
# Adjust the buffer size of the window
$bw = 10000
$bh = 300000
if ($host.name -eq 'ConsoleHost') # or -notmatch 'ISE'
{
[console]::bufferwidth = $bw
[console]::bufferheight = $bh
}
else
{
$pshost = get-host
$pswindow = $pshost.ui.rawui
$newsize = $pswindow.buffersize
$newsize.height = $bh
$newsize.width = $bw
$pswindow.buffersize = $newsize
}
function Get-TextBetweenTwoStrings ([string]$startPattern, [string]$endPattern, [string]$filePath){
# Get content from the input file
$fileContent = Get-Content -Path $filePath -Raw
# Regular expression (Regex) of the given start and end patterns
$pattern = '(?is){0}(.*?){1}' -f [regex]::Escape($startPattern), [regex]::Escape($endPattern)
# Perform the Regex operation and output
[regex]::Match($fileContent,$pattern).Groups[1].Value
}
# Input file path
$inputFile = "THE-LOG-FILE.log"
# The patterns
$startPattern = 'START-OF-PATTERN'
$endPattern = 'END-OF-PATTERN'
Get-TextBetweenTwoStrings -startPattern $startPattern -endPattern $endPattern -filePath $inputFile
You need to perform streaming processing of your Get-Content call, in a pipeline, such as with ForEach-Object, if you want to process lines as they're being read.
This is a must if you're using Get-Content -Wait, because such a call doesn't terminate by itself (it keeps waiting for new lines to be added to the file, indefinitely), but inside a pipeline its output can be processed as it is being received, even before the command terminates.
You're trying to match across multiple lines, which with Get-Content output would only work if you used the -Raw switch - by default, Get-Content reads its input file(s) line by line.
However, -Raw is incompatible with -Wait.
Therefore, you must stick with line-by-line processing, which requires that you match the start and end patterns separately, and keep track of when you're processing lines between those two patterns.
Here's a proof of concept, but note the following:
-Tail 100 is hard-coded - adjust as needed or make it another parameter.
The use of -Wait means that the function will run indefinitely - waiting for new lines to be added to $filePath - so you'll need to use Ctrl-C to stop it.
While you can use a Get-TextBetweenTwoStrings call itself in a pipeline for object-by-object processing, assigning its result to a variable ($result = ...) won't work when terminating with Ctrl-C, because this method of termination also aborts the assignment operation.
To work around this limitation, the function below is defined as an advanced function, which automatically enables support for the common -OutVariable parameter, which is populated even in the event of termination with Ctrl-C; your sample call would then look as follows (as Theo notes, don't use the automatic $input variable as a custom variable):
# Look for blocks of interest in the input file, indefinitely,
# and output them as they're being found.
# After termination with Ctrl-C, $result will also contain the blocks
# found, if any.
Get-TextBetweenTwoStrings -OutVariable result -startPattern $startPattern -endPattern $endPattern -filePath $inputFile
Per your feedback you want the block of lines to encompass the full lines on which the start and end patterns match, so the regexes below are enclosed in .*
The word pattern in your $startPattern and $endPattern parameters is a bit ambiguous in that it suggests that they themselves are regexes that can therefore be used as-is or embedded as-is in a larger regex on the RHS of the -match operator.
However, in the solution below I am assuming that they are be treated as literal strings, which is why they are escaped with [regex]::Escape(); simply omit these calls if these parameters are indeed regexes themselves; i.e.:
$startRegex = '.*' + $startPattern + '.*'
$endRegex = '.*' + $endPattern + '.*'
The solution assumes there is no overlap between blocks and that, in a given block, the start and end patterns are on separate lines.
Each block found is output as a single, multi-line string, using LF ("`n") as the newline character; if you want a CRLF newline sequences instead, use "`r`n"; for the platform-native newline format (CRLF on Windows, LF on Unix-like platforms), use [Environment]::NewLine.
# Note the use of "-" after "Get", to adhere to PowerShell's
# "<Verb>-<Noun>" naming convention.
function Get-TextBetweenTwoStrings {
# Make the function an advanced one, so that it supports the
# -OutVariable common parameter.
[CmdletBinding()]
param(
$startPattern,
$endPattern,
$filePath
)
# Note: If $startPattern and $endPattern are themselves
# regexes, omit the [regex]::Escape() calls.
$startRegex = '.*' + [regex]::Escape($startPattern) + '.*'
$endRegex = '.*' + [regex]::Escape($endPattern) + '.*'
$inBlock = $false
$block = [System.Collections.Generic.List[string]]::new()
Get-Content -Tail 100 -Wait $filePath | ForEach-Object {
if ($inBlock) {
if ($_ -match $endRegex) {
$block.Add($Matches[0])
# Output the block of lines as a single, multi-line string
$block -join "`n"
$inBlock = $false; $block.Clear()
}
else {
$block.Add($_)
}
}
elseif ($_ -match $startRegex) {
$inBlock = $true
$block.Add($Matches[0])
}
}
}
First of all, you should not use $input as self-defined variable name, because this is an Automatic variable.
Then, you are reading the file as a string array, where you would rather read is as a single, multiline string. For that append switch -Raw to the Get-Content call.
The regex you are creating does not allow fgor regex special characters in the start- and end patterns you give, so it I would suggest using [regex]::Escape() on these patterns when creating the regex string.
While your regex does use a group capturing sequence inside the brackets, you are not using that when it comes to getting the value you seek.
Finally, I would recommend using PowerShell naming convention (Verb-Noun) for the function name
Try
function Get-TextBetweenTwoStrings ([string]$startPattern, [string]$endPattern, [string]$filePath){
# Get content from the input file
$fileContent = Get-Content -Path $filePath -Raw
# Regular expression (Regex) of the given start and end patterns
$pattern = '(?is){0}(.*?){1}' -f [regex]::Escape($startPattern), [regex]::Escape($endPattern)
# Perform the Regex operation and output
[regex]::Match($fileContent,$pattern).Groups[1].Value
}
$inputFile = "D:\Test\THE-LOG-FILE.log"
$startPattern = 'START-OF-PATTERN'
$endPattern = 'END-OF-PATTERN'
Get-TextBetweenTwoStrings -startPattern $startPattern -endPattern $endPattern -filePath $inputFile
Would result in something like:
blahblah
more lines here
The (?is) makes the regex case-insensitive and have the dot match linebreaks as well
Nice to see you're using my version of the Get-TextBetweenTwoStrings function, however I believe you are mistaking the output in the console to output as in a dedicated text editor. In the console, too long lines will be truncated, whereas in a text editor like notepad, you can choose to wrap long lines or have a horizontal scrollbar.
If you simply append
| Set-Content -Path 'X:\wherever\theoutput.txt'
to the Get-TextBetweenTwoStrings .. call, you will find the lines are NOT truncated when you open it in Word or notepad for instance.
In fact, you can have that line folowed by
notepad 'X:\wherever\theoutput.txt'
to have notepad open that file straight away.
I have a file prototype as follows:
// <some stuff>
#define KEYWORD release01-11
// <more stuff>
How can I delete the last two characters in the same line as KEYWORD and replace them with two different characters (12 in this case), in order to end up with:
// <some stuff>
#define KEYWORD release01-12
// <more stuff>
I'm trying to use Clear-Content and Add-Content but I cannot get it to do what I need. The rest of the file needs to remain unchanged after these symbols have been replaced. Is there a better alternative?
Use the -replace regex operator to identify the relevant statements and replace/remove the trailing numbers:
# read file into a variable
$code = Get-Content myfile.c
# replace the trailing -XX with 12 in all lines starting with `#define KEYWORD`, with
$code = $code -replace '(?<=#define KEYWORD .+-)\d{2}\s*$','12'
# write the contents back to the file
$code |Set-Content myfile.c
The regex construct (?<=...) is a positive lookbehind - it ensures that the following expression will only match at a position where text right behind it is #define KEYWORD followed by some characters and a -.
If you want to always increment the current value (as opposed to just replacing it with 12), we'll need some way to inspect and evaluate the current value before doing the substitution.
The [Regex]::Replace() method allows for just that:
# read file into a variable
$code = Get-Content myfile.c
$code = $code |ForEach-Object {
# Same as before, but now we can hook into the regex engine's substitution routine
[regex]::Replace($_, '(?<=#define KEYWORD .+-)\d{2}\s*$',{
param($m)
# extract the trailing numbers, convert to a numerical type
$value = $m.Value -as [int]
# increment the value
$value++
# return the new value
return $value
})
}
# write the contents back to the file
$code |Set-Content myfile.c
In PowerShell 6.1 and up, the -replace operator natively supports scriptblock substitutions:
$code = $code |ForEach-Object {
# Same as before, but now we can hook into the regex engine's substitution routine
$_ -replace '(?<=#define KEYWORD .+-)\d{2}\s*$',{
# extract the trailing numbers, convert to a numerical type
$value = $_.Value -as [int]
# increment the value
$value++
# return the new value
return $value
}
}
Just beginning with Powershell. I have a text file that contains the string "CloseYear/2019" and looking for a way to increment the "2019" to "2020". Any advice would be appreciated. Thank you.
If the question is how to update text within a file, you can do the following, which will replace specified text with more specified text. The file (t.txt) is read with Get-Content, the targeted text is updated with the String class Replace method, and the file is rewritten using Set-Content.
(Get-Content t.txt).Replace('CloseYear/2019','CloseYear/2020') | Set-Content t.txt
Additional Considerations:
General incrementing would require a object type that supports incrementing. You can isolate the numeric data using -split, increment it, and create a new, joined string. This solution assumes working with 32-bit integers but can be updated to other numeric types.
$str = 'CloseYear/2019'
-join ($str -split "(\d+)" | Foreach-Object {
if ($_ -as [int]) {
[int]$_ + 1
}
else {
$_
}
})
Putting it all together, the following would result in incrementing all complete numbers (123 as opposed to 1 and 2 and 3 individually) in a text file. Again, this can be tailored to target more specific numbers.
$contents = Get-Content t.txt -Raw # Raw to prevent an array output
-join ($contents -split "(\d+)" | Foreach-Object {
if ($_ -as [int]) {
[int]$_ + 1
}
else {
$_
}
}) | Set-Content t.txt
Explanation:
-split uses regex matching to split on the matched result resulting in an array. By default, -split removes the matched text. Creating a capture group using (), ensures the matched text displays as is and is not removed. \d+ is a regex mechanism matching a digit (\d) one or more (+) successive times.
Using the -as operator, we can test that each item in the split array can be cast to [int]. If successful, the if statement will evaluate to true, the text will be cast to [int], and the integer will be incremented by 1. If the -as operator is not successful, the pipeline object will remain as a string and just be output.
The -join operator just joins the resulting array (from the Foreach-Object) into a single string.
AdminOfThings' answer is very detailed and the correct answer.
I wanted to provide another answer for options.
Depending on what your end goal is, you might need to convert the date to a datetime object for future use.
Example:
$yearString = 'CloseYear/2019'
#convert to datetime
[datetime]$dateConvert = [datetime]::new((($yearString -split "/")[-1]),1,1)
#add year
$yearAdded = $dateConvert.AddYears(1)
#if you want to display "CloseYear" with the new date and write-host
$out = "CloseYear/{0}" -f $yearAdded.Year
Write-Host $out
This approach would allow you to use $dateConvert and $yearAdded as a datetime allowing you to accurately manipulate dates and cultures, for example.
I'm renaming files with a regex replacement:
ls *dat | rename-item -newname {$_.name -replace '(.*)_([0-9]+)(.dat)','$1($2)$3'}
but I want also to add a constant int to [string]$2; something like :
'$1(#{int.parse($2)+3})($3)'
How should I proceed ?
PowerShell Core v6.1.0+ supports passing a script block as the replacement operand of the
-replace operator; the script block is invoked for each match found, and its output forms the replacement string, thereby enabling algorithmic replacements (rather than merely textual ones):
'foo_10.dat' -replace '(.*)_([0-9]+)(\.dat)', {
'{0}{1}{2}' -f $_.Groups[1].Value,
([int] $_.Groups[2].Value + 3),
$_.Groups[3].Value
}
The above yields:
foo13.dat
Note how 3 was added to 10 (and the underscore was removed):
$_ inside the script block is a [System.Text.RegularExpressions.Match] instance representing the results of the match; e.g., $_.Value represents the full match, whereas $_.Groups[1].Value represents what the 1st capture group matched.
The functionality is built on the [regex] .NET type's .Replace() method, which can also used directly (but less easily) in earlier PowerShell versions - see below.
In Windows PowerShell you have two options:
Use the -match operator first and then perform the transformation in a separate statement based on the match information reflected in the automatic $Matches variable, as suggested by kuujinbo:
$null = 'foo_10.dat' -match '(.*)_([0-9]+)(\.dat)' # match first
'{0}{1}{2}' -f $Matches.1, ([int] $Matches.2 + 3), $Matches.3 # build via $Matches
Use the .NET framework directly, with a script block passed as a delegate (callback function) to the above-mentioned [regex]::Replace() method, as suggested by Ansgar Wiechers:
([regex] '(.*)_([0-9]+)(\.dat)').Replace('foo_10.dat', {
param($match)
'{0}{1}{2}' -f $match.Groups[1].Value, ([int] $match.Groups[2].Value + 3), $match.Groups[3].Value
})
Note how a formal parameter - param($match) - must be declared here in order to access the match results, whereas the pure PowerShell solution at the top was able to use $_, as usual.
values.ini looks like
[default]
A=1
B=2
C=3
foo.txt looks like
Now is the %A% for %a% %B% men to come to the %C% of their %c%
I want to use Powershell to search for all of the %x% values in values.ini and then replace every matching instance in foo.txt with the corresponding value, case insensitively; generating the following:
Now is the 1 for 1 2 men to come to the 3 of their 3
Assuming PowerShell version 3.0 or newer, you can use the ConvertFrom-StringData cmdlet to parse the key-value pair in your ini file, but you'll need to filter out the [default] directive:
# grab relevant lines from file
$KeyValPairs = Get-Content .\values.ini | Where {$_ -like "*=*" }
# join strings together as one big string
$KeyValPairString = $KeyValPairs -join [Environment]::NewLine
# create hashtable/dictionary from string with ConvertFrom-StringData
$Dictionary = $KeyValPairString |ConvertFrom-StringData
You can then use the [regex]::Replace() method to do a lookup against the dictionary for each match you want to replace:
Get-Content .\foo.txt |ForEach-Object {
[Regex]::Replace($_, '%(\p{L}+)%', {
param($Match)
# look term up in dictionary
return $Dictionary[$Match.Groups[1].Value]
})
}
To complement Mathias R. Jessen's excellent answer with alternative approaches that also take the later requirement change of limiting values to a specific INI-file section into account (PSv2+, except for Get-Content -Raw; in PSv2, use (Get-Content ...) -join "`n" instead.)
Using PsIni\Get-IniContent and [environment]::ExpandEnvironmentVariables():
# Translate key-value pairs from section the section of interest
# into environment variables.
# After this command, the following environment variables are defined:
# $env:A, with value 1 (cmd.exe equivalent: %A%)
# $env:B, with value 2 (cmd.exe equivalent: %B%)
# $env:C, with value 3 (cmd.exe equivalent: %C%)
$section = 'default' # Specify the INI-file section of interest.
(Get-IniContent values.ini)[$section].GetEnumerator() |
ForEach-Object { Set-Item "env:$($_.Name)" -Value $_.Value }
# Read the template string as a whole from file foo.txt, and expand the
# environment-variable references in it, using the .NET framework.
# With the sample input, this yields
# "Now is the 1 for 1 2 men to come to the 3 of their 3".
[environment]::ExpandEnvironmentVariables((Get-Content -Raw foo.txt))
The 3rd-party Get-IniContent cmdlet, which conveniently reads an INI file (*.ini) into a nested, ordered hashtable, can easily be installed with Install-Module PsIni from an elevated console (alternatively, add -Scope CurrentUser), if you have PS v5+ (or v3 or v4 with PackageManagement installed).
This solution takes advantage of the fact that the placeholders (e.g., %a%) look like cmd.exe-style environment-variable references.
Note the assumptions and caveats:
All ini-file keys / placeholder names are legal environment-variable names.
Preexisting variables may be overwritten, which can be problematic with names such as PATH.
Cross-platform caveat: on Unix-like platforms, environment-variable references are case-sensitive, so the solution won't work the same there.
Using custom INI-file parsing and [environment]::ExpandEnvironmentVariables():
If installing a module for INI-file parsing is not an option, the following solution uses a - rather complex - regular expression to extract the section of interest via the -replace operator.
$section = 'default' # Specify the INI-file section of interest.
# Get all non-empty, non-comment lines from the section using a regex.
$sectLines = (Get-Content -Raw values.ini) -replace ('(?smn)\A.*?(^|\r\n)\[' + [regex]::Escape($section) + '\]\r\n(?<sectLines>.*?)(\r\n\[.*|\Z)'), '${sectLines}' -split "`r`n" -notmatch '(^;|^\s*$)'
# Define the key-value pairs as environment variables.
$sectlines | ForEach-Object { $tokens = $_ -split '=', 2; Set-Item "env:$($tokens[0].Trim())" -Value $tokens[1].Trim() }
# Read the template string as a whole, and expand the environment-variable
# references in it, as before.
[environment]::ExpandEnvironmentVariables((Get-Content -Raw foo.txt))
I found a simpler solution using this INI script called Get-IniContent.
#read from Setup.ini
$INI = Get-IniContent .\Setup.ini
$sec="setup"
#REPLACE VARIABLES
foreach($c in Get-ChildItem -Path .\Application -Recurse -Filter *.config)
{
Write-Output $c.FullName
Write-Output $c.DirectoryName
$configFile = Get-Content $c.FullName -Raw
foreach($v in $INI[$sec].Keys)
{
$k = '%'+$v+'%'
$match = [regex]::IsMatch($configFile, $k)
if($match)
{
$configFile = $configFile -ireplace [regex]::Escape($k), $INI[$sec][$v]
}
}
Set-Content $c.FullName -Value $configFile
}