I'm writing a powershell program to replace strings using
-replace "$in", "$out"
It doesn't work for strings containing a backslash, how can I do to escape it?
The -replace operator uses regular expressions, which treat backslash as a special character. You can use double backslash to get a literal single backslash.
In your case, since you're using variables, I assume that you won't know the contents at design time. In this case, you should run it through [RegEx]::Escape():
-replace [RegEx]::Escape($in), "$out"
That method escapes any characters that are special to regex with whatever is needed to make them a literal match (other special characters include .,$,^,(),[], and more.
You'll need to either escape the backslash in the pattern with another backslash or use the .Replace() method instead of the -replace operator (but be advised they may perform differently):
PS C:\> 'asdf' -replace 'as', 'b'
bdf
PS C:\> 'a\sdf' -replace 'a\s', 'b'
a\sdf
PS C:\> 'a\sdf' -replace 'a\\s', 'b'
bdf
PS C:\> 'a\sdf' -replace ('a\s' -replace '\\','\\'), 'b'
bdf
Note that only the search pattern string needs to be escaped. The code -replace '\\','\\' says, "replace the escaped pattern string '\\', which is a single backslash, with the unescaped literal string '\\' which is two backslashes."
So, you should be able to use:
-replace ("$in" -replace '\\','\\'), "$out"
[Note: briantist's solution is better.]
However, if your pattern has consecutive backslashes, you'll need to test it.
Or, you can use the .Replace() string method, but as I said above, it may not perfectly match the behavior of the -replace operator:
PS C:\> 'a\sdf'.replace('a\\s', 'b')
a\sdf
PS C:\> 'a\sdf'.replace( 'a\s', 'b')
bdf
Related
I have a set of SQL files stored in a folder. These files contain the schema name in the format dbo_xxxxxx where xxxxxx is the year & month e.g. dbo_202001, dbo_202002 etc. I want the powershell script to replace the xxxxx number with a new one in each of these SQL files. I'm using the below script to achieve that. However, the issue is that it seems to partially match on the old string (instead of matching on the full string) and puts the new string in place e.g. instead of replacing [dbo_202001] with [dbo_201902], anywhere it finds d, b, o etc. it replaces it with [dbo_201902]. Anyway to fix this?
$sourceDir = "C:\SQL_Scripts"
$SQLScripts = Get-ChildItem $sourceDir *.sql -rec
foreach ($file in $SQLScripts)
{
(Get-Content $file.PSPath) |
Foreach-Object { $_ -replace "[dbo_202001]", "[dbo_201902]" } |
Set-Content $file.PSPath -NoNewline
}
vonPryz and marsze have provided the crucial pointers: since the -replace operator operates on regexes (regular expressions), you must \-escape special characters such as [ and ] in order to treat them verbatim (as literals):
$_ -replace '\[dbo_202001\]', '[dbo_201902]'
While use of the -replace operator is generally preferable, the [string] type's .Replace() method directly offers verbatim (literal) string replacements and is therefore also faster than -replace.
Typically, this won't matter, but in situations similar to yours, where many iterations are involved, it may (note that the replacement is case-sensitive):
$_.Replace('[dbo_202001]', '[dbo_201902]')
See the bottom section for guidance on when to use -replace vs. .Replace().
The performance of your code can be greatly improved:
$sourceDir = 'C:\SQL_Scripts'
foreach ($file in Get-ChildItem -File $sourceDir -Filter *.sql -Recurse)
{
# CAVEAT: This overwrites the files in-place.
Set-Content -NoNewLine $file.PSPath -Value `
(Get-Content -Raw $file.PSPath).Replace('[dbo_202001]', '[dbo_201902]')
}
Since you're reading the whole file into memory anyway, using Get-Content's -Raw switch to read it as a single, multi-line string (rather than an array of lines) on which you can perform a single .Replace() operation is much faster.
Set-Content's -NoNewLine switch is needed to prevent an additional newline from getting appended on writing back to the file.
Note the use of the -Value parameter rather than the pipeline to provide the file content. Since there's only a single string object to write here, it makes little difference, but in general, with many objects to write that are already collected in memory, Set-Content ... -Value $array is much faster than $array | Set-Content ....
Guidance on use of the -replace operator vs. the .Replace() method:
-replace is the PowerShell-specific regex (regular-expression)-based string replacement operator,
whereas .Replace() is a method of the .NET [string] type. (System.String), which performs verbatim (literal) string replacements.
Note that both features invariably replace all matches they find, and, conversely, return the original string if none are found.
Generally, PowerShell's -replace operator is a more natural fit in PowerShell code - both syntactically and due to its case-insensitivity - and offers more functionality, thanks to being regex-based.
The .Replace() method is limited to verbatim replacements and in Windows PowerShell to case-sensitive ones, but has the advantage of being faster and not having to worry about escaping special characters in its arguments:
Only use the [string] type's .Replace() method:
for invariably verbatim string replacements
with the following case-sensitivity:
PowerShell [Core] v6+: case-sensitive by default, optionally case-insensitive via an additional argument; e.g.:
'FOO'.Replace('o', '#', 'InvariantCultureIgnoreCase')
Windows PowerShell: invariably(!) case-sensitive
if functionally feasible, when performance matters
Otherwise, use PowerShell's -replace operator (covered in more detail here):
for regex-based replacements:
enables sophisticated, pattern-based matching and dynamic construction of replacement strings
To escape metacharacters (characters with special syntactic meaning) in order to treat them verbatim:
in the pattern (regex) argument: \-escape them (e.g., \. or \[)
in the replacement argument: only $ is special, escape it as $$.
To escape an entire operand in order to treat its value verbatim (to effectively perform literal replacement):
in the pattern argument: call [regex]::Escape($pattern)
in the replacement argument: call $replacement.Replace('$', '$$')
with the following case-sensitivity:
case-insensitive by default
optionally case-sensitive via its c-prefixed variant, -creplace
Note: -replace is a PowerShell-friendly wrapper around the [regex]::Replace() method that doesn't expose all of the latter's functionality, notably not its ability to limit the number of replacements; see this answer for how to use it.
Note that -replace can directly operation on arrays (collections) of strings as the LHS, in which case the replacement is performed on each element, which is stringified on demand.
Thanks to PowerShell's fundamental member-access enumeration feature, .Replace() too can operate on arrays, but only if all elements are already strings. Also, unlike -replace, which always also returns an array if the LHS is one, member-access enumeration returns a single string if the input object happens to be a single-element array.
As an aside: similar considerations apply to the use of PowerShell's -split operator vs. the [string] type's .Split() method - see this answer.
Examples:
-replace - see this answer for syntax details:
# Case-insensitive replacement.
# Pattern operand (regex) happens to be a verbatim string.
PS> 'foo' -replace 'O', '#'
f##
# Case-sensitive replacement, with -creplace
PS> 'fOo' -creplace 'O', '#'
f#o
# Regex-based replacement with verbatim replacement:
# metacharacter '$' constrains the matching to the *end*
PS> 'foo' -replace 'o$', '#'
fo#
# Regex-based replacement with dynamic replacement:
# '$&' refers to what the regex matched
PS> 'foo' -replace 'o$', '>>$&<<'
fo>>o<<
# PowerShell [Core] only:
# Dynamic replacement based on a script block.
PS> 'A1' -replace '\d', { [int] $_.Value + 1 }
A2
# Array operation, with elements stringified on demand:
PS> 1..3 -replace '^', '0'
01
02
03
# Escape a regex metachar. to be treated verbatim.
PS> 'You owe me $20' -replace '\$20', '20 dollars'
You owe me 20 dollars.
# Ditto, via a variable and [regex]::Escape()
PS> $var = '$20'; 'You owe me $20' -replace [regex]::Escape($var), '20 dollars'
You owe me 20 dollars.
# Escape a '$' in the replacement operand so that it is always treated verbatim:
PS> 'You owe me 20 dollars' -replace '20 dollars', '$$20'
You owe me $20
# Ditto, via a variable and [regex]::Escape()
PS> $var = '$20'; 'You owe me 20 dollars' -replace '20 dollars', $var.Replace('$', '$$')
You owe me $20.
.Replace():
# Verbatim, case-sensitive replacement.
PS> 'foo'.Replace('o', '#')
f##
# No effect, because matching is case-sensitive.
PS> 'foo'.Replace('O', '#')
foo
# PowerShell [Core] v6+ only: opt-in to case-INsensitivity:
PS> 'FOO'.Replace('o', '#', 'InvariantCultureIgnoreCase')
F##
# Operation on an array, thanks to member-access enumeration:
# Returns a 2 -element array in this case.
PS> ('foo', 'goo').Replace('o', '#')
f##
g##
# !! Fails, because not all array elements are *strings*:
PS> ('foo', 42).Replace('o', '#')
... Method invocation failed because [System.Int32] does not contain a method named 'Replace'. ...
I have the following Regular Expression: ([a-z])([A-Z])
When I plug it into RegEx 101 it seems to work perfectly: https://regex101.com/r/vhifNL/1
But when I plug it into Powershell to have the matches replaced with dashes, it goes crazy:
"JavaScript" -replace '([a-z])([A-Z])', '$1-$2'
I expect to get Java-Script. But instead I get:
J-av-aS-cr-ip-t
Why is it not matching the same way that RegEx101 has it match?
NOTE: This question is not tagged with RegEx on purpose. I would take it as a kindness if no-one added it. The RegEx folks have a different set of rules they run by for questions and will likely close my question.
PowerShell's -replace operator, like all PowerShell operators that can operate on strings (notably -match, -eq, -like, -contains and their negated counterparts), and like PowerShell in general, is case-insensitive by default.
However, all such operators have case-sensitive variants, selected by simply prepending c to the operator name, namely -creplace in the case at hand:
PS> "JavaScript" -creplace '([a-z])([A-Z])', '$1-$2'
Java-Script
As for what you tried:
Due to -replace being case-insensitive (which you can optionally signal explicitly with the
-ireplace alias), your regex was essentially equivalent to:
([a-zA-Z][a-zA-Z])
and therefore matched any two consecutive (ASCII-range) letters, and not the desired transition from a lowercase to an uppercase letter.
I have a set of SQL files stored in a folder. These files contain the schema name in the format dbo_xxxxxx where xxxxxx is the year & month e.g. dbo_202001, dbo_202002 etc. I want the powershell script to replace the xxxxx number with a new one in each of these SQL files. I'm using the below script to achieve that. However, the issue is that it seems to partially match on the old string (instead of matching on the full string) and puts the new string in place e.g. instead of replacing [dbo_202001] with [dbo_201902], anywhere it finds d, b, o etc. it replaces it with [dbo_201902]. Anyway to fix this?
$sourceDir = "C:\SQL_Scripts"
$SQLScripts = Get-ChildItem $sourceDir *.sql -rec
foreach ($file in $SQLScripts)
{
(Get-Content $file.PSPath) |
Foreach-Object { $_ -replace "[dbo_202001]", "[dbo_201902]" } |
Set-Content $file.PSPath -NoNewline
}
vonPryz and marsze have provided the crucial pointers: since the -replace operator operates on regexes (regular expressions), you must \-escape special characters such as [ and ] in order to treat them verbatim (as literals):
$_ -replace '\[dbo_202001\]', '[dbo_201902]'
While use of the -replace operator is generally preferable, the [string] type's .Replace() method directly offers verbatim (literal) string replacements and is therefore also faster than -replace.
Typically, this won't matter, but in situations similar to yours, where many iterations are involved, it may (note that the replacement is case-sensitive):
$_.Replace('[dbo_202001]', '[dbo_201902]')
See the bottom section for guidance on when to use -replace vs. .Replace().
The performance of your code can be greatly improved:
$sourceDir = 'C:\SQL_Scripts'
foreach ($file in Get-ChildItem -File $sourceDir -Filter *.sql -Recurse)
{
# CAVEAT: This overwrites the files in-place.
Set-Content -NoNewLine $file.PSPath -Value `
(Get-Content -Raw $file.PSPath).Replace('[dbo_202001]', '[dbo_201902]')
}
Since you're reading the whole file into memory anyway, using Get-Content's -Raw switch to read it as a single, multi-line string (rather than an array of lines) on which you can perform a single .Replace() operation is much faster.
Set-Content's -NoNewLine switch is needed to prevent an additional newline from getting appended on writing back to the file.
Note the use of the -Value parameter rather than the pipeline to provide the file content. Since there's only a single string object to write here, it makes little difference, but in general, with many objects to write that are already collected in memory, Set-Content ... -Value $array is much faster than $array | Set-Content ....
Guidance on use of the -replace operator vs. the .Replace() method:
-replace is the PowerShell-specific regex (regular-expression)-based string replacement operator,
whereas .Replace() is a method of the .NET [string] type. (System.String), which performs verbatim (literal) string replacements.
Note that both features invariably replace all matches they find, and, conversely, return the original string if none are found.
Generally, PowerShell's -replace operator is a more natural fit in PowerShell code - both syntactically and due to its case-insensitivity - and offers more functionality, thanks to being regex-based.
The .Replace() method is limited to verbatim replacements and in Windows PowerShell to case-sensitive ones, but has the advantage of being faster and not having to worry about escaping special characters in its arguments:
Only use the [string] type's .Replace() method:
for invariably verbatim string replacements
with the following case-sensitivity:
PowerShell [Core] v6+: case-sensitive by default, optionally case-insensitive via an additional argument; e.g.:
'FOO'.Replace('o', '#', 'InvariantCultureIgnoreCase')
Windows PowerShell: invariably(!) case-sensitive
if functionally feasible, when performance matters
Otherwise, use PowerShell's -replace operator (covered in more detail here):
for regex-based replacements:
enables sophisticated, pattern-based matching and dynamic construction of replacement strings
To escape metacharacters (characters with special syntactic meaning) in order to treat them verbatim:
in the pattern (regex) argument: \-escape them (e.g., \. or \[)
in the replacement argument: only $ is special, escape it as $$.
To escape an entire operand in order to treat its value verbatim (to effectively perform literal replacement):
in the pattern argument: call [regex]::Escape($pattern)
in the replacement argument: call $replacement.Replace('$', '$$')
with the following case-sensitivity:
case-insensitive by default
optionally case-sensitive via its c-prefixed variant, -creplace
Note: -replace is a PowerShell-friendly wrapper around the [regex]::Replace() method that doesn't expose all of the latter's functionality, notably not its ability to limit the number of replacements; see this answer for how to use it.
Note that -replace can directly operation on arrays (collections) of strings as the LHS, in which case the replacement is performed on each element, which is stringified on demand.
Thanks to PowerShell's fundamental member-access enumeration feature, .Replace() too can operate on arrays, but only if all elements are already strings. Also, unlike -replace, which always also returns an array if the LHS is one, member-access enumeration returns a single string if the input object happens to be a single-element array.
As an aside: similar considerations apply to the use of PowerShell's -split operator vs. the [string] type's .Split() method - see this answer.
Examples:
-replace - see this answer for syntax details:
# Case-insensitive replacement.
# Pattern operand (regex) happens to be a verbatim string.
PS> 'foo' -replace 'O', '#'
f##
# Case-sensitive replacement, with -creplace
PS> 'fOo' -creplace 'O', '#'
f#o
# Regex-based replacement with verbatim replacement:
# metacharacter '$' constrains the matching to the *end*
PS> 'foo' -replace 'o$', '#'
fo#
# Regex-based replacement with dynamic replacement:
# '$&' refers to what the regex matched
PS> 'foo' -replace 'o$', '>>$&<<'
fo>>o<<
# PowerShell [Core] only:
# Dynamic replacement based on a script block.
PS> 'A1' -replace '\d', { [int] $_.Value + 1 }
A2
# Array operation, with elements stringified on demand:
PS> 1..3 -replace '^', '0'
01
02
03
# Escape a regex metachar. to be treated verbatim.
PS> 'You owe me $20' -replace '\$20', '20 dollars'
You owe me 20 dollars.
# Ditto, via a variable and [regex]::Escape()
PS> $var = '$20'; 'You owe me $20' -replace [regex]::Escape($var), '20 dollars'
You owe me 20 dollars.
# Escape a '$' in the replacement operand so that it is always treated verbatim:
PS> 'You owe me 20 dollars' -replace '20 dollars', '$$20'
You owe me $20
# Ditto, via a variable and [regex]::Escape()
PS> $var = '$20'; 'You owe me 20 dollars' -replace '20 dollars', $var.Replace('$', '$$')
You owe me $20.
.Replace():
# Verbatim, case-sensitive replacement.
PS> 'foo'.Replace('o', '#')
f##
# No effect, because matching is case-sensitive.
PS> 'foo'.Replace('O', '#')
foo
# PowerShell [Core] v6+ only: opt-in to case-INsensitivity:
PS> 'FOO'.Replace('o', '#', 'InvariantCultureIgnoreCase')
F##
# Operation on an array, thanks to member-access enumeration:
# Returns a 2 -element array in this case.
PS> ('foo', 'goo').Replace('o', '#')
f##
g##
# !! Fails, because not all array elements are *strings*:
PS> ('foo', 42).Replace('o', '#')
... Method invocation failed because [System.Int32] does not contain a method named 'Replace'. ...
Suppose I have a file database_partial.xml.
I am trying to strip the file from "_partial" as well as extension (xml) and then capitalize the name so that it becomes DATABASE.
Param($xmlfile)
$xml = Get-ChildItem "C:\Files" -Filter "$xmlfile"
$db = [IO.Path]::GetFileNameWithoutExtension($xml).ToUpper()
That returns DATABASE_PARTIAL, but I don't know how to strip the _PARTIAL part.
You don't need GetFileNameWithoutExtension() for removing the extension. The FileInfo objects returned by Get-ChildItem have a property BaseName that gives you the filename without extension. Uppercase that, then remove the "_PARTIAL" suffix. I would also recommend processing the output of Get-ChildItem in a loop, just in case it doesn't return exactly one result.
Get-ChildItem "C:\Files" -Filter "$xmlfile" | ForEach-Object {
$_.BaseName.ToUpper().Replace('_PARTIAL', '')
}
If the substring after the underscore can vary, use a regular expression replacement instead of a string replacement, e.g. like this:
Get-ChildItem "C:\Files" -Filter "$xmlfile" | ForEach-Object {
$_.BaseName.ToUpper() -replace '_[^_]*$'
}
Ansgar Wiechers's helpful answer provides an effective solution.
To focus on the more general question of how to strip (remove) part of a file name (string):
Use PowerShell's -replace operator, whose syntax is:<stringOrStrings> -replace <regex>, <replacement>:
<regex> is a regex (regular expression) that matches the part to replace,
<replacement> is replacement operand (the string to replace what the regex matched).
In order to effectively remove what the regex matched, specify '' (the empty string) or simply omit the operand altogether - in either case, the matched part is effectively removed from the input string.
For more information about -replace, see this answer.
Applied to your case:
$db = 'DATABASE_PARTIAL' # sample input value
PS> $db -replace '_PARTIAL$', '' # removes suffix '_PARTIAL' from the end (^)
DATABASE
PS> $db -replace '_PARTIAL$' # ditto, with '' implied as the replacement string.
DATABASE
Note:
-replace is case-insensitive by default, as are all PowerShell operators. To explicitly perform case-sensitive matching, use the -creplace variant.
By contrast, the [string] type's .Replace() method (e.g., $db.Replace('_PARTIAL', ''):
matches by string literals only, and therefore offers less flexibility; in this case, you couldn't stipulate that _PARTIAL should only be matched at the end of the string, for instance.
is invariably case-sensitive in the .NET Framework (though .NET Core offers a case-insensitive overload).
Building on Ansgar's answer, your script can therefore be streamlined as follows:
Param($xmlfile)
$db = ((Get-ChildItem C:\Files -Filter $xmlfile).BaseName -replace '_PARTIAL$').ToUpper()
Note that in PSv3+ this works even if $xmlfile should match multiple files, due to member-access enumeration and the ability of -replace to accept an array of strings as input, the desired substring removal would be performed on the base names of all files, as would the subsequent uppercasing - $db would then receive an array of stripped base names.
I want to unescape PHP code that has been escaped with backslashes. I'm using Powershell to do this.
Backslash characters can be used to escape a number of things, but in my sample data I only see examples of double-quotes being escaped in this manner. Ideally my code would support the other cases, but this is all I have for now.
$test = 'some-text.'
$test -replace '\"', '"'
The output of this is not what I expected:
some-text.
Expected output:
some-text.
I've also tried
$test -replace "`\`"", "`""
But the result was the same.
OK it seems backslashes are also used for escaping backslashes in powershell. Therefore the correct code is
$test -replace '\\"', '"'