How to add counter into Powershells ForEach-Object function - powershell

So I have a Pipe that will search a file for a specific stream and if found will replace it with a masked value, I am trying to have a counter for all of the times the oldValue is replaced with the newValue. It doesn't necessarily need to be a one liner just curious how you guys would go about this. TIA!
Get-Content -Path $filePath |
ForEach-Object {
$_ -replace "$oldValue", "$newValue"
} |
Set-Content $filePath

I suggest:
Reading the entire input file as a single string with Get-Content's -Raw switch.
Using -replace / [regex]::Replace() with a script block to determine the substitution text, which allows you to increment a counter variable every time a replacement is made.
Note: Since you're replacing the input file with the results, be sure to make a backup copy first, to be safe.
In PowerShell (Core) 7+, the -replace operator now directly accepts a script block that allows you to determine the substitution text dynamically:
$count = 0
(Get-Content -Raw $filePath) -replace $oldValue, { $newValue; ++$count } |
Set-Content -NoNewLine $filePath
$count now contains the number of replacements, across all lines (including multiple matches on the same line), that were performed.
In Windows PowerShell, direct use of the underlying .NET API, [regex]::Replace(), is required:
$count = 0
[regex]::Replace(
'' + (Get-Content -Raw $filePath),
$oldValue,
{ $newValue; ++(Get-Variable count).Value }
) | Set-Content -NoNewLine $filePath
Note:
'' + ensures that the call succeeds even if file $filePath has no content at all; without it, [regex]::Replace() would complain about the argument being null.
++(Get-Variable count).Value must be used in order to increment the $count variable in the caller's scope (Get-Variable can retrieve variables defined in ancestral scopes; -Scope 1 is implied here, thanks to PowerShell's dynamic scoping). Unlike with -replace in PowerShell 7+, the script block runs in a child scope.
As an aside:
For this use case, the only reason a script block is used is so that the counter variable can be incremented - the substitution text itself is static. See this answer for an example where the substitution text truly needs to be determined dynamically, by deriving it from the match at hand, as passed to the script block.

Changing my answer due to more clarifications in comments. The best way I can think of is to get the count of the $Oldvalue ahead of time. Then replace!
$content = Get-Content -Path $filePath
$toBeReplaced = Select-String -InputObject $content -Pattern $oldValue -AllMatches
$replacedTotal = $toBeReplaced.Matches.Count
$content | ForEach-Object {$_ -replace "$oldValue", "$newValue"} | Set-Content $filePath

Related

PowerShell: Replace string in all .txt files within directory

I am trying to replace every instance of a string within a directory. However my code is not replacing anything.
What I have so far:
Test Folder contains multiple files and folders containing content that I need to change.
The folders contain .txt documents, the .txt documents contain strings like this: Content reference="../../../PartOfPath/EN/EndofPath/Caution.txt" that i need to change into this: Content reference="../../../PartOfPath/FR/EndofPath/Caution.txt"
Before this question comes up, yes it has to be done this way, as there are other similar strings that I don't want to edit. So I cannot just replace all instances of EN with FR.
$DirectoryPath = "C:\TestFolder"
$Parts =#(
#{PartOne="/PartOfPath";PartTwo="EndofPath/Caution.txt"},
#{PartOne="/OtherPartOfPath";PartTwo="EndofPath/Note.txt"},
#{PartOne="/ThirdPartOfPath";PartTwo="OtherEndofPath/Warning.txt"}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
Get-ChildItem $DirectoryPath | ForEach {
foreach($n in $Parts){
[string]$PartOne = $n.PartOne
[string]$PartTwo = $n.PartTwo
$ReplaceThis = "$PartOne/EN/$PartTwo"
$WithThis = "$PartOne/FR/$PartTwo"
(Get-Content $_) | ForEach {$_ -Replace $ReplaceThis, $WithThis} | Set-Content $_
}
}
The code will run and overwrite files, however no edits will have been made.
While troubleshooting I came across this potential cause:
This test worked:
$FilePath = "C:\TestFolder\Test.txt"
$ReplaceThis ="/PartOfPath/EN/Notes/Note.txt"
$WithThis = "/PartOfPath/FR/Notes/Note.txt"
(Get-Content -Path $FilePath) -replace $ReplaceThis, $WithThis | Set-Content $FilePath
But this test did not
$FilePath = "C:\TestFolder\Test.txt"
foreach($n in $Parts){
[string]$PartOne = $n.PartOne
[string]$PartTwo = $n.PartTwo
[string]$ReplaceThis = "$PartOne/EN/$PartTwo"
[string]$WithThis = "$PartOne/FR/$PartTwo"
(Get-Content -Path $FilePath) -replace $ReplaceThis, $WithThis | Set-Content $FilePath
}
If you can help me understand what is wrong here I would greatly appreciate it.
Thanks to #TessellatingHeckler 's comments I revised my code and found this solution:
$DirectoryPath = "C:\TestFolder"
$Parts =#(
#{PartOne="/PartOfPath";PartTwo="EndofPath/Caution.txt"},
#{PartOne="/OtherPartOfPath";PartTwo="EndofPath/Note.txt"},
#{PartOne="/ThirdPartOfPath";PartTwo="OtherEndofPath/Warning.txt"}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
Get-ChildItem $LanguageFolderPath -Filter "*.txt" -Recurse | ForEach {
foreach($n in $Parts){
[string]$PartOne = $n.PartOne
[string]$PartTwo = $n.PartTwo
$ReplaceThis = "$PartOne/EN/$PartTwo"
$WithThis = "$PartOne/FR/$PartTwo"
(Get-Content $_) | ForEach {$_.Replace($ReplaceThis, $WithThis)} | Set-Content $_
}
}
There were two problems:
Replace was not working as I intended, so I had to use .replace instead
The original Get-ChildItem was not returning any values and had to be replaced with the above version.
PowerShell's -replace operator is regex-based and case-insensitive by default:
To perform literal replacements, \-escape metacharacters in the pattern or call [regex]::Escape().
By contrast, the [string] type's .Replace() method performs literal replacement and is case-sensitive, invariably in Windows PowerShell, by default in PowerShell (Core) 7+ (see this answer for more information).
Therefore:
As TessellatingHeckler points out, given that your search strings seem to contain no regex metacharacters (such as . or \) that would require escaping, there is no obvious reason why your original approach didn't work.
Given that you're looking for literal substring replacements, the [string] type's .Replace() is generally the simpler and faster option if case-SENSITIVITY is desired / acceptable (invariably so in Windows PowerShell; as noted, in PowerShell (Core) 7+, you have the option of making .Replace() case-insensitive too).
However, since you need to perform multiple replacements, a more concise, single-pass -replace solution is possible (though whether it actually performs better would have to be tested; if you need case-sensitivity, use -creplace in lieu of -replace):
$oldLang = 'EN'
$newLang = 'FR'
$regex = #(
"(?<prefix>/PartOfPath/)$oldLang(?<suffix>/EndofPath/Caution.txt)",
"(?<prefix>/OtherPartOfPath/)$oldLang(?<suffix>/EndofPath/Note.txt)",
"(?<prefix>/ThirdPartOfPath/)$oldLang(?<suffix>/OtherEndofPath/Warning.txt)"
) -join '|'
Get-ChildItem C:\TestFolder\Test.txt -Filter *.txt -Recurse | ForEach-Object {
($_ |Get-Content -Raw) -replace $regex, "`${prefix}$newLang`${suffix}" |
Set-Content -LiteralPath $_.FullName
}
See this regex101.com page for an explanation of the regex and the ability to experiment with it.
The expression used as the replacement operand, "`${prefix}$newLang`${suffix}", mixes PowerShell's up-front string interpolation ($newLang, which could also be written as ${newLang}) with placeholders referring to the named capture groups (e.g. (?<prefix>...)) in the regex, which only coincidentally use the same notation as PowerShell variables (though enclosing the name in {...} is required; also, here the $ chars. must be `-escaped to prevent PowerShell's string interpolation from interpreting them); see this answer for background information.
Note the use of -Raw with Get-Content, which reads a text file as a whole into memory, as a single, multi-line string. Given that you don't need line-by-line processing in this case, this greatly speeds up the processing of a given file.
As a general tip: you may need to use the -Encoding parameter with Set-Content to ensure the desired character encoding, given that PowerShell never preserves a file's original coding when reading it. By default, you'll get ANSI-encoded files in Windows PowerShell, and BOM-less UTF-8 files in PowerShell (Core) 7+.

How to prepend to a file in PowerShell?

I'm generating two files, userscript.meta.js and userscript.user.js. I need the output of userscript.meta.js to be placed at the very beginning of userscript.user.js.
Add-Content doesn't seem to accept a parameter to prepend and Get-Content | Set-Content will fail because userscript.user.js is being used by Get-Content.
I'd rather not create an intermediate file if it's physically possible to have a clean solution.
How to achieve this?
The Subexpression operator $( ) can evaluate both Get-Content statements which are then enumerated and passed through the pipeline to Set-Content:
$(
Get-Content userscript.meta.js -Raw
Get-Content userscript.user.js -Raw
) | Set-Content userscript.user.js
Consider using the Absolute Path of the files if your current directory is not where those files are.
An even more simplified approach than the above would be to put the paths in the desired order since both, the -Path and -LiteralPath parameters can take multiple values:
(Get-Content userscript.meta.js, userscript.user.js -Raw) |
Set-Content userscript.user.js
And in case you want to get rid of excess leading or trailing white-space, you can include the String.Trim Method:
(Get-Content userscript.meta.js, userscript.user.js -Raw).Trim() |
Set-Content userscript.user.js
Note that in above examples the grouping operator ( ) is mandatory as we need to consume all output from Get-Content before being passed through the pipeline to Set-Content. See Piping grouped expressions for more details.
For future folks, here's a snippet if you need to prepend the same thing to multiple files:
example: prepending an #include directive to a bunch of auto-generated C++ files so it works with my Windows environment.
Get-ChildItem -Path . -Filter *.cpp | ForEach-Object {
$file = $_.FullName
# the -Raw param was important for me as it didn't read the entire
# file properly without it. I even tried [System.IO.File]::ReadAllText
# and got the same thing, so there must have been some characater that
# caused the file read to return prematurely
$content = Get-Content $file -Raw
$prepend = '#include "stdafx.h"' + "`r`n"
#this could also be from a file: aka
# $prepend = Get-Content 'path_to_my_file_used_for_prepending'
$content = $prepend + $content
Set-Content $file $content
}

Replace a non-unique line of text under a unique line of text in a text file using powershell

I have the following txt file.
[AppRemover]
Enable=0
[CleanWipe]
Enable=0
[RerunSetup]
Enable=0
How do I change the Enable=0 to Enable=1 under [CleanWipe] only?
Below is how I plan on using the code with my file.
$Path = C:\temp\file.txt
$File = Get-Content -Path $Path
# Code to update file
$File | Out-File $Path
You can use -replace to update the value if it is 0.
$Path = C:\temp\file.txt
(Get-Content $Path -Raw) -replace "(?<text>\[CleanWipe\]\r?\nEnable=)0",'${text}1' |
Set-Content $Path
Using a module that parses INI files will be the best solution though. I'd recommend trying PsIni.
Explanation:
The -Raw switch reads the file contents as a single string. This makes it easier to work with newline characters.
-replace performs a regex match and then replace. Below is the regex match breakdown.
(?<text>) is a named capture group. Anything matched within that capture group can be recalled in the replace string as '${text}'.
\[CleanWipe\] is a literal match of [CleanWipe] while escaping the [] characters with \.
\r? is optional carriage return
\n is the newline character
Enable= is a literal match
0 is a literal match
The replace string is the capture group contents and 1 when a match exists. Technically, a capture group is not needed if you want to use a positive lookbehind instead. The positive lookbehind assertion is (?<=). That solution would look like the following:
$Path = C:\temp\file.txt
(Get-Content $Path -Raw) -replace "(?<=\[CleanWipe\]\r?\nEnable=)0",'1' |
Set-Content $Path
The problem with the -replace solutions as they written is they will update the file regardless of a change actually being made to the contents. You would need to add an extra comparison to prevent that. Other issues could be extra white space on any of these lines. You can account for that by adding \s* where you think those possibilities may exist.
Alternative With More Steps:
$file = Get-Content $Path
$TargetIndex = $file.IndexOf('[CleanWipe]') + 1
if ($file[$TargetIndex] -match 'Enable=0') {
$file[$TargetIndex] = 'Enable=1'
$file | Set-Content $Path
}
This solution will only update the file if it meets the match condition. It uses the array method IndexOf() to determine where [CleanWipe] is. Then assumes the line you want to change is in the next index.
IndexOf() is not the only way to find an index. The method requires that your line match the string exactly. You can use Select-String (case-insensitive by default) to return a line number. Since it will be a line number and not an index (indexes start at 0 while line numbers start at 1), it will invariably be the index number you want.
$file = Get-Content $Path
$TargetIndex = ($file | Select-String -Pattern '[CleanWipe]' -SimpleMatch).LineNumber
if ($file[$TargetIndex] -match 'Enable=0') {
$file[$TargetIndex] = 'Enable=1'
$file | Set-Content $Path
}

Why are all newlines gone after PowerShell's Get-Content, Regex, and Set-Content?

I want to load a file template into a variable, modify data within the variable and output the modified template to a new location from the variable.
The issue is that PowerShell is removing newlines from my template.
The input file (template file) has Unix line endings which are also required for output since the recipient of the modified version is a Unix-based system.
I have the following code which results into a concatted one-liner:
[String] $replacement = "Foo Bar"
[String] $template = Get-Content -Path "$pwd\template.sh" -Encoding UTF8
$template = $template -replace '<REPLACE_ME>', $replacement
$template | Set-Content -Path "$pwd\script.sh" -Encoding UTF8
Having the template input:
#!/bin/sh
myvar="<REPLACE_ME>"
echo "my variable: $myvar"
exit 0
Resulted into:
#!/bin/sh myvar="Foo Bar" echo "my variable: $myvar" exit 0
It appears to me that somewhere LF where replaced by one simple whitespace. Finally at the end of the script there is an added CR LF which was not present in the template file.
How do I preserve the line endings and prevent adding further (CR LF) wrong line endings to the final script?
For the $replacement variable, you don't really need to specify the type [string], PowerShell will infer that from the assignment.
For the $template variable, [string] is actually wrong. By default, Get-Content will give you an array of strings (i.e. lines) instead of one string.
But in fact you don't even want to split the input into lines in the first place. When Set-Content or Out-File see an array as their input, they will join it with spaces.
Using -Raw makes Get-Content return the entire file as one string, this way also the line endings (like LF for Linux files) will stay the way they are.
$replacement = "Foo Bar"
$template = Get-Content -Path "$pwd\template.sh" -Encoding UTF8 -Raw
$template = $template -replace '<REPLACE_ME>', $replacement
Set-Content -Path "$pwd\script.sh" -Value $template -Encoding UTF8
PowerShell will save all UTF-8 files with a BOM. If you don't want that, you must use a different utility to write the file:
$UTF8_NO_BOM = New-Object System.Text.UTF8Encoding $False
$replacement = "Foo Bar"
$template = Get-Content -Path "$pwd\template.sh" -Encoding UTF8 -Raw
$template = $template -replace '<REPLACE_ME>', $replacement
[System.IO.File]::WriteAllText("$pwd\script.sh", $template, $UTF8_NO_BOM)
Notes:
PowerShell operators (like -replace) silently operate on arrays. $x -replace "search", "replacement" will perform a replace operation on every member of $x, be that a single string or an array of them.
Recommended reading: PowerShell Set-Content and Out-File what is the difference?
Use the -delimiter "`n" option instead of -raw. The -raw option reads/returns the entire content as a single string, although it preserves the new-line characters but it is useless if you need to manipulate the content e.g. skip Header/1st row or skip blank lines etc.
Get-Content - background info:
By default, the Get-Content cmdlet reads & returns content line-by-line, which means if you pipe a Set-Content or Add-Content to instantly write each-line (being read) into the output file - the newline characters are preserved and written as expected, e.g.:
Get-Content $inputFile | Set-Content $outputFilePath
However, if you store the entire content (read) into a $variable, you will receive a single string-array without any separator/delimiter (by default), which means you lose the new-line characters, however, when reading file (using Get-Content) you can use the -delimiter option to specify a newline character, e.g.:
Get-Content -Delimiter "`n" $fileToRead
HTH.
I think you need to use the -Raw switch with Get-Content in order to load the file as a single string:
[String] $replacement = "Foo Bar"
[String] $template = Get-Content -Path "$pwd\template.sh" -Encoding UTF8 -Raw
$template = $template -replace '<REPLACE_ME>', $replacement
To stop the Windows line ending being added to the end of the script, I think you need to use this .NET method for writing the file:
[io.file]::WriteAllText("$pwd\template.sh",$template)
By default PowerShell attempts to convert your input in to an array of strings for each line in the file. I think because of the Unix line endings its not doing this successfully but is subsequently removing the new line characters.
In PowerShell 3.0 we now have a new dynamic parameter, Raw. When
specified, Get-Content ignores newline characters and returns the
entire contents of a file in one string. Raw is a dynamic parameter,
it is available only in file system drives.
https://social.technet.microsoft.com/Forums/windowsserver/en-US/6026b31a-2a0e-4e0a-90b5-355387dce9ac/preventing-newline-with-outfile-or-addcontent?forum=winserverpowershell
I was using Get-Content-Tail, which doesn't allow you to specify -Raw at the same time, but I did have luck with Out-String. So, in your case:
$template = Out-String -InputObject $( Get-Content -Path "$pwd\template.sh" -Encoding UTF8 -Raw)
Or perhaps, if you care about tail:
$template = Out-String -InputObject $(Get-Content -Path "$pwd\template.sh" -tail 4)

replace content of multiple Files with PowerShell

I'm trying to replace a certain Line in multiple logonscripts (>2000 Scripts).
The script works in the current form, but it writes every file to the disk, even when no changes are made, but I don't want this behaviour. It only should write to the disk, if changes are made.
This is what I already have:
$varFiles = Get-ChildItem $varPath*.$VarEnding
foreach ($file in $varFiles)
{
(Get-Content $file) |
Foreach-Object { $_ -replace [regex]::Escape("$varFind"), "$varReplace" } |
Set-Content $file
}
And this is what I already tried, but it seems, that it is not possible to use if in piped commands:
$varFiles = Get-ChildItem $varPath*.$VarEnding
foreach ($file in $varFiles)
{
$control = $file
(Get-Content $file) |
Foreach-Object { $_ -replace [regex]::Escape("$varFind"), "$varReplace" } |
If($control -ne $file){Set-Content $file}
}
The variables $varPath, $varEnding, $varFind and $varReplace are defined by a few Read-Host commands at the start of the script.
I hope you guys can help me :)
For simplicity and speed - although at the expense of memory use - I'd simply cache and operate on whole input files (requires PowerShell v3+, due to use of -Raw[1]); since logon scripts are generally small, this should be acceptable:
$varFindEscaped = [regex]::Escape($varFind)
$varReplaceEscaped = $varReplace -replace '\$', '$$$$'
foreach ($file in Get-ChildItem $varPath*$varEnding) {
$contentBefore = Get-Content -Raw $file
$contentAfter = $contentBefore -replace $varFindEscaped, $varReplaceEscaped
if ($contentBefore -ne $contentAfter) {
Set-Content $file $contentAfter
}
}
To improve performance I've moved escaping of the -regex operands outside the loop.
Note that I'm also escaping $ instances in the replacement value to prevent their interpretation as references to what was matched, such as $& to refer to the entire match.
Note that Set-Content by default uses the system's default code page rather than UTF-8 encoding.
[1]
In PS v2, you may omit -Raw, which turns $contentBefore into an array of strings (lines) on whose elements -replace then operates individually (as in the OP's approach). While probably slightly slower, it does have the advantage of performing the substitution on individual lines only rather than potentially across multiple lines.