How do I add an atomic counter to a powershell ForEach -Parallel loop - powershell

In this question, it was explained how to add to a concurrent ThreadSafe collection Powershell: How to add Result to an Array (ForEach-Object -Parallel)
I have a simpler use case , where I would just like to increment a single value. (Integer).
Is it possible to do in Powershell using some sort of Atomic Integer data type?
$myAtomicCounter = 0
$myItems | ForEach-Object -Parallel {
#...other work
$myAtomicCounter.ThreadSafeAdd(2)
# .. some more work using counter
}
Write-Host($myAtomicCounter)

In PowerShell when updating a single value from multiple threads you must use a locking mechanism, for example Mutex, SemaphoreSlim or even Monitor.Enter otherwise the updating operation will not be thread safe.
For example, supposing you have an array of arrays:
$toProcess = 0..10 | ForEach-Object {
, (Get-Random -Count (Get-Random -Minimum 5 -Maximum 10))
}
And you wanted to keep track of the processed items in each array, here is how you could do it using Mutex:
$processedItems = [hashtable]::Synchronized(#{
Lock = [System.Threading.Mutex]::new()
Counter = 0
})
$toProcess | ForEach-Object -Parallel {
# using sleep as to emulate doing something here
Start-Sleep (Get-Random -Maximum 5)
# bring the local variable to this scope
$ref = $using:processedItems
# lock this thread until I can write
if($ref['Lock'].WaitOne()) {
# when I can write, update the value
$ref['Counter'] += $_.Count
# and realease this lock so others threads can write
$ref['Lock'].ReleaseMutex()
}
}
$processedCount = ($toProcess | Write-Output | Measure-Object).Count
# Should be True:
$processedItems['Counter'] -eq $processedCount
A much simpler approach in PowerShell would be to output from your parallel loop into a linear loop where you can safely update the counter without having to care about thread safety:
$counter = 0
$toProcess | ForEach-Object -Parallel {
# using sleep as to emulate doing something here
Start-Sleep (Get-Random -Maximum 5)
# when this thread is done,
# output this array of processed items
$_
} | ForEach-Object {
# then the output from the parallel loop is received in this linear
# thread safe loop where we can update the counter
$counter += $_.Count
}
$processedCount = ($toProcess | Write-Output | Measure-Object).Count
# Should be True:
$counter -eq $processedCount

Related

Why this script is so slow when creating a random list

I am trying to generate a random directory structure with 10s of thousands of directories, but this script takes way too long. Is it the recursive call to CreateDir that is causing the slowness. I using a file to store the the directory list, so I can create all the directories at once AND because I could not figure out how to use an array. For example, it took 3 minutes for 1200 directories.
Clear-Host
$ParentDir = 'c:\a\'
$MinNumberOfLeaves = 2
$MaxNumberOfLeaves = 10
$MaxDepth = 5
$MinDepth = 2
Function CreateDirName{
$fRandomNames = [System.Collections.Arraylist]#()
# Get-Random -Count ([int]::MaxValue)) Randomizes the entire list of commands
$fRandomNames += (((Get-Command) | Select-Object -Property Name).Name | `
Where-Object {($_ -inotmatch ":") -and ($_ -inotlike "")} )
$fRandomName = ($fRandomNames | Get-Random -Count 1) -replace("-","")
Return $fRandomName
}
Function CreateDir{
Param(
$fParentDir,
$fMinNumberOfLeaves = 2,
$fMaxNumberOfLeaves = 3,
$fMaxDepth = 3,
$fMinDepth = 2,
$fRandomDepth = 2
)
For($d=1;$d -le ($fRandomDepth);$d++)
{
$fNumOfLeaves = Get-Random -Minimum $fMinNumberOfLeaves -Maximum $fMaxNumberOfLeaves
#$fNumOfLeaves = 4
For($l=1;$l -le $fNumOfLeaves;$l++)
{
$fSubDirName = CreateDirName
$fFullDirPath = $fParentDir + '\' + $fSubDirName
$fFullDirPath | Out-File -Append -FilePath c:\a\Paths.txt -Encoding ascii
#New-Item -Path $fFullDirPath -ItemType Directory
$SubDirs = CreateDir -fParentDir $fFullDirPath -fRandomDepth ($fRandomDepth-1)
Out-Null
}
Out-Null
}
Out-Null
}
"Dirs"| Set-content c:\a\Paths.txt -Force -Encoding ascii
$RandomDepth = Get-Random -Minimum $MinDepth -Maximum $MaxDepth
#$RandomDepth = 3
CreateDir -fParentDir $ParentDir -fRandomDepth $RandomDepth
Out-Null
Ok, there are many aspects of your approach that makes this script slow.
I think is a great learning opportunity.
First, let's address your 'CreateDirName' function.
The current workflow is:
Create a new ArrayList
Get a list of ALL commands, Select the property 'Name', then select 'Name' again '().Name', use a post filter for strings that don't match ':' and it's not like ''.
Pipe the result to 'Get-Random' and use '-replace' to remove '-'
First, remove the unnecessary 'Select-Object'. You can expand the property 'Name' like this: '(Get-Command).Name'.
Then let's look at your filter. 'Where-Object', by itself is costly, but you make it worse by using '-inotmatch' and '-inotlike'.
'-match' operators use RegEx.
'-like' operators works with wildcards, which is also costly.
Since a string is an array of characters, we can use '.Contains()' to filter ':', and to avoid empty or null values we can use 'String.IsNullOrEmpty()'.
'-replace' also uses regex, we can replace it with '.Replace()'. (Pun not intended).
Suddenly, our code looks like this:
$fRandomNames = [System.Collections.Arraylist]#()
$fRandomNames += ((Get-Command).Name | Where-Object { !$_.Contains(':') -and ![string]::IsNullOrEmpty($_) })
$fRandomName = ($fRandomNames | Get-Random -Count 1).Replace('-', '')
Now, still on this function, since arrays are immutable objects, the '+=' operator creates a new array to support the object's size.
To improve that, we can use ArrayList.AddRange().
And since we are talking performance, using the pipeline always adds cost, so let's borrow LINQ from C#, and change this 'Where-Object'.
And now we have:
$fRandomNames = [System.Collections.Arraylist]#()
$fRandomNames.AddRange([System.Linq.Enumerable]::Where((Get-Command).Name, [Func[object, bool]] { param($c) !$c.Contains(':') -and ![string]::IsNullOrEmpty($c) }).ToList())
$fRandomName = (Get-Random -InputObject $fRandomNames -Count 1).Replace('-', '')
Getting better. I ran some tests with these options, and we got a 36% improvement.
Now let's attack the main function body.
You don't need to call 'Get-Command' every time, you can get the list and re-use it for further operations.
With this, we can ditch 'CreateDirName' completely, which avoids a memory lap, and contributes to our performance.
We can also avoid using 'for' loops, specially if you don't need the index number.
Let's use a 'do-while' loop instead.
Before I paste the next code session, let's look at how you write your data into the file.
You are calling 'Out-File' in every single operation.
That involves:
Getting the string and pass it to the pipe line.
Check if the file and directory exists.
Open a file stream to write data.
Write data.
Close the stream.
Dispose of unmanaged objects.
We can store the results in another ArrayList and write everything at the end.
And now we have:
$fRandomNames = [System.Collections.Arraylist]#()
$fRandomNames.AddRange([System.Linq.Enumerable]::Where((Get-Command).Name, [Func[object, bool]] { param($c) !$c.Contains(':') -and ![string]::IsNullOrEmpty($c) }).ToList())
$newPathList = [System.Collections.Arraylist]#()
$currentDepth = 0
do {
$leafNumber = 0
do {
$newPathList.Add("$fParentDir\$((Get-Random -InputObject $fRandomNames -Count 1).Replace('-', ''))")
New-CustomDirectory -fParentDir $fFullDirPath -fRandomDepth ($fRandomDepth - 1)
$leafNumber++
} while ($leafNumber -lt (Get-Random -Minimum $fMinNumberOfLeaves -Maximum $fMaxNumberOfLeaves))
$currentDepth++
} while ($currentDepth -lt $fRandomDepth)
Last thing I want to talk about is 'Out-File' and 'Out-Null'.
We now the steps for writing a file, why don't we use pure .NET instead?
And what's with all the 'Out-Null's?????
Every 'Out-Null' implies a cost. You should consider never using 'Out-Null'.
Instead, use $result = $null, or [void]$result.DoWork().
And you only need to do it once, at the main function call.
Let's tackle the file writing first.
Ran another sample to show you the benefits of using pure .NET:
And at the end, we have something like this:
$ParentDir = 'c:\a\'
$MaxDepth = 5
$MinDepth = 2
function New-CustomDirectory {
Param(
$fParentDir,
[int]$fMinNumberOfLeaves = 2,
[int]$fMaxNumberOfLeaves = 10,
[int]$fMaxDepth = 3,
[int]$fMinDepth = 2,
[int]$fRandomDepth = 2
)
$fRandomNames = [System.Collections.Arraylist]#()
$fRandomNames.AddRange([System.Linq.Enumerable]::Where((Get-Command).Name, [Func[object, bool]] { param($c) !$c.Contains(':') -and ![string]::IsNullOrEmpty($c) }).ToList())
$newPathList = [System.Collections.Arraylist]#()
$currentDepth = 0
do {
$leafNumber = 0
do {
$newPathList.Add("$fParentDir\$((Get-Random -InputObject $fRandomNames -Count 1).Replace('-', ''))")
New-CustomDirectory -fParentDir $fFullDirPath -fRandomDepth ($fRandomDepth - 1)
$leafNumber++
} while ($leafNumber -lt (Get-Random -Minimum $fMinNumberOfLeaves -Maximum $fMaxNumberOfLeaves))
$currentDepth++
} while ($currentDepth -lt $fRandomDepth)
$stream = [System.IO.File]::AppendText('C:\a\Paths.txt')
$stream.Write($newPathList)
$stream.Dispose()
}
"Dirs"| Set-content c:\a\Paths.txt -Force -Encoding ascii
$RandomDepth = Get-Random -Minimum $MinDepth -Maximum $MaxDepth
[void](New-CustomDirectory -fParentDir $ParentDir -fRandomDepth $RandomDepth)
DISCLAIMER! I didn't study what your script does and didn't test my version's output. This was only a block performance study.
You might need to change it to suit your needs.
Source:
PowerShell scripting performance considerations
High performance PowerShell with LINQ
Under the stairs: Performance with PowerShell
Hope it helps!
Happy scripting!
The comments are correct in pointing out multiple issues. But the first and primary needed change is removing these lines from the CreateDirName function and placing them near the top of the script:
$fRandomNames = [System.Collections.Arraylist]#()
# Get-Random -Count ([int]::MaxValue)) Randomizes the entire list of commands
$fRandomNames += (((Get-Command) | Select-Object -Property Name).Name | `
Where-Object {($_ -inotmatch ":") -and ($_ -inotlike "")} )
The (Get-Command) takes about 11 seconds on my computer, and calling it over and over again is insane! Build the $fRandomNames arraylist when the script starts, and use CreateDirName to only pick a random name from the already created $fRandomNames arraylist.

Why my code is not executed after the "break" when it is outside the loop? [duplicate]

I have the following code:
$project.PropertyGroup | Foreach-Object {
if($_.GetAttribute('Condition').Trim() -eq $propertyGroupConditionName.Trim()) {
$a = $project.RemoveChild($_);
Write-Host $_.GetAttribute('Condition')"has been removed.";
}
};
Question #1: How do I exit from ForEach-Object? I tried using "break" and "continue", but it doesn't work.
Question #2: I found that I can alter the list within a foreach loop... We can't do it like that in C#... Why does PowerShell allow us to do that?
First of all, Foreach-Object is not an actual loop and calling break in it will cancel the whole script rather than skipping to the statement after it.
Conversely, break and continue will work as you expect in an actual foreach loop.
Item #1. Putting a break within the foreach loop does exit the loop, but it does not stop the pipeline. It sounds like you want something like this:
$todo=$project.PropertyGroup
foreach ($thing in $todo){
if ($thing -eq 'some_condition'){
break
}
}
Item #2. PowerShell lets you modify an array within a foreach loop over that array, but those changes do not take effect until you exit the loop. Try running the code below for an example.
$a=1,2,3
foreach ($value in $a){
Write-Host $value
}
Write-Host $a
I can't comment on why the authors of PowerShell allowed this, but most other scripting languages (Perl, Python and shell) allow similar constructs.
There are differences between foreach and foreach-object.
A very good description you can find here: MS-ScriptingGuy
For testing in PS, here you have scripts to show the difference.
ForEach-Object:
# Omit 5.
1..10 | ForEach-Object {
if ($_ -eq 5) {return}
# if ($_ -ge 5) {return} # Omit from 5.
Write-Host $_
}
write-host "after1"
# Cancels whole script at 15, "after2" not printed.
11..20 | ForEach-Object {
if ($_ -eq 15) {continue}
Write-Host $_
}
write-host "after2"
# Cancels whole script at 25, "after3" not printed.
21..30 | ForEach-Object {
if ($_ -eq 25) {break}
Write-Host $_
}
write-host "after3"
foreach
# Ends foreach at 5.
foreach ($number1 in (1..10)) {
if ($number1 -eq 5) {break}
Write-Host "$number1"
}
write-host "after1"
# Omit 15.
foreach ($number2 in (11..20)) {
if ($number2 -eq 15) {continue}
Write-Host "$number2"
}
write-host "after2"
# Cancels whole script at 25, "after3" not printed.
foreach ($number3 in (21..30)) {
if ($number3 -eq 25) {return}
Write-Host "$number3"
}
write-host "after3"
To stop the pipeline of which ForEach-Object is part just use the statement continue inside the script block under ForEach-Object. continue behaves differently when you use it in foreach(...) {...} and in ForEach-Object {...} and this is why it's possible. If you want to carry on producing objects in the pipeline discarding some of the original objects, then the best way to do it is to filter out using Where-Object.
Since ForEach-Object is a cmdlet, break and continue will behave differently here than with the foreach keyword. Both will stop the loop but will also terminate the entire script:
break:
0..3 | foreach {
if ($_ -eq 2) { break }
$_
}
echo "Never printed"
# OUTPUT:
# 0
# 1
continue:
0..3 | foreach {
if ($_ -eq 2) { continue }
$_
}
echo "Never printed"
# OUTPUT:
# 0
# 1
So far, I have not found a "good" way to break a foreach script block without breaking the script, except "abusing" exceptions, although powershell core uses this approach:
throw:
class CustomStopUpstreamException : Exception {}
try {
0..3 | foreach {
if ($_ -eq 2) { throw [CustomStopUpstreamException]::new() }
$_
}
} catch [CustomStopUpstreamException] { }
echo "End"
# OUTPUT:
# 0
# 1
# End
The alternative (which is not always possible) would be to use the foreach keyword:
foreach:
foreach ($_ in (0..3)) {
if ($_ -eq 2) { break }
$_
}
echo "End"
# OUTPUT:
# 0
# 1
# End
If you insist on using ForEach-Object, then I would suggest adding a "break condition" like this:
$Break = $False;
1,2,3,4 | Where-Object { $Break -Eq $False } | ForEach-Object {
$Break = $_ -Eq 3;
Write-Host "Current number is $_";
}
The above code must output 1,2,3 and then skip (break before) 4. Expected output:
Current number is 1
Current number is 2
Current number is 3
Below is a suggested approach to Question #1 which I use if I wish to use the ForEach-Object cmdlet.
It does not directly answer the question because it does not EXIT the pipeline.
However, it may achieve the desired effect in Q#1.
The only drawback an amateur like myself can see is when processing large pipeline iterations.
$zStop = $false
(97..122) | Where-Object {$zStop -eq $false} | ForEach-Object {
$zNumeric = $_
$zAlpha = [char]$zNumeric
Write-Host -ForegroundColor Yellow ("{0,4} = {1}" -f ($zNumeric, $zAlpha))
if ($zAlpha -eq "m") {$zStop = $true}
}
Write-Host -ForegroundColor Green "My PSVersion = 5.1.18362.145"
I hope this is of use.
Happy New Year to all.
There is a way to break from ForEach-Object without throwing an exception. It employs a lesser-known feature of Select-Object, using the -First parameter, which actually breaks the pipeline when the specified number of pipeline items have been processed.
Simplified example:
$null = 1..5 | ForEach-Object {
# Do something...
Write-Host $_
# Evaluate "break" condition -> output $true
if( $_ -eq 2 ) { $true }
} | Select-Object -First 1 # Actually breaks the pipeline
Output:
1
2
Note that the assignment to $null is there to hide the output of $true, which is produced by the break condition. The value $true could be replaced by 42, "skip", "foobar", you name it. We just need to pipe something to Select-Object so it breaks the pipeline.
I found this question while looking for a way to have fine grained flow control to break from a specific block of code. The solution I settled on wasn't mentioned...
Using labels with the break keyword
From: about_break
A Break statement can include a label that lets you exit embedded
loops. A label can specify any loop keyword, such as Foreach, For, or
While, in a script.
Here's a simple example
:myLabel for($i = 1; $i -le 2; $i++) {
Write-Host "Iteration: $i"
break myLabel
}
Write-Host "After for loop"
# Results:
# Iteration: 1
# After for loop
And then a more complicated example that shows the results with nested labels and breaking each one.
:outerLabel for($outer = 1; $outer -le 2; $outer++) {
:innerLabel for($inner = 1; $inner -le 2; $inner++) {
Write-Host "Outer: $outer / Inner: $inner"
#break innerLabel
#break outerLabel
}
Write-Host "After Inner Loop"
}
Write-Host "After Outer Loop"
# Both breaks commented out
# Outer: 1 / Inner: 1
# Outer: 1 / Inner: 2
# After Inner Loop
# Outer: 2 / Inner: 1
# Outer: 2 / Inner: 2
# After Inner Loop
# After Outer Loop
# break innerLabel Results
# Outer: 1 / Inner: 1
# After Inner Loop
# Outer: 2 / Inner: 1
# After Inner Loop
# After Outer Loop
# break outerLabel Results
# Outer: 1 / Inner: 1
# After Outer Loop
You can also adapt it to work in other situations by wrapping blocks of code in loops that will only execute once.
:myLabel do {
1..2 | % {
Write-Host "Iteration: $_"
break myLabel
}
} while ($false)
Write-Host "After do while loop"
# Results:
# Iteration: 1
# After do while loop
You have two options to abruptly exit out of ForEach-Object pipeline in PowerShell:
Apply exit logic in Where-Object first, then pass objects to Foreach-Object, or
(where possible) convert Foreach-Object into a standard Foreach looping construct.
Let's see examples: Following scripts exit out of Foreach-Object loop after 2nd iteration (i.e. pipeline iterates only 2 times)":
Solution-1: use Where-Object filter BEFORE Foreach-Object:
[boolean]$exit = $false;
1..10 | Where-Object {$exit -eq $false} | Foreach-Object {
if($_ -eq 2) {$exit = $true} #OR $exit = ($_ -eq 2);
$_;
}
OR
1..10 | Where-Object {$_ -le 2} | Foreach-Object {
$_;
}
Solution-2: Converted Foreach-Object into standard Foreach looping construct:
Foreach ($i in 1..10) {
if ($i -eq 3) {break;}
$i;
}
PowerShell should really provide a bit more straightforward way to exit or break out from within the body of a Foreach-Object pipeline. Note: return doesn't exit, it only skips specific iteration (similar to continue in most programming languages), here is an example of return:
Write-Host "Following will only skip one iteration (actually iterates all 10 times)";
1..10 | Foreach-Object {
if ($_ -eq 3) {return;} #skips only 3rd iteration.
$_;
}
HTH
Answer for Question #1 -
You could simply have your if statement stop being TRUE
$project.PropertyGroup | Foreach {
if(($_.GetAttribute('Condition').Trim() -eq $propertyGroupConditionName.Trim()) -and !$FinishLoop) {
$a = $project.RemoveChild($_);
Write-Host $_.GetAttribute('Condition')"has been removed.";
$FinishLoop = $true
}
};

PowerShell : Return keyword inside Foreach block not returning line of command to caller but acts as continue [duplicate]

I have the following code:
$project.PropertyGroup | Foreach-Object {
if($_.GetAttribute('Condition').Trim() -eq $propertyGroupConditionName.Trim()) {
$a = $project.RemoveChild($_);
Write-Host $_.GetAttribute('Condition')"has been removed.";
}
};
Question #1: How do I exit from ForEach-Object? I tried using "break" and "continue", but it doesn't work.
Question #2: I found that I can alter the list within a foreach loop... We can't do it like that in C#... Why does PowerShell allow us to do that?
First of all, Foreach-Object is not an actual loop and calling break in it will cancel the whole script rather than skipping to the statement after it.
Conversely, break and continue will work as you expect in an actual foreach loop.
Item #1. Putting a break within the foreach loop does exit the loop, but it does not stop the pipeline. It sounds like you want something like this:
$todo=$project.PropertyGroup
foreach ($thing in $todo){
if ($thing -eq 'some_condition'){
break
}
}
Item #2. PowerShell lets you modify an array within a foreach loop over that array, but those changes do not take effect until you exit the loop. Try running the code below for an example.
$a=1,2,3
foreach ($value in $a){
Write-Host $value
}
Write-Host $a
I can't comment on why the authors of PowerShell allowed this, but most other scripting languages (Perl, Python and shell) allow similar constructs.
There are differences between foreach and foreach-object.
A very good description you can find here: MS-ScriptingGuy
For testing in PS, here you have scripts to show the difference.
ForEach-Object:
# Omit 5.
1..10 | ForEach-Object {
if ($_ -eq 5) {return}
# if ($_ -ge 5) {return} # Omit from 5.
Write-Host $_
}
write-host "after1"
# Cancels whole script at 15, "after2" not printed.
11..20 | ForEach-Object {
if ($_ -eq 15) {continue}
Write-Host $_
}
write-host "after2"
# Cancels whole script at 25, "after3" not printed.
21..30 | ForEach-Object {
if ($_ -eq 25) {break}
Write-Host $_
}
write-host "after3"
foreach
# Ends foreach at 5.
foreach ($number1 in (1..10)) {
if ($number1 -eq 5) {break}
Write-Host "$number1"
}
write-host "after1"
# Omit 15.
foreach ($number2 in (11..20)) {
if ($number2 -eq 15) {continue}
Write-Host "$number2"
}
write-host "after2"
# Cancels whole script at 25, "after3" not printed.
foreach ($number3 in (21..30)) {
if ($number3 -eq 25) {return}
Write-Host "$number3"
}
write-host "after3"
To stop the pipeline of which ForEach-Object is part just use the statement continue inside the script block under ForEach-Object. continue behaves differently when you use it in foreach(...) {...} and in ForEach-Object {...} and this is why it's possible. If you want to carry on producing objects in the pipeline discarding some of the original objects, then the best way to do it is to filter out using Where-Object.
Since ForEach-Object is a cmdlet, break and continue will behave differently here than with the foreach keyword. Both will stop the loop but will also terminate the entire script:
break:
0..3 | foreach {
if ($_ -eq 2) { break }
$_
}
echo "Never printed"
# OUTPUT:
# 0
# 1
continue:
0..3 | foreach {
if ($_ -eq 2) { continue }
$_
}
echo "Never printed"
# OUTPUT:
# 0
# 1
So far, I have not found a "good" way to break a foreach script block without breaking the script, except "abusing" exceptions, although powershell core uses this approach:
throw:
class CustomStopUpstreamException : Exception {}
try {
0..3 | foreach {
if ($_ -eq 2) { throw [CustomStopUpstreamException]::new() }
$_
}
} catch [CustomStopUpstreamException] { }
echo "End"
# OUTPUT:
# 0
# 1
# End
The alternative (which is not always possible) would be to use the foreach keyword:
foreach:
foreach ($_ in (0..3)) {
if ($_ -eq 2) { break }
$_
}
echo "End"
# OUTPUT:
# 0
# 1
# End
If you insist on using ForEach-Object, then I would suggest adding a "break condition" like this:
$Break = $False;
1,2,3,4 | Where-Object { $Break -Eq $False } | ForEach-Object {
$Break = $_ -Eq 3;
Write-Host "Current number is $_";
}
The above code must output 1,2,3 and then skip (break before) 4. Expected output:
Current number is 1
Current number is 2
Current number is 3
Below is a suggested approach to Question #1 which I use if I wish to use the ForEach-Object cmdlet.
It does not directly answer the question because it does not EXIT the pipeline.
However, it may achieve the desired effect in Q#1.
The only drawback an amateur like myself can see is when processing large pipeline iterations.
$zStop = $false
(97..122) | Where-Object {$zStop -eq $false} | ForEach-Object {
$zNumeric = $_
$zAlpha = [char]$zNumeric
Write-Host -ForegroundColor Yellow ("{0,4} = {1}" -f ($zNumeric, $zAlpha))
if ($zAlpha -eq "m") {$zStop = $true}
}
Write-Host -ForegroundColor Green "My PSVersion = 5.1.18362.145"
I hope this is of use.
Happy New Year to all.
There is a way to break from ForEach-Object without throwing an exception. It employs a lesser-known feature of Select-Object, using the -First parameter, which actually breaks the pipeline when the specified number of pipeline items have been processed.
Simplified example:
$null = 1..5 | ForEach-Object {
# Do something...
Write-Host $_
# Evaluate "break" condition -> output $true
if( $_ -eq 2 ) { $true }
} | Select-Object -First 1 # Actually breaks the pipeline
Output:
1
2
Note that the assignment to $null is there to hide the output of $true, which is produced by the break condition. The value $true could be replaced by 42, "skip", "foobar", you name it. We just need to pipe something to Select-Object so it breaks the pipeline.
I found this question while looking for a way to have fine grained flow control to break from a specific block of code. The solution I settled on wasn't mentioned...
Using labels with the break keyword
From: about_break
A Break statement can include a label that lets you exit embedded
loops. A label can specify any loop keyword, such as Foreach, For, or
While, in a script.
Here's a simple example
:myLabel for($i = 1; $i -le 2; $i++) {
Write-Host "Iteration: $i"
break myLabel
}
Write-Host "After for loop"
# Results:
# Iteration: 1
# After for loop
And then a more complicated example that shows the results with nested labels and breaking each one.
:outerLabel for($outer = 1; $outer -le 2; $outer++) {
:innerLabel for($inner = 1; $inner -le 2; $inner++) {
Write-Host "Outer: $outer / Inner: $inner"
#break innerLabel
#break outerLabel
}
Write-Host "After Inner Loop"
}
Write-Host "After Outer Loop"
# Both breaks commented out
# Outer: 1 / Inner: 1
# Outer: 1 / Inner: 2
# After Inner Loop
# Outer: 2 / Inner: 1
# Outer: 2 / Inner: 2
# After Inner Loop
# After Outer Loop
# break innerLabel Results
# Outer: 1 / Inner: 1
# After Inner Loop
# Outer: 2 / Inner: 1
# After Inner Loop
# After Outer Loop
# break outerLabel Results
# Outer: 1 / Inner: 1
# After Outer Loop
You can also adapt it to work in other situations by wrapping blocks of code in loops that will only execute once.
:myLabel do {
1..2 | % {
Write-Host "Iteration: $_"
break myLabel
}
} while ($false)
Write-Host "After do while loop"
# Results:
# Iteration: 1
# After do while loop
You have two options to abruptly exit out of ForEach-Object pipeline in PowerShell:
Apply exit logic in Where-Object first, then pass objects to Foreach-Object, or
(where possible) convert Foreach-Object into a standard Foreach looping construct.
Let's see examples: Following scripts exit out of Foreach-Object loop after 2nd iteration (i.e. pipeline iterates only 2 times)":
Solution-1: use Where-Object filter BEFORE Foreach-Object:
[boolean]$exit = $false;
1..10 | Where-Object {$exit -eq $false} | Foreach-Object {
if($_ -eq 2) {$exit = $true} #OR $exit = ($_ -eq 2);
$_;
}
OR
1..10 | Where-Object {$_ -le 2} | Foreach-Object {
$_;
}
Solution-2: Converted Foreach-Object into standard Foreach looping construct:
Foreach ($i in 1..10) {
if ($i -eq 3) {break;}
$i;
}
PowerShell should really provide a bit more straightforward way to exit or break out from within the body of a Foreach-Object pipeline. Note: return doesn't exit, it only skips specific iteration (similar to continue in most programming languages), here is an example of return:
Write-Host "Following will only skip one iteration (actually iterates all 10 times)";
1..10 | Foreach-Object {
if ($_ -eq 3) {return;} #skips only 3rd iteration.
$_;
}
HTH
Answer for Question #1 -
You could simply have your if statement stop being TRUE
$project.PropertyGroup | Foreach {
if(($_.GetAttribute('Condition').Trim() -eq $propertyGroupConditionName.Trim()) -and !$FinishLoop) {
$a = $project.RemoveChild($_);
Write-Host $_.GetAttribute('Condition')"has been removed.";
$FinishLoop = $true
}
};

PowerShell 6 Break from ForEach-Object Loop without exiting script [duplicate]

I have the following code:
$project.PropertyGroup | Foreach-Object {
if($_.GetAttribute('Condition').Trim() -eq $propertyGroupConditionName.Trim()) {
$a = $project.RemoveChild($_);
Write-Host $_.GetAttribute('Condition')"has been removed.";
}
};
Question #1: How do I exit from ForEach-Object? I tried using "break" and "continue", but it doesn't work.
Question #2: I found that I can alter the list within a foreach loop... We can't do it like that in C#... Why does PowerShell allow us to do that?
First of all, Foreach-Object is not an actual loop and calling break in it will cancel the whole script rather than skipping to the statement after it.
Conversely, break and continue will work as you expect in an actual foreach loop.
Item #1. Putting a break within the foreach loop does exit the loop, but it does not stop the pipeline. It sounds like you want something like this:
$todo=$project.PropertyGroup
foreach ($thing in $todo){
if ($thing -eq 'some_condition'){
break
}
}
Item #2. PowerShell lets you modify an array within a foreach loop over that array, but those changes do not take effect until you exit the loop. Try running the code below for an example.
$a=1,2,3
foreach ($value in $a){
Write-Host $value
}
Write-Host $a
I can't comment on why the authors of PowerShell allowed this, but most other scripting languages (Perl, Python and shell) allow similar constructs.
There are differences between foreach and foreach-object.
A very good description you can find here: MS-ScriptingGuy
For testing in PS, here you have scripts to show the difference.
ForEach-Object:
# Omit 5.
1..10 | ForEach-Object {
if ($_ -eq 5) {return}
# if ($_ -ge 5) {return} # Omit from 5.
Write-Host $_
}
write-host "after1"
# Cancels whole script at 15, "after2" not printed.
11..20 | ForEach-Object {
if ($_ -eq 15) {continue}
Write-Host $_
}
write-host "after2"
# Cancels whole script at 25, "after3" not printed.
21..30 | ForEach-Object {
if ($_ -eq 25) {break}
Write-Host $_
}
write-host "after3"
foreach
# Ends foreach at 5.
foreach ($number1 in (1..10)) {
if ($number1 -eq 5) {break}
Write-Host "$number1"
}
write-host "after1"
# Omit 15.
foreach ($number2 in (11..20)) {
if ($number2 -eq 15) {continue}
Write-Host "$number2"
}
write-host "after2"
# Cancels whole script at 25, "after3" not printed.
foreach ($number3 in (21..30)) {
if ($number3 -eq 25) {return}
Write-Host "$number3"
}
write-host "after3"
To stop the pipeline of which ForEach-Object is part just use the statement continue inside the script block under ForEach-Object. continue behaves differently when you use it in foreach(...) {...} and in ForEach-Object {...} and this is why it's possible. If you want to carry on producing objects in the pipeline discarding some of the original objects, then the best way to do it is to filter out using Where-Object.
Since ForEach-Object is a cmdlet, break and continue will behave differently here than with the foreach keyword. Both will stop the loop but will also terminate the entire script:
break:
0..3 | foreach {
if ($_ -eq 2) { break }
$_
}
echo "Never printed"
# OUTPUT:
# 0
# 1
continue:
0..3 | foreach {
if ($_ -eq 2) { continue }
$_
}
echo "Never printed"
# OUTPUT:
# 0
# 1
So far, I have not found a "good" way to break a foreach script block without breaking the script, except "abusing" exceptions, although powershell core uses this approach:
throw:
class CustomStopUpstreamException : Exception {}
try {
0..3 | foreach {
if ($_ -eq 2) { throw [CustomStopUpstreamException]::new() }
$_
}
} catch [CustomStopUpstreamException] { }
echo "End"
# OUTPUT:
# 0
# 1
# End
The alternative (which is not always possible) would be to use the foreach keyword:
foreach:
foreach ($_ in (0..3)) {
if ($_ -eq 2) { break }
$_
}
echo "End"
# OUTPUT:
# 0
# 1
# End
If you insist on using ForEach-Object, then I would suggest adding a "break condition" like this:
$Break = $False;
1,2,3,4 | Where-Object { $Break -Eq $False } | ForEach-Object {
$Break = $_ -Eq 3;
Write-Host "Current number is $_";
}
The above code must output 1,2,3 and then skip (break before) 4. Expected output:
Current number is 1
Current number is 2
Current number is 3
Below is a suggested approach to Question #1 which I use if I wish to use the ForEach-Object cmdlet.
It does not directly answer the question because it does not EXIT the pipeline.
However, it may achieve the desired effect in Q#1.
The only drawback an amateur like myself can see is when processing large pipeline iterations.
$zStop = $false
(97..122) | Where-Object {$zStop -eq $false} | ForEach-Object {
$zNumeric = $_
$zAlpha = [char]$zNumeric
Write-Host -ForegroundColor Yellow ("{0,4} = {1}" -f ($zNumeric, $zAlpha))
if ($zAlpha -eq "m") {$zStop = $true}
}
Write-Host -ForegroundColor Green "My PSVersion = 5.1.18362.145"
I hope this is of use.
Happy New Year to all.
There is a way to break from ForEach-Object without throwing an exception. It employs a lesser-known feature of Select-Object, using the -First parameter, which actually breaks the pipeline when the specified number of pipeline items have been processed.
Simplified example:
$null = 1..5 | ForEach-Object {
# Do something...
Write-Host $_
# Evaluate "break" condition -> output $true
if( $_ -eq 2 ) { $true }
} | Select-Object -First 1 # Actually breaks the pipeline
Output:
1
2
Note that the assignment to $null is there to hide the output of $true, which is produced by the break condition. The value $true could be replaced by 42, "skip", "foobar", you name it. We just need to pipe something to Select-Object so it breaks the pipeline.
I found this question while looking for a way to have fine grained flow control to break from a specific block of code. The solution I settled on wasn't mentioned...
Using labels with the break keyword
From: about_break
A Break statement can include a label that lets you exit embedded
loops. A label can specify any loop keyword, such as Foreach, For, or
While, in a script.
Here's a simple example
:myLabel for($i = 1; $i -le 2; $i++) {
Write-Host "Iteration: $i"
break myLabel
}
Write-Host "After for loop"
# Results:
# Iteration: 1
# After for loop
And then a more complicated example that shows the results with nested labels and breaking each one.
:outerLabel for($outer = 1; $outer -le 2; $outer++) {
:innerLabel for($inner = 1; $inner -le 2; $inner++) {
Write-Host "Outer: $outer / Inner: $inner"
#break innerLabel
#break outerLabel
}
Write-Host "After Inner Loop"
}
Write-Host "After Outer Loop"
# Both breaks commented out
# Outer: 1 / Inner: 1
# Outer: 1 / Inner: 2
# After Inner Loop
# Outer: 2 / Inner: 1
# Outer: 2 / Inner: 2
# After Inner Loop
# After Outer Loop
# break innerLabel Results
# Outer: 1 / Inner: 1
# After Inner Loop
# Outer: 2 / Inner: 1
# After Inner Loop
# After Outer Loop
# break outerLabel Results
# Outer: 1 / Inner: 1
# After Outer Loop
You can also adapt it to work in other situations by wrapping blocks of code in loops that will only execute once.
:myLabel do {
1..2 | % {
Write-Host "Iteration: $_"
break myLabel
}
} while ($false)
Write-Host "After do while loop"
# Results:
# Iteration: 1
# After do while loop
You have two options to abruptly exit out of ForEach-Object pipeline in PowerShell:
Apply exit logic in Where-Object first, then pass objects to Foreach-Object, or
(where possible) convert Foreach-Object into a standard Foreach looping construct.
Let's see examples: Following scripts exit out of Foreach-Object loop after 2nd iteration (i.e. pipeline iterates only 2 times)":
Solution-1: use Where-Object filter BEFORE Foreach-Object:
[boolean]$exit = $false;
1..10 | Where-Object {$exit -eq $false} | Foreach-Object {
if($_ -eq 2) {$exit = $true} #OR $exit = ($_ -eq 2);
$_;
}
OR
1..10 | Where-Object {$_ -le 2} | Foreach-Object {
$_;
}
Solution-2: Converted Foreach-Object into standard Foreach looping construct:
Foreach ($i in 1..10) {
if ($i -eq 3) {break;}
$i;
}
PowerShell should really provide a bit more straightforward way to exit or break out from within the body of a Foreach-Object pipeline. Note: return doesn't exit, it only skips specific iteration (similar to continue in most programming languages), here is an example of return:
Write-Host "Following will only skip one iteration (actually iterates all 10 times)";
1..10 | Foreach-Object {
if ($_ -eq 3) {return;} #skips only 3rd iteration.
$_;
}
HTH
Answer for Question #1 -
You could simply have your if statement stop being TRUE
$project.PropertyGroup | Foreach {
if(($_.GetAttribute('Condition').Trim() -eq $propertyGroupConditionName.Trim()) -and !$FinishLoop) {
$a = $project.RemoveChild($_);
Write-Host $_.GetAttribute('Condition')"has been removed.";
$FinishLoop = $true
}
};

foreach loops: How to update collection variable within loop?

Is there a way to change to behaviour that the collection variable for a loop cannot be updated from within its loop and use the new values in the next iteration?
For example:
$items = #(1,1,1,2)
$counter = 0
foreach ($item in $items) {
$counter += 1
Write-Host "Iteration:" $counter " | collection variable:" $items
$item
$items = $items | Where-Object {$_ -ne $item}
}
$counter
If you run this code the loop will execute for times.
However, since with the first iteration $items is changed from 1,1,1,2 to only contain 2, the loop should only run once more.
I suspect this is because the collection variable $items is not updated in the foreach part.
Is there a way to fix this?
You cannot use a foreach loop with a collection that is being modified in the loop body.
Attempting to do so will actually result in an error (Collection was modified; enumeration operation may not execute.)
The reason you're not seeing an error is that you're not actually modifying the original collection itself; you're assigning a new collection instance to the same variable, but that has no bearing on the original collection instance being enumerated.
You should use a while loop instead, in whose condition the $items variable reference is re-evaluated in every iteration:
$items = 1, 1, 1, 2
$counter = 0
while ($items) { # Loop as long as the collection has at last 1 item.
$counter += 1
Write-Host "Iteration: $counter | collection variable: $items"
$item = $items[0] # access the 1st element
$item # output it
$items = $items | Where-Object {$_ -ne $item} # filter out all elements with the same val.
}
Now you get just 2 iterations:
Iteration: 1 | collection variable: 1 1 1 2
1
Iteration: 2 | collection variable: 2
2