How to Execute Powershell Pipeline Asynchronously - powershell

Question

How to make Powershell execute pipelines asynchronously instead of 'sequentially'?
An example to illustrate this:
Function PipeDelay($Name, $Miliseconds){
Process {
Sleep -miliseconds $Miliseconds
Write-Host "$Name : $_"
$_
}
}
1..7 | PipeDelay Fast 100 | PipeDelay Slow 300 | PipeDelay Slowest 900
The output in both ISE and powershell are the following:
Fast : 1
Slow : 1
Slowest : 1
1
Fast : 2
Slow : 2
Slowest : 2
2
Fast : 3
Slow : 3
Slowest : 3
3
...
It is as if Powershell sequentially run the whole pipeline for one input object before processing next input object. The execution looks something like this :
Fast : 1 2 3
Slow : 1---1 2---2 3---3
Slowest : 1---------------1 2---------------2 3-----...
Is it possible with some settings/environment variables/etc. to make pipelines to run independently/asynchronously? perhaps with setting about pipeline buffer size, etc.? So that the execution will looks something like this:
Fast : 1 2 3 4 5 6 7
Slow : 1---1 2---2 3---3 4---4 5---5 6---6 7---7
Slowest : 1---------------1 2---------------2 3-----...
NOTE
I thought it is because of STA/MTA mode. I don't understand them completely but same result in ISE (STA) / Powershell Shell (MTA) seems to eliminate STA/MTA mode as the cause.
Also, I thought Write-Host is the issue that force pipeline to be processed sequentially, but even if I substitute Write-Host with New-Event, the sequential processing still applies.

Answer 1

I don't think you will be able to leverage the pipeline in an asynchronous manner in the way you desire without being quiet expensive with resource usage. I tried to capture the spirit of what you are trying to accomplish, but in a different way. I used a slightly different example to illustrate how [Automation.PowerShell] Async works.
#1. You have a list of room requests you want to process in a function. TimeToClean was added as a controllable thread block.
$roomsToClean = #( ([psCustomObject]#{Name='Bedroom';TimeToClean=2}),
([psCustomObject]#{Name='Kitchen';TimeToClean=5}),
([psCustomObject]#{Name='Bathroom';TimeToClean=3}),
([psCustomObject]#{Name='Living room';TimeToClean=1}),
([psCustomObject]#{Name='Dining room';TimeToClean=1}),
([psCustomObject]#{Name='Foyier';TimeToClean=1})
)
#2. We will clean three rooms and return a custom PowerShell object with a message.
Function Clean-Room{
param([string]$RoomName,[int]$Seconds)
Sleep -Seconds $Seconds
Write-Output [psCustomObject] #{Message= "The robot cleaned the $RoomName in $Seconds seconds."}
}
#3. Executing this list synchronously will result in an approximate 13 second runtime.
Write-Host "===== Synchronous Results =====" -ForegroundColor Green
$stopwatch = [system.diagnostics.stopwatch]::StartNew()
foreach($item in $roomsToClean){
$obj = Clean-Room $item.Name $item.TimeToClean
Write-Output $obj.Message
}
$stopwatch.Stop()
Write-Host "Execution time for synchronous function was $($stopwatch.Elapsed)." -ForegroundColor Green
#4. Now let's run this function asynchronously for all of these items. Expected runtime will be approximately 5 seconds.
#=============== Setting up an ansynchronous powerShell Automation object and attaching it to a runspace.
#Many [Automation.PowerShell] objects can be attached to a given Runspace pool and the Runspace pool will manage queueing/dequeueing of each PS object as it completes.
#Create a RunSpace pool with 5 runspaces. The pool will manage the
#Many PowerShell autom
$minRunSpaces = 2
$maxRunsSpaces = 5
$runspacePool = [RunspaceFactory]::CreateRunspacePool($minRunSpaces, $maxRunsSpaces)
$runspacePool.ApartmentState = 'STA' #MTA = Multithreaded apartment #STA = Singl-threaded apartment.
$runspacePool.Open() #runspace pool must be opened before it can be used.
#For each room object, create an [Automation.PowerShell] object and attach it to the runspace pool.
#Asynchronously invoke the function for all objects in the collection.
$ps_collections = foreach($room in $roomsToClean){
try{
$ps = [System.Management.Automation.PowerShell]::Create()
$ps.RunspacePool = $runspacePool
#Add your custom functions to the [Automation.PowerShell] object.
#Add argument with parameter name for readability. You may just use AddArgument as an alternative but know your positional arguments.
[void] $ps.AddScript(${function:Clean-Room})
[void] $ps.AddParameter('RoomName',$room.Name) #Add parameterName,value
[void] $ps.AddParameter('Seconds',$room.TimeToClean) #Add parameterName,value
#extend the ps management object to include AsyncResult and attach the AsyncResult object for receiving results at a later time.
$ps | Add-Member -MemberType NoteProperty -Name 'AsyncResult' -Value $ps.BeginInvoke() #invoke asynchronously
$ps | Add-Member -MemberType ScriptMethod -Name 'GetAsyncResult' -Value {$this.EndInvoke($this.AsyncResult) } -PassThru
}
catch{
throw $_ #handle custom error here.
}
}
#After the function has been asynchronously called for all room objects, Grab results from asynchronous function calls.
Write-Host "===== Asynchronous Results =====" -ForegroundColor Green
$stopwatch = [system.diagnostics.stopwatch]::StartNew()
foreach($ps in $ps_collections){
$obj = $ps.GetAsyncResult()
[void] $ps.Dispose() #dispose of object after use.
Write-Output $obj.Message
}
$stopwatch.Stop()
Write-Host "Execution time for asynchronous function was
$($stopwatch.Elapsed)." -ForegroundColor Green
#Runspace cleanup.
If($runspacePool){
[void] $runspacePool.Close()
[void] $runspacePool.Dispose()
}
Result times will vary slightly but should look similar to this:
===== Synchronous Results =====
The robot cleaned the Bedroom in 2 seconds.
The robot cleaned the Kitchen in 5 seconds.
The robot cleaned the Bathroom in 3 seconds.
The robot cleaned the Living room in 1 seconds.
The robot cleaned the Dining room in 1 seconds.
The robot cleaned the Foyier in 1 seconds.
Execution time for synchronous function was 00:00:13.0719157.
####===== Asynchronous Results =====
The robot cleaned the Bedroom in 2 seconds.
The robot cleaned the Kitchen in 5 seconds.
The robot cleaned the Bathroom in 3 seconds.
The robot cleaned the Living room in 1 seconds.
The robot cleaned the Dining room in 1 seconds.
The robot cleaned the Foyier in 1 seconds.
Execution time for asynchronous function was 00:00:04.9909951.