Related
At [Asynchronous start][1] I had a question about starting a power-shell script asynchronously which creates a form. As answered in that question this can be solved using start-job
eg
Start-Job -ScriptBlock { test NW -NoWarning -Paranoia:2 }
So I have tried to write the Test.PS1 script routine so it re-calls itself with "Start-Job Test -NoSpawn" The switch nospawn then means it runs without a second call. I have tested this with the example code the above line now has to be and it works
Start-Job -ScriptBlock { test NW -NoSpawn -NoWarning -Paranoia:2 }
However I'm struggling to get the parameters from the original command line to passthrough to the job
I have tried creating a string in the correct format , an array , list the arguments manually , I either get repeated arguments being passed or all of the string ending up in the first Parameter $ComputerList -
A summary of the parameters and the attempts are
Param ([string]$ComputerList = 'status\edi.csv',[switch]$NoSpawn,[switch]$NoWarning,[switch]$Debug,[INT]$Paranoia=6)
...... <Snip>
Start-Job -ScriptBlock { test $ComputerList -NoSpawn -NoWarning:$NoWarning -Paranoia:$Paranoia }
Doesn't work due to scope - also switches are wrong way to do this
Start-Job -ScriptBlock { test -NoSpawn $Args } -argumentlist $ComputerList
Insufficent arguments but works - But I think One Argument is possible ?
Start-Job -ScriptBlock { test $Args -NoSpawn } -argumentlist #("-NoWarning:$NoWarning","-ComputerList:$ComputerList","-Paranoia:$Paranoia")
Everything ends up in $ComputerList
Start-Job -ScriptBlock { test $Args -NoSpawn } -argumentlist "-NoWarning:$NoWarning -ComputerList:$ComputerList -Paranoia:$Paranoia"
Everything ends up in $ComputerList
Full code follows
Param ([string]$ComputerList = 'status\edi.csv',[switch]$NoSpawn,[switch]$NoWarning,[switch]$Debug,[INT]$Paranoia=6)
$Log_Paranoia=$Paranoia
If ($Debug) { $debugPreference="Continue"} #enable debug messages if -debug is specified
If ($NoWarning) { $WarningPreference="SilentlyContinue"} #turn off warning messages
function Write-Paranoid($Level, $message) {
$CS=Get-PSCallStack
$Caller = $CS[1]
$Module = "$($Caller.FunctionName)[$($Caller.ScriptLineNumber)]"
$Diff=$level - $Log_Paranoia
$MSG= "$Module($($Level),$($Log_Paranoia)):$message"
if ($level - $Log_Paranoia -le 0 ) {
Write-host $MSG
}
if($Error.Count -gt 0 ) {
$MSG= "$Module($Level)ERROR:$Error[0]"
Write-Error $MSG
}
$error.clear()
}
Function AddStatusBar($form , $Txt) {
Write-Paranoid 10 "Enter"
$statusBar = New-Object System.Windows.Forms.StatusBar
$statusBar.DataBindings.DefaultDataSourceUpdateMode = 0
$statusBar.TabIndex = 4
$statusBar.Size = SDS 428 22
$statusBar.Location = SDP 0 337
$statusBar.Text = $Txt
$form.Controls.Add($statusBar)
$statusBar
Write-Paranoid 10 "Exit"
}
Function Create-Form ($Title)
{
Write-Paranoid 10 "Enter"
$form1 = New-Object System.Windows.Forms.Form
$form1.Text = $Title
$form1.DataBindings.DefaultDataSourceUpdateMode = 0
$form1.ClientSize = SDS 890 359
$form1.StartPosition = 0
$form1.BackColor = [System.Drawing.Color]::FromArgb(255,185,209,234)
$form1
Write-Paranoid 10 "Exit"
}
Function GenerateTestForm
{
Write-Paranoid 10 "Enter"
[reflection.assembly]::loadwithpartialname("System.Drawing") | Out-Null
[reflection.assembly]::loadwithpartialname("System.Windows.Forms") | Out-Null
$Form1 = Create-Form "Test Form"
$Alist = Get-CommandLine
$StatusBar = AddStatusBar $form1 $AList
$form1.ShowDialog() | Out-Null # Suspends calller
Write-Paranoid 10 "Exit"
}
if ($NoSpawn )
{
Write-Paranoid 3 " NoSpawn "
Write-Paranoid 5 "Call GenerateForm"
if ($Test) {
GenerateTestForm
} else {
GenerateTestForm
}
} else {
Write-Paranoid 3 "NOT NoSpawn restarting as job"
# note that test.ps1 is in the path so it will restart this script
# Start-Job -ScriptBlock { test $ComputerList -NoSpawn -NoWarning:$NoWarning -Paranoia:$Paranoia } #Wrong scope
# Start-Job -ScriptBlock { test -NoSpawn $Args } -argumentlist $ComputerList # Insufficent aruments but works - ONLY One Argument possible -
# Start-Job -ScriptBlock { test $Args -NoSpawn } -argumentlist #("-NoWarning:$NoWarning","-ComputerList:$ComputerList","-Paranoia:$Paranoia") # Everything ends up in $ComputerList
# Start-Job -ScriptBlock { test $Args -NoSpawn } -argumentlist "-NoWarning:$NoWarning -ComputerList:$ComputerList -Paranoia:$Paranoia" # Everything ends up in $ComputerList
}
Your problem can be reduced to this:
How can I re-invoke the script at hand as a background job, passing all original arguments (parameter values, including default parameter values) through?
A simplified example:
param (
[string] $ComputerList = 'status\edi.csv',
[switch] $NoSpawn,
[switch] $NoWarning,
[switch] $Debug,
[int] $Paranoia=6
)
if ($NoSpawn) { # already running as a background job.
"I'm now running in the background with the following arguments:"
$PSBoundParameters
} else { # must re-invoke via a background job
# Add *default* parameter values, if necessary, given that
# they're *not* reflected in $PSBoundParameters.
foreach ($paramName in $MyInvocation.MyCommand.Parameters.Keys) {
if (-not $PSBoundParameters.ContainsKey($paramName)) {
$defaultValue = Get-Variable -Scope Local -ValueOnly $paramName
if (-not ($null -eq $defaultValue -or ($defaultValue -is [switch] -and -not $defaultValue))) {
$PSBoundParameters[$paramName] = $defaultValue
}
}
}
# Start a background job that reinvokes this script with the original
# arguments / default values.
Start-Job {
$argsHash = $using:PSBoundParameters
& $using:PSCommandPath -NoSpawn #argsHash
} |
Receive-Job -Wait -AutoRemoveJob
}
Note:
For demonstration purposes, the initial call waits for the re-invocation via a background job to finish, using Receive-Job -Wait -AutoRemoveJob
In your real code, you can simply discard Start-Job's output (a job-information object) with $null = Start-Job { ... }, and then rely on the job getting cleaned up when the caller's session as a whole exits.
The extra code needed to propagate parameter default values is somewhat cumbersome, but necessary, given that the automatic $PSBoundParameters variable does not reflect default values.
GitHub issue #3285 discusses this limitation, and suggests a potential future solution.
I need to initialize a job using the shell. The job will be a delay plus a call to a vbScript. The following code works fine. For my example, the vbScript is just a single line with a MsgBox "Hello world!"
$functions = {
Function execute_vbs {
param ([string]$path_VBScript, [int]$secs)
Start-Sleep -Seconds $secs
cscript /nologo $path_VBScript
}
}
$seconds = 2
Start-Job -InitializationScript $functions -ScriptBlock {execute_vbs -path_VBScript 'C:\Users\[USERNAME]\Desktop\hello_world.vbs' -secs $seconds} -Name MyJob
The problem comes the moment I want to parameterize the vbScript path. (the idea is to do several different calls to some different vbScripts).
When I do this, the command seems to ignore the parameter input. I did other tests with int parameter and they work fine, the problem looks to be only with the string parameters. The following code does not work:
$functions = {
Function execute_vbs {
param ([string]$path_VBScript, [int]$secs)
Start-Sleep -Seconds $secs
cscript /nologo $path_VBScript
}
}
$input = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
$seconds = 2
Start-Job -InitializationScript $functions -ScriptBlock {execute_vbs -path_VBScript $input -secs $seconds} -Name MyJob
I've also tried using the [-ArgumentList] command, but it has the same problem.
Any idea?
The problem is that the $input and $seconds variables inside your script block are in a different scope and are effectively different variables to the ones in the main script.
I've modified your script slightly to remove the call to VBScript to make it easier to reproduce here - my example code is:
$functions = {
Function execute_vbs {
param ([string]$path_VBScript, [int]$secs)
Start-Sleep -Seconds $secs
write-output "filename = '$path_VBScript'"
write-output "secs = '$secs'"
}
}
Here's two ways to fix it:
The Using: scope modifier
See https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_scopes?view=powershell-7#the-using-scope-modifier for the full details, but basically:
For any script or command that executes out of session, you need the Using scope modifier to embed variable values from the calling session scope, so that out of session code can access them.
$filename = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
$seconds = 2
$job = Start-Job -InitializationScript $functions -ScriptBlock {
execute_vbs -path_VBScript $using:filename -secs $using:seconds
} -Name MyJob
wait-job $job
receive-job $job
# output:
# filename = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
# secs = '2'
Note the $using before the variable names inside the script block - this allows you to "inject" the variables from your main script into the scriptblock.
ScriptBlock Parameters
You can define parameters on the script block similar to how you do it with a function, and then provide the values in the -ArgumentList parameter when you invoke Start-Job.
$filename = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
$seconds = 2
$job = Start-Job -InitializationScript $functions -ScriptBlock {
param( [string] $f, [int] $s )
execute_vbs -path_VBScript $f -secs $s
} -ArgumentList #($filename, $seconds) -Name MyJob
wait-job $job
receive-job $job
# output:
# filename = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
# secs = '2'
``
When code is invoked though Invoke-Command using the -ScriptBlock, -ArgumentList and -Computer parameters, only a single item is returned from each call to the servers.
Two examples can be found below highlighting the problem.
$s = New-PSSession -ComputerName Machine01, Machine02
# when called, this block only retuns a single item from the script block
# notice that the array variable is being used
Invoke-Command -Session $s -ScriptBlock {
param( $array )
$array | % { $i = $_ ; Get-culture | select #{name='__id'; ex={$i} } , DisplayName
}
} -ArgumentList 1,2,3
write-host "`r`n======================================`r`n"
# when called, this block retuns all items from the script block
# notice that the call is the same but instead of using the array variable we use a local array
Invoke-Command -Session $s -ScriptBlock {
param( $array )
1,2,3 | % { $i = $_ ; Get-culture | select #{name='__id'; ex={$i} } , DisplayName
}
} -ArgumentList 1,2,3
$s | Remove-PSSession
Can anyone explain to me what i am doing wrong? I cant be the only person caught out by this.
-ArgumentList does what the name implies, it passes a list of arguments to the command. Each value from that list is, if possible, assigned to a defined parameter. But you only have one parameter defined: $array. Therefore, you only get the first value from the arg list.
See, this is actually how it's supposed to work (3 arguments bound to 3 parameters):
Invoke-Command -Session $s -ScriptBlock {
param ($p1, $p2, $p3)
$p1, $p2, $p3 | % { $i = $_ ; Get-culture | select #{name='__id'; ex={$i} } , DisplayName }
} -ArgumentList 1, 2, 3
So, what you actually want to do is pass one array as one single parameter.
One way to accomplish that would be:
-ArgumentList (,(1, 2, 3))
Final code:
Invoke-Command -Session $s -ScriptBlock {
param ($array)
$array | % { $i = $_ ; Get-culture | select #{n = '__id'; e = {$i}}, DisplayName }
} -ArgumentList (, (1, 2, 3))
Another way (in this simple case) would be using the automatic $args variable:
Invoke-Command -ScriptBlock {
$args | % { $i = $_ ; Get-culture | select #{n = '__id'; e = {$i}}, DisplayName }
} -ArgumentList 1, 2, 3
I'm not sure whether to call this a need for multi-threading, job-based, or async, but basically I have a Powershell script function that takes several parameters and I need to call it several times with different parameters and have these run in parallel.
Currently, I call the function like this:
Execute "param1" "param2" "param3" "param4"
How can I call this multiple times without waiting for each call to Execute return to the caller?
Currently I'm running v2.0 but I can update if necessary
EDIT: here's what I have so far, which doesn't work:
$cmd = {
param($vmxFilePath,$machineName,$username,$password,$scriptTpath,$scriptFile,$uacDismissScript,$snapshotName)
Execute $vmxFilePath $machineName $username $password $scriptTpath $scriptFile $uacDismissScript $snapshotName
}
Start-Job -ScriptBlock $cmd -ArgumentList $vmxFilePath, $machineName, $username $password, $scriptTpath, $scriptFile, $uacDismissScript, $snapshotName
I get an error:
cannot convert 'system.object[]' to the type 'system.management.automation.scriptblock' required by parameter 'initializationscript'. specified method is not supported
EDIT2: I've modified my script but I still get the error mentioned above. Here's my mod:
$cmd = {
param($vmxFilePath,$machineName,$username,$password,$scriptTpath,$scriptFile,$uacDismissScript,$snapshotName)
Execute $vmxFilePath $machineName $username $password $scriptTpath $scriptFile $uacDismissScript $snapshotName
}
Start-Job -ScriptBlock $cmd -ArgumentList $vmxFilePath, $machineName, $username $password, $scriptTpath, $scriptFile, $uacDismissScript, $snapshotName
No update necessary for this. Define a script block and use Start-Job to run the script block as many times as necessary. Example:
$cmd = {
param($a, $b)
Write-Host $a $b
}
$foo = "foo"
1..5 | ForEach-Object {
Start-Job -ScriptBlock $cmd -ArgumentList $_, $foo
}
The script block takes 2 parameters $a and $b which are passed by the -ArgumentList option. In the example above, the assignments are $_ → $a and $foo → $b. $foo is just an example for a configurable, but static parameter.
Run Get-Job | Remove-Job at some point to remove the finished jobs from the queue (or Get-Job | % { Receive-Job $_.Id; Remove-Job $_.Id } if you want to retrieve the output).
Here's a quick bogus scriptblock for the purpose of testing:
$Code = {
param ($init)
$start = Get-Date
(1..30) | % { Start-Sleep -Seconds 1; $init +=1 }
$stop = Get-Date
Write-Output "Counted from $($init - 30) until $init in $($stop - $start)."
}
This scriptblock can then be passed on to Start-Job, with for example 3 parameters (10, 15, 35)
$jobs = #()
(10,15,35) | % { $jobs += Start-Job -ArgumentList $_ -ScriptBlock $Code }
Wait-Job -Job $jobs | Out-Null
Receive-Job -Job $jobs
This creates 3 jobs, assign them to the $jobs variable, runs them in parallel and then waits for these 3 jobs to finish, and retrieves the results:
Counted from 10 until 40 in 00:00:30.0147167.
Counted from 15 until 45 in 00:00:30.0057163.
Counted from 35 until 65 in 00:00:30.0067163.
This did not take 90 seconds to execute, only 30.
One of the tricky parts is to provide -Argumentlist to Start-Job, and include a param() block inside the ScriptBlock. Otherwise, your values are never seen by the scriptblock.
You can use an alternative which may be faster than invoking jobs if the function is not a long running one. Max thread is 25 and I'm only invoking this function 10 times, so I expect my total runtime to be 5 seconds. You can wrap Measure-Command around the 'results=' statement to view stats.
Example:
$ScriptBlock = {
Param ( [int]$RunNumber )
Start-Sleep -Seconds 5
Return $RunNumber
}
$runNumbers = #(1..10)
$MaxThreads = 25
$runspacePool = [RunspaceFactory ]::CreateRunspacePool(1, $MaxThreads)
$runspacePool.Open()
$pipeLines = foreach($num in $runNumbers){
$pipeline = [powershell]::Create()
$pipeline.RunspacePool = $runspacePool
$pipeline.AddScript($ScriptBlock) | Out-Null
$pipeline.AddArgument($num) | Out-Null
$pipeline | Add-Member -MemberType NoteProperty -Name 'AsyncResult' -Value $pipeline.BeginInvoke() -PassThru
}
#obtain results as they come.
$results = foreach($pipeline in $pipeLines){
$pipeline.EndInvoke($pipeline.AsyncResult )
}
#cleanup code.
$pipeLines | % { $_.Dispose()}
$pipeLines = $null
if ( $runspacePool ) { $runspacePool.Close()}
#your results
$results
I'm sorry that everyone missed your issue - I know it is far too late now, but...
This error is caused because you are missing a comma between $username and $password in your list.
You can test it out with this snippet, which I modelled off of the previous answers:
$cmd = {
param($a, $b, $c, $d)
}
$foo = "foo"
$bar = "bar"
start-job -scriptblock $cmd -ArgumentList "a", $foo, $bar, "gold" #added missing comma for this to work
I've made a very versitile function to do this for you, and unlike the other answers you dont need to rejig your code to get it to work.
Just pass your function as a parameter to Async and pipe your input in, each item on the pipeline will run your scriptblock in parallel asynchronously, and emit them as each one is completed.
For your question specifically it would look something like this
#(
#{vmxFilePath='a';machineName='b';username='c';password='d';scriptTpath='e';scriptFile='f';uacDismissScript='g';snapshotName'h'},
#{vmxFilePath='i';machineName='j';username='k';password='l';scriptTpath='m';scriptFile='n';uacDismissScript='o';snapshotName'p'}
...
) `
| Async `
-Func { Process {
Execute $_.vmxFilePath $_.machineName $_.username $_.password $_.scriptTpath $_.scriptFile $_.uacDismissScript $_.snapshotName
} }
On top of this my function supports not only automatic construction of [powershell] (#binarySalt's answer) but also Jobs (utilised in #Joost's, but dont use these as they are much slower than runspaces) and Tasks if you're using other people's code that already spawn them (use the -AsJob flag, which I explain at the bottom of this answer).
So, that's not useful for new visitors to this question, lets do something more demonstrable that you can run on your machine and see real-world results.
Take this simple code for example, it just takes in some test data for websites and checks if they're up.
$in=TestData | ?{ $_.proto -eq 'tcp' }
$in `
| %{
$WarningPreference='SilentlyContinue'
$_ `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Test-NetConnection `
-ComputerName $_.address `
-Port $_.port `
-InformationLevel Quiet
)
} `
| Timer -name 'normal' `
| Format-Table
Here's the test data, just a good few of the same websites on repeat.
And also a timing function to see how performant it is.
Function TestData {
1..20 | %{
[PsCustomObject]#{proto='tcp' ; address='www.w3.org' ; port=443},
[PsCustomObject]#{proto='https'; address='www.w3.org' ; port=443},
[PsCustomObject]#{proto='icmp' ; address='www.w3.org' ; },
[PsCustomObject]#{proto='tcp' ; address='developer.mozilla.org' ; port=443},
[PsCustomObject]#{proto='https'; address='developer.mozilla.org' ; port=443},
[PsCustomObject]#{proto='icmp' ; address='developer.mozilla.org' ; },
[PsCustomObject]#{proto='tcp' ; address='help.dottoro.com' ; port=80 },
[PsCustomObject]#{proto='http' ; address='help.dottoro.com' ; port=80 },
[PsCustomObject]#{proto='icmp' ; address='help.dottoro.com' ; }
}
}
Function Timer {
Param ($name)
Begin {
$timer=[system.diagnostics.stopwatch]::StartNew()
}
Process { $_ }
End {
#(
$name,
' '
[math]::Floor($timer.Elapsed.TotalMinutes),
':',
($timer.Elapsed.Seconds -replace '^(.)$','0$1')
) -join '' | Out-Host
}
}
Okay, 15seconds, so how much faster can this get if we use Async?
And how much do we have to change to get it to work?
$in=TestData | ?{ $_.proto -eq 'tcp' }
$in `
| Async `
-Expected $in.Count `
-Func { Process {
$WarningPreference='SilentlyContinue'
$_ `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Test-NetConnection `
-ComputerName $_.address `
-Port $_.port `
-InformationLevel Quiet
)
} } `
| Timer -name 'async' `
| Format-Table
It looks basically identical..
Okay, what's the speed?
Wow, cut it by two thirds!
Not only that, but because we know how many items are in the pipeline I wrote in some smarts to give you a progressbar and an ETA
Don't believe me? Have a video
Or run the code yourself :)
#Requires -Version 5.1
#asynchronously run a pool of tasks,
#and aggregate the results back into a synchronous output
#without waiting to pool all input before seeing the first result
Function Async { Param(
#maximum permitted simultaneous background tasks
[int]$BatchSize=[int]$env:NUMBER_OF_PROCESSORS * 3,
#the task that accepts input on a pipe to execute in the background
[scriptblock]$Func,
#because your task is in a subshell you wont have access to your outer scope,
#you may pass them in here
[array]$ArgumentList=#(),
[System.Collections.IDictionary]$Parameters=#{},
#the title of the progress bar
[string]$Name='Processing',
#your -Func may return a [Job] instead of being backgrounded itself,
#if so it must return #(job;input;args)
#optionally job may be a [scriptblock] to be backgrounded, or a [Task]
[switch]$AsJob,
#if you know the number of tasks ahead of time,
#providing it here will have the progress bar show an ETA
[int]$Expected,
#outputs of this stream will be #(job;input) where job is the result
[switch]$PassThru,
#the time it takes to give up on one job type if there are others waiting
[int]$Retry=5
)
Begin {
$ArgumentList=[Array]::AsReadOnly($ArgumentList)
$Parameters=$Parameters.GetEnumerator() `
| &{
Begin { $params=[ordered]#{} }
Process { $params.Add($_.Key, $_.Value) }
End { $params.AsReadOnly() }
}
#the currently running background tasks
$running=#{}
$counts=[PSCustomObject]#{
completed=0;
jobs=0;
tasks=0;
results=0;
}
#a lazy attempt at uniquely IDing this instance for Write-Progress
$asyncId=Get-Random
#a timer for Write-Progress
$timer=[system.diagnostics.stopwatch]::StartNew()
$pool=[RunspaceFactory]::CreateRunspacePool(1, $BatchSize)
$pool.Open()
#called whenever we want to update the progress bar
Function Progress { Param($Reason)
#calculate ETA if applicable
$eta=-1
$total=[math]::Max(1, $counts.completed + $running.Count)
if ($Expected) {
$total=[math]::Max($total, $Expected)
if ($counts.completed) {
$eta=`
($total - $counts.completed) * `
$timer.Elapsed.TotalSeconds / `
$counts.completed
}
}
$Reason=Switch -regex ($Reason) {
'^done$' { "Finishing up the final $($running.Count) jobs." }
'^(do|next)$' { "
Running
$($running.Count)
jobs concurrently.
$(#('Adding','Waiting to add')[!($Reason -eq 'do')])
job #
$($counts.completed + $running.Count + 1)
" -replace '\r?\n\t*','' }
Default { "
Running $($running.Count) jobs concurrently.
Emitting
$($counts.completed)
$(#{1='st';2='nd';3='rd'}[$counts.completed % 10] -replace '^$','th')
result.
" -replace '\r?\n\t*','' }
}
Write-Progress `
-Id $asyncId `
-Activity $Name `
-SecondsRemaining $eta `
-Status ("
$($counts.completed)
jobs completed in
$([math]::Floor($timer.Elapsed.TotalMinutes))
:
$($timer.Elapsed.Seconds -replace '^(.)$','0$1')
" -replace '\r?\n\t*','') `
-CurrentOperation $Reason `
-PercentComplete (100 * $counts.completed / $total)
}
#called with the [Job]'s that have completed
Filter Done {
++$counts.completed
$out=$running.Item($_.Id)
$running.Remove($_.Id)
Progress
$out.job=`
if ($_ -is [System.Management.Automation.Job]) {
--$counts.jobs
$_ | Receive-Job
}
elseif ($_.pwsh) {
--$counts.results
try {
$_.pwsh.EndInvoke($_)
}
catch {
#[System.Management.Automation.MethodInvocationException]
$_.Exception.InnerException
}
finally {
$_.pwsh.Dispose()
}
}
elseif ($_.IsFaulted) {
--$counts.tasks
#[System.AggregateException]
$_.Exception.InnerException
}
else {
--$counts.tasks
$_.Result
}
if ($PassThru) {
$out
}
else {
$out.job
}
}
$isJob={
$_ -is [System.Management.Automation.Job]
}
$isTask={
$_ -is [System.Threading.Tasks.Task]
}
$isResult={
$_ -is [IAsyncResult]
}
$isFinished={
$_.IsCompleted -or `
(
$_.JobStateInfo.State -gt 1 -and
$_.JobStateInfo.State -ne 6 -and
$_.JobStateInfo.State -ne 8
)
}
$handle={
$_.AsyncWaitHandle
}
Function Jobs { Param($Filter)
$running.Values | %{ $_.job } | ? $Filter
}
#called whenever we need to wait for at least one task to completed
#outputs the completed tasks
Function Wait { Param([switch]$Finishing)
#if we are at the max background tasks this instant
while ($running.Count -ge $BatchSize) {
Progress -Reason #('done','next')[!$Finishing]
$value=#('jobs', 'tasks', 'results') `
| %{ $counts.($_) } `
| measure -Maximum -Sum
$wait=if ($value.Maximum -lt $value.Sum) {
$Retry
}
else {
-1
}
$value=Switch -exact ($value.Maximum) {
$counts.jobs {
(Wait-Job `
-Any `
-Job (Jobs -Filter $isJob) `
-Timeout $wait
).Count -lt 1
break
}
Default {
[System.Threading.WaitHandle]::WaitAny(
(Jobs -Filter $handle | % $handle),
[math]::Max($wait * 1000, -1)
) -eq [System.Threading.WaitHandle]::WaitTimeout
break
}
}
(Jobs -Filter $isFinished) | Done
}
}
}
#accepts inputs to spawn a new background task with
Process {
Wait
Progress -Reason 'do'
$run=[PSCustomObject]#{
input=$_;
job=$Func;
args=$ArgumentList;
params=$Parameters;
}
if ($AsJob) {
$run.job=$NULL
Invoke-Command `
-ScriptBlock $Func `
-ArgumentList #($run) `
| Out-Null
}
if ($run.job | % $isJob) {
++$counts.jobs
}
elseif ($run.job | % $isTask) {
++$counts.tasks
}
#if we weren't given a [Job] we need to spawn it for them
elseif ($run.job -is [ScriptBlock]) {
$pwsh=[powershell]::Create().AddScript($run.job)
$run.args | %{ $pwsh.AddArgument($_) } | Out-Null
$pwsh.RunspacePool=$pool
$run.job=$pwsh.AddParameters($run.params).BeginInvoke(
[System.Management.Automation.PSDataCollection[PSObject]]::new(
[PSObject[]]($run.input)
)
)
$run.job | Add-Member `
-MemberType NoteProperty `
-Name pwsh `
-Value $pwsh `
-PassThru `
| Add-Member `
-MemberType NoteProperty `
-Name Id `
-Value $run.job.AsyncWaitHandle.Handle.ToString()
++$counts.results
}
else {
throw "$($run.job.GetType()) needs to be a ScriptBlock"
}
$running.Add($run.job.Id, $run) | Out-Null
}
End {
#wait for the remaining running processes
$BatchSize=1
Wait -Finishing
Write-Progress -Id $asyncId -Activity $Name -Completed
$pool.Close()
$pool.Dispose()
}
}
So you may have noticed three things above, I alluded to -AsJob (using a mix of Jobs, Tasks and scriptblocks), there are unused protocols mentioned in the test data, and the video had a third test in it.
Here it is. Instead of doing the basic tcp test, using the test data we will also do a http/s check and a icmp ping (idk, maybe the https fails, but you want to narrow down if it's because the machine is down or just the service).
Test-Connection -AsJob is a cmdlet that returns a Job and checks ping.
WebRequest.GetResponseAsync() which connects to a web resource, returning a Task
And finally Test-NetConnection, which will be the same as before, ran in what we programmed as a synchronous scriptblock, which will be ran asynchronously for us.
$in=TestData
$in `
| Async `
-Expected $in.Count `
-PassThru `
-AsJob `
<#this would be accessible as a named parameter if needed#>`
-Parameters #{proxy=[System.Net.WebRequest]::GetSystemWebProxy()} `
-Func { Param([parameter(Position=0)]$x)
$x.job=Switch -regex ($x.input.proto) {
'^icmp$' {
Test-Connection `
-ComputerName $x.input.address `
-Count 1 `
-ThrottleLimit 1 `
-AsJob
}
'^tcp$' {
$x.params=#{address=$x.input.address; port=$x.input.port}
{ Param($address, $port)
$WarningPreference='SilentlyContinue'
Test-NetConnection `
-ComputerName $address `
-Port $port `
-InformationLevel Quiet
}
}
'^(http|https)$' {
[Net.ServicePointManager]::SecurityProtocol=[Net.SecurityProtocolType]::Tls12
$request=[System.Net.HttpWebRequest]::Create((#(
$x.input.proto,
'://',
$x.input.address,
':',
$x.input.port
) -join ''))
$request.Proxy=$NULL
$request.Method='Get'
$request.GetResponseAsync()
}
}
} `
| %{
$result=$_
$result.input `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Switch -regex (#($result.input.proto, $result.job.message)[$result.job -is [Exception]]) {
#[Win32_PingStatus]
'^icmp$' { $result.job.StatusCode -eq 0 }
#[bool]
'^tcp$' { $result.job }
#[System.Net.HttpWebResponse]
'^(http|https)$' {
$result.job.Close()
Switch ($result.job.StatusCode.value__) {
{ $_ -ge 200 -and $_ -lt 400 } { $True }
Default {$False}
}
}
#[Exception]
Default { $False }
})
} `
| Timer -name 'async asjob' `
| Format-Table
As you may have seen, this did more than double the work of the original code, but still finished in around half the time at 8seconds.
How do I Start a job of a function i just defined?
function FOO { write-host "HEY" } Start-Job -ScriptBlock { FOO } |
Receive-Job
Receive-Job: The term 'FOO' is not recognized as the name of cmdlet,
function ,script file or operable program.
What do I do?
Thanks.
As #Shay points out, FOO needs to be defined for the job. Another way to do this is to use the -InitializationScript parameter to prepare the session.
For your example:
$functions = {
function FOO { write-host "HEY" }
}
Start-Job -InitializationScript $functions -ScriptBlock {FOO}|
Wait-Job| Receive-Job
This can be useful if you want to use the same functions for different jobs.
#Rynant's suggestion of InitializationScript is great
I thought the purpose of (script) blocks is so that you can pass them around. So depending on how you are doing it, I would say go for:
$FOO = {write-host "HEY"}
Start-Job -ScriptBlock $FOO | wait-job |Receive-Job
Of course you can parameterize script blocks as well:
$foo = {param($bar) write-host $bar}
Start-Job -ScriptBlock $foo -ArgumentList "HEY" | wait-job | receive-job
It worked for me as:
Start-Job -ScriptBlock ${Function:FOO}
An improvement to #Rynant's answer:
You can define the function as normal in the main body of your script:
Function FOO
{
Write-Host "HEY"
}
and then recycle this definition within a scriptblock:
$export_functions = [scriptblock]::Create(#"
Function Foo { $function:FOO }
"#)
(makes more sense if you have a substantial function body) and then pass them to Start-Job as above:
Start-Job -ScriptBlock {FOO} -InitializationScript $export_functions| Wait-Job | Receive-Job
I like this way, as it is easier to debug jobs by running them locally under the debugger.
The function needs to be inside the scriptblock:
Start-Job -ScriptBlock { function FOO { write-host "HEY" } ; FOO } | Wait-Job | Receive-Job
A slightly different take. A function is just a scriptblock assigned to a variable. Oh, it has to be a threadjob. It can't be foreach-object -parallel.
$func = { 'hi' } # or
function hi { 'hi' }; $func = $function:hi
start-threadjob { & $using:func } | receive-job -auto -wait
hi
#Ben Power's comment under the accepted answer was my concern also, so I googled how to get function definitions, and I found Get-Command - though this gets only the function body. But it can be used also if the function is coming from elsewhere, like a dot-sourced file. So I came up with the following (hold my naming convention :)), the idea is to re-build the function definitions delimited by newlines:
Filter Greeting {param ([string]$Greeting) return $Greeting}
Filter FullName {param ([string]$FirstName, [string]$LastName) return $FirstName + " " + $LastName}
$ScriptText = ""
$ScriptText += "Filter Greeting {" + (Get-Command Greeting).Definition + "}`n"
$ScriptText += "Filter FullName {" + (Get-Command FullName).Definition + "}`n"
$Job = Start-Job `
-InitializationScript $([ScriptBlock]::Create($ScriptText)) `
-ScriptBlock {(Greeting -Greeting "Hello") + " " + (FullName -FirstName "PowerShell" -LastName "Programmer")}
$Result = $Job | Wait-Job | Receive-Job
$Result
$Job | Remove-Job
As long as the function passed to the InitializationScript param on Start-Job isn't large Rynant's answer will work, but if the function is large you may run into the below error.
[localhost] There is an error launching the background process. Error
reported: The filename or extension is too long"
Capturing the function's definition and then using Invoke-Expression on it in the ScriptBlock is a better alternative.
function Get-Foo {
param
(
[string]$output
)
Write-Output $output
}
$getFooFunc = $(Get-Command Get-Foo).Definition
Start-Job -ScriptBlock {
Invoke-Expression "function Get-Foo {$using:getFooFunc}"
Get-Foo -output "bar"
}
Get-Job | Receive-Job
PS C:\Users\rohopkin> Get-Job | Receive-Job
bar