I'm using the below code to display the results of PowerShell Jobs with a timeout of 120 seconds. I would like to enhance this code by incorporating Write-Progress (based on number of jobs completed). I tried using this example as a reference, however, when I try to incorporate that code, the progress bar displays briefly after all the jobs are all done already.
$Jobs = #()
$ForceStoppedIds = #{}
$Jobs += Get-Job
$Jobs | Wait-Job -Timeout 120 | Out-Null
$Jobs | ?{$_.State -eq 'Running'} | Stop-Job -PassThru | %{$ForceStoppedIds[$_.Id] = $true}
foreach ($Job in $Jobs) {
$Name = $Job.Name
$Output = (Get-Job -Name $Name | Receive-Job)
if ($ForceStoppedIds.Contains($Job.Id)) {
Write-Output "$($Name) - Device unable to process request within 2 minutes"
} else {
Write-Output $Output
}
}
Wait-Job -Timeout 120 will block the thread until the specified timeout or all jobs have completed, hence, is not possible to display progress and wait for them at the same time.
There are 2 alternatives that I can think of, the first one would be to create a proxy command / proxy function around this cmdlet to extend it's functionality.
These blogs demonstrate how to do it:
https://devblogs.microsoft.com/scripting/proxy-functions-spice-up-your-powershell-core-cmdlets/
https://devblogs.microsoft.com/powershell/extending-andor-modifing-commands-with-proxies/
proxy function
You can also follow the indications from this helpful answer.
The other alternative is to define your own function that does a similar work as Wait-Job but, instead of blocking the thread, you can add a loop that will run based on 2 conditions:
That the elapsed time is lower than or equal to the Timeout we passed as argument to the function (we can use Diagnostics.Stopwatch for this).
And, that the jobs are still Running (the $jobs List<T> is still populated).
Note, the function below should work in most cases however is purely for demonstration purposes only and should not be relied upon.
First we define a new function that can be used to display progress as well as wait for our jobs based on a timeout:
using namespace System.Collections.Generic
using namespace System.Diagnostics
using namespace System.Threading
using namespace System.Management.Automation
function Wait-JobWithProgress {
[cmdletbinding()]
param(
[parameter(Mandatory, ValueFromPipeline)]
[object[]] $InputObject,
[parameter()]
[double] $TimeOut
)
begin {
$jobs = [List[object]]::new()
}
process {
foreach($job in $InputObject) {
$jobs.Add($job)
}
}
end {
$timer = [Stopwatch]::StartNew()
$total = $jobs.Count
$completed = 0.1
$expression = { $true }
if($PSBoundParameters.ContainsKey('TimeOut')) {
$expression = { $timer.Elapsed.TotalSeconds -le $TimeOut }
}
while((& $expression) -and $jobs) {
$remaining = $total - $completed
$average = $timer.Elapsed.TotalSeconds / $completed
$estimate = [math]::Round($remaining * $average)
$status = 'Completed Jobs: {0:0} of {1} - ETC: {2}s' -f $completed, $total, $estimate
$progress = #{
Activity = 'Waiting for Jobs'
PercentComplete = $completed / $total * 100
Status = $status
}
Write-Progress #progress
$id = [WaitHandle]::WaitAny($jobs.Finished, 200)
if($id -eq [WaitHandle]::WaitTimeout) {
continue
}
# output this job
$jobs[$id]
# remove this job
$jobs.RemoveAt($id)
$completed++
}
# Stop the jobs not yet Completed and remove them
$jobs | Stop-Job -PassThru | ForEach-Object {
Remove-Job -Job $_
"Job [#{0} - {1}] did not complete on time and was removed." -f $_.Id, $_.Name
} | Write-Warning
Write-Progress #progress -Completed
}
}
Then for testing it, we can create a few jobs with a random timer:
0..10 | ForEach-Object {
Start-Job {
Start-Sleep (Get-Random -Minimum 5 -Maximum 15)
[pscustomobject]#{
Job = $using:_
Result = 'Hello from [Job #{0:D2}]' -f $using:_
}
}
} | Wait-JobWithProgress -TimeOut 10 |
Receive-Job -AutoRemoveJob -Wait | Format-Table -AutoSize
i got a question to my existing script.
the script does exactly what it should to do
This is what i got.
<#
Ping mass huge amount of hosts in parallel multithreading
#>
function Parallel-Ping {
[CmdletBinding()]
param(
[parameter(mandatory=$true)][ValidateNotNullOrEmpty()][string[]]$hosts,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][int]$timeout = 2000,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][ValidateRange(1,255)][int]$ttl = 128,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][bool]$DontFragment = $false,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][byte[]]$data = ([byte[]]0),
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][int]$MaxConcurrent = [System.Environment]::ProcessorCount,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][switch]$showprogress
)
begin{
$rspool = [runspacefactory]::CreateRunspacePool(1,$MaxConcurrent)
$rspool.ApartmentState = 'STA'
$rspool.Open()
$jobs = New-Object System.Collections.ArrayList
}
process{
foreach ($hostname in $hosts){
$ps = [Powershell]::Create()
$ps.RunspacePool = $rspool
[void]$ps.AddScript({
param($hostname,$timeout,$ttl,$DontFragment,$data)
$result = [ordered]#{Host=$hostname;Address=$null;Status=[System.Net.NetworkInformation.IPStatus]::Unknown;RoundtripTime = -1;TTL=$ttl;DontFragment=$DontFragment;ResponseBuffer='';Time=(get-date)}
$ping = New-Object System.Net.NetworkInformation.Ping
try{
$reply = $ping.Send($hostname,$timeout,$data,(New-Object System.Net.NetworkInformation.PingOptions($ttl,$DontFragment)))
$result.RoundtripTime = $reply.RoundtripTime
$result.Status = $reply.Status
$result.Address = $reply.Address
$result.ResponseBuffer = $reply.Buffer
}catch [System.Net.NetworkInformation.PingException] {
$result.Status = [System.Net.NetworkInformation.IPStatus]::DestinationHostUnreachable
}finally{
$ping.Dispose()
}
[pscustomobject]$result
}).AddParameters(#($hostname,$timeout,$ttl,$DontFragment,$data))
$job = $ps.BeginInvoke()
[void]$jobs.Add(([pscustomobject]#{Handle = $job; Powershell = $ps}))
}
}
end{
write-verbose "Waiting for all jobs to complete."
while(($jobs | ?{!$_.Handle.IsCompleted})){
if ($showprogress.IsPresent){
$completed = ($jobs | ?{$_.Handle.IsCompleted}).Count
Write-Progress -Activity $PSCmdlet.MyInvocation.InvocationName -Status "Pinging a total of $($hosts.Count) hosts." -PercentComplete (($completed / $hosts.Count) * 100) -CurrentOperation "Completed $completed from $($hosts.Count)."
}
}
# get results of jobs
$results = $jobs | %{
$_.Powershell.EndInvoke($_.handle)
$_.Powershell.Dispose()
}
# cleanup
$rspool.Close();$rspool.Dispose()
$results
}
}
$hosts = Get-Content 'E:\ips.txt'
Parallel-Ping -hosts $hosts -timeout 500 -MaxConcurrent 50 -showprogress | ogv
I would like to mention another function in my script.
My goal is it, to add the function that i can see the actual logged in user to the pinged machines?
Unfortunately i dont know where to add.
$fullnamexp = ((net user $winxp /domain | Select-String "Full Name") -replace "Full Name","").Trim();
If $winxp cannot be found, the command will hang, is there a timeout I can use with this to make it move on after 5-10 seconds? Not sure where I would put it.
Edit- I use this to pull the username:
$reg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey('LocalMachine', $tag1)
$key = $reg.OpenSubKey('SOFTWARE\Microsoft\Windows NT\CurrentVersion\WinLogon')
$winxp = $key.GetValue('DefaultUserName') -replace '^.*?\\'
$winxp is then a login name such as ajstepanik then I put it into: $fullnamexp = ((net user $winxp /domain | Select-String "Full Name") -replace "Full Name","").Trim();
1.21.2014 Update
$timeoutSeconds = 5
$code = {
((net user $winxp /domain | Select-String "Full Name") -replace "Full Name","").Trim(); # your commands here, e.g.
}
$j = Start-Job -ScriptBlock $code
if (Wait-Job $j -Timeout $timeoutSeconds) { $fullnamexp = Receive-Job $j }
Remove-Job -force $j
While #mjolinor may have indeed provided you an alternative approach, here is a direct answer to your general question: how do you force a timeout in PowerShell?
Wrap whatever you wish to time-limit in a script block, run that as a job, then use the Wait-Job cmdlet to time-limit the operation. Wait-Job will return either at the end of the timeout period or when the script block completes, whichever occurs first. After Wait-Job returns, you can examine the job state ($j.state) to determine whether it was interrupted or not, if it matters to you.
$timeoutSeconds = 5 # set your timeout value here
$j = Start-Job -ScriptBlock {
# your commands here, e.g.
Get-Process
}
"job id = " + $j.id # report the job id as a diagnostic only
Wait-Job $j -Timeout $timeoutSeconds | out-null
if ($j.State -eq "Completed") { "done!" }
elseif ($j.State -eq "Running") { "interrupted" }
else { "???" }
Remove-Job -force $j #cleanup
2014.01.18 Update
Here is a bit more streamlining approach that also includes the practical step of getting information out of the script block with Receive-Job, assuming what you want is generated on stdout:
$timeoutSeconds = 3
$code = {
# your commands here, e.g.
Get-ChildItem *.cs | select name
}
$j = Start-Job -ScriptBlock $code
if (Wait-Job $j -Timeout $timeoutSeconds) { Receive-Job $j }
Remove-Job -force $j
You can use Start-Sleep to pause the script:
Start-Sleep -s 5
net doesn't explicitly allow you to set a time out on it's operations, but you could check out this link on changing the ipv4 timeout for your sockets:
http://www.cyberciti.biz/tips/linux-increasing-or-decreasing-tcp-sockets-timeouts.html
The only thing else I could imagine is spawning a worker thread but I don't even know if that's possible in bash, I'm not fluid enough in it to answer that; plus it opens you up to sync problems and all sorts of multi threaded issues beyond what you're trying to accomplish quickly in a bash script to begin with! :P
Does this help?
$query = (dsquery user -samid $winxp)
if ($query) {$fullnamexp = ($query | dsget user -display)[1].trim()}
$fullnamexp
This solution doesn't work for me. remove-job -force $j takes over 5 seconds in this example.
$timeoutseconds = 1
$start = get-date
$j = start-job -scriptblock { Resolve-DnsName 1.1.1.1 }
if (wait-job $j -timeout $timeoutseconds) { $fullnamexp = receive-job $j }
remove-job -force $j
(get-date) - $start
Days : 0
Hours : 0
Minutes : 0
Seconds : 5
Milliseconds : 342
Ticks : 53426422
TotalDays : 6.18361365740741E-05
TotalHours : 0.00148406727777778
TotalMinutes : 0.0890440366666667
TotalSeconds : 5.3426422
TotalMilliseconds : 5342.6422
Here's a simple timeout example with notepad:
notepad
if (-not $(wait-process notepad 10; $?)) { stop-process -name notepad }
$watchdog = 10 #seconds
$start_time = Get-Date
$j = Start-Job -ScriptBlock{
#timeout command
if ($true) {
$i = 0
while($true) {
Write-Host "Count: $i"
Start-Sleep -Milliseconds 100
$i++
}
}
write-host "Hello"
}
while($true) {
if ($j.HasMoreData) {
Receive-Job $j
Start-Sleep -Milliseconds 200
}
$current = Get-Date
$time_span = $current - $start_time
if ($time_span.TotalSeconds -gt $watchdog) {
write-host "TIMEOUT!"
Stop-Job $j
break
}
if (-not $j.HasMoreData -and $j.State -ne 'Running') {
write-host "Finished"
break
}
}
Remove-Job $j
So I have written a simple pinger that ping all that a network. I'm using test-connection, but I what to also get the devises name as well.
So I'm using cmd's nbtstat to get the name but (lets say it's not clean)
Is there a way to do this a bit cleaner?
this is what i have got
$a = New-Object System.Collections.ArrayList
for ($i =0;$i -le 225; $i++){
if (Test-Connection 192.168.1.$i -Count 1 -Quiet){
echo "live 192.168.1.$i"
$a.add("192.168.1.$i")
}else {
echo "dead 192.168.1.$i"
}
}
echo $a
foreach ($i in $a){
Test-Connection $i -Count 1
}
foreach ($i in $a){
nbtstat -a $i
}
There are always many ways to do something, one such example:
$Pinger = New-Object system.Net.NetworkInformation.Ping
$PC = "192.168.1.$i"
try{
$ErrorActionPreference = "Stop"
$PingResult = $Pinger.send($PC)
$ResultAddress = $PingResult.Address
$PingStatus = $PingResult.Status
$DNSObject = [System.Net.Dns]::GetHostEntry($PC).HostName
}
catch
{
}
finally
{
Write-Host "$PC $PingStatus $PingHost $? $DNSObject"
}
This gives more than you requested, but also I think it might also give you some ideas:
.NET is at your disposal, not just native cmdlets and Windows Commands
Do use built in operators like $?
I'd do it a bit differently. Every time you contact a host, an ARP entry is created. Why not leverage that?
Get-WmiObject Win32_PingStatus -Filter "Address='192.168.1.2' AND ResolveAddressNames='true'" |
Select IPV4Address, ProtocolAddressResolved, #{
Name="PhysicalAddress";Expression={
([regex]::Matches($(arp -a $_.IPV4Address),"[0-9A-F]{2}-[0-9A-F]{2}-[0-9A-F]{2}-[0-9A-F]{2}-[0-9A-F]{2}-[0-9A-F]{2}",#('MultiLine', 'Ignorecase'))).value
}
}
Test-Connection and Test-NetConnection now support returning more information, including names. The latter requires Power Shell 5.
Test-Connection
EXAMPLES
Example 1: Send echo requests to a remote computer
PS C:\> Test-Connection "Server01"
Source Destination IPV4Address IPV6Address Bytes Time(ms)
------ ----------- ----------- ----------- ----- --------
ADMIN1 Server01 10.59.137.44 32 0
ADMIN1 Server01 10.59.137.44 32 0
ADMIN1 Server01 10.59.137.44 32 0
ADMIN1 Server01 10.59.137.44 32 1
Source
Test-NetConnection
Test ping connectivity with detailed results
PS C:\> Test-NetConnection -InformationLevel "Detailed"
ComputerName : internetbeacon.msedge.net
RemoteAddress : 2a01:111:2003::52
NameResolutionResults : 2a01:111:2003::52
13.107.4.52
InterfaceAlias : Ethernet
SourceAddress : 2001:4898:d8:33:81e8:7b49:8bf5:8710
NetRoute (NextHop) : fe80::200:5eff:fe00:203
PingSucceeded : True
PingReplyDetails (RTT) : 6 ms
Source
Also, you mention that you are pinging a whole network. That's how found your question (among others), because I was trying to do a ping sweep too. I settles on this script. The guy calls it Fast Ping Sweep Asynchronous.
Even being a Power Shell n00b, I was able to pipe its output, and then modify its output to only include what I wanted. I came across other scripts, but could not decipher their scripts to modify them for my purposes.
I'm not sure what version Power Shell this requires, but it works on v4 and v5.
I have seen most of the Powershell IP scanner, ping sweeping scripts
but none of them uses the PingASync method.The "problem" with
synchronous scripts is that they have to wait until a node replies or
times out before continuing to the next address.Using this approach
may take s
function Global:Ping-IPRange {
<#
.SYNOPSIS
Sends ICMP echo request packets to a range of IPv4 addresses between two given addresses.
.DESCRIPTION
This function lets you sends ICMP echo request packets ("pings") to
a range of IPv4 addresses using an asynchronous method.
Therefore this technique is very fast but comes with a warning.
Ping sweeping a large subnet or network with many swithes may result in
a peak of broadcast traffic.
Use the -Interval parameter to adjust the time between each ping request.
For example, an interval of 60 milliseconds is suitable for wireless networks.
The RawOutput parameter switches the output to an unformated
[System.Net.NetworkInformation.PingReply[]].
.INPUTS
None
You cannot pipe input to this funcion.
.OUTPUTS
The function only returns output from successful pings.
Type: System.Net.NetworkInformation.PingReply
The RawOutput parameter switches the output to an unformated
[System.Net.NetworkInformation.PingReply[]].
.NOTES
Author : G.A.F.F. Jakobs
Created : August 30, 2014
Version : 6
.EXAMPLE
Ping-IPRange -StartAddress 192.168.1.1 -EndAddress 192.168.1.254 -Interval 20
IPAddress Bytes Ttl ResponseTime
--------- ----- --- ------------
192.168.1.41 32 64 371
192.168.1.57 32 128 0
192.168.1.64 32 128 1
192.168.1.63 32 64 88
192.168.1.254 32 64 0
In this example all the ip addresses between 192.168.1.1 and 192.168.1.254 are pinged using
a 20 millisecond interval between each request.
All the addresses that reply the ping request are listed.
.LINK
http://gallery.technet.microsoft.com/Fast-asynchronous-ping-IP-d0a5cf0e
#>
[CmdletBinding(ConfirmImpact='Low')]
Param(
[parameter(Mandatory = $true, Position = 0)]
[System.Net.IPAddress]$StartAddress,
[parameter(Mandatory = $true, Position = 1)]
[System.Net.IPAddress]$EndAddress,
[int]$Interval = 30,
[Switch]$RawOutput = $false
)
$timeout = 2000
function New-Range ($start, $end) {
[byte[]]$BySt = $start.GetAddressBytes()
[Array]::Reverse($BySt)
[byte[]]$ByEn = $end.GetAddressBytes()
[Array]::Reverse($ByEn)
$i1 = [System.BitConverter]::ToUInt32($BySt,0)
$i2 = [System.BitConverter]::ToUInt32($ByEn,0)
for($x = $i1;$x -le $i2;$x++){
$ip = ([System.Net.IPAddress]$x).GetAddressBytes()
[Array]::Reverse($ip)
[System.Net.IPAddress]::Parse($($ip -join '.'))
}
}
$IPrange = New-Range $StartAddress $EndAddress
$IpTotal = $IPrange.Count
Get-Event -SourceIdentifier "ID-Ping*" | Remove-Event
Get-EventSubscriber -SourceIdentifier "ID-Ping*" | Unregister-Event
$IPrange | foreach{
[string]$VarName = "Ping_" + $_.Address
New-Variable -Name $VarName -Value (New-Object System.Net.NetworkInformation.Ping)
Register-ObjectEvent -InputObject (Get-Variable $VarName -ValueOnly) -EventName PingCompleted -SourceIdentifier "ID-$VarName"
(Get-Variable $VarName -ValueOnly).SendAsync($_,$timeout,$VarName)
Remove-Variable $VarName
try{
$pending = (Get-Event -SourceIdentifier "ID-Ping*").Count
}catch [System.InvalidOperationException]{}
$index = [array]::indexof($IPrange,$_)
Write-Progress -Activity "Sending ping to" -Id 1 -status $_.IPAddressToString -PercentComplete (($index / $IpTotal) * 100)
Write-Progress -Activity "ICMP requests pending" -Id 2 -ParentId 1 -Status ($index - $pending) -PercentComplete (($index - $pending)/$IpTotal * 100)
Start-Sleep -Milliseconds $Interval
}
Write-Progress -Activity "Done sending ping requests" -Id 1 -Status 'Waiting' -PercentComplete 100
While($pending -lt $IpTotal){
Wait-Event -SourceIdentifier "ID-Ping*" | Out-Null
Start-Sleep -Milliseconds 10
$pending = (Get-Event -SourceIdentifier "ID-Ping*").Count
Write-Progress -Activity "ICMP requests pending" -Id 2 -ParentId 1 -Status ($IpTotal - $pending) -PercentComplete (($IpTotal - $pending)/$IpTotal * 100)
}
if($RawOutput){
$Reply = Get-Event -SourceIdentifier "ID-Ping*" | ForEach {
If($_.SourceEventArgs.Reply.Status -eq "Success"){
$_.SourceEventArgs.Reply
}
Unregister-Event $_.SourceIdentifier
Remove-Event $_.SourceIdentifier
}
}else{
$Reply = Get-Event -SourceIdentifier "ID-Ping*" | ForEach {
If($_.SourceEventArgs.Reply.Status -eq "Success"){
$_.SourceEventArgs.Reply | select #{
Name="IPAddress" ; Expression={$_.Address}},
#{Name="Bytes" ; Expression={$_.Buffer.Length}},
#{Name="Ttl" ; Expression={$_.Options.Ttl}},
#{Name="ResponseTime"; Expression={$_.RoundtripTime}}
}
Unregister-Event $_.SourceIdentifier
Remove-Event $_.SourceIdentifier
}
}
if($Reply -eq $Null){
Write-Verbose "Ping-IPrange : No ip address responded" -Verbose
}
return $Reply
}
I'm not sure whether to call this a need for multi-threading, job-based, or async, but basically I have a Powershell script function that takes several parameters and I need to call it several times with different parameters and have these run in parallel.
Currently, I call the function like this:
Execute "param1" "param2" "param3" "param4"
How can I call this multiple times without waiting for each call to Execute return to the caller?
Currently I'm running v2.0 but I can update if necessary
EDIT: here's what I have so far, which doesn't work:
$cmd = {
param($vmxFilePath,$machineName,$username,$password,$scriptTpath,$scriptFile,$uacDismissScript,$snapshotName)
Execute $vmxFilePath $machineName $username $password $scriptTpath $scriptFile $uacDismissScript $snapshotName
}
Start-Job -ScriptBlock $cmd -ArgumentList $vmxFilePath, $machineName, $username $password, $scriptTpath, $scriptFile, $uacDismissScript, $snapshotName
I get an error:
cannot convert 'system.object[]' to the type 'system.management.automation.scriptblock' required by parameter 'initializationscript'. specified method is not supported
EDIT2: I've modified my script but I still get the error mentioned above. Here's my mod:
$cmd = {
param($vmxFilePath,$machineName,$username,$password,$scriptTpath,$scriptFile,$uacDismissScript,$snapshotName)
Execute $vmxFilePath $machineName $username $password $scriptTpath $scriptFile $uacDismissScript $snapshotName
}
Start-Job -ScriptBlock $cmd -ArgumentList $vmxFilePath, $machineName, $username $password, $scriptTpath, $scriptFile, $uacDismissScript, $snapshotName
No update necessary for this. Define a script block and use Start-Job to run the script block as many times as necessary. Example:
$cmd = {
param($a, $b)
Write-Host $a $b
}
$foo = "foo"
1..5 | ForEach-Object {
Start-Job -ScriptBlock $cmd -ArgumentList $_, $foo
}
The script block takes 2 parameters $a and $b which are passed by the -ArgumentList option. In the example above, the assignments are $_ → $a and $foo → $b. $foo is just an example for a configurable, but static parameter.
Run Get-Job | Remove-Job at some point to remove the finished jobs from the queue (or Get-Job | % { Receive-Job $_.Id; Remove-Job $_.Id } if you want to retrieve the output).
Here's a quick bogus scriptblock for the purpose of testing:
$Code = {
param ($init)
$start = Get-Date
(1..30) | % { Start-Sleep -Seconds 1; $init +=1 }
$stop = Get-Date
Write-Output "Counted from $($init - 30) until $init in $($stop - $start)."
}
This scriptblock can then be passed on to Start-Job, with for example 3 parameters (10, 15, 35)
$jobs = #()
(10,15,35) | % { $jobs += Start-Job -ArgumentList $_ -ScriptBlock $Code }
Wait-Job -Job $jobs | Out-Null
Receive-Job -Job $jobs
This creates 3 jobs, assign them to the $jobs variable, runs them in parallel and then waits for these 3 jobs to finish, and retrieves the results:
Counted from 10 until 40 in 00:00:30.0147167.
Counted from 15 until 45 in 00:00:30.0057163.
Counted from 35 until 65 in 00:00:30.0067163.
This did not take 90 seconds to execute, only 30.
One of the tricky parts is to provide -Argumentlist to Start-Job, and include a param() block inside the ScriptBlock. Otherwise, your values are never seen by the scriptblock.
You can use an alternative which may be faster than invoking jobs if the function is not a long running one. Max thread is 25 and I'm only invoking this function 10 times, so I expect my total runtime to be 5 seconds. You can wrap Measure-Command around the 'results=' statement to view stats.
Example:
$ScriptBlock = {
Param ( [int]$RunNumber )
Start-Sleep -Seconds 5
Return $RunNumber
}
$runNumbers = #(1..10)
$MaxThreads = 25
$runspacePool = [RunspaceFactory ]::CreateRunspacePool(1, $MaxThreads)
$runspacePool.Open()
$pipeLines = foreach($num in $runNumbers){
$pipeline = [powershell]::Create()
$pipeline.RunspacePool = $runspacePool
$pipeline.AddScript($ScriptBlock) | Out-Null
$pipeline.AddArgument($num) | Out-Null
$pipeline | Add-Member -MemberType NoteProperty -Name 'AsyncResult' -Value $pipeline.BeginInvoke() -PassThru
}
#obtain results as they come.
$results = foreach($pipeline in $pipeLines){
$pipeline.EndInvoke($pipeline.AsyncResult )
}
#cleanup code.
$pipeLines | % { $_.Dispose()}
$pipeLines = $null
if ( $runspacePool ) { $runspacePool.Close()}
#your results
$results
I'm sorry that everyone missed your issue - I know it is far too late now, but...
This error is caused because you are missing a comma between $username and $password in your list.
You can test it out with this snippet, which I modelled off of the previous answers:
$cmd = {
param($a, $b, $c, $d)
}
$foo = "foo"
$bar = "bar"
start-job -scriptblock $cmd -ArgumentList "a", $foo, $bar, "gold" #added missing comma for this to work
I've made a very versitile function to do this for you, and unlike the other answers you dont need to rejig your code to get it to work.
Just pass your function as a parameter to Async and pipe your input in, each item on the pipeline will run your scriptblock in parallel asynchronously, and emit them as each one is completed.
For your question specifically it would look something like this
#(
#{vmxFilePath='a';machineName='b';username='c';password='d';scriptTpath='e';scriptFile='f';uacDismissScript='g';snapshotName'h'},
#{vmxFilePath='i';machineName='j';username='k';password='l';scriptTpath='m';scriptFile='n';uacDismissScript='o';snapshotName'p'}
...
) `
| Async `
-Func { Process {
Execute $_.vmxFilePath $_.machineName $_.username $_.password $_.scriptTpath $_.scriptFile $_.uacDismissScript $_.snapshotName
} }
On top of this my function supports not only automatic construction of [powershell] (#binarySalt's answer) but also Jobs (utilised in #Joost's, but dont use these as they are much slower than runspaces) and Tasks if you're using other people's code that already spawn them (use the -AsJob flag, which I explain at the bottom of this answer).
So, that's not useful for new visitors to this question, lets do something more demonstrable that you can run on your machine and see real-world results.
Take this simple code for example, it just takes in some test data for websites and checks if they're up.
$in=TestData | ?{ $_.proto -eq 'tcp' }
$in `
| %{
$WarningPreference='SilentlyContinue'
$_ `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Test-NetConnection `
-ComputerName $_.address `
-Port $_.port `
-InformationLevel Quiet
)
} `
| Timer -name 'normal' `
| Format-Table
Here's the test data, just a good few of the same websites on repeat.
And also a timing function to see how performant it is.
Function TestData {
1..20 | %{
[PsCustomObject]#{proto='tcp' ; address='www.w3.org' ; port=443},
[PsCustomObject]#{proto='https'; address='www.w3.org' ; port=443},
[PsCustomObject]#{proto='icmp' ; address='www.w3.org' ; },
[PsCustomObject]#{proto='tcp' ; address='developer.mozilla.org' ; port=443},
[PsCustomObject]#{proto='https'; address='developer.mozilla.org' ; port=443},
[PsCustomObject]#{proto='icmp' ; address='developer.mozilla.org' ; },
[PsCustomObject]#{proto='tcp' ; address='help.dottoro.com' ; port=80 },
[PsCustomObject]#{proto='http' ; address='help.dottoro.com' ; port=80 },
[PsCustomObject]#{proto='icmp' ; address='help.dottoro.com' ; }
}
}
Function Timer {
Param ($name)
Begin {
$timer=[system.diagnostics.stopwatch]::StartNew()
}
Process { $_ }
End {
#(
$name,
' '
[math]::Floor($timer.Elapsed.TotalMinutes),
':',
($timer.Elapsed.Seconds -replace '^(.)$','0$1')
) -join '' | Out-Host
}
}
Okay, 15seconds, so how much faster can this get if we use Async?
And how much do we have to change to get it to work?
$in=TestData | ?{ $_.proto -eq 'tcp' }
$in `
| Async `
-Expected $in.Count `
-Func { Process {
$WarningPreference='SilentlyContinue'
$_ `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Test-NetConnection `
-ComputerName $_.address `
-Port $_.port `
-InformationLevel Quiet
)
} } `
| Timer -name 'async' `
| Format-Table
It looks basically identical..
Okay, what's the speed?
Wow, cut it by two thirds!
Not only that, but because we know how many items are in the pipeline I wrote in some smarts to give you a progressbar and an ETA
Don't believe me? Have a video
Or run the code yourself :)
#Requires -Version 5.1
#asynchronously run a pool of tasks,
#and aggregate the results back into a synchronous output
#without waiting to pool all input before seeing the first result
Function Async { Param(
#maximum permitted simultaneous background tasks
[int]$BatchSize=[int]$env:NUMBER_OF_PROCESSORS * 3,
#the task that accepts input on a pipe to execute in the background
[scriptblock]$Func,
#because your task is in a subshell you wont have access to your outer scope,
#you may pass them in here
[array]$ArgumentList=#(),
[System.Collections.IDictionary]$Parameters=#{},
#the title of the progress bar
[string]$Name='Processing',
#your -Func may return a [Job] instead of being backgrounded itself,
#if so it must return #(job;input;args)
#optionally job may be a [scriptblock] to be backgrounded, or a [Task]
[switch]$AsJob,
#if you know the number of tasks ahead of time,
#providing it here will have the progress bar show an ETA
[int]$Expected,
#outputs of this stream will be #(job;input) where job is the result
[switch]$PassThru,
#the time it takes to give up on one job type if there are others waiting
[int]$Retry=5
)
Begin {
$ArgumentList=[Array]::AsReadOnly($ArgumentList)
$Parameters=$Parameters.GetEnumerator() `
| &{
Begin { $params=[ordered]#{} }
Process { $params.Add($_.Key, $_.Value) }
End { $params.AsReadOnly() }
}
#the currently running background tasks
$running=#{}
$counts=[PSCustomObject]#{
completed=0;
jobs=0;
tasks=0;
results=0;
}
#a lazy attempt at uniquely IDing this instance for Write-Progress
$asyncId=Get-Random
#a timer for Write-Progress
$timer=[system.diagnostics.stopwatch]::StartNew()
$pool=[RunspaceFactory]::CreateRunspacePool(1, $BatchSize)
$pool.Open()
#called whenever we want to update the progress bar
Function Progress { Param($Reason)
#calculate ETA if applicable
$eta=-1
$total=[math]::Max(1, $counts.completed + $running.Count)
if ($Expected) {
$total=[math]::Max($total, $Expected)
if ($counts.completed) {
$eta=`
($total - $counts.completed) * `
$timer.Elapsed.TotalSeconds / `
$counts.completed
}
}
$Reason=Switch -regex ($Reason) {
'^done$' { "Finishing up the final $($running.Count) jobs." }
'^(do|next)$' { "
Running
$($running.Count)
jobs concurrently.
$(#('Adding','Waiting to add')[!($Reason -eq 'do')])
job #
$($counts.completed + $running.Count + 1)
" -replace '\r?\n\t*','' }
Default { "
Running $($running.Count) jobs concurrently.
Emitting
$($counts.completed)
$(#{1='st';2='nd';3='rd'}[$counts.completed % 10] -replace '^$','th')
result.
" -replace '\r?\n\t*','' }
}
Write-Progress `
-Id $asyncId `
-Activity $Name `
-SecondsRemaining $eta `
-Status ("
$($counts.completed)
jobs completed in
$([math]::Floor($timer.Elapsed.TotalMinutes))
:
$($timer.Elapsed.Seconds -replace '^(.)$','0$1')
" -replace '\r?\n\t*','') `
-CurrentOperation $Reason `
-PercentComplete (100 * $counts.completed / $total)
}
#called with the [Job]'s that have completed
Filter Done {
++$counts.completed
$out=$running.Item($_.Id)
$running.Remove($_.Id)
Progress
$out.job=`
if ($_ -is [System.Management.Automation.Job]) {
--$counts.jobs
$_ | Receive-Job
}
elseif ($_.pwsh) {
--$counts.results
try {
$_.pwsh.EndInvoke($_)
}
catch {
#[System.Management.Automation.MethodInvocationException]
$_.Exception.InnerException
}
finally {
$_.pwsh.Dispose()
}
}
elseif ($_.IsFaulted) {
--$counts.tasks
#[System.AggregateException]
$_.Exception.InnerException
}
else {
--$counts.tasks
$_.Result
}
if ($PassThru) {
$out
}
else {
$out.job
}
}
$isJob={
$_ -is [System.Management.Automation.Job]
}
$isTask={
$_ -is [System.Threading.Tasks.Task]
}
$isResult={
$_ -is [IAsyncResult]
}
$isFinished={
$_.IsCompleted -or `
(
$_.JobStateInfo.State -gt 1 -and
$_.JobStateInfo.State -ne 6 -and
$_.JobStateInfo.State -ne 8
)
}
$handle={
$_.AsyncWaitHandle
}
Function Jobs { Param($Filter)
$running.Values | %{ $_.job } | ? $Filter
}
#called whenever we need to wait for at least one task to completed
#outputs the completed tasks
Function Wait { Param([switch]$Finishing)
#if we are at the max background tasks this instant
while ($running.Count -ge $BatchSize) {
Progress -Reason #('done','next')[!$Finishing]
$value=#('jobs', 'tasks', 'results') `
| %{ $counts.($_) } `
| measure -Maximum -Sum
$wait=if ($value.Maximum -lt $value.Sum) {
$Retry
}
else {
-1
}
$value=Switch -exact ($value.Maximum) {
$counts.jobs {
(Wait-Job `
-Any `
-Job (Jobs -Filter $isJob) `
-Timeout $wait
).Count -lt 1
break
}
Default {
[System.Threading.WaitHandle]::WaitAny(
(Jobs -Filter $handle | % $handle),
[math]::Max($wait * 1000, -1)
) -eq [System.Threading.WaitHandle]::WaitTimeout
break
}
}
(Jobs -Filter $isFinished) | Done
}
}
}
#accepts inputs to spawn a new background task with
Process {
Wait
Progress -Reason 'do'
$run=[PSCustomObject]#{
input=$_;
job=$Func;
args=$ArgumentList;
params=$Parameters;
}
if ($AsJob) {
$run.job=$NULL
Invoke-Command `
-ScriptBlock $Func `
-ArgumentList #($run) `
| Out-Null
}
if ($run.job | % $isJob) {
++$counts.jobs
}
elseif ($run.job | % $isTask) {
++$counts.tasks
}
#if we weren't given a [Job] we need to spawn it for them
elseif ($run.job -is [ScriptBlock]) {
$pwsh=[powershell]::Create().AddScript($run.job)
$run.args | %{ $pwsh.AddArgument($_) } | Out-Null
$pwsh.RunspacePool=$pool
$run.job=$pwsh.AddParameters($run.params).BeginInvoke(
[System.Management.Automation.PSDataCollection[PSObject]]::new(
[PSObject[]]($run.input)
)
)
$run.job | Add-Member `
-MemberType NoteProperty `
-Name pwsh `
-Value $pwsh `
-PassThru `
| Add-Member `
-MemberType NoteProperty `
-Name Id `
-Value $run.job.AsyncWaitHandle.Handle.ToString()
++$counts.results
}
else {
throw "$($run.job.GetType()) needs to be a ScriptBlock"
}
$running.Add($run.job.Id, $run) | Out-Null
}
End {
#wait for the remaining running processes
$BatchSize=1
Wait -Finishing
Write-Progress -Id $asyncId -Activity $Name -Completed
$pool.Close()
$pool.Dispose()
}
}
So you may have noticed three things above, I alluded to -AsJob (using a mix of Jobs, Tasks and scriptblocks), there are unused protocols mentioned in the test data, and the video had a third test in it.
Here it is. Instead of doing the basic tcp test, using the test data we will also do a http/s check and a icmp ping (idk, maybe the https fails, but you want to narrow down if it's because the machine is down or just the service).
Test-Connection -AsJob is a cmdlet that returns a Job and checks ping.
WebRequest.GetResponseAsync() which connects to a web resource, returning a Task
And finally Test-NetConnection, which will be the same as before, ran in what we programmed as a synchronous scriptblock, which will be ran asynchronously for us.
$in=TestData
$in `
| Async `
-Expected $in.Count `
-PassThru `
-AsJob `
<#this would be accessible as a named parameter if needed#>`
-Parameters #{proxy=[System.Net.WebRequest]::GetSystemWebProxy()} `
-Func { Param([parameter(Position=0)]$x)
$x.job=Switch -regex ($x.input.proto) {
'^icmp$' {
Test-Connection `
-ComputerName $x.input.address `
-Count 1 `
-ThrottleLimit 1 `
-AsJob
}
'^tcp$' {
$x.params=#{address=$x.input.address; port=$x.input.port}
{ Param($address, $port)
$WarningPreference='SilentlyContinue'
Test-NetConnection `
-ComputerName $address `
-Port $port `
-InformationLevel Quiet
}
}
'^(http|https)$' {
[Net.ServicePointManager]::SecurityProtocol=[Net.SecurityProtocolType]::Tls12
$request=[System.Net.HttpWebRequest]::Create((#(
$x.input.proto,
'://',
$x.input.address,
':',
$x.input.port
) -join ''))
$request.Proxy=$NULL
$request.Method='Get'
$request.GetResponseAsync()
}
}
} `
| %{
$result=$_
$result.input `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Switch -regex (#($result.input.proto, $result.job.message)[$result.job -is [Exception]]) {
#[Win32_PingStatus]
'^icmp$' { $result.job.StatusCode -eq 0 }
#[bool]
'^tcp$' { $result.job }
#[System.Net.HttpWebResponse]
'^(http|https)$' {
$result.job.Close()
Switch ($result.job.StatusCode.value__) {
{ $_ -ge 200 -and $_ -lt 400 } { $True }
Default {$False}
}
}
#[Exception]
Default { $False }
})
} `
| Timer -name 'async asjob' `
| Format-Table
As you may have seen, this did more than double the work of the original code, but still finished in around half the time at 8seconds.