I would like to force-terminate/timeout a PowerShell Invoke-Command remote session after 20 minutes regardless of whether it's busy or idle.
How do I achieve this? Thanks.
Something like :
$job = Start-Job -ScriptBlock { <# Invoke-Command #> }
$job | Wait-Job -Timeout ( 20 * 60 ) | Remove-Job
PS> invoke_command_responsive -Machine pv3040 -Cmd "get-location"
function invoke_command_responsive {
param(
[string]$Machine,
[string]$Cmd
)
$start_time = Get-Date
$start_dir = better_resolve_path(".")
$watchdog = 8 #seconds
[ScriptBlock]$sb = [ScriptBlock]::Create($opt_cmd)
$j = Start-Job -ScriptBlock {
set-location $using:start_dir | out-null
[ScriptBlock]$sb = [ScriptBlock]::Create($using:cmd)
invoke-command -Computer $using:Machine -ScriptBlock:$sb | out-host
}
# Wait for Job to Complete or TIMEOUT!
while($true) {
if ($j.HasMoreData) {
Receive-Job $j
Start-Sleep -Milliseconds 50
}
$current = Get-Date
$time_span = $current - $start_time
if ($time_span.TotalSeconds -gt $watchdog) {
write-host "TIMEOUT!"
Stop-Job $j
break
}
if (-not $j.HasMoreData -and $j.State -ne 'Running') {
write-host "Finished"
break
}
}
Remove-Job $j
}
function better_resolve_path {
param([string]$path)
$pathfix = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($path)
return $pathfix
}
Related
This question already has answers here:
Getting an error when executing a nested ScriptBlock from Invoke-Command
(2 answers)
Closed 1 year ago.
Trying to create an invoke-command with a 10 second timeout incase one of my computers is in a zombie state and doesn't respond. Here's what I have so far below. But, one problem i'm having is that it doesn't behave like the normal invoke-command by reporting the output to the terminal as the command runs...
PS> Invoke_command_responsive -Computer pv3039 -ScriptBlock {gci -recurse -Path C:\ | out-host}
Cannot bind parameter 'ScriptBlock'. Cannot convert the "gci -recurse -Path C:\ | out-host" value of type "System.String" to type "System.Management.Automation.ScriptBlock".
+ CategoryInfo : InvalidArgument: (:) [Invoke-Command], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeCommandCommand
+ PSComputerName : localhost
Finished
Here's my code:
function invoke_command_responsive {
$start_time = Get-Date
$start_dir = better_resolve_path(".")
$watchdog = 60
$j = Start-Job -ScriptBlock {
set-location $using:start_dir | out-null
$iargs = $using:args
invoke-command #iargs | out-host
}
# Wait for Job to Complete or TIMEOUT!
while($true) {
if ($j.HasMoreData) {
Receive-Job $j
Start-Sleep -Milliseconds 200
}
$current = Get-Date
$time_span = $current - $start_time
if ($time_span.TotalSeconds -gt $watchdog) {
write-host "TIMEOUT!"
Stop-Job $j
break
}
if (-not $j.HasMoreData -and $j.State -ne 'Running') {
write-host "Finished"
break
}
}
Remove-Job $j
}
function better_resolve_path {
param([string]$path)
$pathfix = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($path)
return $pathfix
}
It kind of looks like it can't take a ScriptBlock from "using:"
Powershell just has trouble passing a scriptblock using "using:" syntax.
PS> invoke_command_responsive -Machine pv3040 -Cmd "get-location"
function invoke_command_responsive {
param(
[string]$Machine,
[string]$Cmd
)
$start_time = Get-Date
$start_dir = better_resolve_path(".")
$watchdog = 8 #seconds
[ScriptBlock]$sb = [ScriptBlock]::Create($opt_cmd)
$j = Start-Job -ScriptBlock {
set-location $using:start_dir | out-null
[ScriptBlock]$sb = [ScriptBlock]::Create($using:cmd)
invoke-command -Computer $using:Machine -ScriptBlock:$sb | out-host
}
# Wait for Job to Complete or TIMEOUT!
while($true) {
if ($j.HasMoreData) {
Receive-Job $j
Start-Sleep -Milliseconds 50
}
$current = Get-Date
$time_span = $current - $start_time
if ($time_span.TotalSeconds -gt $watchdog) {
write-host "TIMEOUT!"
Stop-Job $j
break
}
if (-not $j.HasMoreData -and $j.State -ne 'Running') {
write-host "Finished"
break
}
}
Remove-Job $j
}
function better_resolve_path {
param([string]$path)
$pathfix = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($path)
return $pathfix
}
When I run get-job | receive-job -keep I get no results and have no way to verify that my function is passed and working. Can I pass a function like this into a psjob? How do I capture the return value afterwards?
$servers = #("xxxx", "xxxx")
$jobs = New-Object System.Collections.ArrayList;
foreach ($server in $servers)
{
$jobName = $server + "_job";
$scriptBlock =
{
param($server)
param($portNumber)
Function testPort ($server, $portNumber)
{
$testPort = New-Object System.Net.Sockets.TCPClient # -ArgumentList $server, 3389;
$testPort.SendTimeout = 3;
try
{
$testPort.Connect($server, 3389);
}
catch
{
#do nothing;
}
$result = $testPort.Connected;
$testPort.Close();
return $result;
}
testPort -server $server -portNumber 3389; sleep 10;
}
$portNumber = "3389";
#Start-Job -Name $jobName -ScriptBlock {$scriptBlock} -ArgumentList $server, $portNumber;
$jobs.Add((Start-Job -Name $jobName -ScriptBlock {$scriptBlock} -ArgumentList $server, $portNumber | Out-Null));
}
$jobsReturnValues = New-Object System.Collections.ArrayList;
foreach ($job in $jobs)
{
$jobsReturnValues.Add(($job | Wait-Job | Receive-Job | Out-Null));
}
The code is almost fine, you did not have many errors. The main issue was the use of Out-Null, the { } on -ScriptBlock {$scriptBlock} and the param( ) block was defined 2 times in your function. I did a little modification to your function too :)
$servers = #('google.com','twitter.com')
$jobs = New-Object System.Collections.ArrayList;
$testPort = 80
foreach ($server in $servers)
{
$jobName = $server + "_job";
$scriptBlock = {
param(
[string]$server,
[int]$portNumber
)
Function testPort ($server, $portNumber)
{
$testPort = New-Object System.Net.Sockets.TCPClient # -ArgumentList $server, 3389;
$testPort.SendTimeout = 3;
try
{
$testPort.Connect($server, $portNumber);
}
catch
{
#do nothing;
}
$result = $testPort.Connected;
$testPort.Close();
[pscustomobject]#{
ServerName = $server
Port = $portNumber
TestConnection = $result
}
}
testPort -server $server -portNumber $portNumber
}
$jobs.Add((Start-Job -Name $jobName -ScriptBlock $scriptBlock -ArgumentList $server, $testPort)) > $null;
}
$jobsReturnValues = New-Object System.Collections.ArrayList;
foreach ($job in $jobs)
{
$jobsReturnValues.Add(($job | Receive-Job -Wait -AutoRemoveJob)) > $null;
}
Looks like this:
PS C:\> $jobsReturnValues|select * -ExcludeProperty RunspaceID|ft
ServerName Port TestConnection
---------- ---- --------------
google.com 80 True
twitter.com 80 True
If you go to the docs you will see that there are a family of cmdlets that work with Jobs.
$job = Start-Job...
$data = $job | Wait-Job | Receive-Job
Wait-Job waits for the job until it is not in a Running State.
Receive-Job returns the output of the job.
Update
After you updated your code I can say the following:
Start-Job -Name $jobName -ScriptBlock {$scriptBlock} -ArgumentList $server, $portNumber | Out-Null
Why do you pipe it into Out-Null? This makes the output of the Start-Job disappear, and you are appending into $job only null values. Remove the pipe into the Out-Null. Start-Job returns a System.Management.Automation.PSRemotingJob object, which you need so you can reference it later.
Also, you do the same thing here: $job | Wait-Job | Receive-Job | Out-Null. This makes the job's output disappear.
Start-Job -Name $jobName -ScriptBlock {$scriptBlock}
needs to be
Start-Job -Name $jobName -ScriptBlock $scriptBlock
Basically, you put a script block inside a script block with the first approach.
I think you should first test with some basic examples until you get the hang of how jobs work.
I'm using PowerShell job to process activity in parallel. I attached a sample code of it below.
Function Wait-UntilJobFailOrSuccess {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline)]
[System.Management.Automation.Job[]]$Job
)
begin {
$jobs = #()
$abort = $false
}
process {
$jobs += $Job
}
end {
while ('Running' -In $jobs.State) {
if ('Failed' -in $jobs.State) {
$jobs | Stop-Job
$abort = $true
break
}
Start-Sleep -Milliseconds 500
}
foreach ($job in $jobs) {
if ($job.State -eq 'Failed') {
$job.ChildJobs | ForEach-Object {
Write-Host ($_.JobStateInfo.Reason.Message) -ForegroundColor Red
}
}
else {
Write-Host "$($job.Name) $($job.State) successfully"
}
}
if ($abort) {
exit 1
}
}
}
The above method will be used for a job which will be parallelly executed.
$packageSpecs | ForEach-Object -Begin {
$job = #()
} -Process {
$builditem = $_
$job += Start-Job -Name $("Pack" + $builditem.name) -ScriptBlock $scriptoExecute -ArgumentList $args
if ($job.Count -eq $maxNumberOfThread) {
$job |Wait-UntilJobFailOrSuccess
$job = #()
}
} -End {
if ($job.Count -gt 0) {
$job |Wait-UntilJobFailOrSuccess
$job = #()
}
}
If you see above code I'm running the job based on the number of CPU present in a machine, $maxNumberOfThread contain number of logical processes allowed in the system.
Now I will tell you my problem statement.
Suppose $maxNumberOfThread =4. It will process 4 jobs at a time. What I want is if I am processing 4 jobs and anyone of the job is finished. I should be able to queue the next one. At present it is processing all 4 job then next batch.
I made a few tweaks, i cut out the Wait-UntilJobFailOrSuccess function. But this should work. Try and let me know. If it works you then made can tweak it for your function.
$packageSpecs | ForEach-Object -Begin {
$job = #()
} -Process {
$builditem = $_
$job += Start-Job -Name $("Pack" + $builditem.name) -ScriptBlock $scriptoExecute -ArgumentList $args
$running = #($job | Where-Object { $_.State -eq 'Running' })
if ($running.Count -ge $maxNumberOfThread) {
$running | Wait-Job -Any | Out-Null
Write-Verbose "Waiting on jobs" -Verbose
}
} -End {
if ($job.Count -gt 0) {
$job |Wait-UntilJobFailOrSuccess
$job = #()
}
}
I made some changes to your function, pls let me know how it works out. The function now needs the $MaxNumberOfThread passed to it.
Function Wait-UntilJobFailOrSuccess {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline)]
[System.Management.Automation.Job[]]$Job,
[Parameter()]
[int]$maxNumberOfThread
)
begin {
$jobs = #()
$abort = $false
}
process {
$jobs += $Job
foreach ($j in $Jobs) {
$running = #($jobs | Where-Object { $_.State -eq 'Running' })
if ($running.Count -ge $maxNumberOfThread) {
if ('Failed' -in $jobs.State) {
$jobs | Stop-Job
$abort = $true
break
}
else {
$running | Wait-Job -Any | Out-Null
Write-Verbose "Waiting on jobs" -Verbose
}
}
if ($abort) {
exit 1
}
}
}
end {
foreach ($j in $jobs) {
if ($j.State -eq 'Failed') {
$j.ChildJobs | ForEach-Object {
Write-Host ($_.JobStateInfo.Reason.Message) -ForegroundColor Red
}
}
else {
Write-Host "$($j.Name) $($j.State) successfully"
}
}
}
}
I'm trying to run a virus scan on a list of servers in our environment. There are hundreds of machines, so we'd like to run the scan (using a command line prompt that we already have) around 10 at a time. We're totally new to PowerShell so any help would be really appreciated. We have a general idea of what commands we need to use -- here's how we think it might work for now:
$server = Get-Content "serverlist.txt"
$server | % {
$VirusScan = { Scan32.exe }
Invoke-Command -ScriptBlock { $VirusScan } -computerName $server -ThrottleLimit 10 -Authentication domain/admin
}
Does anyone have any suggestions on how we might orchestrate this?
I'm using something like this for running tasks in parallel on remote hosts:
$maxSlots = 10
$hosts = "foo", "bar", "baz", ...
$job = {
Invoke-Command -ScriptBlock { Scan32.exe } -Computer $ARGV[0] -ThrottleLimit 10 -Authentication domain/admin
}
$queue = [System.Collections.Queue]::Synchronized((New-Object System.Collections.Queue))
$hosts | ForEach-Object { $queue.Enqueue($_) }
while ( $queue.Count -gt 0 -or #(Get-Job -State Running).Count -gt 0 ) {
$freeSlots = $maxSlots - #(Get-Job -State Running).Count
for ( $i = $freeSlots; $i -gt 0 -and $queue.Count -gt 0; $i-- ) {
Start-Job -ScriptBlock $job -ArgumentList $queue.Dequeue() | Out-Null
}
Get-Job -State Completed | ForEach-Object {
Receive-Job -Id $_.Id
Remove-Job -Id $_.Id
}
Sleep -Milliseconds 100
}
# Remove all remaining jobs.
Get-Job | ForEach-Object {
Receive-Job -Id $_.Id
Remove-Job -Id $_.Id
}
I'm trying to make jobs with powershell but they get stuck for a reason.
Untitled2.ps1:
$A = "papa"
$B = "mama"
Start-Job -ScriptBlock { MainHA $args[0] $args[1] } -ArgumentList #($A, $B) -InitializationScript { . "C:\Tools\Untitled3.ps1" }
While (Get-Job -State "Running")
{
write-host Still working...
Get-Job | Receive-Job
Start-Sleep 1
}
Get-Job | Receive-Job
Remove-Job *
Untitled3.ps1:
Function MainHA ($x, $y)
{
write-host $x, $y
}
Any idea?
Are there any other jobs running?
One way to wait for the specific job you are starting is to store the job you start in a variable.
$job = Start-Job -ScriptBlock { MainHA $args[0] $args[1] } -ArgumentList #($A, $B) -InitializationScript { . "C:\Tools\Untitled3.ps1" }
While ($job.State -eq "Running") {
write-host Still working...
Receive-Job $job
Start-Sleep 1
}
Receive-Job $job
Remove-Job $job