PowerShell - How to force timeout Invoke-Command - powershell

I would like to force-terminate/timeout a PowerShell Invoke-Command remote session after 20 minutes regardless of whether it's busy or idle.
How do I achieve this? Thanks.

Something like :
$job = Start-Job -ScriptBlock { <# Invoke-Command #> }
$job | Wait-Job -Timeout ( 20 * 60 ) | Remove-Job

PS> invoke_command_responsive -Machine pv3040 -Cmd "get-location"
function invoke_command_responsive {
param(
[string]$Machine,
[string]$Cmd
)
$start_time = Get-Date
$start_dir = better_resolve_path(".")
$watchdog = 8 #seconds
[ScriptBlock]$sb = [ScriptBlock]::Create($opt_cmd)
$j = Start-Job -ScriptBlock {
set-location $using:start_dir | out-null
[ScriptBlock]$sb = [ScriptBlock]::Create($using:cmd)
invoke-command -Computer $using:Machine -ScriptBlock:$sb | out-host
}
# Wait for Job to Complete or TIMEOUT!
while($true) {
if ($j.HasMoreData) {
Receive-Job $j
Start-Sleep -Milliseconds 50
}
$current = Get-Date
$time_span = $current - $start_time
if ($time_span.TotalSeconds -gt $watchdog) {
write-host "TIMEOUT!"
Stop-Job $j
break
}
if (-not $j.HasMoreData -and $j.State -ne 'Running') {
write-host "Finished"
break
}
}
Remove-Job $j
}
function better_resolve_path {
param([string]$path)
$pathfix = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($path)
return $pathfix
}

Related

Invoke-Command with timeout [duplicate]

This question already has answers here:
Getting an error when executing a nested ScriptBlock from Invoke-Command
(2 answers)
Closed 1 year ago.
Trying to create an invoke-command with a 10 second timeout incase one of my computers is in a zombie state and doesn't respond. Here's what I have so far below. But, one problem i'm having is that it doesn't behave like the normal invoke-command by reporting the output to the terminal as the command runs...
PS> Invoke_command_responsive -Computer pv3039 -ScriptBlock {gci -recurse -Path C:\ | out-host}
Cannot bind parameter 'ScriptBlock'. Cannot convert the "gci -recurse -Path C:\ | out-host" value of type "System.String" to type "System.Management.Automation.ScriptBlock".
+ CategoryInfo : InvalidArgument: (:) [Invoke-Command], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeCommandCommand
+ PSComputerName : localhost
Finished
Here's my code:
function invoke_command_responsive {
$start_time = Get-Date
$start_dir = better_resolve_path(".")
$watchdog = 60
$j = Start-Job -ScriptBlock {
set-location $using:start_dir | out-null
$iargs = $using:args
invoke-command #iargs | out-host
}
# Wait for Job to Complete or TIMEOUT!
while($true) {
if ($j.HasMoreData) {
Receive-Job $j
Start-Sleep -Milliseconds 200
}
$current = Get-Date
$time_span = $current - $start_time
if ($time_span.TotalSeconds -gt $watchdog) {
write-host "TIMEOUT!"
Stop-Job $j
break
}
if (-not $j.HasMoreData -and $j.State -ne 'Running') {
write-host "Finished"
break
}
}
Remove-Job $j
}
function better_resolve_path {
param([string]$path)
$pathfix = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($path)
return $pathfix
}
It kind of looks like it can't take a ScriptBlock from "using:"
Powershell just has trouble passing a scriptblock using "using:" syntax.
PS> invoke_command_responsive -Machine pv3040 -Cmd "get-location"
function invoke_command_responsive {
param(
[string]$Machine,
[string]$Cmd
)
$start_time = Get-Date
$start_dir = better_resolve_path(".")
$watchdog = 8 #seconds
[ScriptBlock]$sb = [ScriptBlock]::Create($opt_cmd)
$j = Start-Job -ScriptBlock {
set-location $using:start_dir | out-null
[ScriptBlock]$sb = [ScriptBlock]::Create($using:cmd)
invoke-command -Computer $using:Machine -ScriptBlock:$sb | out-host
}
# Wait for Job to Complete or TIMEOUT!
while($true) {
if ($j.HasMoreData) {
Receive-Job $j
Start-Sleep -Milliseconds 50
}
$current = Get-Date
$time_span = $current - $start_time
if ($time_span.TotalSeconds -gt $watchdog) {
write-host "TIMEOUT!"
Stop-Job $j
break
}
if (-not $j.HasMoreData -and $j.State -ne 'Running') {
write-host "Finished"
break
}
}
Remove-Job $j
}
function better_resolve_path {
param([string]$path)
$pathfix = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($path)
return $pathfix
}

Can I pass a function and call it in a PSJobs scriptblock, and receive a return value?

When I run get-job | receive-job -keep I get no results and have no way to verify that my function is passed and working. Can I pass a function like this into a psjob? How do I capture the return value afterwards?
$servers = #("xxxx", "xxxx")
$jobs = New-Object System.Collections.ArrayList;
foreach ($server in $servers)
{
$jobName = $server + "_job";
$scriptBlock =
{
param($server)
param($portNumber)
Function testPort ($server, $portNumber)
{
$testPort = New-Object System.Net.Sockets.TCPClient # -ArgumentList $server, 3389;
$testPort.SendTimeout = 3;
try
{
$testPort.Connect($server, 3389);
}
catch
{
#do nothing;
}
$result = $testPort.Connected;
$testPort.Close();
return $result;
}
testPort -server $server -portNumber 3389; sleep 10;
}
$portNumber = "3389";
#Start-Job -Name $jobName -ScriptBlock {$scriptBlock} -ArgumentList $server, $portNumber;
$jobs.Add((Start-Job -Name $jobName -ScriptBlock {$scriptBlock} -ArgumentList $server, $portNumber | Out-Null));
}
$jobsReturnValues = New-Object System.Collections.ArrayList;
foreach ($job in $jobs)
{
$jobsReturnValues.Add(($job | Wait-Job | Receive-Job | Out-Null));
}
The code is almost fine, you did not have many errors. The main issue was the use of Out-Null, the { } on -ScriptBlock {$scriptBlock} and the param( ) block was defined 2 times in your function. I did a little modification to your function too :)
$servers = #('google.com','twitter.com')
$jobs = New-Object System.Collections.ArrayList;
$testPort = 80
foreach ($server in $servers)
{
$jobName = $server + "_job";
$scriptBlock = {
param(
[string]$server,
[int]$portNumber
)
Function testPort ($server, $portNumber)
{
$testPort = New-Object System.Net.Sockets.TCPClient # -ArgumentList $server, 3389;
$testPort.SendTimeout = 3;
try
{
$testPort.Connect($server, $portNumber);
}
catch
{
#do nothing;
}
$result = $testPort.Connected;
$testPort.Close();
[pscustomobject]#{
ServerName = $server
Port = $portNumber
TestConnection = $result
}
}
testPort -server $server -portNumber $portNumber
}
$jobs.Add((Start-Job -Name $jobName -ScriptBlock $scriptBlock -ArgumentList $server, $testPort)) > $null;
}
$jobsReturnValues = New-Object System.Collections.ArrayList;
foreach ($job in $jobs)
{
$jobsReturnValues.Add(($job | Receive-Job -Wait -AutoRemoveJob)) > $null;
}
Looks like this:
PS C:\> $jobsReturnValues|select * -ExcludeProperty RunspaceID|ft
ServerName Port TestConnection
---------- ---- --------------
google.com 80 True
twitter.com 80 True
If you go to the docs you will see that there are a family of cmdlets that work with Jobs.
$job = Start-Job...
$data = $job | Wait-Job | Receive-Job
Wait-Job waits for the job until it is not in a Running State.
Receive-Job returns the output of the job.
Update
After you updated your code I can say the following:
Start-Job -Name $jobName -ScriptBlock {$scriptBlock} -ArgumentList $server, $portNumber | Out-Null
Why do you pipe it into Out-Null? This makes the output of the Start-Job disappear, and you are appending into $job only null values. Remove the pipe into the Out-Null. Start-Job returns a System.Management.Automation.PSRemotingJob object, which you need so you can reference it later.
Also, you do the same thing here: $job | Wait-Job | Receive-Job | Out-Null. This makes the job's output disappear.
Start-Job -Name $jobName -ScriptBlock {$scriptBlock}
needs to be
Start-Job -Name $jobName -ScriptBlock $scriptBlock
Basically, you put a script block inside a script block with the first approach.
I think you should first test with some basic examples until you get the hang of how jobs work.

Powershell job execution handle batch processing properly without waiting for whole batch to finish

I'm using PowerShell job to process activity in parallel. I attached a sample code of it below.
Function Wait-UntilJobFailOrSuccess {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline)]
[System.Management.Automation.Job[]]$Job
)
begin {
$jobs = #()
$abort = $false
}
process {
$jobs += $Job
}
end {
while ('Running' -In $jobs.State) {
if ('Failed' -in $jobs.State) {
$jobs | Stop-Job
$abort = $true
break
}
Start-Sleep -Milliseconds 500
}
foreach ($job in $jobs) {
if ($job.State -eq 'Failed') {
$job.ChildJobs | ForEach-Object {
Write-Host ($_.JobStateInfo.Reason.Message) -ForegroundColor Red
}
}
else {
Write-Host "$($job.Name) $($job.State) successfully"
}
}
if ($abort) {
exit 1
}
}
}
The above method will be used for a job which will be parallelly executed.
$packageSpecs | ForEach-Object -Begin {
$job = #()
} -Process {
$builditem = $_
$job += Start-Job -Name $("Pack" + $builditem.name) -ScriptBlock $scriptoExecute -ArgumentList $args
if ($job.Count -eq $maxNumberOfThread) {
$job |Wait-UntilJobFailOrSuccess
$job = #()
}
} -End {
if ($job.Count -gt 0) {
$job |Wait-UntilJobFailOrSuccess
$job = #()
}
}
If you see above code I'm running the job based on the number of CPU present in a machine, $maxNumberOfThread contain number of logical processes allowed in the system.
Now I will tell you my problem statement.
Suppose $maxNumberOfThread =4. It will process 4 jobs at a time. What I want is if I am processing 4 jobs and anyone of the job is finished. I should be able to queue the next one. At present it is processing all 4 job then next batch.
I made a few tweaks, i cut out the Wait-UntilJobFailOrSuccess function. But this should work. Try and let me know. If it works you then made can tweak it for your function.
$packageSpecs | ForEach-Object -Begin {
$job = #()
} -Process {
$builditem = $_
$job += Start-Job -Name $("Pack" + $builditem.name) -ScriptBlock $scriptoExecute -ArgumentList $args
$running = #($job | Where-Object { $_.State -eq 'Running' })
if ($running.Count -ge $maxNumberOfThread) {
$running | Wait-Job -Any | Out-Null
Write-Verbose "Waiting on jobs" -Verbose
}
} -End {
if ($job.Count -gt 0) {
$job |Wait-UntilJobFailOrSuccess
$job = #()
}
}
I made some changes to your function, pls let me know how it works out. The function now needs the $MaxNumberOfThread passed to it.
Function Wait-UntilJobFailOrSuccess {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline)]
[System.Management.Automation.Job[]]$Job,
[Parameter()]
[int]$maxNumberOfThread
)
begin {
$jobs = #()
$abort = $false
}
process {
$jobs += $Job
foreach ($j in $Jobs) {
$running = #($jobs | Where-Object { $_.State -eq 'Running' })
if ($running.Count -ge $maxNumberOfThread) {
if ('Failed' -in $jobs.State) {
$jobs | Stop-Job
$abort = $true
break
}
else {
$running | Wait-Job -Any | Out-Null
Write-Verbose "Waiting on jobs" -Verbose
}
}
if ($abort) {
exit 1
}
}
}
end {
foreach ($j in $jobs) {
if ($j.State -eq 'Failed') {
$j.ChildJobs | ForEach-Object {
Write-Host ($_.JobStateInfo.Reason.Message) -ForegroundColor Red
}
}
else {
Write-Host "$($j.Name) $($j.State) successfully"
}
}
}
}

Use PowerShell to run virus scan on multiple servers

I'm trying to run a virus scan on a list of servers in our environment. There are hundreds of machines, so we'd like to run the scan (using a command line prompt that we already have) around 10 at a time. We're totally new to PowerShell so any help would be really appreciated. We have a general idea of what commands we need to use -- here's how we think it might work for now:
$server = Get-Content "serverlist.txt"
$server | % {
$VirusScan = { Scan32.exe }
Invoke-Command -ScriptBlock { $VirusScan } -computerName $server -ThrottleLimit 10 -Authentication domain/admin
}
Does anyone have any suggestions on how we might orchestrate this?
I'm using something like this for running tasks in parallel on remote hosts:
$maxSlots = 10
$hosts = "foo", "bar", "baz", ...
$job = {
Invoke-Command -ScriptBlock { Scan32.exe } -Computer $ARGV[0] -ThrottleLimit 10 -Authentication domain/admin
}
$queue = [System.Collections.Queue]::Synchronized((New-Object System.Collections.Queue))
$hosts | ForEach-Object { $queue.Enqueue($_) }
while ( $queue.Count -gt 0 -or #(Get-Job -State Running).Count -gt 0 ) {
$freeSlots = $maxSlots - #(Get-Job -State Running).Count
for ( $i = $freeSlots; $i -gt 0 -and $queue.Count -gt 0; $i-- ) {
Start-Job -ScriptBlock $job -ArgumentList $queue.Dequeue() | Out-Null
}
Get-Job -State Completed | ForEach-Object {
Receive-Job -Id $_.Id
Remove-Job -Id $_.Id
}
Sleep -Milliseconds 100
}
# Remove all remaining jobs.
Get-Job | ForEach-Object {
Receive-Job -Id $_.Id
Remove-Job -Id $_.Id
}

powershells' stuck jobs

I'm trying to make jobs with powershell but they get stuck for a reason.
Untitled2.ps1:
$A = "papa"
$B = "mama"
Start-Job -ScriptBlock { MainHA $args[0] $args[1] } -ArgumentList #($A, $B) -InitializationScript { . "C:\Tools\Untitled3.ps1" }
While (Get-Job -State "Running")
{
write-host Still working...
Get-Job | Receive-Job
Start-Sleep 1
}
Get-Job | Receive-Job
Remove-Job *
Untitled3.ps1:
Function MainHA ($x, $y)
{
write-host $x, $y
}
Any idea?
Are there any other jobs running?
One way to wait for the specific job you are starting is to store the job you start in a variable.
$job = Start-Job -ScriptBlock { MainHA $args[0] $args[1] } -ArgumentList #($A, $B) -InitializationScript { . "C:\Tools\Untitled3.ps1" }
While ($job.State -eq "Running") {
write-host Still working...
Receive-Job $job
Start-Sleep 1
}
Receive-Job $job
Remove-Job $job