How to know when a command isn't responding in powershell - powershell

I have a powershell script which launch a command line utility DacIESvcCli.exe
DacIESvcCli.exe sends me a response and when I receive it I take the status, which can be "Running" or "Completed"
My problem is that sometimes the call hangs and I never get a response. The following script can run 3 days without interruption.
How can I prevent this from happening?
$myCounter = 0
while($myCounter -lt 5){
Write "start of the while counter : " $myCounter
$exportResponse = C:\DAC\DacIESvcCli.exe -s "myserver.database.windows.net" -u "mylogin#myserver" -p "mypassword" -requestid 'e1e34eee-1aaa-4cc9-8c48-3a2239fe1bff' -status
$exportStatus = $exportResponse[10].split(" ")[1].toString()
Write $exportStatus
$myCounter++
}
Here is the output
start of the while counter :
0
Completed
start of the while counter :
1
Completed
start of the while counter :
2
Completed
start of the while counter :
3
_
... and it never ends.

Here is part of my script for Travis Heseman
param($username, $password, $serverName, $requestId)
$consecutiveFailedAttempt = 0
while( $exportStatus -ne "Completed" -and $exportStatus -ne "Failed" -and $exportStatus -ne "TooLong"){
if($exportStatus -ne "FirstRun"){
Start-Sleep -m 60000 # Wait 1 min
}
$currentNow = Get-Date
$job = Start-Job { param($s, $u, $p, $r) C:\DAC\DacIESvcCli.exe -s $s -User $u -p $p -requestid $r -status } -ArgumentList #($serverName, $username, $password, $requestId)
Wait-Job $job -Timeout 60 # if the command takes longer than 60 sec we timeout and retry
Stop-Job $job
$exportResponse = Receive-Job $job
Remove-Job $job
if($exportResponse[10]){
$exportStatus = $exportResponse[10].split(" ")[1].toString()
$consecutiveFailedAttempt = 0;
}else{
$currentNow = Get-Date
$whileMessage = "Time out we retry " + $currentNow
$whileMessage | Out-File $File -append
$exportStatus = "Unknown"
$consecutiveFailedAttempt++;
}
if($consecutiveFailedAttempt -gt 10){
$exportStatus = "TooLong"
}
}
# do wantever you want with the export status

Related

Can't get wait-job result

I'm trying to achieve this with a code:
☑ Run a command
☑ Wait X seconds
☐ If command completed and is successful i return 1
☐ If command completed and is not successful i return 0
☑ If command did not completed I return 0.
I don't need it to be asynchronous. This is my first code:
$timeoutSeconds = 10
$code = {
ping something
}
$job = Start-Job -ScriptBlock $code
if (Wait-Job $job -Timeout $timeoutSeconds) { Receive-Job $job }
Remove-Job -force $job
if ($job.State -eq 'Completed'){
$ExitCode = 0
}
else {
$ExitCode = 1
}
My Powershell
[System.Environment]::OSVersion.Version
Major Minor Build Revision
----- ----- ----- --------
6 2 9200 0
I would like this to be not only for my powershell version.
I tried to add a return inside my $code
$code = {
ping something
return $LASTEXITCODE
}
But I can't find anything inside my $job.
I tried to assign Receive-Job to a variable, but it doesn't get never populated.
if (Wait-Job $job -Timeout $timeoutSeconds) { $r = Receive-Job $job }
$r is alwasys empty (in both cases: Wait-Job reach or not the timeout).
I want to get the $LASTEXITCODE i'm returning in my $code block, I would like something like:
$timeoutSeconds = 10
$code = {
ping 2.3.1.2
return $LASTEXITCODE
}
$job = Start-Job -ScriptBlock $code
if (Wait-Job $job -Timeout $timeoutSeconds) { Receive-Job $job }
Remove-Job -force $job
if ($job.State -eq 'Completed'){
$ExitCode = $job.ReturnedValue #this doesn't exists
}
else {
$ExitCode = 1
}
Write-Output $ExitCode
My problem is how to discern if:
Job completed in less than timeout because it succeeded
Job completed in less than timeout because it failed faster than I thought.
The command I'm running is a software command I would like to leave unkown, it is a something like a ping but:
the command finish in less than 10 seconds when it works (return 0)
the command finish in more than 10 seconds (30 or more) when the resource is unreachable (return 1)
the command finish in less than 10 seconds when something went wrong (return 1)
If the case is 2 the job is not completed.
If the case is either 1 or 2 I need the returned value inside the $code block to understand if the command is good or not.
How do I get the returned value inside the $code block outside after the job is completed? (if job is not completed I'm aware I can't have it, but I should be able to use the result of a job)
Thanks
If Wait-Job -Timeout <timeoutInSeconds> returns the job object then you know the job completed in time. If a terminating error occurred that caused the job to exit prematurely, the State property on the job object will reflect this, so simply do:
$timeoutSeconds = 10
$completedInTime = $false
$code = {
MyCommands
}
# start job, wait for timeout
$job = Start-Job -ScriptBlock $code
if($job |Wait-Job -Timeout $timeoutSeconds){
# If it completed successfully, State will have the value `Completed`
$completedInTime = $job.State -eq 'Completed'
$results = $job |Receive-Job
}
$job |Remove-Job -Force
# `$true` = 1
# `$false` = 0
$ExitCode = [int]$completedInTime
Thanks to #Mathias R. Jessen I achieved this:
$timeoutSeconds = 10
$completedInTime = $false
$code = {
ping 2.13.1.2 /n 1 > null # this way results won't have this output that I don't need
return $LASTEXITCODE
}
# start job, wait for timeout
$job = Start-Job -ScriptBlock $code
if($job |Wait-Job -Timeout $timeoutSeconds){
# If it completed successfully, State will have the value `Completed`
if($job.State -eq 'Completed'){
# Only in this case results will hold the LASTEXITCODE I returned in code block
$results = $job |Receive-Job
if ($results -eq 0){
$ExitCode = 0
}
else {
$ExitCode = 1
}
}
else{
# In this case I know the command is failing
$ExitCode = 1
}
}
$job |Remove-Job -Force
Write-Output $ExitCode

Powershell Animation Till the last command

My Script has a lot of commands that taking time.
How to make powershell animation till the last commands runs. I don't want start-sleep
I tried Wait-Job but not working.
I just got this function and it uses Start-sleep I can change it if it will affect
function ProcessingAnimation($scriptBlock) {
$cursorTop = [Console]::CursorTop
try {
[Console]::CursorVisible = $false
$counter = 0
$frames = '|', '/', '-', '\'
$jobName = Start-Job -ScriptBlock $scriptBlock
while($jobName.JobStateInfo.State -eq "Running") {
$frame = $frames[$counter % $frames.Length]
Write-Host "$frame" -NoNewLine
[Console]::SetCursorPosition(0, $cursorTop)
$counter += 1
Start-Sleep -Milliseconds 125
}
# Only needed if you use a multiline frames
Write-Host ($frames[0] -replace '[^\s+]', ' ')
}
finally {
[Console]::SetCursorPosition(0, $cursorTop)
[Console]::CursorVisible = $true
}
}
Write-Host "Getting System Information..."
ProcessingAnimation { Wait-Job -Name $WindowsVer }
command1
command2
command3
command4
command5
command6
$WindowsVer = (Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion").ReleaseId

How to set timeout to a PowerShell function?

I have a script which is fetching the disk space information of many machines in a call center. However, sometimes this function takes too long. I need to create a time out for that function and have been unable to. I tried using start-job but to be honest, I don't fully understand it and so I am not getting the desired results.
try {
$timeoutSeconds = 30
$code = {
param($currentPCname, $activeDriveTypes)
// Function which takes computer name as input and outputs the ComputerName,DeviceID, Size, Freespace and DriveType (3 = local)
function get-FDS {
[cmdLetBinding()]
Param([string]$hostName)
Get-WmiObject Win32_LogicalDisk -ComputerName $hostName |
Where-object {$_.DriveType -in $activeDriveTypes} |
select-Object #{Name="ComputerName";Expression={$hostName}}, #{Name="Date";Expression={Get-Date}}, DeviceID, Size, Freespace, DriveType
}
get-FDS($currentPCname) -errorAction stop
}
$processTime = measure-command {
$j = Start-Job -ScriptBlock $code -Arg $currentPCname, $activeDriveTypes
if (Wait-Job $j -Timeout $timeoutSeconds) { $tempData = Receive-Job $j }
$jobState = $j.state
Remove-Job -force $j
}
if ($jobState -ne 'Completed') {
$pcTurnedOn = $false
$errorMessage = "ERROR talking to $currentPCname : Query timed-out"
$query = "CALL sp_fds_insert_log(" + $currentRunID + ", '" + $errorMessage + "', '" + $scriptName + "');"
$cmd = new-object mysql.data.mysqlclient.mysqlcommand($query, $conn)
$cmd.executenonquery()
}
} catch {
$pcTurnedOn = $false
$errorMessage = "ERROR talking to $currentPCname : $_"
$query = "CALL sp_fds_insert_log(" + $currentRunID + ", '" + $errorMessage + "', '" + $scriptName + "');"
$cmd = new-object mysql.data.mysqlclient.mysqlcommand($query, $conn)
$cmd.executenonquery()
}
The main point of the code above is that if the line below which is calling the $code segment
$processTime = measure-command {
$j = Start-Job -ScriptBlock $code -Arg $currentPCname, $activeDriveTypes
if (Wait-Job $j -Timeout $timeoutSeconds) { $tempData = Receive-Job $j }
$jobState = $j.state
Remove-Job -force $j
}
takes more than 30 seconds which is the $timeoutSeconds, the last IF statement would be called, if the line above does not work for some reason, the catch statement would be called and if it works in less than 30 seconds, nothing would be called.
If the job finished before the timeout is reached, Wait-Job will return the job itself - if the timeout is exceeded it won't return anything:
$timeout = 2
$job = Start-Job { Start-Sleep -Seconds 3 }
$done = $job |Wait-Job -TimeOut $timeout
if($done){
# It returned within the timeout
}
else {
# Nothing was returned, job timed out
}

adding a timeout to batch/powershell

$fullnamexp = ((net user $winxp /domain | Select-String "Full Name") -replace "Full Name","").Trim();
If $winxp cannot be found, the command will hang, is there a timeout I can use with this to make it move on after 5-10 seconds? Not sure where I would put it.
Edit- I use this to pull the username:
$reg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey('LocalMachine', $tag1)
$key = $reg.OpenSubKey('SOFTWARE\Microsoft\Windows NT\CurrentVersion\WinLogon')
$winxp = $key.GetValue('DefaultUserName') -replace '^.*?\\'
$winxp is then a login name such as ajstepanik then I put it into: $fullnamexp = ((net user $winxp /domain | Select-String "Full Name") -replace "Full Name","").Trim();
1.21.2014 Update
$timeoutSeconds = 5
$code = {
((net user $winxp /domain | Select-String "Full Name") -replace "Full Name","").Trim(); # your commands here, e.g.
}
$j = Start-Job -ScriptBlock $code
if (Wait-Job $j -Timeout $timeoutSeconds) { $fullnamexp = Receive-Job $j }
Remove-Job -force $j
While #mjolinor may have indeed provided you an alternative approach, here is a direct answer to your general question: how do you force a timeout in PowerShell?
Wrap whatever you wish to time-limit in a script block, run that as a job, then use the Wait-Job cmdlet to time-limit the operation. Wait-Job will return either at the end of the timeout period or when the script block completes, whichever occurs first. After Wait-Job returns, you can examine the job state ($j.state) to determine whether it was interrupted or not, if it matters to you.
$timeoutSeconds = 5 # set your timeout value here
$j = Start-Job -ScriptBlock {
# your commands here, e.g.
Get-Process
}
"job id = " + $j.id # report the job id as a diagnostic only
Wait-Job $j -Timeout $timeoutSeconds | out-null
if ($j.State -eq "Completed") { "done!" }
elseif ($j.State -eq "Running") { "interrupted" }
else { "???" }
Remove-Job -force $j #cleanup
2014.01.18 Update
Here is a bit more streamlining approach that also includes the practical step of getting information out of the script block with Receive-Job, assuming what you want is generated on stdout:
$timeoutSeconds = 3
$code = {
# your commands here, e.g.
Get-ChildItem *.cs | select name
}
$j = Start-Job -ScriptBlock $code
if (Wait-Job $j -Timeout $timeoutSeconds) { Receive-Job $j }
Remove-Job -force $j
You can use Start-Sleep to pause the script:
Start-Sleep -s 5
net doesn't explicitly allow you to set a time out on it's operations, but you could check out this link on changing the ipv4 timeout for your sockets:
http://www.cyberciti.biz/tips/linux-increasing-or-decreasing-tcp-sockets-timeouts.html
The only thing else I could imagine is spawning a worker thread but I don't even know if that's possible in bash, I'm not fluid enough in it to answer that; plus it opens you up to sync problems and all sorts of multi threaded issues beyond what you're trying to accomplish quickly in a bash script to begin with! :P
Does this help?
$query = (dsquery user -samid $winxp)
if ($query) {$fullnamexp = ($query | dsget user -display)[1].trim()}
$fullnamexp
This solution doesn't work for me. remove-job -force $j takes over 5 seconds in this example.
$timeoutseconds = 1
$start = get-date
$j = start-job -scriptblock { Resolve-DnsName 1.1.1.1 }
if (wait-job $j -timeout $timeoutseconds) { $fullnamexp = receive-job $j }
remove-job -force $j
(get-date) - $start
Days : 0
Hours : 0
Minutes : 0
Seconds : 5
Milliseconds : 342
Ticks : 53426422
TotalDays : 6.18361365740741E-05
TotalHours : 0.00148406727777778
TotalMinutes : 0.0890440366666667
TotalSeconds : 5.3426422
TotalMilliseconds : 5342.6422
Here's a simple timeout example with notepad:
notepad
if (-not $(wait-process notepad 10; $?)) { stop-process -name notepad }
$watchdog = 10 #seconds
$start_time = Get-Date
$j = Start-Job -ScriptBlock{
#timeout command
if ($true) {
$i = 0
while($true) {
Write-Host "Count: $i"
Start-Sleep -Milliseconds 100
$i++
}
}
write-host "Hello"
}
while($true) {
if ($j.HasMoreData) {
Receive-Job $j
Start-Sleep -Milliseconds 200
}
$current = Get-Date
$time_span = $current - $start_time
if ($time_span.TotalSeconds -gt $watchdog) {
write-host "TIMEOUT!"
Stop-Job $j
break
}
if (-not $j.HasMoreData -and $j.State -ne 'Running') {
write-host "Finished"
break
}
}
Remove-Job $j

Managing the running time of background jobs. Timing out if not completed after x seconds,

I would like to time my background jobs (started with start-job) and time them out after x seconds. I find it hard however to keep track of the running time on each separate job (I am running aprox 400 jobs).
I wish there was a way to time out the job and set it to failed if not completed in X seconds, but I find no timeout-parameter.
What would be a good way to track the individual run-time of the jobs?
I guess I could create a hashtable with start-time of each job and the job-id and check against the running state and do a manual timeout, but that sounds kinda "inventing the wheel".
Any ideas?
Edit
Thank you everyone for a fruitful discussion and great inspiration on this topic!
You can use a hash table of timers:
$jobtimer = #{}
foreach ($job in $jobs){
start-job -name $job -ScriptBlock {scriptblock commands}
$jobtimer[$job] = [System.Diagnostics.Stopwatch]::startnew()
}
The running time of each job will be in $jobtimer[$job].elapsed
Just walk through the list of running jobs and stop any that have run past your timeout spec e.g.:
$timeout = [timespan]::FromMinutes(1)
$now = Get-Date
Get-Job | Where {$_.State -eq 'Running' -and
(($now - $_.PSBeginTime) -gt $timeout)} | Stop-Job
BTW there are more properties to a job object than the default formatting shows e.g.:
3 > $job | fl *
State : Running
HasMoreData : True
StatusMessage :
Location : localhost
Command : Start-sleep -sec 30
JobStateInfo : Running
Finished : System.Threading.ManualResetEvent
InstanceId : de370ea8-763b-4f3b-ba0e-d45f402c8bc4
Id : 3
Name : Job3
ChildJobs : {Job4}
PSBeginTime : 3/18/2012 11:07:20 AM
PSEndTime :
PSJobType : BackgroundJob
Output : {}
Error : {}
Progress : {}
Verbose : {}
Debug : {}
Warning : {}
You can specify the timeout option of Wait-Job:
-Timeout
Determines the maximum wait time for each background job, in seconds.
The default, -1, waits until the job completes, no matter how long it
runs. The timing starts when you submit the Wait-Job command, not the
Start-Job command.
If this time is exceeded, the wait ends and the command prompt
returns, even if the job is still running. No error message is
displayed.
Here's some example code:
This part just makes some test jobs:
Remove-Job -Name *
$jobs = #()
1..10 | % {
$jobs += Start-Job -ScriptBlock {
Start-Sleep -Seconds (Get-Random -Minimum 5 -Maximum 20)
}
}
The variable $timedOutJobs contains jobs that timed out. You can then restart them or what have you.
$jobs | Wait-Job -Timeout 10
$timedOutJobs = Get-Job | ? {$_.State -eq 'Running'} | Stop-Job -PassThru
For completeness, this answer combines the maximum seconds per job and the maximum concurrent jobs running. As this is what most people are after.
The example below retrieves the printer configuration for each print server. There can be over 3000 printers, so we added throttling.
$i = 0
$maxConcurrentJobs = 40
$maxSecondsPerJob = 60
$jobTimer = #{ }
$StopLongRunningJobs = {
$jobTimer.GetEnumerator().where( {
($_.Value.IsRunning) -and
($_.Value.Elapsed.TotalSeconds -ge $maxSecondsPerJob)
}).Foreach( {
$_.Value.Stop()
Write-Verbose "Stop job '$($_.Name.Name)' that ran for '$($_.Value.Elapsed.TotalSeconds)' seconds"
Stop-Job $_.Name
})
}
Foreach ($Computer in #($GetPrinterJobResults.Where( { $_.Data }) )) {
foreach ($Printer in $Computer.Data) {
do {
& $StopLongRunningJobs
$running = #(Get-Job -State Running)
$Wait = $running.Count -ge $maxConcurrentJobs
if ($Wait) {
Write-Verbose 'Waiting for jobs to fininsh'
$null = $running | Wait-Job -Any -Timeout 5
}
} while ($Wait)
$i++
Write-Verbose "$I $($Computer.ComputerName) Get print config '$($Printer.Name)'"
$Job = $Printer | Get-PrintConfiguration -AsJob -EA Ignore
$jobtimer[$Job] = [System.Diagnostics.Stopwatch]::StartNew()
}
}
$JobResult = Get-Job | Wait-Job -Timeout $maxSecondsPerJob -EA Ignore
$JobResult = Get-Job | Stop-Job -EA Ignore # Add this line if you want to stop the Jobs that reached the max time to wait (TimeOut)
$JobResult = Get-Job | Receive-Job -EA Ignore
$JobResult.count