I'm very new to powershell.
My question is very simple if you know anything about powershell.
In the code below I'm trying to fire an event from a piece of code running asynchronously as a job. For some reason, the code doesn't work.
$callback = {
param($event);
Write "IN CALLBACK";
};
$jobScript = {
while ($true) {
sleep -s 1;
"IN JOB SCRIPT";
New-Event myEvent;
}
}
Register-EngineEvent -SourceIdentifier myEvent -Action $callback;
Start-Job -Name Job -ScriptBlock $jobScript;
while ($true) {
sleep -s 1;
"IN LOOP";
}
Expected output:
IN LOOP
IN JOB SCRIPT
IN CALLBACK
IN LOOP
IN JOB SCRIPT
IN CALLBACK
...
Actual output:
IN LOOP
IN LOOP
IN LOOP
IN LOOP
...
After some reading, I changed this line
Start-Job -Name Job -ScriptBlock $jobScript
to
Start-Job -Name Job -ScriptBlock $jobScript | Wait-Job | Receive-Job;
and I get no output at all, because job never finishes.
It's kind of asynchoronous, but not really.
It would be fairly simple to acomplish in JS.
const fireEvent = (eventName) => { ... }
const subscribeToEvent = (eventName, callback) => { ... }
const callback = () => console.log('IN CALLBACK')
subscribeToEvent('myEvent', callback);
setInterval(() => {
console.log('IN LOOP')
fireEvent('myEvent');
}, 1000)
Please, help!
Running:
while ($true) {
sleep -s 1;
"IN LOOP";
}
will always only give you:
IN LOOP
IN LOOP
IN LOOP
IN LOOP
Running this block:
$callback = {
param($event);
Write "IN CALLBACK";
};
$jobScript = {
while ($true) {
sleep -s 1;
"IN JOB SCRIPT";
New-Event myEvent;
}
}
Register-EngineEvent -SourceIdentifier myEvent -Action $callback;
Start-Job -Name Job -ScriptBlock $jobScript;
gives you a job called Job. This job will run untill you stop it. Output of the job will be something like this:
get-job Job | receive-job
IN JOB SCRIPT
RunspaceId : cf385728-926c-4dda-983e-6a5cfd4fd67f
ComputerName :
EventIdentifier : 1
Sender :
SourceEventArgs :
SourceArgs : {}
SourceIdentifier : myEvent
TimeGenerated : 6/15/2019 3:35:09 PM
MessageData :
IN JOB SCRIPT
RunspaceId : cf385728-926c-4dda-983e-6a5cfd4fd67f
ComputerName :
EventIdentifier : 2
Sender :
SourceEventArgs :
SourceArgs : {}
SourceIdentifier : myEvent
TimeGenerated : 6/15/2019 3:35:10 PM
MessageData :
...
JavaScript is really Greek to me, but it seems your issue is that the event is not registered in the scope of the job. If you move the registration to the job, it behaves a bit more like you expect.
If you do this:
$jobScript = {
$callback = {
Write-Host "IN CALLBACK"
}
$null = Register-EngineEvent -SourceIdentifier myEvent -Action $callback
while ($true) {
sleep -s 1
Write-Host "IN JOB SCRIPT"
$Null = New-Event myEvent
}
}
$Null = Start-Job -Name Job -ScriptBlock $jobScript;
while ($true) {
sleep -s 1
"IN LOOP"
Get-Job -Name Job | Receive-Job
}
When running $null = Register-EngineEvent ... or $null = Start-Job ... you avoid the objects these commands creates to be displayed at the console. Furthermore you do not need to terminate your lines with ; in PowerShell.
To complement Axel Anderson's helpful answer:
Register-EngineEvent subscribes to an event in the current session, whereas commands launched with Start-Job run in a background job that is a child process, which by definition is a separate session.
Similarly, any events you raise with New-Event are only seen in the same session, which is why the calling session never saw the events.
By moving all event logic into the background job, as in Axel's answer, the events are processed in the background job, but there's an important limitation - which may or may not matter to you:
You won't be able to capture output from the event handler: while you can make its output print to the console using Write-Host, that output cannot be captured for programmatic processing. Also, such Write-Host output cannot be suppressed.
By contrast, output sent to the success output stream directly from the background job - such as "IN JOB SCRIPT" is, implicitly (implied use of Write-Output) - can be captured via Receive-Job, which retrieves the output from background jobs.
Therefore, perhaps the following is sufficient in your case, which doesn't require use of events at all:
# Code to execute in the background, defined a script block.
$jobScript = {
while ($true) {
Start-Sleep -Seconds 1
"JOB OUTPUT #$(($i++))"
}
}
# Launch the background job and save the job object in variable $job.
$job = Start-Job -Name Job -ScriptBlock $jobScript;
# Periodically relay the background job's output.
while ($true) {
Start-Sleep -Seconds 1
"IN LOOP, about to get latest job output:"
$latestOutput = Receive-Job $job
"latest output: $latestOutput"
}
For an explanation of the $(($i++)) construct, see this answer.
The above yields the following:
IN LOOP, about to get latest job output:
latest output:
IN LOOP, about to get latest job output:
latest output: JOB OUTPUT #0
IN LOOP, about to get latest job output:
latest output: JOB OUTPUT #1
IN LOOP, about to get latest job output:
latest output: JOB OUTPUT #2
IN LOOP, about to get latest job output:
latest output: JOB OUTPUT #3
IN LOOP, about to get latest job output:
latest output: JOB OUTPUT #4
IN LOOP, about to get latest job output:
latest output: JOB OUTPUT #5
IN LOOP, about to get latest job output:
...
Related
It sounds like a reasonable expectation that events fired from one and the same thread should be received in the order in which they were fired. However, that doesn't seem to be the case. Is this a known/documented behavior, and is there any recourse to correct it?
Below are two ready-to-run code snippets that exhibit the issue, tested with PS v5.1 under both Win7 and Win10.
(a) Events fired from a thread in a separate job (i.e. a different process).
$events = 1000
$recvd = 0
$ooseq = 0
$job = Register-EngineEvent -SourceIdentifier 'Posted' -Action {
$global:recvd++
if($global:recvd -ne $event.messageData) {
$global:ooseq++
("-?- event #{0} received as #{1}" -f $event.messageData, $global:recvd)
} }
$run = Start-Job -ScriptBlock {
Register-EngineEvent -SourceIdentifier 'Posted' -Forward
for($n = 1; $n -le $using:events; $n++) {
[void] (New-Event -SourceIdentifier 'Posted' -MessageData $n)
} }
Receive-Job -Job $run -Wait -AutoRemoveJob
Unregister-Event -SourceIdentifier 'Posted'
Receive-Job -Job $job -Wait -AutoRemoveJob
if($events -eq $script:recvd) {
("total {0} events" -f $events)
} else {
("total {0} events events, {1} received" -f $events, $recvd)
}
if($ooseq -ne 0) {
("{0} out-of-sequence events" -f $ooseq)
}
Sample output from a failure case (out of a batch of 100 consecutive runs).
-?- event #545 received as #543
-?- event #543 received as #544
-?- event #546 received as #545
-?- event #544 received as #546
total 1000 events
4 out-of-sequence events
(b) Events fired from a separate runspace (i.e. a different thread).
$recvd = 0
$ooseq = 0
$job = Register-EngineEvent -SourceIdentifier 'Posted' -Action {
$global:recvd++
if($recvd -ne $event.messageData) {
$global:ooseq++
("-?- event #{0} received as #{1}" -f $event.messageData, $recvd)
}}
$sync = [hashTable]::Synchronized(#{})
$sync.Host = $host
$sync.events = 1000
$sync.posted = 0
$rs = [runspaceFactory]::CreateRunspace()
$rs.ApartmentState = "STA"
$rs.ThreadOptions = "ReUseThread"
$rs.Open()
$rs.SessionStateProxy.SetVariable("sync",$sync)
$ps = [powerShell]::Create().AddScript({
for($n = 1; $n -le $sync.events; $n++) {
$sync.Host.Runspace.Events.GenerateEvent('Posted', $null, $null, $n)
$sync.posted++
}})
$ps.runspace = $rs
$thd = $ps.BeginInvoke()
$ret = $ps.EndInvoke($thd)
$ps.Dispose()
Unregister-Event -SourceIdentifier 'Posted'
Receive-Job -Job $job -Wait -AutoRemoveJob
if($sync.events -eq $recvd) {
("total {0} events" -f $sync.events)
} else {
("total {0} events fired, {1} posted, {2} received" -f $sync.events, $sync.posted, $recvd)
}
if($ooseq -ne 0) {
("{0} out-of-sequence events" -f $ooseq)
}
Failure cases resemble the sample one posted under (a) above, except a few runs also had events dropped altogether. This, however, is more likely related to the other question Action-based object events sometimes lost.
total 1000 events fired, 1000 posted, 999 received
484 out-of-sequence events
[ EDIT ] I ran some additional tests for case (b) specifically, and confirmed that:
the receiving Action (where $global:recvd++) is always called on the same managed thread (this was confirmed by saving and comparing the [System.Threading.Thread]::CurrentThread.ManagedThreadId between calls);
the receiving Action is not re-entered during execution (this was confirmed by adding a global "nesting" counter, wrapping the Action between [System.Threading.Interlocked]::Increment/Decrement calls and checking that the counter never takes any values other than 0 and 1).
These eliminate a couple of possible race conditions, but still do not explain why the observed behavior is happening or how to correct it, so the original question remains open.
Is this a known/documented behavior?
"Normally" event handling is asynchronous by design. And this is the case in PowerShell with cmdlets like Register-EngineEvent -Action. This is indeed known and intended behaviour. You can read more about PowerShell eventing here and here. Both Microsoft sources point out, that this way of event handling is asynchronous:
PowerShell Eventing lets you respond to the asynchronous notifications
that many objects support.
and
NOTE These cmdlets can only be used for asynchronous .NET events. It’s
not possible to set up event handlers for synchronous events using the
PowerShell eventing cmdlets. This is because synchronous events all
execute on the same thread and the cmdlets expect (require) that the
events will happen on another thread. Without the second thread, the
PowerShell engine will simply block the main thread and nothing will
ever get executed.
So that's basically what you are doing. You forward events from your background job to the event subscriber that has an action defined and perform the action without blocking your background job. As far as I can tell, there is nothing more to expect. There is no requirement specified to process the forwarded events in any special order. Even the -Forward switch does not ensure anything more, except passing the events:
Indicates that the cmdlet sends events for this subscription to the
session on the local computer. Use this parameter when you are
registering for events on a remote computer or in a remote session.
It is hard and maybe impossible to find any documentation on the internals of the cmdlets. Keep in mind that Microsoft does not publish any documentation about internals afaik from the past, instead it is up to the MVPs to guess what happens inside and write books about it (drastically expressed).
So as there is no requirement to process the events in a certain order, and PowerShell just has the task to perfrom actions on an event queue, it is also allowed to perform those actions in parallel to accelerate the processing of the event queue.
Test your scripts on a VM with only one vCPU. The wrong order will still occur sometimes, but way more rarely. So less (real) parallelism, less possibilities to scramble the order. Of course, you cannot prevent the logical parallelism, implemented by different threads executed on one physical core. So some "errors" remain.
Is there any recourse to correct it?
I put "normally" into quotation marks, because there are ways to implement it synchronously. You will have to implement your own event handler of type System.EventHandler. I recommend reading this article to get an example for an implementation.
Another workaround is to store the events in an own event queue and sort them after collection (runs in ISE, not yet in PS):
$events = 10000
$recvd = 0
$ooseq = 0
$myEventQueue = #()
$job = Register-EngineEvent -SourceIdentifier 'Posted' -Action {$global:myEventQueue += $event}
$run = Start-Job -ScriptBlock {
Register-EngineEvent -SourceIdentifier 'Posted' -Forward
for($n = 1; $n -le $using:events; $n++) {
[void] (New-Event -SourceIdentifier 'Posted' -MessageData $n)
}
}
Receive-Job -Job $run -Wait -AutoRemoveJob
Unregister-Event -SourceIdentifier 'Posted'
Receive-Job -Job $job -Wait -AutoRemoveJob
Write-Host "Wrong event order in unsorted queue:"
$i = 1
foreach ($event in $myEventQueue) {
if ($i -ne $event.messageData) {
Write-Host "Event $($event.messageData) received as event $i"
}
$i++
}
$myEventQueue = $myEventQueue | Sort-Object -Property EventIdentifier
Write-Host "Wrong event order in sorted queue:"
$i = 1
foreach ($event in $myEventQueue) {
if ($i -ne $event.messageData) {
Write-Host "Event $($event.messageData) received as event $i"
}
$i++
}
Archived links:
PowerShell eventing async 1
PowerShell eventing async 2
PowerShell eventing sync
I tried to run few Azure cmdlets in background (i tried it by switch parameter "-asJob" and by with Start-Job cmdlet), i push cmdlet operations to the hashtable with certian keys and then iterate running jobs.
The problem is that all cmdlet change state to "Completed" in the same time, even if some job in hashtable actually ended by "Completed".
Feels like that all jobs end with last ending job.
Below i write some example code with problem
$vmssInstances = Get-AzVmssVM -ResourceGroupName $(agentPoolName) -VMScaleSetName $(agentPoolName)
$groupedVmssInstancesByComputerName = #{}
foreach($vmssInstance in $vmssInstances)
{
$groupedVmssInstancesByComputerName[$vmssInstance.OsProfile.ComputerName] = #{vmmsInstance = $vmssInstance; agents = #(); isFullyUpdated = $false}
}
foreach($key in $groupedVmssInstancesByComputerName.Keys)
{
$vmssInstance = $groupedVmssInstancesByComputerName[$key]["vmmsInstance"]
Write-Host "Trying to reimage instance $($vmssInstance.InstanceId)..."
$groupedVmssInstancesByComputerName[$key]["reimageOperation"] = Set-AzVmssVM -Reimage -InstanceId $vmssInstance.InstanceId -ResourceGroupName $(agentPoolName) -VMScaleSetName $(agentPoolName) -AsJob
}
while($true)
{
Get-Job
Start-Sleep -Seconds 10
}
I cant understand what is going on.
Maybe i dont know some features of powershell.
Please help me, guyes!
Try using Wait-Job - Suppresses the command prompt until one or all of the PowerShell background jobs running in the session are completed.
Get-Job | Wait-Job
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/wait-job?view=powershell-6
Let's say you have the following script saved in a file outermost.ps1:
powershell.exe -Command "while ( 1 -eq 1 ) {} "
echo "Done"
When running outermost.ps1 you can only abort this by pressing Ctrl+C, and no output will be written to console. How can I modify it so that the outermost script continues and executes echo "Done" when Ctrl+C is pressed?
This is a simplified version of a real-life scenario where the inner script is actually an executable which only is stoppable by pressing Ctrl+C.
Edit: The script could also be:
everlooping.exe
echo "Done"
but I wanted to provide an example everyone could copy-paste into an editor if they wanted to "try at home".
Start your infinite command/statement as a job, make your PowerShell script process Ctrl+C as regular input (see here), and stop the job when that input is received:
[Console]::TreatControlCAsInput = $true
$job = Start-Job -ScriptBlock {
powershell.exe -Command "while ( 1 -eq 1 ) {} "
}
while ($true) {
if ([Console]::KeyAvailable) {
$key = [Console]::ReadKey($true)
if (($key.Modifiers -band [ConsoleModifiers]::Control) -and $key.Key -eq 'c') {
$job.StopJob()
break
}
}
Start-Sleep -Milliseconds 100
}
Receive-Job -Id $job.Id
Remove-Job -Id $job.Id
echo "Done"
If you need to retrieve output from the job while it's running, you can do so like this in an else branch to the outer if statement:
if ($job.HasMoreData) { Receive-Job -Id $job.Id }
The simplest solution to this is:
Start-Process -Wait "powershell.exe" -ArgumentList "while ( 1 -eq 1 ) {}"
echo "Done"
This will spawn a second window unlinke ansgar-wiechers solution, but solves my problem with the least amount of code.
Thanks to Jaqueline Vanek for the tip
I have a powershell script that starts a job
start-job -scriptblock {
while($true) {
echo "Running"
Start-Sleep 2
}
}
and then it continues executing the rest of the script.
That job, is kind of a monitoring one for the PID of that process.
I would like to synchronously print the PID every n seconds, without having to end the job.
For example, as the rest of the script is being executed, i would like to see output in my console.
Is something like that possible in powershell?
Thanks.
Yes, you can use events:
$job = Start-Job -ScriptBlock {
while($true) {
Register-EngineEvent -SourceIdentifier MyNewMessage -Forward
Start-Sleep -Seconds 3
$null = New-Event -SourceIdentifier MyNewMessage -MessageData "Pingback from job."
}
}
$event = Register-EngineEvent -SourceIdentifier MyNewMessage -Action {
Write-Host $event.MessageData;
}
for($i=0; $i -lt 10; $i++) {
Start-Sleep -Seconds 1
Write-Host "Pingback from main."
}
$job,$event| Stop-Job -PassThru| Remove-Job #stop the job and event listener
Credit goes to this answer. Other useful links:
How to Get Windows PowerShell to Notify You When a Job is Complete
Manage Event Subscriptions with PowerShell - Hey, Scripting Guy! Blog
A script is executing the following steps in a loop, assume both steps take a long time to complete:
$x = DoSomeWork;
Start-Job -Name "Process $x" { DoSomeMoreWork $x; };
Step 1 blocks the script and step 2 does not, of course.
I can easily monitor the progress/state of the loop and step 1 through the console.
What I'd also like to do is monitor the job status of jobs started by step 2 while the batch is still executing.
In general, it is possible to 'attach' or query another powershell session from another session? (Assuming the monitoring session does not spawn the worker session)
If I'm following you, then you cannot share state between two different console instances. That is to say, it's not possible in the way you want to do it. However, it's not true that you cannot monitor a job from the same session. You can signal with events from within the job:
Start-Job -Name "bgsignal" -ScriptBlock {
# forward events named "progress" back to job owner
# this even works across machines ;-)
Register-EngineEvent -SourceIdentifier Progress -Forward
$percent = 0
while ($percent -lt 100) {
$percent += 10
# raise a new progress event, redirecting to $null to prevent
# it ending up in the job's output stream
New-Event -SourceIdentifier Progress -MessageData $percent > $null
# wait 5 seconds
sleep -Seconds 5
}
}
Now you have the choice to either use Wait-Event [-SourceIdentifier Progress], Register-EngineEvent -SourceIdentifier Progress [-Action { ... }] or plain old interactive Get-Event to see and/or act on progress from the same session (or a different machine if you started the job on a remote server.)
It's also entirely possible you don't need the Jobs infrastructure if all work is being done on the local machine. Take a look at an old blog post of mine on the RunspaceFactory and PowerShell objects for a rudimentary script "threadpool" implementation:
http://www.nivot.org/2009/01/22/CTP3TheRunspaceFactoryAndPowerShellAccelerators.aspx
Hope this helps,
-Oisin
State is easy to monitor:
$job = Start-Job -Name "Process $x" { DoSomeMoreWork $x }
$job.state
If you don't need to retrieve any output data from the function then you can write to output like so:
$job = Start-Job {$i=0; while (1) { "Step $i"; $i++; Start-Sleep -sec 1 }}
while ($job.State -eq 'Running')
{
Receive-Job $job.id
}
If you do need to capture the output, then you could use the progress stream I think:
$job = Start-Job {$i=0; while (1) {
Write-Progress Activity "Step $i"; $i++; Start-Sleep -sec 1 }}
while ($job.State -eq 'Running') {
$progress=$job.ChildJobs[0].progress;
$progress | %{$_.StatusDescription};
$progress.Clear(); Start-Sleep 1 }