./backup.ps1
$job = Start-Job {$i=0; $c=0; while (1) {
Write-Progress Activity "Step $i"; $i++; Start-Sleep -sec 1 }}
while ($job.State -eq 'Running' -And $c -lt 5) {
$c++;
$progress=$job.ChildJobs[0].progress;
$progress | %{$_.StatusDescription};
$progress.Clear(); Start-Sleep 1 }
I have been trying to work around a script which waits for backup to complete and than executes the next block of code, but i could not do it with start-job and wait-job, i found the above code and pasted into my document and it worked, but as i am new to powershell i dont know what exactly this script is doing
Can't you just pipe the job to out-null using this syntax?
& ./backup.ps1|Out-Null
This will wait to do next process until $job is done and released.
comment out everything after ./backup that is in regards to waiting for the script call to be finished.
You're just callling a PS script inside a PS script. if you want to wait for the first one to be done before you start your next task, add the |Out-Null to the end
Also - it's not going to have all the verbose output you have in your current set up. But it's quick, efficient, and much less code.
Related
Hello all and good afternoon!
I had a quick question regarding -asjob running with invoke-command.
If I run 2 Invoke-Command's using -asjob, does it run simultaneously when I try to receive the ouput? Does this mean wait-job waits till the first job specified is finished running to get the next results?
Write-Host "Searching for PST and OST files. Please be patient!" -BackgroundColor White -ForegroundColor DarkBlue
$pSTlocation = Invoke-Command -ComputerName localhost -ScriptBlock {Get-Childitem "C:\" -Recurse -Filter "*.pst" -ErrorAction SilentlyContinue | % {Write-Host $_.FullName,$_.lastwritetime}} -AsJob
$OSTlocation = Invoke-Command -ComputerName localhost -ScriptBlock {Get-Childitem "C:\Users\me\APpdata" -Recurse -Filter "*.ost" -ErrorAction SilentlyContinue | % {Write-Host $_.FullName,$_.lastwritetime} } -AsJob
$pSTlocation | Wait-Job | Receive-Job
$OSTlocation | Wait-Job | Receive-Job
Also, another question: can i save the output of the jobs to a variable without it showing to the console? Im trying to make it where it checks if theres any return, and if there is output it, but if theres not do something else.
I tried:
$job1 = $pSTlocation | Wait-Job | Receive-Job
if(!$job1){write-host "PST Found: $job1"} else{ "No PST Found"}
$job2 = $OSTlocation | Wait-Job | Receive-Job
if(!$job2){write-host "OST Found: $job2"} else{ "No OST Found"}
No luck, it outputs the following:
Note: This answer does not directly answer the question - see the other answer for that; instead, it shows a reusable idiom for a waiting for multiple jobs to finish in a non-blocking fashion.
The following sample code uses the child-process-based Start-Job cmdlet to create local jobs, but the solution equally works with local thread-based jobs created by Start-ThreadJob as well as jobs based on remotely executing Invoke-Command -ComputerName ... -AsJob commands, as used in the question.
It shows a reusable idiom for a waiting for multiple jobs to finish in a non-blocking fashion that allows for other activity while waiting, along with collecting per-job output in an array.
Here, the output is only collected after each job completes, but note that collecting it piecemeal, as it becomes available, is also an option, using (potentially multiple) Receive-Job calls even before a job finishes.
# Start two jobs, which run in parallel, and store the objects
# representing them in array $jobs.
# Replace the Start-Job calls with your
# Invoke-Command -ComputerName ... -AsJob
# calls.
$jobs = (Start-Job { Get-Date; sleep 1 }),
(Start-Job { Get-Date '1970-01-01'; sleep 2 })
# Initialize a helper array to keep track of which jobs haven't finished yet.
$remainingJobs = $jobs
# Wait iteratively *without blocking* until any job finishes and receive and
# output its output, until all jobs have finished.
# Collect all results in $jobResults.
$jobResults =
while ($remainingJobs) {
# Check if at least 1 job has terminated.
if ($finishedJob = $remainingJobs | Where State -in Completed, Failed, Stopped, Disconnected | Select -First 1) {
# Output the just-finished job's results as part of custom object
# that also contains the original command and the
# specific termination state.
[pscustomobject] #{
Job = $finishedJob.Command
State = $finishedJob.State
Result = $finishedJob | Receive-Job
}
# Remove the just-finished job from the array of remaining ones...
$remainingJobs = #($remainingJobs) -ne $finishedJob
# ... and also as a job managed by PowerShell.
Remove-Job $finishedJob
} else {
# Do other things...
Write-Host . -NoNewline
Start-Sleep -Milliseconds 500
}
}
# Output the jobs' results
$jobResults
Note:
It's tempting to try $remainingJobs | Wait-Job -Any -Timeout 0 to momentarily check for termination of any one job without blocking execution, but as of PowerShell 7.1 this doesn't work as expected: even already completed jobs are never returned - this appears to be bug, discussed in GitHub issue #14675.
If I run 2 Invoke-Command's using -asjob, does it run simultaneously when I try to receive the output?
Yes, PowerShell jobs always run in parallel, whether they're executing remotely, as in your case (with Invoke-Command -AsJob, assuming that localhost in the question is just a placeholder for the actual name of a different computer), or locally (using Start-Job or Start-ThreadJob).
However, by using (separate) Wait-Job calls, you are synchronously waiting for each jobs to finish (in a fixed sequence, too). That is, each Wait-Job calls blocks further execution until the target job terminates.[1]
Note, however, that both jobs continue to execute while you're waiting for the first one to finish.
If, instead of waiting in a blocking fashion, you want to perform other operations while you wait for both jobs to finish, you need a different approach, detailed in the the other answer.
can i save the output of the jobs to a variable without it showing to the console?
Yes, but the problem is that in your remotely executing script block ({ ... }) you're mistakenly using Write-Host in an attempt to output data.
Write-Host is typically the wrong tool to use, unless the intent is to write to the display only, bypassing the success output stream and with it the ability to send output to other commands, capture it in a variable, or redirect it to a file. To output a value, use it by itself; e.g., $value instead of Write-Host $value (or use Write-Output $value, though that is rarely needed); see this answer.
Therefore, your attempt to collect the job's output in a variable failed, because the Write-Host output bypassed the success output stream that variable assignments capture and went straight to the host (console):
# Because the job's script block uses Write-Host, its output goes to the *console*,
# and nothing is captured in $job1
$job1 = $pSTlocation | Wait-Job | Receive-Job
(Incidentally, the command could be simplified to
$job1 = $pSTlocation | Receive-Job -Wait).
[1] Note that Wait-Job has an optional -Timeout parameter, which allows you to limit waiting to at most a given number of seconds and return without output if the target job hasn't finished yet. However, as of PowerShell 7.1, -Timeout 0 for non-blocking polling for whether jobs have finished does not work - see GitHub issue #14675.
I have a PowerShell script that connects to a database and pulls a list of user data. I take this data and create a foreach loop to run a script for the data.
This is working but its slow as the results could be 1000+ entries, and it has to complete the Script.bat for User A before it can start User B. The Script.bat for a single user is independent from another and takes ~30s for each user.
Is there a way to speed this up at all? I've been playing with -Parallel, ForEach-Object and workflow but I can't get it to work, likely due to me being a noob in PS.
foreach ($row in $Dataset.tables[0].rows)
{
$UserID=$row.value
$DeviceID=$row.value1
$EmailAddress=$row.email_address
cmd.exe /c "`"$PSScriptRoot`"\bin\Script.bat -c `" -Switch $UserID`" >> `"$PSScriptRoot`"\${FileName3}_REST_${DateTime}.txt 2> nul";
}
You said it yourself, your bottleneck is with the batch file in your script, not the loop itself. foreach (as opposed to ForEach-Object) is already the faster foreach loop mechanism in PowerShell. Investigate your batch file to find out why it takes 30 seconds to complete, and optimize it where you can.
Using Jobs
Note: Start-Job will run the job under another process. If you have PowerShell Core you can make use of the Start-ThreadJob cmdlet in lieu of Start-Job. This will start your job as part of another thread of the same process instead of starting another process.
If you can't optimize your batch script or optimize it to meet your needs, then you can consider using Start-Job to kick off the job to execute asynchronously, and then check the result and get any output from it using Receive-Job. For example:
# Master list of jobs you need to check the result of later
$jobs = New-Object System.Collections.Generic.List[System.Management.Automation.Job]
# Run your script for each row
foreach ($row in $Dataset.tables[0].rows)
{
$UserID=$row.value
$DeviceID=$row.value1
$EmailAddress=$row.email_address
# Use Start-Job here to kick off the script and store the job information
# for later retrieval.
# The $using: scope modifier allows you to make use of variables that were
# defined in the session calling Start-Job
$job = Start-Job -ScriptBlock { cmd.exe /c "`"${using:PSScriptRoot}`"\bin\Script.bat -c `" -Switch ${using:UserID}`" >> `"${using:PSScriptRoot}`"\${using:FileName3}_REST_${DateTime}.txt 2> nul"; }
# Add the execution to the $jobs list to check the result of later
# Casting to void here prevents the Add method from returning the object
# we've added.
[void]$jobs.Add($job)
}
# Wait for the jobs to be done
Write-Host 'Waiting for all jobs to complete...'
while( $jobs | Where-Object { $_.State -eq 'Running' } ){
Start-Sleep -s 10
}
# Retrieve the output of the jobs
foreach( $j in $jobs ) {
Receive-Job $j
}
Note: Since you have ~1000 times you need to execute this script, you may want to consider writing your logic to only run a certain number of jobs at a time. My example above starts all necessary jobs without regarding the number that may execute at once.
For more information about jobs and the properties you can inspect on a running/completed job, check the links below:
About Jobs
Job Class
Using Scope*
* The documentation states that the using scope can only be declared when working with remote sessions, but this seems to work fine with Start-Job even if the job is local.
I have been writing a script in PowerShell V4 on Windows 8.1 which makes use of background processes and events. I have found something which is a little strange. Rather than post the 2,500 lines or so of my script I have included a much shorter program which exhibits the odd behaviour. I expect it is something I am doing wrong but I cannot see what the problem is. The code is as follows:
`
# Scriptblocks to simulate the background task and the action to
# take when an event is raised
[scriptblock] $MyScript = {
for ($i = 0;$i -lt 30;$i++)
{
[console]::writeline("This is a test - $i")
start-sleep -m 500
}
}
[scriptblock] $StateChanged = {
[console]::writeline("The state changed")
}
# Create a runspace pool
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, [int] $env:NUMBER_OF_PROCESSORS + 1)
$RunspacePool.ApartmentState = "MTA"
$RunspacePool.Open()
# Create and start the background task to run
$PS = [powershell]::Create()
[void] $PS.AddScript($MyScript)
$PS.RunspacePool = $RunspacePool
$Asyncresult = $PS.BeginInvoke()
# Register an interest in the InvocationStateChanged event for
# the background task. Should the event happen (which it will)
# run the $StateChanged scriptblock
Register-ObjectEvent -InputObject $PS -EventName InvocationStateChanged -Action $StateChanged
# The loop that simulates the main purpose of the script
[int] $j = 0
while ($PS.InvocationStateInfo.State -eq [System.Management.Automation.PSInvocationState]::Running)
{
if ($j -eq 2)
{
[void] $PS.BeginStop($NULL, $NULL)
}
"Running: $j" | out-host
sleep -m 400
$j = $j + 1
}
sleep 10
`
Essentially all it does is create a runspace to run a powershell scriptblock and while that is running something else happens in the foreground. I simulate someone pressing a button or, for whatever reason, a beginstop method being executed to stop the background process. That all works and the background process duly stops. However, I have registered an event for the background powershell script which runs a scriptblock when the background job changes state. The strange thing is that the scriptblock gets invoked twice and I cannot work out why.
Here is some output from running the script:
E:\Test Programs>powershell -file .\strange.ps1
This is a test - 0
Running: 0
This is a test - 1
Running: 1
The state changed
Running: 2
The state changed
E:\Test Programs>
As you can see it displays "The state changed" twice. They are a fraction of a second apart. I put a sleep 10 at the end to eliminate the possibility that it is the script stopping that is causing the second "The state changed" message.
If anyone can explain what is wrong I would be very grateful.
The InvocationStateChanged event is likely being called when on Running and Completed
If you change $StateChanged to also include the $Sender.InvocationStateInfo.State automatic variable properties, like this:
[scriptblock] $StateChanged = {
[console]::writeline("The state changed to: $($Sender.InvocationStateInfo.State)")
}
Your output will probably look like:
This is a test - 0
Running: 0
This is a test - 1
Running: 1
The state changed to: Running
Running: 2
The state changed: Completed
I am very sorry that I didn't reply a year and a half ago as I should have done. For some reason I never had a notification that there was an answer and, to be honest, I just forgot to check.
Anyway, thanks for the answer. I still have my little test script although the thing I was actually writing has been dumped and rewritten but I did check with my little test script and what I get is:
Running: 0
Running: 1
This is a test - 1
The state changed to: Stopped
The state changed to: Stopped
Running: 2
This is now on Windows 10.
Oh well, thanks for the suggestion anyway. As I said I've rewritten the original script so this is now just for interest.
Best wishes........
Colin
I'm trying to create a script that can export a user's mailbox to a PST, remotely (Exchange Server 2010 console is installed on the server we're running this from, and the module is loaded correctly). It's being done using a script so our L2 admins do not have to manually perform the task. Here's the MWE.
$UserID = Read-Host "Enter username"
$PstDestination = "\\ExServer\Share\$UserID.pst"
$Date = Get-Date -Format "yyyyMMddhhmmss"
$ExportName = "$UserID" + "$Date"
try {
New-MailboxExportRequest -Mailbox $UserID -FilePath $PstDestination -Name $ExportName -ErrorAction Stop -WarningAction SilentlyContinue | Out-Null
# Loop through the process to track its status and write progress
do {
$Percentage = (Get-MailboxExportRequest -Name $ExportName | Get-MailboxExportRequestStatistics).PercentComplete
Write-Progress "Mailbox export is in progress." -Status "Export $Percentage% complete" -PercentComplete "$Percentage"
}
while ($Percentage -ne 100)
Write-Output "$UserID`'s mailbox has been successfully exported. The archive can be found at $PstDestination."
}
catch {
Write-Output "There was an error exporting the mailbox. The process was aborted."
}
The problem is, as soon as we initiate the export, the task gets Queued. Sometimes, the export remains queued for a very long time, and the script is currently unable to figure out when the task begins, and when it does, is unable to display the progress correctly. The export happens in the background, but the script remains stuck there. So anything after the export, does not get executed, and the whole thing then has to be done manually.
Please suggest a way to handle this?
I tried adding a wait timer and then a check to see if the export has begun. It didn't quite work as expected.
Two things. First one is more about performance/hammering Exchange with unnesacary requests in do/while loop. Start-Sleep -Seconds 1 (or any other delay that makes sense depending on the mailbox size(s)) inside the loop is a must.
Second: rather than wait for job to start, just resume it yourself:
if ($request.Status -eq 'Queued') {
$request | Resume-MailboxExportRequest
}
Is it possible to force the execution of some code if a PowerShell script is forcefully terminated? I have tried try..finally and Traps, but they both don't seem to work, at least when I press Ctrl-C from PowerShell ISE.
Basically, I have a Jenkins build that executes a PowerShell script. If for any reason I want to stop the build from within Jenkins, I don't want any subprocess to lock the files, hence keeping my build project in a broken state until an admin manually kill the offending processes (nunit-agent.exe in my case). So I want to be able to force the execution of a code that terminates nunit-agent.exe if this happens.
UPDATE: As #Frode suggested below, I tried to use try..finally:
$sleep = {
try {
Write-Output "In the try block of the job."
Start-Sleep -Seconds 10
}
finally {
Write-Output "In the finally block of the job."
}
}
try {
$sleepJob = Start-Job -ScriptBlock $sleep
Start-Sleep -Seconds 5
}
finally {
Write-Output "In the finaly block of the script."
Stop-Job $sleepJob
Write-Output "Receiving the output from the job:"
$content = Receive-Job $sleepJob
Write-Output $content
}
Then when I executed this and broke the process using Ctrl-C, I got no output. I thought that what I should got is:
In the finally block of the script.
Receiving the output from the job:
In the try block of the job.
In the finally block of the job.
I use try {} finally {} for this. The finally-block runs when try is done or if you use ctrl+c, so you need to either run commands that are safe to run either way, ex. it doesn't matter if you kill a process that's already dead..
Or you could add a test to see if the last command was a success using $?, ex:
try {
Write-Host "Working"
Start-Sleep -Seconds 100
} finally {
if(-not $?) { Write-Host "Cleanup on aisle 5" }
Write-Host "Done"
}
Or create your own test (just in case the last command in try failed for some reason):
try {
$IsDone = $false
Write-Host "Working"
Start-Sleep -Seconds 100
#.....
$IsDone = $true
} finally {
if(-not $IsDone) { Write-Host "Cleanup on aisle 5" }
Write-Host "Done"
}
UPDATE: The finally block will not work for output as the pipeline is stopped on CTRL+C.
Note that pressing CTRL+C stops the pipeline. Objects that are sent to
the pipeline will not be displayed as output. Therefore, if you
include a statement to be displayed, such as "Finally block has run",
it will not be displayed after you press CTRL+C, even if the Finally
block ran.
Source: about_Try_Catch_Finally
However, if you save the output from Receive-Job to a global variable like $global:content = Receive-Job $sleepJob you can read it after the finally-block. The variable is normally created in a different local scope and lost after the finally-block.