Powershell Execute program in parallel and wait end of execution - powershell

I need to execute a program (.exe) in a powershell script in a foreach loop, I need to wait the end of execution before doing some other tasks.
I tried this solution, the program is launched but it's closing immediately
$jobArray = New-Object -TypeName System.Collections.ArrayList
ForEach ($item in Get-Content C:\items.txt) {
$job = Start-Job -ScriptBlock {Start-Process "C:\Development\Console.exe" -ArgumentList /count, /Id:$item, /verbose }
$jobArray.Add($job)
}
Write-Verbose "started" -Verbose
#Wait for all jobs
ForEach ($job in $jobArray) {
$job | Wait-Job
}

A process already is like a job, it runs in parallel. In fact, PowerShell jobs are also just processes, so you currently start a process just to start another process.
Use Start-Process -PassThru without -Wait to capture a process object on which you can wait using Wait-Process.
$processes = ForEach ($item in Get-Content C:\items.txt) {
Start-Process -PassThru -FilePath 'C:\Development\Console.exe' -ArgumentList '/count', "/Id:$item", '/verbose'
}
Write-Verbose "started" -Verbose
# Wait for all processes
$processes | Wait-Process
As another optimization, most of the times you use a foreach or ForEach-Object loop you don't need to use ArrayList explicitly. Just assign the loop statement or pipeline to a variable and let PowerShell create an array for you in an efficient way.

Related

PowerShell Script - Run multiple executables in parallel and wait for all launched executables to terminate before proceeding

I have an executable file (.exe) which has to be run multiple times with different arguments in parallel (ideally on different cores) from a PowerShell script, and at the end wait for all launched executables to terminate. To implement that in my PowerShell script I have used the Start-Job command that runs multiple threads in parallel. And as the script needs to wait for all jobs to finish their execution I used Start-Job in the combination with Get-Job | Wait-Job. This makes the script wait for all of the jobs running in the session to finish:
$SCRIPT_PATH = "path/to/Program.exe"
$jobs = Get-ChildItem -Path $DIR | Foreach-Object {
if ($_ -like "Folder") {
# Do nothing
}
else {
$ARG1_VAR = "Directory\$($_.BaseName)"
$ARG2_VAR = "Directory\$($_.BaseName)\SubDirectory"
$ARG3_VAR = "Directory\$($_.BaseName)\SubSubDirectory"
if (Test-Path -Path $ARG1_VAR)
{
Start-Job -Name -ScriptBlock {
& $using:SCRIPT_PATH -arg1 $using:ARG1_VAR -arg2 $using:ARG2_VAR
}
}
else
{
Start-Job -Name -ScriptBlock {
& $using:SCRIPT_PATH -arg1 $using:ARG1_VAR -arg3 $using:ARG3_VAR
}
}
}
}
$jobs | Receive-Job -Wait -AutoRemoveJob
However, it seems that -FilePath argument of Start-Job does NOT accept .exe files, but only .ps1 files, and therefore I get an exception.
Thus, I decided to use Start-Process command instead which spawns seperate processes instead of seperate threads. But I was not able to find a command that can wait for the termination of all started processed from my script. Therefore, I tried to do it manually by storing all started processes in an array list. And then I tried to wait for each process (using process ID) to terminate. However, that does not seem to work either, because Start-Process -FilePath Program.exe -ArgumentList $ARG_LIST returns NULL, and therefore nothing is saved in the $Process_List.
$SCRIPT_PATH = "path/to/Program.exe"
$procs = Get-ChildItem -Path $DIR | Foreach-Object {
if ($_ -like "Folder") {
# Do nothing
}
else {
$ARG1_VAR = "Directory\$($_.BaseName)"
$ARG2_VAR = "Directory\$($_.BaseName)\SubDirectory"
$ARG3_VAR = "Directory\$($_.BaseName)\SubSubDirectory"
if (Test-Path -Path $ARG1_VAR)
{
$ARG_LIST = #( "-arg1 $ARG1_VAR", "-arg2 $ARG2_VAR")
Start-Process -FilePath $SCRIPT_PATH -ArgumentList $ARG_LIST -PassThru -NoNewWindow
}
else
{
$ARG_LIST = #( "-arg1 $ARG1_VAR", "-arg3 $ARG3_VAR")
Start-Process -FilePath $SCRIPT_PATH -ArgumentList $ARG_LIST -PassThru -NoNewWindow
}
}
}
$procs | Wait-Process
I would appreciate any help. Please note I am using Powershell 5.1, thus ForEach-Object -Parallelconstruct is not supported on my machine.
Thank you!
Regarding your first example with Start-Job, instead of using the -FilePath parameter you could use the -ScriptBlock parameter:
$path = 'path/to/my.exe'
$jobs = Get-ChildItem -Path $DIR | Foreach-Object {
Start-Job -ScriptBlock {
& $using:path -arg1 $using:_ -arg2 $using:ARG2_VAR
}
}
$jobs | Receive-Job -Wait -AutoRemoveJob
Regarding your second example, using Start-Process you should note that, this cmdlet produces no output without the -PassThru switch, hence you're adding effectively nothing to your list.
$processes = Get-ChildItem -Path $DIR | Foreach-Object {
Start-Process -FilePath Program.exe -ArgumentList $ARG_LIST -PassThru
}
With this minor addition of the -PassThru switch you can either use a while loop checking the .HasExited Property of the objects in case you need to do something else with your code while waiting for the processes:
# block the thread until all processes have finished
while($processes.HasExited -contains $false) {
# do something else here if needed
Start-Sleep -Milliseconds 200
}
Or even simpler, as mklement0 points out, if you only need to wait for the processes, you can use Wait-Process:
$processes | Wait-Process

Script-Block parsing issues

I am running some code that is read from an encrypted file and converted into a ScriptBlock. The code will be a full complex script, but for simplicity let's assume it is the following:
"$(date) Agent started." | Out-File -FilePath 'C:\TMP\test_agent.log' -Append
while($true) {
'$(date) Will check back in 30 seconds...' | Out-File -FilePath 'C:\TMP\test_agent.log' -Append
Start-Sleep -Seconds 30
}
Below is the simple code that launches it, and it works just fine (the $sStr variable contains the above script as a string):
$job = Start-Job -ScriptBlock {
$sb = $executioncontext.invokecommand.NewScriptBlock($args[0])
Invoke-Command -ScriptBlock $sb
} -ArgumentList $sStr | Wait-Job -Timeout 1 | Receive-Job
Again, this works fine. However, I need this to run as a new PowerShell process. But when I try the below the ScriptBlock is not parsed properly and I get errors. Here is the modified launcher:
$job = Start-Job -ScriptBlock {
$sb = $executioncontext.invokecommand.NewScriptBlock($args[0])
powershell "Invoke-Command -ScriptBlock $sb"
} -ArgumentList $sStr | Wait-Job -Timeout 1 | Receive-Job
How to start a new powershell process (and kill the parent) so that the ScriptBlock is correctly parsed and executed?
Thanks!
Maybe you could solve your problem with a PowerShell background job, which can be created via Start-Job. Start-Job also has a -ScriptBlock parameter.
Example 7 of Get-Job shows you how to check if the job is already completed, and how you can retrieve the results of the job.
Hope that helps.

Automatically Removing a PowerShell Job when it has finished (asynchronously)

I have a PowerShell cmdlet which I use to simplify connecting to another computer with RDP.
Within that cmdlet I run do the following:
if ($null -ne $Username -and $null -ne $Password) {
Start-Process -FilePath "cmdkey.exe" -ArgumentList #("/generic:`"TERMSRV/$ComputerName`"", "/user:`"$Username`"", "/pass:`"$Password`"") -WindowStyle Hidden
}
Start-Job -ScriptBlock {
param($InstallPath, $ComputerName, $Port, $Username, $Password)
$arguments = #("`"$(Join-Path $InstallPath '\Support Files\MSTSC\Default.rdp')`"")
if ($null -ne $Port) {
$arguments += "/v:`"$($ComputerName):$($Port)`""
} else {
$arguments += "/v:`"$($ComputerName)`""
}
Start-Process -FilePath "mstsc.exe" -ArgumentList $arguments -Wait
if ($null -ne $Username -and $null -ne $Password) {
Start-Process -FilePath "cmdkey.exe" -ArgumentList #("/delete:`"TERMSRV/$ComputerName`"") -WindowStyle Hidden
}
} -ArgumentList #($InstallPath, $ComputerName, $Port, $Username, $Password)
As you can see I add the credentials used to connect to the remote machine, then start a job which executes mstsc.exe, waits for it to finish then removes the credentials.
The problem is I have to wait for mstsc to close before deleting the credentials, as otherwise they get removed before mstsc has a chance to establish the connection and I want this cmdlet to be self contained - returning control immediately to the users command prompt so I can run other commands while I am also using the RDP session which means I can't wait for the job to finish as that mean I am stuck waiting until I disconnect from the remote session:
| Wait-Job | Remove-Job
What I want to do is to be able to Remove-Job once it has completed, perhaps using some kind of callback so I don't have to manually run another command to clean up the Job once I log out of the RDP session and the Job isn't left in a Completed state (which is what I am currently doing but obviously this isn't 'clean).
For the full cmdlet you can see it here for more context:
https://github.com/paulmarsy/Console/blob/master/AdvancedPowerShellConsole/Exports/Functions/Connect-Remote.ps1
I know this is a very old question... You can use Register-ObjectEvent to clean up after jobs. Jobs have a StateChanged event that has an EventSubscriber parameter passed to it containing details of the event and the source job.
Here's an example. Once the job completes the callback will remove both itself and the source job.
$job = Start-Job { Start-Sleep -Seconds 2 }
Register-ObjectEvent -InputObject $job -EventName StateChanged -Action {
Unregister-Event $EventSubscriber.SourceIdentifier
Remove-Job $EventSubscriber.SourceIdentifier
Remove-Job -Id $EventSubscriber.SourceObject.Id
} | Out-Null
So register a scheduledjob to run in 5 minutes that will remove the completed job. I am pretty sure that you can do something like:
Register-ScheduledJob -ScriptBlock {param($computername); Wait-Job -Name $ComputerName|remove-job} -Trigger #{Frequency="Once";At=(get-date).AddMinutes(5).ToString("h:MM tt")} -argumentlist $computername
Then just give your connection a name when you do your Start-Job by appending -Name $ComputerName to the end of the command. That way 5 minutes after you launch it a scheduled task kicks off that finds and clears out that job by name.

Wait for process to end

I have a powershell script that will automatically print all .pdf files in a folder with Foxit. I have it set to wait until Foxit has exited until the script continues. Every once in a while, the script will pause and wait even though Foxit has already exited. Is there a way to make it time out after a certain amount of time?
Here is the code I have:
Start-Process $foxit -ArgumentList $argument -Wait
Move-Item $filePath $printed[$location]
Add-Content "$printLogDir\$logFileName" $logEntry
I've tried the recommendations here and they don't seem to work. For example if I do:
$proc = Start-Process $foxit -ArgumentList $argument
$proc.WaitForExit()
Move-Item $filePath $printed[$location]
Add-Content "$printLogDir\$logFileName" $logEntry
I get:
You cannot call a method on a null-valued expression.
Any help would be greatly appreciated.
I think I figured it out. If I start it with Invoke-WmiMethod I can get the process ID and wait for it, then ask it to time out.
$proc = Invoke-WmiMethod -Class win32_process -Name create -ArgumentList "$foxit $argument"
Wait-Process -Id $proc.ProcessId -Timeout 120
Move-Item $filePath $printed[$location]
Add-Content "$printLogDir\$logFileName" $logEntry
This seems to work pretty consistantly.
One way to implement a timeout is to use a background job:
$Job = start-job -ScriptBlock { write-output 'start';Start-Sleep -Seconds 15 }
$timeout = New-TimeSpan -Seconds 10
$timer = [diagnostics.stopwatch]::StartNew()
While ($timer.Elapsed -le $timeout)
{
if ($Job.State -eq 'Completed' )
{
Receive-Job $Job
Remove-Job $Job
Return
}
Start-Sleep -Seconds 1
}
write-warning 'Job timed out. Stopping job'
Stop-Job $Job
Receive-Job $Job
Remove-Job $Job
I ran into this same problem and found a slightly simpler solution that also captures the output of the child process. The -PassThru argument is key as it returns a process object for each process that the cmdlet starts.
$proc = Start-Process $foxit -ArgumentList $argument -PassThru
Wait-Process -Id $proc.Id -Timeout 120
Move-Item $filePath $printed[$location]
Add-Content "$printLogDir\$logFileName" $logEntry
I prefer this style to have better control what to do during the runtime of the external process:
$waitTime = 60
$dism = Start-Process "$env:windir\system32\dism.exe" -ArgumentList "/Online /Cleanup-Image /ScanHealth" -PassThru -WindowStyle Hidden
while (!$dism.HasExited -or $waitTime -gt 0) {sleep -Seconds 1; $waitTime--}
"done."
By cheking the HasExited-attribute I can continue with my code with any other tasks in parallel.

Getting output from a job

I'm failing miserably to get any output from jobs - what is wrong with the following code?
$test = {
DIR
}
$mjob = Start-Job -ScriptBlock {$test}
while (Get-Job -State Running){}
Receive-Job -Job $mjob -OutVariable $otest
Write-Host($otest)
When you use -OutVariable supply only the name of the variable e.g.:
... -OutVariable otest
unless $otest happens to contain the name of the variable you want to save the output to.
A few other suggestions. $test represents a scriptblock so you don't need to put {} around it. And rather than wait using a while loop, just use Wait-Job e.g.:
$test = { get-childitem }
$job = Start-Job $test
Wait-Job $job
Receive-Job $job -OutVariable otest
$otest
You can use the pipeline to wait for the job to finish and then receive its result. Make sure to remove the braces when you pass a scriptblock to the ScriptBlock parameter, otherwise you're creating a nested scriptblock:
$test = { DIR }
Start-Job -ScriptBlock $test | Wait-Job | Receive-Job