Limit number of Start-Process running in powershell - powershell

Question

I have tried to limit the number of Start-Process running from a Powershell, but I can't seem to get it to work.
I tried to follow this process: https://exchange12rocks.org/2015/05/24/how-to-limit-a-number-of-powershell-jobs-running-simultaneously/ and Run N parallel jobs in powershell
But these are for Jobs not Processes, and I would like to remove the -Wait from the Start-Process
My concern with the script is that if there are 1000 audio files in the folder, then FFMpeg would crash the system.
# get the folder for conversion
function mbAudioConvert {
[Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms") | Out-Null
[System.Windows.Forms.Application]::EnableVisualStyles()
$fileBrowser = New-Object System.Windows.Forms.FolderBrowserDialog
$fileBrowser.SelectedPath = "B:\"
$fileBrowser.ShowNewFolderButton = $false
$fileBrowser.Description = "Select the folder with the audio which you wish to convert to Avid DNxHD 120 25P 48kHz"
$mbLoop = $true
$mbCount = 0001
$mbMaxJob = 4
while( $mbLoop ) {
if( $fileBrowser.ShowDialog() -eq "OK" ) {
$mbLoop = $false
$mbImage = ( Get-Item -Path "C:\Users\user\Desktop\lib\AudioOnly.jpg" )
$mbff32 = ( Get-Item -Path "C:\Users\user\Desktop\lib\ffmpeg32.exe" )
$mbff64 = ( Get-Item -Path "C:\Users\user\Desktop\lib\ffmpeg64.exe" )
$mbFolder = $fileBrowser.SelectedPath
$mbItemInc = ( ls $mbFolder\* -Include *.mp3, *.MP3, *.wav*, *.WAV*, *.ogg, *.OGG, *.wma, *.WMA, *.flac, *.FLAC, *.m4a, *.M4a )
$mbProgress = ( Get-ChildItem -Path $mbItemInc )
$mbHasRaw = ( $mbFolder + "\RAW" )
if( !( Test-Path -Path $mbHasRaw ) ) {
# force create a RAW folder if it does not exist
New-Item -ItemType Directory -Force -Path "$mbHasRaw"
}
foreach( $mbItem in $mbItemInc ) {
$mbCheck = $false
# output the progress
# Suggestion: You might want to consider updating this after starting the job and do the final update after running ex. Get-Job | Wait-Job to make the progress-bar stay until all processes are finished
#Write-Progress -Activity "Counting files for conversion" -status "Currently processing: $mbCount" -percentComplete ($mbCount / $mbItemInc.count*100)
# limit the run number
while ($mbCheck -eq $false) {
if( (Get-Job -State 'Running').count -lt $mbMaxJob) {
$mbScriptBlock = {
$mbItemName = $using:mbItem.BaseName
$mbNewItem = ( $using:mbFolder + "\RAW\" + $mbItemName + ".mov" )
$mbArgs = " -loop 1 -i $using:mbImage -i $using:mbItem -shortest -c:v dnxhd -b:v 120M -s 1920x1080 -pix_fmt yuv422p -r 25 -c:a pcm_s16le -ar 48k -af loudnorm=I=-12 $mbNewItem"
Start-Process -FilePath $using:mbff32 -ArgumentList $mbArgs -NoNewWindow -Wait
}
Start-Job -ScriptBlock $mbScriptBlock
#The job-thread doesn't know about $mbCount, better to increment it after starting the job
$mbCount++
$mbCheck = $true
}
}
}
} else {
$mbResponse = [System.Windows.Forms.MessageBox]::Show("You have exited out of the automation process!", "User has cancelled")
if( $mbResponse -eq "OK" ) {
return
}
}
}
$fileBrowser.SelectedPath
$fileBrowser.Dispose()
}
# call to function
mbAudioConvert

Answer 1

You edit $mbCheck, but the while loop is testing $Check which means the while-loop will never execute as $Check -eq $false is $false when $Check is not defined
Variables created outside the job script-block needs to be passed as an argument or you need to use the using: variable-scope to pass them in (PowerShell 3.0 or later). Added it to $mbItem, $mbff32, $mbImage and $mbFolder in the example which were not defined.
$mbMaxJob is not defined. The get running jobs-check will never be true and no processes will start
$mbCount not defined. Progress bar won't work
echo "$mbCount. $mbNewItem" won't return anything unless you use Receive-Job at some point to get the output from a job
Try:
#DemoValues
$mbItemInc = 1..10 | % { New-Item -ItemType File -Name "File$_.txt" }
$mbff32 = "something32"
$mbFolder = "c:\FooFolder"
$mbImage = "BarImage"
$mbMaxJob = 2
$mbCount = 0
foreach( $mbItem in $mbItemInc ) {
$mbCheck = $false
# output the progress
# Suggestion: You might want to consider updating this after starting the job and do the final update after running ex. Get-Job | Wait-Job to make the progress-bar stay until all processes are finished
Write-Progress -Activity "Counting files for conversion" -status "Currently processing: $mbCount" -percentComplete ($mbCount / $mbItemInc.count*100)
# limit the run number
while ($mbCheck -eq $false) {
if ((Get-Job -State 'Running').count -lt $mbMaxJob) {
$mbScriptBlock = {
Param($mbItem, $mbFolder, $mbImage, $mbff32)
#Filename without extension is already available in a FileInfo-object using the BaseName-property
$mbItemName = $mbItem.BaseName
$mbNewItem = ( $mbFolder + "\RAW\" + $mbItemName + ".mov" )
$mbArgs = "-loop 1 -i $mbImage -i $mbItem -shortest -c:v dnxhd -b:v 120M -s 1920x1080 -pix_fmt yuv422p -r 25 -c:a pcm_s16le -ar 48k -af loudnorm=I=-12 $mbNewItem"
Start-Process -FilePath $mbff32 -ArgumentList $mbArgs -NoNewWindow -Wait
}
Start-Job -ScriptBlock $mbScriptBlock -ArgumentList $mbItem, $mbFolder, $mbImage, $mbff32
#The job-thread doesn't know about $mbCount, better to increment it after starting the job
$mbCount++
$mbCheck = $true
}
}
}

Answer 2

I propose you my solution :
cls
$FormatNameJob="FFMPEG"
$maxConcurrentJobs = 100
$DirWithFile="C:\temp"
$DestFolder="C:\temp2"
$TraitmentDir="C:\temp\traitment"
$PathFFMpeg="C:\Temp\ffmpeg\ffmpeg\bin\ffmpeg.exe"
$HistoFolder="C:\temp\histo"
#create dir if dont exists
New-Item -ItemType Directory -Path $TraitmentDir -Force | Out-Null
New-Item -ItemType Directory -Path $DestFolder -Force | Out-Null
New-Item -ItemType Directory -Path $HistoFolder -Force | Out-Null
while ($true)
{
"Loop File"
$ListeFile=Get-ChildItem $DirWithFile -file -Filter "*.avi"
if ($ListeFile.count -eq 0 )
{
Start-Sleep -Seconds 1
continue
}
#loop file to trait
$ListeFile | %{
while ((get-job -State Running | where Name -eq $FormatNameJob ).Count -ge $maxConcurrentJobs)
{
Start-Sleep -Seconds 1
get-job -State Completed | where Name -eq $FormatNameJob | Remove-Job
}
"traitment file : {0}" -f $_.Name
#build newname and move item into traitment dir
$NewfileName="{0:yyyyMMddHHmmssfffff}_{1}" -f (get-date), $_.Name
$ItemTraitment=[io.path]::Combine($TraitmentDir, $NewfileName)
$mbNewItem ="{0}.mov" -f [io.path]::Combine($DestFolder, $_.BaseName)
Move-item $_.FullName -Destination $ItemTraitment
#build arguments and command
$mbArgs = " -loop 1 -i $ItemTraitment -shortest -c:v dnxhd -b:v 120M -s 1920x1080 -pix_fmt yuv422p -r 25 -c:a pcm_s16le -ar 48k -af loudnorm=I=-12 $mbNewItem"
$ScriptBlock=[scriptblock]::Create("Start-Process $PathFFMpeg -ArgumentList $mbArgs -Wait")
#add job
Start-Job -ScriptBlock $ScriptBlock -Name $FormatNameJob
}
}