I have a powershell script that at one point will call 2 other powershell scripts to run. It runs one script to completion, then the other, but this causes it to take longer. Can I force the script to execute the other scripts and continue cycling through? When I used to run these scripts manually I would have 20-30 sessions running and walk away while it worked. What I wrote took the monotony of clicking through them manually
Here's the parent script:
$List = Get-Content C:\archive\${env:id}.txt
$Batch = New-Object System.Collections.ArrayList
foreach ($Data in $List){
if ($Data -eq "" -or $data -eq $List[-1]){
$ProjectName = $Batch[0]
out-file C:\archive\"$ProjectName".txt
foreach($Data in $Batch -ne $Batch[0]){
Add-Content -Path C:\archive\"$ProjectName".txt -Exclude
$Batch[0] -Value $Data
}
--> C:\archive\GetPrograms.ps1 $ProjectName
--> C:\archive\GetNetwork.ps1 $ProjectName
$Batch = New-Object System.Collections.ArrayList
}
else{
[void]$Batch.Add($Data)
}
}
The parent script is not contingent on the data produced by the other 2 scripts. It simply executes them by passing in data
Honestly, based on your use case description, you really want to be looking at Parallel job/task/script processing.
Here is a post along the lines of what should satisfy your goals.
How do I run my PowerShell scripts in parallel without using Jobs?
Update - While this answer explains the process and mechanics of
PowerShell runspaces and how they can help you multi-thread
non-sequential workloads, fellow PowerShell aficionado Warren 'Cookie
Monster' F has gone the extra mile and incorporated these same
concepts into a single tool called Invoke-Parallel - it does what I
describe below, and he has since expanded it with optional switches
for logging and prepared session state including imported modules,
really cool stuff - I strongly recommend you check it out before
building you own shiny solution!
https://serverfault.com/questions/626711/how-do-i-run-my-powershell-scripts-in-parallel-without-using-jobs
Related
Context
On a build server, a PowerShell 7 script script.ps1 will be started and will be running in the background in the remote computer.
What I want
A safenet to ensure that at most 1 instance of the script.ps1 script is running at once on the build server or remote computer, at all times.
What I tried:
I tried meddling with PowerShell 7 background jobs (by executing the script.ps1 as a job inside a wrapper script wrapper.ps1), however that didn't solve the problem as jobs do not carry over (and can't be accessed) in other PowerShell sessions.
What I tried looks like this:
# inside wrapper.ps1
$running_jobs = $(Get-Job -State Running) | Where-Object {$_.Name -eq "ImportantJob"}
if ($running_jobs.count -eq 0) {
Start-Job .\script.ps1 -Name "ImportantJob" -ArgumentList #($some_variables)
} else {
Write-Warning "Could not start new job; Existing job detected must be terminated beforehand."
}
To reiterate, the problem with that is that $running_jobs only returns the jobs running in the current session, so this code only limits one job per session, allowing for multiple instances to be ran if multiple sessions were mistakenly opened.
What I also tried:
I tried to look into Get-CimInstance:
$processes = Get-CimInstance -ClassName Win32_Process | Where-Object {$_.Name -eq "pwsh.exe"}
While this does return the current running PowerShell instances, these elements carry no information on the script that is being executed, as shown after I run:
foreach ($p in $processes) {
$p | Format-List *
}
I'm therefore lost and I feel like I'm missing something.
I appreciate any help or suggestions.
I like to define a config path in the $env:ProgramData location using a CompanyName\ProjectName scheme so I can put "per system" configuration.
You could use a similar scheme with a defined location to store a lock file created when the script run and deleted at the end of it (as suggested already within the comments).
Then, it is up to you to add additional checks if needed (What happen if the script exit prematurely while the lock is still present ?)
Example
# Define default path (Not user specific)
$ConfigLocation = "$Env:ProgramData\CompanyName\ProjectName"
# Create path if it does not exist
New-Item -ItemType Directory -Path $ConfigLocation -EA 0 | Out-Null
$LockFilePath = "$ConfigLocation\Instance.Lock"
$Locked = $null -ne (Get-Item -Path $LockFilePath -EA 0)
if ($Locked) {Exit}
# Lock
New-Item -Path $LockFilePath
# Do stuff
# Remove lock
Remove-Item -Path $LockFilePath
Alternatively, on Windows, you could also use a scheduled task without a schedule and with the setting "If the task is already running, then the following rule applies: Do not start a new instance". From there, instead of calling the original script, you call a proxy script that just launch the scheduled task.
I have to following part of my script:
$active_processes = (Get-WmiObject -Class Win32_Process | where path -like $path | Select-Object -ExpandProperty Path | split-path -leaf | Select-Object -Unique)
It's working fine but I need to check if the process I get after all the script is running with elevated rights to launch another process with elevated rights if neccesary so it can interact with said process. I don't see any information about elevated rights with Get-WmiObject, I was wondering if I'm missing it or if there's another way to get that information
I don't need to run the powershell script as administrator. What I need is to find ff any executable requires elevated rights when launched and I need to find this information via powershell.
After some research on how windows knows if it needs admin to run an executable, I concluded that there are a couple ways but the most recommended and reliable is reading the executable manifest, so I wrote the following function:
function Get-ManifestFromExe{
Param(
[Parameter(Mandatory=$true,Position=0,ValueFromPipelineByPropertyName=$true)]
[Alias("Path")]
[ValidateScript({Test-Path $_ -IsValid})]
[String]$FullName
)
begin{
$stringStart = '<assembly'
$stringEnd = 'assembly>'
}
process{
$content = Get-Content $FullName -Raw
$indexStart = $content.IndexOf($stringStart)
$content = $content.Substring($indexStart)
$indexEnd = ($content.IndexOf($stringEnd)) + $stringEnd.Length
$content = $content.Substring(0,$indexEnd)
if($content -match "$stringStart(.|\s)+(?=$stringEnd)$stringEnd"){
return [XML]$Matches[0]
}
}
}
function Test-IsAdminRequired{
Param(
[Parameter(Mandatory=$true,Position=0)]
[XML]$xml
)
$value = $xml.assembly.trustInfo.security.requestedPrivileges.requestedExecutionLevel.level
if(-not [String]::IsNullOrEmpty($value)){
return ($value -eq "requireAdministrator" -or $value -eq "highestAvailable")
}else{
Write-Error "Provided xml does not contain requestedExecutionLevel node or level property"
}
}
$exe = '.\Firefox Installer.exe'
Get-ManifestFromExe -Path $exe
Test-IsAdminRequired -xml $exeManifest
It works by extracting the manifest XML from the executable and checking requestedExecutionLevel node's level property, the values accepted for this property are in this page, and quoted here:
asInvoker, requesting no additional permissions. This level requires
no additional trust prompts.
highestAvailable, requesting the highest permissions available to the
parent process.
requireAdministrator, requesting full administrator permissions.
So from this we can conclude that only highestAvailable and requireAdministrator would need admin privileges, so I check those, with that we will be done EXCEPT that some executables I tested (mostly installers) don't require admin to run but instead they prompt the UAC when they ruin their child executable, I don't really see a way to check this.. sorry.
BTW I really enjoyed this question (specially the research), hope this can help you.
SOURCES
What is the difference between "asInvoker" and "highestAvailable" execution levels?
reading an application's manifest file?
https://learn.microsoft.com/en-us/visualstudio/deployment/trustinfo-element-clickonce-application?view=vs-2019#requestedexecutionlevel
It's in the System.Security.Principal classes. This returns $true if the current user is elevated to local Administrator:
(New-Object System.Security.Principal.WindowsPrincipal([System.Security.Principal.WindowsIdentity]::GetCurrent())).IsInRole([System.Security.Principal.WindowsBuiltInRole]::Administrator)
I have a PowerShell script that connects to a database and pulls a list of user data. I take this data and create a foreach loop to run a script for the data.
This is working but its slow as the results could be 1000+ entries, and it has to complete the Script.bat for User A before it can start User B. The Script.bat for a single user is independent from another and takes ~30s for each user.
Is there a way to speed this up at all? I've been playing with -Parallel, ForEach-Object and workflow but I can't get it to work, likely due to me being a noob in PS.
foreach ($row in $Dataset.tables[0].rows)
{
$UserID=$row.value
$DeviceID=$row.value1
$EmailAddress=$row.email_address
cmd.exe /c "`"$PSScriptRoot`"\bin\Script.bat -c `" -Switch $UserID`" >> `"$PSScriptRoot`"\${FileName3}_REST_${DateTime}.txt 2> nul";
}
You said it yourself, your bottleneck is with the batch file in your script, not the loop itself. foreach (as opposed to ForEach-Object) is already the faster foreach loop mechanism in PowerShell. Investigate your batch file to find out why it takes 30 seconds to complete, and optimize it where you can.
Using Jobs
Note: Start-Job will run the job under another process. If you have PowerShell Core you can make use of the Start-ThreadJob cmdlet in lieu of Start-Job. This will start your job as part of another thread of the same process instead of starting another process.
If you can't optimize your batch script or optimize it to meet your needs, then you can consider using Start-Job to kick off the job to execute asynchronously, and then check the result and get any output from it using Receive-Job. For example:
# Master list of jobs you need to check the result of later
$jobs = New-Object System.Collections.Generic.List[System.Management.Automation.Job]
# Run your script for each row
foreach ($row in $Dataset.tables[0].rows)
{
$UserID=$row.value
$DeviceID=$row.value1
$EmailAddress=$row.email_address
# Use Start-Job here to kick off the script and store the job information
# for later retrieval.
# The $using: scope modifier allows you to make use of variables that were
# defined in the session calling Start-Job
$job = Start-Job -ScriptBlock { cmd.exe /c "`"${using:PSScriptRoot}`"\bin\Script.bat -c `" -Switch ${using:UserID}`" >> `"${using:PSScriptRoot}`"\${using:FileName3}_REST_${DateTime}.txt 2> nul"; }
# Add the execution to the $jobs list to check the result of later
# Casting to void here prevents the Add method from returning the object
# we've added.
[void]$jobs.Add($job)
}
# Wait for the jobs to be done
Write-Host 'Waiting for all jobs to complete...'
while( $jobs | Where-Object { $_.State -eq 'Running' } ){
Start-Sleep -s 10
}
# Retrieve the output of the jobs
foreach( $j in $jobs ) {
Receive-Job $j
}
Note: Since you have ~1000 times you need to execute this script, you may want to consider writing your logic to only run a certain number of jobs at a time. My example above starts all necessary jobs without regarding the number that may execute at once.
For more information about jobs and the properties you can inspect on a running/completed job, check the links below:
About Jobs
Job Class
Using Scope*
* The documentation states that the using scope can only be declared when working with remote sessions, but this seems to work fine with Start-Job even if the job is local.
I posted a question a couple ago, I needed a powershell script that would start a service if it was stopped, stop the process if running longer than an hour then start it again, and if running less than an hour do nothing. I was given a great script that really helped, but I'm trying to convert it to a "process". I have the following code (below) but am getting the following error
Error
"cmdlet Start-Process at command pipeline position 3
Supply values for the following parameters:
FilePath: "
Powershell
# for debugging
$PSDefaultParameterValues['*Process:Verbose'] = $true
$str = Get-Process -Name "Chrome"
if ($str.Status -eq 'stopped') {
$str | Start-Process
} elseif ($str.StartTime -lt (Get-Date).AddHours(-1)) {
$str | Stop-Process -PassThru | Start-Process
} else {
'Chrome is running and StartTime is within the past hour!'
}
# other logic goes here
Your $str is storing a list of all processes with the name "Chrome", so I imagine you want a single process. You'll need to specify an ID in Get-Process or use $str[0] to single out a specific process in the list.
When you store a single process in $str, if you try to print your $str.Status, you'll see that it would output nothing, because Status isn't a property of a process. A process is either running or it doesn't exist. That said, you may want to have your logic instead check if it can find the process and then start the process if it can't, in which case it needs the path to the executable to start the process. More info with examples can be found here: https://technet.microsoft.com/en-us/library/41a7e43c-9bb3-4dc2-8b0c-f6c32962e72c?f=255&MSPPError=-2147217396
If you're using Powershell ISE, try storing the process in a variable in the terminal, type the variable with a dot afterwards, and Intellisense (if it's on) should give a list of all its available properties.
I use $env:homedrive in my scripts in Azure's CustomScriptExtension.
Strangely sometimes I got blank from this variable but sometimes OK.
Is this environment variable not reliable?
It's difficult to say without looking at the script. But you can put a $null condition check before proceeding.
if($env:HOMEDRIVE -eq $null){
$env:HOMEDRIVE = "H:" # Whatever you want
}
else{
Do-WhatYouWant
}
Alternative you can set $env:HOMEDRIVE directly to your desired location at the beginning of the script or function. But still, I would prefer to put the $null check condition always.
#Dave Wu
If it concerns a script at logon, it is possible that network is not loaded yet due to windows fast logon. Maybe directly pulling it from the registry is more reliable:
(Get-Item -ErrorAction SilentlyContinue "HKCU:\Volatile Environment").GetValue("HomeDrive")