I was reading this post about getting functions passed into a scriptblock for use with jobs:
Powershell start-job -scriptblock cannot recognize the function defined in the same file?
I get how that works by passing the function in as variable and it works for the simple example. What about a real world solution though, is there a more elegant way of handling this?
I have script I'm using to deploy changes to vendor software. It reads an xml that tells it how to navigate the environment and performs the various tasks, ie: map drives, stop services, call a perl installation script. I would like to provide a parameter to the script to allow it to run concurrently, this way if the perl script takes 5 minutes (not uncommon) and you're rolling out to 11 servers you're not waiting for the script to run for an hour.
I'm just going to post some snippets since the full script is a little lengthy. A log function:
function Log
{
Param(
[parameter(ValueFromPipeline=$true)]
$InputObject,
[parameter()]
[alias("v")]
$verbosity = $debug
)
$messageIndex = [array]::IndexOf($verbosityArray, $verbosity)
$verbosityIndex = [array]::IndexOf($verbosityArray, $loggingVerbosity)
if($messageIndex -ge $VerbosityIndex)
{
switch($verbosity)
{
$debug {Write-Host $verbosity ": " $InputObject}
$info {Write-Host $verbosity ": " $InputObject}
$warn {Write-Host $verbosity ": " $InputObject -ForegroundColor yellow}
$error {Write-Host $verbosity ": " $InputObject -ForegroundColor red}
}
}
}
Here's another function that calls the log function:
function ExecuteRollout
{
param(
[parameter(Mandatory=$true)]
[alias("ses")]
$session,
[parameter(Mandatory=$true)]
$command
)
#invoke command
Invoke-Command -session $session -ScriptBlock {$res = cmd /v /k `"$args[0]`"} -args $command
#get the return code from the remote session
$res = Invoke-Command -session $session {$res}
Log ("Command Output: "+$res)
$res = [string] $res
$exitCode = $res.substring($res.IndexOf("ExitCode:"), 10)
$exitCode = $exitCode.substring(9,1)
Log ("Exit code: "+$exitCode)
return $exitCode
}
And lastly a snippet from my main so you can get an idea of what's going on. $target.Destinations.Destination will contain all the servers and relevant information about them that the deployment will go to. I removed some variable setup and logging to make this more compact so yes you'll see variables referenced that are never defined:
#Execute concurrently
$target.Destinations.Destination | %{
$ScriptBlock = {
$destination = $args[0]
Log -v $info ("Starting remote session on: "+$destination.Server)
$session = New-PSSession -computerName $destination.Server
$InitializeRemote -session $session -destination $destination
#Gets a little tricky here, we need to keep the cmd session so it doesn't lose the sys vars set by env.bat
#String everything together with &'s
$cmdString = $destDrive + ": & call "+$lesDestDir+"data\env.bat & cd "+$rolloutDir+" & perl ..\JDH-rollout-2010.pl "+$rollout+" NC,r:\les & echo ExitCode:!errorlevel!"
Log ("cmdString: "+$cmdString)
Log -v $info ("Please wait, executing the rollout now...")
$exitCode = $ExecuteRollout -session $session -command $cmdString
Log ("ExitCode: "+$exitCode)
#respond to return code from rollout script
$HandleExitCode -session $session -destination $destination -exitCode $exitCode
$CleanUpRemote -session $session -destination $destination
}
Start-Job $ScriptBlock -Args $_
}
So if i go with the approach in the link I'd be converting all my functions to variables and passing them in to the script block. Currently, my log function will by default log in DEBUG unless the verbosity parameter is explicitly passed as a different verbosity. If I convert my functins to variables however powershell doesn't seem to like this syntax:
$Log ("Print this to the log")
So I think I'd need to use the parameter all the time now:
$Log ("Print this to the log" -v $debug
So bottom line it looks like I just need to pass all my functions as variables to the script block and change some formatting when I call them. It's not a huge effort, but I'd like to know if there's a better way before I start hacking my script up. Thanks for the input and for looking, I know this is quite a long post.
I started another post about passing parameters to functions stored as variables, the answer to that also resolves this issue. That post can be found here:
Powershell: passing parameters to functions stored in variables
The short answer is you can use the initializationscript parameter of Start-Job to feed all your functions in if you wrap them in a block and store that in a variable.
Example:
# concurrency
$func = {
function Logx
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
}
# Execution starts here
cls
$colors = #("red","blue","green")
$colors | %{
$scriptBlock =
{
Logx $args[0]
Start-Sleep 9
}
Write-Host "Processing: " $_
Start-Job -InitializationScript $func -scriptblock $scriptBlock -args $_
}
Get-Job
while(Get-Job -State "Running")
{
write-host "Running..."
Start-Sleep 2
}
# Output
Get-Job | Receive-Job
# Cleanup jobs
Remove-Job *
Related
I need to initialize a job using the shell. The job will be a delay plus a call to a vbScript. The following code works fine. For my example, the vbScript is just a single line with a MsgBox "Hello world!"
$functions = {
Function execute_vbs {
param ([string]$path_VBScript, [int]$secs)
Start-Sleep -Seconds $secs
cscript /nologo $path_VBScript
}
}
$seconds = 2
Start-Job -InitializationScript $functions -ScriptBlock {execute_vbs -path_VBScript 'C:\Users\[USERNAME]\Desktop\hello_world.vbs' -secs $seconds} -Name MyJob
The problem comes the moment I want to parameterize the vbScript path. (the idea is to do several different calls to some different vbScripts).
When I do this, the command seems to ignore the parameter input. I did other tests with int parameter and they work fine, the problem looks to be only with the string parameters. The following code does not work:
$functions = {
Function execute_vbs {
param ([string]$path_VBScript, [int]$secs)
Start-Sleep -Seconds $secs
cscript /nologo $path_VBScript
}
}
$input = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
$seconds = 2
Start-Job -InitializationScript $functions -ScriptBlock {execute_vbs -path_VBScript $input -secs $seconds} -Name MyJob
I've also tried using the [-ArgumentList] command, but it has the same problem.
Any idea?
The problem is that the $input and $seconds variables inside your script block are in a different scope and are effectively different variables to the ones in the main script.
I've modified your script slightly to remove the call to VBScript to make it easier to reproduce here - my example code is:
$functions = {
Function execute_vbs {
param ([string]$path_VBScript, [int]$secs)
Start-Sleep -Seconds $secs
write-output "filename = '$path_VBScript'"
write-output "secs = '$secs'"
}
}
Here's two ways to fix it:
The Using: scope modifier
See https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_scopes?view=powershell-7#the-using-scope-modifier for the full details, but basically:
For any script or command that executes out of session, you need the Using scope modifier to embed variable values from the calling session scope, so that out of session code can access them.
$filename = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
$seconds = 2
$job = Start-Job -InitializationScript $functions -ScriptBlock {
execute_vbs -path_VBScript $using:filename -secs $using:seconds
} -Name MyJob
wait-job $job
receive-job $job
# output:
# filename = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
# secs = '2'
Note the $using before the variable names inside the script block - this allows you to "inject" the variables from your main script into the scriptblock.
ScriptBlock Parameters
You can define parameters on the script block similar to how you do it with a function, and then provide the values in the -ArgumentList parameter when you invoke Start-Job.
$filename = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
$seconds = 2
$job = Start-Job -InitializationScript $functions -ScriptBlock {
param( [string] $f, [int] $s )
execute_vbs -path_VBScript $f -secs $s
} -ArgumentList #($filename, $seconds) -Name MyJob
wait-job $job
receive-job $job
# output:
# filename = 'C:\Users\[USERNAME]\Desktop\hello_world.vbs'
# secs = '2'
``
This lambda function executes as expected:
$WriteServerName = {
param($server)
Write-Host $server
}
$server = "servername"
$WriteServerName.invoke($server)
servername
However, using the same syntax, the following script prompts for credentials and then exits to the command line (running like this: .\ScriptName.ps1 -ConfigFile Chef.config), implying that the lambda functions aren't executing properly (for testing, each should just output the server name).
Why does the former lambda function return the server name, but the ones in the script don't?
Param(
$ConfigFile
)
Function Main {
#Pre-reqs: get credential, load config from file, and define lambda functions.
$jobs = #()
$Credential = Get-Credential
$Username = $Credential.username
$ConvertedPassword = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($Credential.password)
$Password = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($ConvertedPassword)
$Config = Get-Content $ConfigFile -Raw | Out-String | Invoke-Expression
#Define lambda functions
$BootStrap = {
param($Server)
write-host $server
}
$RunChefClient = {
param($Server)
write-host $server
}
$SetEnvironment = {
param($Server)
write-host $server
}
#Create bootstrap job for each server and pass lambda functions to Scriptblock for execution.
if(($Username -ne $null) -and ($Password -ne $null))
{
ForEach($HashTable in $Config)
{
$Server = $HashTable.Server
$Roles = $HashTable.Roles
$Tags = $HashTable.Tags
$Environment = $HashTable.Environment
$ScriptBlock = {
param ($Server,$BootStrap,$RunChefClient,$SetEnvironment)
$BootStrap.invoke($Server)
$RunChefClient.invoke($Server)
$SetEnvironment.invoke($Server)
}
$Jobs += Start-Job -ScriptBlock $ScriptBlock -ArgumentList #($Server,$BootStrap,$RunChefClient,$SetEnvironment)
}
}
else {Write-Host "Username or password is missing, exiting..." -ForegroundColor Red; exit}
}
Main
Without testing, I am going to go ahead and say it's because you are putting your scriptblock executions in PowerShell Jobs and then not doing anything with them. When you start a job, it starts a new PowerShell instance and executes the code you give it along with the parameters you give it. Once it completes, the completed PSRemotingJob object sits there and does nothing until you actually do something with it.
In your code, all the jobs you start are assigned to the $Jobs variable. You can also get all your running jobs with Get-Job:
Get-Job -State Running
If you want to get any of the data returned from your jobs, you'll have to use Receive-Job
# Either
$Jobs | Receive-Job
# Or
Get-Job -State Running | Receive-Job
I have multiple jobs and for every job I want to have the same initialization script that sets some things up. I'd like to pass some arguments to the initialization script, but unfortunately the arguments passed using -ArgumentList seem to be only accessible in the actual job script.
Here's an example that demonstrates the argument only being accessible in the actual script:
function StartJob([ScriptBlock] $script, [string] $name, [ScriptBlock] $initialization_script = $null, $argument = $null)
{
Start-Job -ScriptBlock $script -Name $name -InitializationScript $initialization_script -ArgumentList $argument | Out-Null
}
[ScriptBlock] $initialization_script =
{
# The argument given to StartJob should be accessible here
param($test)
echo "Test: $test"
}
[ScriptBlock] $actual_script =
{
param($test)
echo "Test: $test"
}
StartJob $actual_script "Test job" $initialization_script "Have this string in the `$initialization_script"
#(Get-Job).ForEach({
# Wait for the job to finish, remove it and output its results
Write-Host "$($_.Name) results:"
Receive-Job -Job $_ -Wait -AutoRemoveJob | Write-Host
})
How would I be able to be access the arguments passed in the $initialization_script?
AFAIK it's not possible to pass parameters to initialization scripts. Init scripts are designed to be reusable scripblocks to load known resources. If something can't be defined once, then it's unique to that job's scriptblock and doesn't belong in a init. script. You have a few alternatives:
If you have a module (.psm1 and maybe a .psd1), then place it in one of the module-folders (see $env:PSModulePath for paths) so you could simply write Import-Module MyImportantModule in your initialization script.
If you can't use the solution above, I would add a paramter to the actual script and pass in the path as a regular argument.
[ScriptBlock] $actual_script =
{
# The argument given to StartJob should be accessible here
param($test, $ModulePath)
#Import-Module $ModulePath
echo "Test: $test"
}
Start-Job -ScriptBlock $actual_script -Name "Test job" -ArgumentList "First argument", "c:\mymodule.ps1"
Or you could generate the initialization scriptblock in your script so it's dynamic:
$ModulePath = "c:\mymodule.ps1"
$init = #"
#Import-Module "$ModulePath"
#Something-Else
"#
$initsb = [scriptblock]::Create($init)
I'm trying to get a simple working example of using functions inside of jobs. I've managed to pass my function into the scriptblock used for my job, but I can't seem to get parameters to the function.
# concurrency
$Logx =
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
# Execution starts here
cls
$colors = #("red","blue","green")
$colors | %{
$scriptBlock =
{
Invoke-Expression -Command $args[1]
Start-Sleep 3
}
Write-Host "Processing: " $_
Start-Job -scriptblock $scriptBlock -args $_, $Logx
}
Get-Job
while(Get-Job -State "Running")
{
write-host "Running..."
Start-Sleep 2
}
# Output
Get-Job | Receive-Job
# Cleanup jobs
Remove-Job *
Here's the output:
Processing: red
Id Name State HasMoreData Location Command
-- ---- ----- ----------- -------- -------
175 Job175 Running True localhost ...
Processing: blue
177 Job177 Running True localhost ...
Processing: green
179 Job179 Running True localhost ...
179 Job179 Running True localhost ...
177 Job177 Running True localhost ...
175 Job175 Running True localhost ...
Running...
Running...
OUT:
OUT:
OUT:
So as evidenced by the OUT: x3 in the output my function is getting called, but I haven't found any syntax that allows me to get the parameter to the function. Thoughts?
EDIT:
Note in Shawn's observation below and my response I tried using functions as variables because using a traditional function does not seem to work. If there is a way to get that working I'd be more than happy to not have to pass my functions around as variables.
The answer is to use the initializationscript parameter of Start-Job. If you define all your functions in a block and pass the block they become available.
Solution was found in this post:
How do I Start a job of a function i just defined?
Here is my example from before, now working:
# concurrency
$func = {
function Logx
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
}
# Execution starts here
cls
$colors = #("red","blue","green")
$colors | %{
$scriptBlock =
{
Logx $args[0]
Start-Sleep 9
}
Write-Host "Processing: " $_
Start-Job -InitializationScript $func -scriptblock $scriptBlock -args $_
}
Get-Job
while(Get-Job -State "Running")
{
write-host "Running..."
Start-Sleep 2
}
# Output
Get-Job | Receive-Job
# Cleanup jobs
Remove-Job *
If you do not prefix your function name with keyword function, PowerShell does not know to treat it as such. As you have written your script it is basically a variable with some special text in it. Which as your output shows it is only executing the commands it recognizes within that variable's content: Write-Host "OUT:".
Using the correct syntax will tell PowerShell it is a function and that you have variables to pass into it that you need executed:
function Logx
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
Then when you call it within your script you will just use Logx
Got this far. Have to run out, will try back later.
PS: What is getting passed at args[1], I am getting a lot of red,
CategoryInfo : InvalidData: (:) [Invoke-Expression], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.InvokeExpressionCommand
here is what I've managed so far.
# concurrency
$Logx =
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
# Execution starts here
cls
$colors = #("red","blue","green")
$colors | %{
& $scriptBlock =
{ Invoke-Expression -Command $args[1]
Start-Sleep 3
}
Write-Host "Processing: " $_
Start-Job -scriptblock $scriptBlock -ArgumentList #($_, $Logx)
}
# Get-Job
while(Get-Job -State "Running")
{
write-host "Running..."
Start-Sleep 2
}
# Output
Get-Job | Receive-Job
# Cleanup jobs
Remove-Job *
I'm trying to run a job in a background which is a .exe with parameters and the destination has spaces. For example:
$exec = "C:\Program Files\foo.exe"
and I want to run this with parameters:
foo.exe /param1 /param2, etc.
I know that Start-Job does this but I've tried tons of different combinations and it either gives me an error because of the white space or because of the parameters. Can someone help me out with the syntax here? I need to assume that $exec is the path of the executable because it is part of a configuration file and could change later on.
One way to do this is use a script block with a param block.
If there is a single argument with a space in it such as a file/folder path it should be quoted to treat it as a single item. The arguments are an array passed to the script block.
This example uses a script block but you can also use a PowerShell script using the -FilePath parameter of the Start-Job cmdlet instead of the -ScriptBlock parameter.
Here is another example that has arguments with spaces:
$scriptBlock = {
param (
[string] $Source,
[string] $Destination
)
$output = & xcopy $Source $Destination 2>&1
return $output
}
$job = Start-Job -scriptblock $scriptBlock -ArgumentList 'C:\My Folder', 'C:\My Folder 2'
Wait-Job $job
Receive-Job $job
Here is an example using the $args built-in variable instead of the param block.
$scriptBlock = {
$output = & xcopy $args 2>&1
return $output
}
$path1 = "C:\My Folder"
$path2 = "C:\My Folder 2"
"hello world" | Out-File -FilePath "$path1\file.txt"
$job = Start-Job -scriptblock $scriptBlock -ArgumentList $path1, $path2
Wait-Job $job
Receive-Job $job
Andy's trick generally works very well. If you have parameter sets, or otherwise want to move complex information into the job, you can also try this technique:
$jobArgs = #{Source="foo"; Destination="bar"}
$jobArgs |Export-CliXml -Path $env:\Temp\MyArgs.clixml
and in the job...
Start-Job {
.... $jobArgs = Import-CliXml -Path $env:\Temp\MyArgs.clixml
} | Wait-Job | Receive-Job
I use both approaches routinely.
I use -ArgumentList / ScriptBlock parameters when:
I am not dealing with parameter sets
I'm using an In-Memory Job (like the -AsJob capability in ShowUI or WPK) where the arguments are real objects, and they cannot die
I'm running in a user context where I can run a job, but can't store to disk (web servers, ISO compliant labs, etc)
If I need complex arguments, and they don't need to be passed in memory (or its otherwise convenient to have them on disk later), I'll use the hashtable approach.
Hope this Helps