Start-job invoke function from different powershell - powershell

In Powershell with the function Start-Job you can call functions if you initialize them beforehand.
$export_functions =
{
Function Log($message) {Write-Host $message}
Function MyFunction($message) {Log($message)}
}
$param1="Hello stack overflow!"
Start-Job -ScriptBlock {MyFunction $using:param1} -InitializationScript $export_functions -ArgumentList #($param1) | Wait-Job | Receive-Job
Would it be possible to use a 'global' function inside the script block as well?
So something like:
Function Log($message) {Write-Host $message}
$export_functions =
{
Function Log($message) {$Function:Log($message)}
Function MyFunction($message) {Log($message)}
}
$param1="Hello stack overflow!"
Start-Job -ScriptBlock {MyFunction $using:param1} -InitializationScript $export_functions -ArgumentList #($param1) | Wait-Job | Receive-Job
Or is that not intended after all?

One way to capture the function definition from the caller's scope (defined outside the initialization script block) is to use an expandable (double-quoted) string ("..."), i.e. string interpolation, and then use [scriptblock]::Create() to create a script block from the result.
You can combine this technique with defining the code you want to place directly inside the script block as usual:
Function Log($message) { Write-Host $message }
# Use string interpolation to capture the body of function Log ($function:Log)
$export_functions = [scriptblock]::Create(
"Function Log { $function:Log }`n" +
{
Function MyFunction($message) { Log $message }
}
)
Start-Job -InitializationScript $export_functions { Log 'hi1'; MyFunction 'hi2' } |
Receive-Job -Wait -AutoRemoveJob
Note the use of $function:Log to refer to the Log function's body, which is an example of namespace variable notation.

Related

powershell: function call inside a function while procesing them paralelly [duplicate]

Assuming Get-Foo and Get-Foo2 and Deploy-Jobs are 3 functions that are part of a very large module. I would like to use Get-Foo and Get-Foo2 in Deploy-Jobs's Start-ThreadJob (below) without reloading the entire module each time.
Is an working example available for how to do this?
function Deploy-Jobs {
foreach ($Device in $Devices) {
Start-ThreadJob -Name $Device -ThrottleLimit 50 -InitializationScript $initScript -ScriptBlock {
param($Device)
Get-Foo | Get-Foo2 -List
} -ArgumentList $Device | out-null
}
}
The method you can use to pass the function's definition to a different scope is the same for Invoke-Command (when PSRemoting), Start-Job, Start-ThreadJob and ForeEach-Object -Parallel. Since you want to invoke 2 different functions in your job's script block, I don't think -InitializationScript is an option, and even if it is, it might make the code even more complicated than it should be.
You can use this as an example of how you can store 2 function definitions in an array ($def), which is then passed to the scope of each TreadJob, this array is then used to define each function in said scope to be later used by each Job.
function Say-Hello {
"Hello world!"
}
function From-ThreadJob {
param($i)
"From ThreadJob # $i"
}
$def = #(
${function:Say-Hello}.ToString()
${function:From-ThreadJob}.ToString()
)
function Run-Jobs {
param($numerOfJobs, $functionDefinitions)
$jobs = foreach($i in 1..$numerOfJobs) {
Start-ThreadJob -ScriptBlock {
# bring the functions definition to this scope
$helloFunc, $threadJobFunc = $using:functionDefinitions
# define them in this scope
${function:Say-Hello} = $helloFunc
${function:From-ThreadJob} = $threadJobFunc
# sleep random seconds
Start-Sleep (Get-Random -Maximum 10)
# combine the output from both functions
(Say-Hello) + (From-ThreadJob -i $using:i)
}
}
Receive-Job $jobs -AutoRemoveJob -Wait
}
Run-Jobs -numerOfJobs 10 -functionDefinitions $def

Reuse 2 functions in Start-ThreadJob

Assuming Get-Foo and Get-Foo2 and Deploy-Jobs are 3 functions that are part of a very large module. I would like to use Get-Foo and Get-Foo2 in Deploy-Jobs's Start-ThreadJob (below) without reloading the entire module each time.
Is an working example available for how to do this?
function Deploy-Jobs {
foreach ($Device in $Devices) {
Start-ThreadJob -Name $Device -ThrottleLimit 50 -InitializationScript $initScript -ScriptBlock {
param($Device)
Get-Foo | Get-Foo2 -List
} -ArgumentList $Device | out-null
}
}
The method you can use to pass the function's definition to a different scope is the same for Invoke-Command (when PSRemoting), Start-Job, Start-ThreadJob and ForeEach-Object -Parallel. Since you want to invoke 2 different functions in your job's script block, I don't think -InitializationScript is an option, and even if it is, it might make the code even more complicated than it should be.
You can use this as an example of how you can store 2 function definitions in an array ($def), which is then passed to the scope of each TreadJob, this array is then used to define each function in said scope to be later used by each Job.
function Say-Hello {
"Hello world!"
}
function From-ThreadJob {
param($i)
"From ThreadJob # $i"
}
$def = #(
${function:Say-Hello}.ToString()
${function:From-ThreadJob}.ToString()
)
function Run-Jobs {
param($numerOfJobs, $functionDefinitions)
$jobs = foreach($i in 1..$numerOfJobs) {
Start-ThreadJob -ScriptBlock {
# bring the functions definition to this scope
$helloFunc, $threadJobFunc = $using:functionDefinitions
# define them in this scope
${function:Say-Hello} = $helloFunc
${function:From-ThreadJob} = $threadJobFunc
# sleep random seconds
Start-Sleep (Get-Random -Maximum 10)
# combine the output from both functions
(Say-Hello) + (From-ThreadJob -i $using:i)
}
}
Receive-Job $jobs -AutoRemoveJob -Wait
}
Run-Jobs -numerOfJobs 10 -functionDefinitions $def

How to pass $_ ($PSItem) in a ScriptBlock

I'm basically building my own parallel foreach pipeline function, using runspaces.
My problem is: I call my function like this:
somePipeline | MyNewForeachFunction { scriptBlockHere } | pipelineGoesOn...
How can I pass the $_ parameter correctly into the ScriptBlock? It works when the ScriptBlock contains as first line
param($_)
But as you might have noticed, the powershell built-in ForEach-Object and Where-Object do not need such a parameter declaration in every ScriptBlock that is passed to them.
Thanks for your answers in advance
fjf2002
EDIT:
The goal is: I want comfort for the users of function MyNewForeachFunction - they shoudln't need to write a line param($_) in their script blocks.
Inside MyNewForeachFunction, The ScriptBlock is currently called via
$PSInstance = [powershell]::Create().AddScript($ScriptBlock).AddParameter('_', $_)
$PSInstance.BeginInvoke()
EDIT2:
The point is, how does for example the implementation of the built-in function ForEach-Object achieve that $_ need't be declared as a parameter in its ScriptBlock parameter, and can I use that functionality, too?
(If the answer is, ForEach-Object is a built-in function and uses some magic I can't use, then this would disqualify the language PowerShell as a whole in my opinion)
EDIT3:
Thanks to mklement0, I could finally build my general foreach loop. Here's the code:
function ForEachParallel {
[CmdletBinding()]
Param(
[Parameter(Mandatory)] [ScriptBlock] $ScriptBlock,
[Parameter(Mandatory=$false)] [int] $PoolSize = 20,
[Parameter(ValueFromPipeline)] $PipelineObject
)
Begin {
$RunspacePool = [runspacefactory]::CreateRunspacePool(1, $poolSize)
$RunspacePool.Open()
$Runspaces = #()
}
Process {
$PSInstance = [powershell]::Create().
AddCommand('Set-Variable').AddParameter('Name', '_').AddParameter('Value', $PipelineObject).
AddCommand('Set-Variable').AddParameter('Name', 'ErrorActionPreference').AddParameter('Value', 'Stop').
AddScript($ScriptBlock)
$PSInstance.RunspacePool = $RunspacePool
$Runspaces += New-Object PSObject -Property #{
Instance = $PSInstance
IAResult = $PSInstance.BeginInvoke()
Argument = $PipelineObject
}
}
End {
while($True) {
$completedRunspaces = #($Runspaces | where {$_.IAResult.IsCompleted})
$completedRunspaces | foreach {
Write-Output $_.Instance.EndInvoke($_.IAResult)
$_.Instance.Dispose()
}
if($completedRunspaces.Count -eq $Runspaces.Count) {
break
}
$Runspaces = #($Runspaces | where { $completedRunspaces -notcontains $_ })
Start-Sleep -Milliseconds 250
}
$RunspacePool.Close()
$RunspacePool.Dispose()
}
}
Code partly from MathiasR.Jessen, Why PowerShell workflow is significantly slower than non-workflow script for XML file analysis
The key is to define $_ as a variable that your script block can see, via a call to Set-Variable.
Here's a simple example:
function MyNewForeachFunction {
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[scriptblock] $ScriptBlock
,
[Parameter(ValueFromPipeline)]
$InputObject
)
process {
$PSInstance = [powershell]::Create()
# Add a call to define $_ based on the current pipeline input object
$null = $PSInstance.
AddCommand('Set-Variable').
AddParameter('Name', '_').
AddParameter('Value', $InputObject).
AddScript($ScriptBlock)
$PSInstance.Invoke()
}
}
# Invoke with sample values.
1, (Get-Date) | MyNewForeachFunction { "[$_]" }
The above yields something like:
[1]
[10/26/2018 00:17:37]
What I think you're looking for (and what I was looking for) is to support a "delay-bind" script block, supported in PowerShell 5.1+. The Microsoft documentation tells a bit about what's required, but doesn't provide any user-script examples (currently).
The gist is that PowerShell will implicitly detect that your function can accept a delay-bind script block if it defines an explicitly typed pipeline parameter (either by Value or by PropertyName), as long as it's not of type [scriptblock] or type [object].
function Test-DelayedBinding {
param(
# this is our typed pipeline parameter
# per doc this cannot be of type [scriptblock] or [object],
# but testing shows that type [object] may be permitted
[Parameter(ValueFromPipeline, Mandatory)][string]$string,
# this is our scriptblock parameter
[Parameter(Position=0)][scriptblock]$filter
)
Process {
if (&$filter $string) {
Write-Output $string
}
}
}
# sample invocation
>'foo', 'fi', 'foofoo', 'fib' | Test-DelayedBinding { return $_ -match 'foo' }
foo
foofoo
Note that the delay-bind will only be applied if input is piped into the function, and that the script block must use named parameters (not $args) if additional parameters are desired.
The frustrating part is that there is no way to explicitly specify that delay-bind should be used, and errors resulting from incorrectly structuring your function may be non-obvious.
Maybe this can help.
I'd normally run auto-generated jobs in parallel this way:
Get-Job | Remove-Job
foreach ($param in #(3,4,5)) {
Start-Job -ScriptBlock {param($lag); sleep $lag; Write-Output "slept for $lag seconds" } -ArgumentList #($param)
}
Get-Job | Wait-Job | Receive-Job
If I understand you correctly, you are trying to get rid of param() inside the scriptblock. You may try to wrap that SB with another one. Below is the workaround for my sample:
Get-Job | Remove-Job
#scriptblock with no parameter
$job = { sleep $lag; Write-Output "slept for $lag seconds" }
foreach ($param in #(3,4,5)) {
Start-Job -ScriptBlock {param($param, $job)
$lag = $param
$script = [string]$job
Invoke-Command -ScriptBlock ([Scriptblock]::Create($script))
} -ArgumentList #($param, $job)
}
Get-Job | Wait-Job | Receive-Job
# I was looking for an easy way to do this in a scripted function,
# and the below worked for me in PSVersion 5.1.17134.590
function Test-ScriptBlock {
param(
[string]$Value,
[ScriptBlock]$FilterScript={$_}
)
$_ = $Value
& $FilterScript
}
Test-ScriptBlock -Value 'unimportant/long/path/to/foo.bar' -FilterScript { [Regex]::Replace($_,'unimportant/','') }

Run functions in parallel

Is there any way to run parallel programmed functions in PowerShell?
Something like:
Function BuildParallel($configuration)
{
$buildJob = {
param($configuration)
Write-Host "Building with configuration $configuration."
RunBuilder $configuration;
}
$unitJob = {
param()
Write-Host "Running unit."
RunUnitTests;
}
Start-Job $buildJob -ArgumentList $configuration
Start-Job $unitJob
While (Get-Job -State "Running")
{
Start-Sleep 1
}
Get-Job | Receive-Job
Get-Job | Remove-Job
}
Does not work because it complains about not recognizing "RunUnitTests" and "RunBuilder", which are functions declared in the same script file. Apparently this happens because the script block is a new context and does not know anything about the scripts declared in the same file.
I could try to use -InitializationScript in Start-Job, but both RunUnitTests and RunBuilder call more functions declared in the same file or referred from other files, so...
I'm sure there's a way to do this, since it's just modular programming (functions, routines and all that stuff).
You could have the functions in a separate file and import them into the current context wherever needed via dot sourcing. I do this in my Powershell profile, so some of my custom functions are available.
$items = Get-ChildItem "$PSprofilePath\functions"
$items | ForEach-Object {
. $_.FullName
}
If you wanted import one file, it would just be:
. C:\some\path\RunUnitTests.ps1

Synax Highlighting w/ Dynamic Scriptblock

I'm creating a dynamic ScriptBlock the way below so I can use local functions and variables and easily pass them to remote computers via Invoke-Command. The issue is that since all the text inside Create is enclosed with double quotes, I loose all my syntax highlighting since all editors see the code as one big string.
While this is only a cosmetic issue, I'd like to find a work around that allow my code to be passed without having double quotes. I've tried passing a variable inside Create instead of the actually text, but it does not get interpreted.
function local_admin($a, $b) {
([adsi]"WinNT://localhost/Administrators,group").Add("WinNT://$a/$b,user")
}
$SB = [ScriptBlock]::Create(#"
#Define Function
function local_admin {$Function:local_admin}
local_admin domain username
"#)
Invoke-Command -ComputerName server2 -ScriptBlock $SB
You can pass the function into the remote session using the following example. This allows you to define the ScriptBlock using curly braces instead of as a string.
# Define the function
function foo {
"bar";
}
$sb = {
# Import the function definition into the remote session
[void](New-Item -Path $args[0].PSPath -Value $args[0].Definition);
# Call the function
foo;
};
#(gi function:foo) | select *
Invoke-Command -ComputerName . -ScriptBlock $sb -ArgumentList (Get-Item -Path function:foo);
Here is a modified version of your function. Please take note that the domain and username can be dynamically passed into the remote ScriptBlock using the -ArgumentList parameter. I am using the $args automatic variable to pass objects into the ScriptBlock.
function local_admin($a, $b) {
([adsi]"WinNT://localhost/Administrators,group").Add("WinNT://$a/$b,user")
}
$SB = {
#Define Function
[void](New-Item -Path $args[0].PSPath -Value $args[0].Definition);
# Call the function
local_admin $args[1] $args[2];
}
Invoke-Command -ComputerName server2 -ScriptBlock $SB -ArgumentList (Get-Item -Path function:local_admin), 'domain', 'username';