How to pass variable by reference to Powershell job or runspace? - powershell

I have Powershell job.
$cmd = {
param($a, $b)
$a++
$b++
}
$a = 1
$b = 2
Start-Job -ScriptBlock $cmd -ArgumentList $a, $b
How to pass $a and $b by a reference so when the job is done they will be updated? Alternatively how to pass variables by reference to runspaces?

Simple sample I just wrote (don't mind the messy code)
# Test scriptblock
$Scriptblock = {
param([ref]$a,[ref]$b)
$a.Value = $a.Value + 1
$b.Value = $b.Value + 1
}
$testValue1 = 20 # set initial value
$testValue2 = 30 # set initial value
# Create the runspace
$Runspace = [runspacefactory]::CreateRunspace()
$Runspace.ApartmentState = [System.Threading.ApartmentState]::STA
$Runspace.Open()
# create the PS session and assign the runspace
$PS = [powershell]::Create()
$PS.Runspace = $Runspace
# add the scriptblock and add the argument as reference variables
$PS.AddScript($Scriptblock)
$PS.AddArgument([ref]$testValue1)
$PS.AddArgument([ref]$testValue2)
# Invoke the scriptblock
$PS.BeginInvoke()
After running this the for the testvalues are updated since they are passed by ref.

Passing parameters by reference is always awkward in PowerShell, and probably won't work for PowerShell jobs anyway, as #bluuf pointed out.
I would probably do something like this instead:
$cmd = {
Param($x, $y)
$x+1
$y+1
}
$a = 1
$b = 2
$a, $b = Start-Job -ScriptBlock $cmd -ArgumentList $a, $b |
Wait-Job |
Receive-Job
The above code passes the variables $a and $b to the scriptblock and assigns the modified values back to the variables after receiving the job output.

a more comprehensive script with example .
it should also include ability to pass $host or something, to make write-host from the passed script, output to the console . but i don't have time to figure out how to do this .
$v = 1
function newThread ([scriptblock]$script, [Parameter(ValueFromPipeline)]$param, [Parameter(ValueFromRemainingArguments)]$args) {
process {
$Powershell = [powershell]::Create()
$Runspace = [runspacefactory]::CreateRunspace()
# allows to limit commands available there
# $InitialSessionState = InitialSessionState::Create()
# $Runspace.InitialSessionState = $InitialSessionState
$Powershell.Runspace = $Runspace
$null = $Powershell.AddScript($script)
$null = $Powershell.AddArgument($param)
foreach ($v_f in $args) {
$null = $Powershell.AddArgument($v_f)
}
$Runspace.Open()
$Job = $Powershell.BeginInvoke()
[PSCustomObject]#{
Job=$Job
Powershell=$Powershell
}
}
}
$script = {
param([ref]$v,$v2)
$v.Value++
$v2
}
$thread = newThread $script ([ref]$v) 3
do {} until ($thread.Job.IsCompleted)
$v1 = $thread.Powershell.EndInvoke($thread.Job)
$thread.Powershell.Dispose()
write-host "end $($v,$v1[0])"

Related

Pass command line arguments on to to "self" but with Start-Job

At [Asynchronous start][1] I had a question about starting a power-shell script asynchronously which creates a form. As answered in that question this can be solved using start-job
eg
Start-Job -ScriptBlock { test NW -NoWarning -Paranoia:2 }
So I have tried to write the Test.PS1 script routine so it re-calls itself with "Start-Job Test -NoSpawn" The switch nospawn then means it runs without a second call. I have tested this with the example code the above line now has to be and it works
Start-Job -ScriptBlock { test NW -NoSpawn -NoWarning -Paranoia:2 }
However I'm struggling to get the parameters from the original command line to passthrough to the job
I have tried creating a string in the correct format , an array , list the arguments manually , I either get repeated arguments being passed or all of the string ending up in the first Parameter $ComputerList -
A summary of the parameters and the attempts are
Param ([string]$ComputerList = 'status\edi.csv',[switch]$NoSpawn,[switch]$NoWarning,[switch]$Debug,[INT]$Paranoia=6)
...... <Snip>
Start-Job -ScriptBlock { test $ComputerList -NoSpawn -NoWarning:$NoWarning -Paranoia:$Paranoia }
Doesn't work due to scope - also switches are wrong way to do this
Start-Job -ScriptBlock { test -NoSpawn $Args } -argumentlist $ComputerList
Insufficent arguments but works - But I think One Argument is possible ?
Start-Job -ScriptBlock { test $Args -NoSpawn } -argumentlist #("-NoWarning:$NoWarning","-ComputerList:$ComputerList","-Paranoia:$Paranoia")
Everything ends up in $ComputerList
Start-Job -ScriptBlock { test $Args -NoSpawn } -argumentlist "-NoWarning:$NoWarning -ComputerList:$ComputerList -Paranoia:$Paranoia"
Everything ends up in $ComputerList
Full code follows
Param ([string]$ComputerList = 'status\edi.csv',[switch]$NoSpawn,[switch]$NoWarning,[switch]$Debug,[INT]$Paranoia=6)
$Log_Paranoia=$Paranoia
If ($Debug) { $debugPreference="Continue"} #enable debug messages if -debug is specified
If ($NoWarning) { $WarningPreference="SilentlyContinue"} #turn off warning messages
function Write-Paranoid($Level, $message) {
$CS=Get-PSCallStack
$Caller = $CS[1]
$Module = "$($Caller.FunctionName)[$($Caller.ScriptLineNumber)]"
$Diff=$level - $Log_Paranoia
$MSG= "$Module($($Level),$($Log_Paranoia)):$message"
if ($level - $Log_Paranoia -le 0 ) {
Write-host $MSG
}
if($Error.Count -gt 0 ) {
$MSG= "$Module($Level)ERROR:$Error[0]"
Write-Error $MSG
}
$error.clear()
}
Function AddStatusBar($form , $Txt) {
Write-Paranoid 10 "Enter"
$statusBar = New-Object System.Windows.Forms.StatusBar
$statusBar.DataBindings.DefaultDataSourceUpdateMode = 0
$statusBar.TabIndex = 4
$statusBar.Size = SDS 428 22
$statusBar.Location = SDP 0 337
$statusBar.Text = $Txt
$form.Controls.Add($statusBar)
$statusBar
Write-Paranoid 10 "Exit"
}
Function Create-Form ($Title)
{
Write-Paranoid 10 "Enter"
$form1 = New-Object System.Windows.Forms.Form
$form1.Text = $Title
$form1.DataBindings.DefaultDataSourceUpdateMode = 0
$form1.ClientSize = SDS 890 359
$form1.StartPosition = 0
$form1.BackColor = [System.Drawing.Color]::FromArgb(255,185,209,234)
$form1
Write-Paranoid 10 "Exit"
}
Function GenerateTestForm
{
Write-Paranoid 10 "Enter"
[reflection.assembly]::loadwithpartialname("System.Drawing") | Out-Null
[reflection.assembly]::loadwithpartialname("System.Windows.Forms") | Out-Null
$Form1 = Create-Form "Test Form"
$Alist = Get-CommandLine
$StatusBar = AddStatusBar $form1 $AList
$form1.ShowDialog() | Out-Null # Suspends calller
Write-Paranoid 10 "Exit"
}
if ($NoSpawn )
{
Write-Paranoid 3 " NoSpawn "
Write-Paranoid 5 "Call GenerateForm"
if ($Test) {
GenerateTestForm
} else {
GenerateTestForm
}
} else {
Write-Paranoid 3 "NOT NoSpawn restarting as job"
# note that test.ps1 is in the path so it will restart this script
# Start-Job -ScriptBlock { test $ComputerList -NoSpawn -NoWarning:$NoWarning -Paranoia:$Paranoia } #Wrong scope
# Start-Job -ScriptBlock { test -NoSpawn $Args } -argumentlist $ComputerList # Insufficent aruments but works - ONLY One Argument possible -
# Start-Job -ScriptBlock { test $Args -NoSpawn } -argumentlist #("-NoWarning:$NoWarning","-ComputerList:$ComputerList","-Paranoia:$Paranoia") # Everything ends up in $ComputerList
# Start-Job -ScriptBlock { test $Args -NoSpawn } -argumentlist "-NoWarning:$NoWarning -ComputerList:$ComputerList -Paranoia:$Paranoia" # Everything ends up in $ComputerList
}
Your problem can be reduced to this:
How can I re-invoke the script at hand as a background job, passing all original arguments (parameter values, including default parameter values) through?
A simplified example:
param (
[string] $ComputerList = 'status\edi.csv',
[switch] $NoSpawn,
[switch] $NoWarning,
[switch] $Debug,
[int] $Paranoia=6
)
if ($NoSpawn) { # already running as a background job.
"I'm now running in the background with the following arguments:"
$PSBoundParameters
} else { # must re-invoke via a background job
# Add *default* parameter values, if necessary, given that
# they're *not* reflected in $PSBoundParameters.
foreach ($paramName in $MyInvocation.MyCommand.Parameters.Keys) {
if (-not $PSBoundParameters.ContainsKey($paramName)) {
$defaultValue = Get-Variable -Scope Local -ValueOnly $paramName
if (-not ($null -eq $defaultValue -or ($defaultValue -is [switch] -and -not $defaultValue))) {
$PSBoundParameters[$paramName] = $defaultValue
}
}
}
# Start a background job that reinvokes this script with the original
# arguments / default values.
Start-Job {
$argsHash = $using:PSBoundParameters
& $using:PSCommandPath -NoSpawn #argsHash
} |
Receive-Job -Wait -AutoRemoveJob
}
Note:
For demonstration purposes, the initial call waits for the re-invocation via a background job to finish, using Receive-Job -Wait -AutoRemoveJob
In your real code, you can simply discard Start-Job's output (a job-information object) with $null = Start-Job { ... }, and then rely on the job getting cleaned up when the caller's session as a whole exits.
The extra code needed to propagate parameter default values is somewhat cumbersome, but necessary, given that the automatic $PSBoundParameters variable does not reflect default values.
GitHub issue #3285 discusses this limitation, and suggests a potential future solution.

multithread with runspaces instead of foreach cycle

I have a script with a foreach cycle. It has about a dozen functions, each collecting information from remote machines' C$ share (cutting text files, checking file version, etc.)
This is however taking some time, since each machine's data collected after one by one. (sometimes it runs with 500+ input)
Wish to put this into runspaces with parallel execution, but so far no examples worked. I am quite new to the concept.
Current script's outline
$inputfile = c:\temp\computerlist.txt
function 1
function 2
function 3, etc
foreach cycle
function 1
function 2
function 3
All results written to screen with write-host for now.
This example pings a number of server in parallel, so you easily can modify it for your demands:
Add-Type -AssemblyName System.Collections
$GH = [hashtable]::Synchronized(#{})
[System.Collections.Generic.List[PSObject]]$GH.results = #()
[System.Collections.Generic.List[string]]$GH.servers = #('server1','server2','server3');
[System.Collections.Generic.List[string]]$GH.functions = #('Check-Server');
[System.Collections.Generic.List[PSObject]]$jobs = #()
#-----------------------------------------------------------------
function Check-Server {
#-----------------------------------------------------------------
# a function which runs parallel
param(
[string]$server
)
$result = Test-Connection $server -Count 1 -Quiet
$GH.results.Add( [PSObject]#{ 'Server' = $server; 'Result' = $result } )
}
#-----------------------------------------------------------------
function Create-InitialSessionState {
#-----------------------------------------------------------------
param(
[System.Collections.Generic.List[string]]$functionNameList
)
# Setting up an initial session state object
$initialSessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
foreach( $functionName in $functionNameList ) {
# Getting the function definition for the functions to add
$functionDefinition = Get-Content function:\$functionName
$functionEntry = New-Object System.Management.Automation.Runspaces.SessionStateFunctionEntry -ArgumentList $functionName, $functionDefinition
# And add it to the iss object
[void]$initialSessionState.Commands.Add($functionEntry)
}
return $initialSessionState
}
#-----------------------------------------------------------------
function Create-RunspacePool {
#-----------------------------------------------------------------
param(
[InitialSessionState]$initialSessionState
)
$runspacePool = [RunspaceFactory]::CreateRunspacePool(1, ([int]$env:NUMBER_OF_PROCESSORS + 1), $initialSessionState, $Host)
$runspacePool.ApartmentState = 'MTA'
$runspacePool.ThreadOptions = "ReuseThread"
[void]$runspacePool.Open()
return $runspacePool
}
#-----------------------------------------------------------------
function Release-Runspaces {
#-----------------------------------------------------------------
$runspaces = Get-Runspace | Where { $_.Id -gt 1 }
foreach( $runspace in $runspaces ) {
try{
[void]$runspace.Close()
[void]$runspace.Dispose()
}
catch {
}
}
}
$initialSessionState = Create-InitialSessionState -functionNameList $GH.functions
$runspacePool = Create-RunspacePool -initialSessionState $initialSessionState
foreach ($server in $GH.servers)
{
Write-Host $server
$job = [System.Management.Automation.PowerShell]::Create($initialSessionState)
$job.RunspacePool = $runspacePool
$scriptBlock = { param ( [hashtable]$GH, [string]$server ); Check-Server -server $server }
[void]$job.AddScript( $scriptBlock ).AddArgument( $GH ).AddArgument( $server )
$jobs += New-Object PSObject -Property #{
RunNum = $jobCounter++
JobObj = $job
Result = $job.BeginInvoke() }
do {
Sleep -Seconds 1
} while( $runspacePool.GetAvailableRunspaces() -lt 1 )
}
Do {
Sleep -Seconds 1
} While( $jobs.Result.IsCompleted -contains $false)
$GH.results
Release-Runspaces | Out-Null
[void]$runspacePool.Close()
[void]$runspacePool.Dispose()
This would be concurrent and run in about 10 seconds total. The computer could be localhost three times if you got it working.
invoke-command comp1,comp2,comp3 { sleep 10; 'done' }
Simple attempt at api (threads):
$a = [PowerShell]::Create().AddScript{sleep 5;'a done'}
$b = [PowerShell]::Create().AddScript{sleep 5;'b done'}
$c = [PowerShell]::Create().AddScript{sleep 5;'c done'}
$r1 = $a.BeginInvoke(); $r2 = $b.BeginInvoke() $r3 = $c.BeginInvoke()
$a.EndInvoke($r1); $b.EndInvoke($r2); $c.EndInvoke($r3)
a done
b done
c done

How to add Parameters into a Job using argument list

I'm trying to create a responsive form, using jobs, and the Add-Jobtracker matrix.
The problem that I have is when I try to call variables inside the job. I used the -Argumentlist parameter but it is always Null, and I'm sure that the way that I'm using it is wrong, could you please guide me with this?
This is just an example, as my original script is a little bit more complex, below script will only write text from 2 variables to a textbox. I've created the same outside and inside the jobscript, used argument list and param inside it and it didn't work.
$Var1 = "sdaasdsadsa"
$Var2 = "asdasdsadsadsa"
$JobScript = {
Param($Var1, $Var2)
Write-Host $Var1, $Var2
}
$UpdateScript = {
Param($Job)
$texbox.Text = 'Working...'
}
$CompletedScript = {
Param($Job)
$results = Receive-Job -Job $Job
$textbox.Text = $results
}
Add-JobTracker -Name "test" -JobScript $JobScript -UpdateScript $UpdateScript -CompletedScript $CompletedScript -ArgumentList $var1, $Var2
At this point it is not doing anything, I have an alternative code but it is longer than this one, and I did not want to bother you guys with a lot of lines.
The job is running under a different scope than the rest of your script, and in that scope the variables aren't defined and therefore $null.
To call variables inside a job, use the $Using:Varname syntax. Eg:
$Test = "TEST!"
$null = Start-Job {
$Test
}
$Job = Get-Job
Start-Sleep -Seconds 2
Receive-Job -Job $Job
Yields no output at all, but:
$Test = "TEST!"
$null = Start-Job {
$Using:Test
}
$Job = Get-Job
Start-Sleep -Seconds 2
Receive-Job -Job $Job
Yields:
TEST!
Read more about Remote Variables here: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_remote_variables?view=powershell-6

Expand variable with scriptblock inside variable in a loop with runspaces

$RunspaceCollection = #()
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1,5)
$RunspacePool.Open()
$code = #({'somecode'},{'someothercode'})
Foreach ($test in $case) {
$finalcode= {
Invoke-Command -ScriptBlock [scriptblock]::create($code[$test])
}.GetNewClosure()
$Powershell = [PowerShell]::Create().AddScript($finalcode)
$Powershell.RunspacePool = $RunspacePool
[Collections.Arraylist]$RunspaceCollection += New-Object -TypeName PSObject -Property #{
Runspace = $PowerShell.BeginInvoke()
PowerShell = $PowerShell
}}
The finalcode variable doesn't expand when the GetNewClosure() happens, so $code[$test] gets into the runspace instead of actual code and I can't get my desired results. Any advice?
Using the method from the answer I'm getting this in the runspace, but it doesn't execute properly. I can confirm that my command is loaded into runspace (at least while in debugger inside runspace I can execute it without dot sourcing)
[System.Management.Automation.PSSerializer]::Deserialize('<ObjsVersion="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">
<SBK> My-Command -Par1 "egweg" -Par2 "qwrqwr" -Par3 "wegweg"</SBK>
</Objs>')
This is what I see in debugger in runspace
Stopped at: $a = Invoke-Command -ScriptBlock { $([System.Management.Automation.PSSerializer]::Deserialize('<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">
[DBG]: [Process:8064]: [Runspace12]: PS C:\git\infrastructure_samples>>
Stopped at: $a = Invoke-Command -ScriptBlock { $([System.Management.Automation.PSSerializer]::Deserialize('<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">
[DBG]: [Process:8064]: [Runspace12]: PS C:\git\infrastructure_samples>> s
Stopped at: </Objs>')) }
The problem with your code is that AddScript method of PowerShell class is expecting a string, not ScriptBlock. And any closure will be lost when you convert ScriptBlock to string. To solve this, you can pass argument to script with AddArgument method:
$RunspaceCollection = New-Object System.Collections.Generic.List[Object]
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1,5)
$RunspacePool.Open()
$code = #({'somecode'},{'someothercode'})
$finalcode= {
param($Argument)
Invoke-Command -ScriptBlock ([scriptblock]::create($Argument))
}
Foreach ($test in $case) {
$Powershell = [PowerShell]::Create().AddScript($finalcode).AddArgument($code[$test])
$Powershell.RunspacePool = $RunspacePool
$RunspaceCollection.Add((New-Object -TypeName PSObject -Property #{
Runspace = $PowerShell.BeginInvoke()
PowerShell = $PowerShell
}))
}
I'm not sure if there's a better way off the top of my head, but you can replace the variables yourself with serialized versions of the same.
You can't use $Using: in this case, but I wrote a function that replaces all $Using: variables manually.
My use case was with DSC, but it would work in this case as well. It allows you to still write your script blocks as scriptblocks (not as strings), and supports variables with complex types.
Here's the code from my blog (also available as a GitHub gist):
function Replace-Using {
[CmdletBinding(DefaultParameterSetName = 'AsString')]
[OutputType([String], ParameterSetName = 'AsString')]
[OutputType([ScriptBlock], ParameterSetName = 'AsScriptBlock')]
param(
[Parameter(
Mandatory,
ValueFromPipeline
)]
[String]
$Code ,
[Parameter(
Mandatory,
ParameterSetName = 'AsScriptBlock'
)]
[Switch]
$AsScriptBlock
)
Process {
$cb = {
$m = $args[0]
$ser = [System.Management.Automation.PSSerializer]::Serialize((Get-Variable -Name $m.Groups['var'] -ValueOnly))
"`$([System.Management.Automation.PSSerializer]::Deserialize('{0}'))" -f $ser
}
$newCode = [RegEx]::Replace($code, '\$Using:(?<var>\w+)', $cb, [System.Text.RegularExpressions.RegexOptions]::IgnoreCase)
if ($AsScriptBlock.IsPresent) {
[ScriptBlock]::Create($newCode)
} else {
$newCode
}
}
}
A better way for me to do this replacement would probably be to use the AST instead of string replacement, but.. effort.
Your Use Case
$finalcode= {
Invoke-Command -ScriptBlock [scriptblock]::create($Using:code[$Using:test])
} | Replace-Using
For better results you might assign a variable first and then just insert that:
$value = [scriptblock]::Create($code[$test])
$finalcode= {
Invoke-Command -ScriptBlock $Using:value
} | Replace-Using

Start() Error In Powershell (Starting a Shortcut)

I have created a Powershell script to do a timed restart on my server. This is a snippet of my code and more specifically the startServers Function.
I have added the global variables into the function in an effort to have an example.
Basically each server name is the name of a shortcut that starts an exe in a separate directory. The names are exactly that and have no .lnk after (I have tried both ways).
The aff array is for processor affinity.
This script should:
For Each Server
Create a New Sysem Diag Process with start info
Start the process
Set affinity
This is not the case: I get an error revolving around the "arg0" and it not being able to start the process. I suppose if anyone has some further explanation of what is going on with args0 and what it is would be helpful.
Code Snippet:
function startServers
{
#Added For Example Purposes
$servers = #( "public", "private" )
$aff = #( 196, 56 )
#End
$i = 0
foreach($s in $servers)
{
$app_name = "$s"
$a = $aff[$i]
$app_arguments = "arg0"
$pinfo = New-Object System.Diagnostics.ProcessStartInfo
$pinfo.Arguments = $app_arguments
$pinfo.FileName = $app_name
$p = New-Object System.Diagnostics.Process
$p.StartInfo = $pinfo
$p.Start()
$p.ProcessorAffinity=$a
$i = $i + 1
#Start "$s"
Write-Output "Starting $s"
}
}
function startServers {
$i = 0
$global:pids = #( )
foreach($s in $servers)
{
$a = $aff[$i]
$Starter = Start-Process -FilePath $s -WindowStyle Minimized -PassThru
$Process = Get-Process -Id $Starter.ID
$global:pids += $Process.ID
$Process.ProcessorAffinity = $a
$CurrentTime = updateTime
Write-Output "Starting $s at $CurrentTime"
$i = $i + 1
Start-Sleep -s 10
}
return
}
This is how I ended up solving the problem. I switched over to tracking the process by PID.
I used -PassThru on my Start-Process and I made it a variable.
I then made a variable that would hold the PID after it was started called Process.
Then you do the basic $foo.ProcessorAffinity = foo and you get your result.