Powershell: Runspace/Pool how do i pass a function and shared variable? - powershell

I am trying to create a thread that will use a shared variable (between main session and the thread)
plus the give the thread ability to use outside function from the main code
I have managed to pass the function for the thread to use
and i have managed to pass read only variable.
my problem is that if i am changing the value of the variable inside the thread and then i try to read it from the main session - i can not see the change in the value, therefore its not shared.
How do i do that?
My goal is to have one thread in the end.
this is my code:
$x = [Hashtable]::Synchronized(#{})
$global:yo = "START"
Function ConvertTo-Hex {
#Write-Output "Function Ran"
write-host "hi"
$x.host.ui.WriteVerboseLine("===========")
write-host $yo
$yo="TEST"
write-host $yo
}
#endregion
$rs = [RunspaceFactory]::CreateRunspace()
$rs.ApartmentState,$rs.ThreadOptions = "STA","ReUseThread"
$rs.Open()
$rs.SessionStateProxy.SetVariable("x",$x)
ls
# create an array and add it to session state
$arrayList = New-Object System.Collections.ArrayList
$arrayList.AddRange(('a','b','c','d','e'))
$x.host = $host
$sessionstate = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
$sessionstate.Variables.Add((New-Object System.Management.Automation.Runspaces.SessionStateVariableEntry('arrayList', $arrayList, $null)))
$sessionstate.Variables.Add((New-Object System.Management.Automation.Runspaces.SessionStateVariableEntry('x', $x, $null)))
#$sessionstate.Variables.Add((New-Object System.Management.Automation.Runspaces.SessionStateVariableEntry('yo', $yo, $true)))
$sessionstate.Commands.Add((New-Object System.Management.Automation.Runspaces.SessionStateFunctionEntry -ArgumentList 'ConvertTo-Hex', (Get-Content Function:\ConvertTo-Hex -ErrorAction Stop)))
$runspacepool = [runspacefactory]::CreateRunspacePool(1, 2, $sessionstate, $Host)
$runspacepool.Open()
$ps1 = [powershell]::Create()
$ps1.RunspacePool = $runspacepool
$ps1.AddScript({
for ($i = 1; $i -le 15; $i++)
{
$letter = Get-Random -InputObject (97..122) | % {[char]$_} # a random lowercase letter
$null = $arrayList.Add($letter)
start-sleep -s 1
}
})
$ps1.AddParameter(#($yo))
# on the first thread start a process that adds values to $arrayList every second
$handle1 = $ps1.BeginInvoke()
# now on the second thread, output the value of $arrayList every 1.5 seconds
$ps2 = [powershell]::Create()
$ps2.RunspacePool = $runspacepool
$ps2.AddScript({
Write-Host "ArrayList contents is "
foreach ($i in $arrayList)
{
Write-Host $i -NoNewline
Write-Host " " -NoNewline
}
Write-Host ""
$yo = "BAH"
ConvertTo-Hex
})
$ps2.AddParameter(#($yo))
1..10 | % {
$handle2 = $ps2.BeginInvoke()
if ($handle2.AsyncWaitHandle.WaitOne())
{
$ps2.EndInvoke($handle2)
}
start-sleep -s 1.5
write-host "====================" + $yo
}

Related

my Powershell script doesn't do multi-thread job , why?

#my script is a excerpt of https://www.codeproject.com/Tips/895840/Multi-Threaded-PowerShell-Cookbook
#the first example
#the issue is where ".AddScript($secb)" is. All jobs are finished sequentially , Could anyone explain ???
#why in my script , .AddScript($sb) is concurrent ??
$numThreads = 5
# Create session state
$myString = "this is session state!"
$sessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
$sessionstate.Variables.Add((New-Object -TypeName System.Management.Automation.Runspaces.SessionStateVariableEntry -ArgumentList "myString" ,$myString, "example string"))
# Create runspace pool consisting of $numThreads runspaces
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, 5, $sessionState, $Host)
$RunspacePool.Open()
$Jobs = #()
$sb={
param ($data)
$r=Get-Random
Write-Host "before $r"
Start-Sleep -Seconds 3
Write-Host "after $r"
}
$secb={
param ($block)
Invoke-Command -ScriptBlock $block
}
1..5 | % {
#below line is not concurrent , i don't know why
$Job = [powershell]::Create().AddScript($secb) # $Job = [powershell]::Create().AddScript($sb) could do multi-thread
$Job.AddArgument($sb)
$Job.RunspacePool = $RunspacePool
$Jobs += New-Object PSObject -Property #{
RunNum = $_
Job = $Job
Result = $Job.BeginInvoke()
}
}
Write-Host "Waiting.." -NoNewline
Do {
Write-Host "." -NoNewline
Start-Sleep -Seconds 1
} While ( $Jobs.Result.IsCompleted -contains $false) #Jobs.Result is a collection
I'm by far not an expert on RunSpaces, usually whenever I need multithreading I use the ThreadJob module. I'm not sure what you are doing wrong in your code but I can show you a working example for what you're trying to do. I took this example from this answer (credits to Mathias) which is excellent and modified it a bit.
Code:
cls
$elapsedTime = [System.Diagnostics.Stopwatch]::StartNew()
$numThreads = 5
# Create session state
$myString = "this is session state!"
$sessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
$sessionstate.Variables.Add((New-Object -TypeName System.Management.Automation.Runspaces.SessionStateVariableEntry -ArgumentList "myString" ,$myString, "example string"))
# Create runspace pool consisting of $numThreads runspaces
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, 5, $sessionState, $Host)
$RunspacePool.Open()
$runspaces = foreach($i in 1..5)
{
$PSInstance = [powershell]::Create().AddScript({
param ($TestNumber)
Write-Output "Test Number: $TestNumber"
$r={Get-Random}
Write-Output "before $(& $r)"
Start-Sleep -Seconds 3
Write-Output "after $(& $r)"
}).AddParameter('TestNumber',$i)
$PSInstance.RunspacePool = $RunspacePool
[pscustomobject]#{
Instance = $PSInstance
Result = $PSInstance.BeginInvoke()
}
}
while($runspaces|Where-Object{-not $_.Result.IsCompleted})
{
Start-Sleep -Milliseconds 500
}
$resultRunspace = [collections.generic.list[string]]::new()
$Runspaces|ForEach-Object {
$resultRunspace.Add($_.Instance.EndInvoke($_.Result))
}
$elapsedTime.Stop()
"Elapsed Time: {0}" -f $elapsedTime.Elapsed.TotalSeconds
""
$resultRunspace
Result:
Elapsed Time: 3.1271587
Test Number: 1 before 474010429 after 2055432874
Test Number: 2 before 1639634857 after 1049683678
Test Number: 3 before 72786850 after 2046654322
Test Number: 4 before 1958738687 after 1832326064
Test Number: 5 before 1944958392 after 1652518661
Now, again, if you are able to install modules I would recommend ThreadJob as it is a lot easier to use and performs equally as fast as RunSpaces.
I wrote a script some time ago which would loop through all the directories in $HOME and get the number of files on each one, the script was meant to compare Linear Loops vs ThreadJob vs RunSpace here are the results and why I would always recommend ThreadJob

ping computers and list logged in user

i got a question to my existing script.
the script does exactly what it should to do
This is what i got.
<#
Ping mass huge amount of hosts in parallel multithreading
#>
function Parallel-Ping {
[CmdletBinding()]
param(
[parameter(mandatory=$true)][ValidateNotNullOrEmpty()][string[]]$hosts,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][int]$timeout = 2000,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][ValidateRange(1,255)][int]$ttl = 128,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][bool]$DontFragment = $false,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][byte[]]$data = ([byte[]]0),
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][int]$MaxConcurrent = [System.Environment]::ProcessorCount,
[parameter(mandatory=$false)][ValidateNotNullOrEmpty()][switch]$showprogress
)
begin{
$rspool = [runspacefactory]::CreateRunspacePool(1,$MaxConcurrent)
$rspool.ApartmentState = 'STA'
$rspool.Open()
$jobs = New-Object System.Collections.ArrayList
}
process{
foreach ($hostname in $hosts){
$ps = [Powershell]::Create()
$ps.RunspacePool = $rspool
[void]$ps.AddScript({
param($hostname,$timeout,$ttl,$DontFragment,$data)
$result = [ordered]#{Host=$hostname;Address=$null;Status=[System.Net.NetworkInformation.IPStatus]::Unknown;RoundtripTime = -1;TTL=$ttl;DontFragment=$DontFragment;ResponseBuffer='';Time=(get-date)}
$ping = New-Object System.Net.NetworkInformation.Ping
try{
$reply = $ping.Send($hostname,$timeout,$data,(New-Object System.Net.NetworkInformation.PingOptions($ttl,$DontFragment)))
$result.RoundtripTime = $reply.RoundtripTime
$result.Status = $reply.Status
$result.Address = $reply.Address
$result.ResponseBuffer = $reply.Buffer
}catch [System.Net.NetworkInformation.PingException] {
$result.Status = [System.Net.NetworkInformation.IPStatus]::DestinationHostUnreachable
}finally{
$ping.Dispose()
}
[pscustomobject]$result
}).AddParameters(#($hostname,$timeout,$ttl,$DontFragment,$data))
$job = $ps.BeginInvoke()
[void]$jobs.Add(([pscustomobject]#{Handle = $job; Powershell = $ps}))
}
}
end{
write-verbose "Waiting for all jobs to complete."
while(($jobs | ?{!$_.Handle.IsCompleted})){
if ($showprogress.IsPresent){
$completed = ($jobs | ?{$_.Handle.IsCompleted}).Count
Write-Progress -Activity $PSCmdlet.MyInvocation.InvocationName -Status "Pinging a total of $($hosts.Count) hosts." -PercentComplete (($completed / $hosts.Count) * 100) -CurrentOperation "Completed $completed from $($hosts.Count)."
}
}
# get results of jobs
$results = $jobs | %{
$_.Powershell.EndInvoke($_.handle)
$_.Powershell.Dispose()
}
# cleanup
$rspool.Close();$rspool.Dispose()
$results
}
}
$hosts = Get-Content 'E:\ips.txt'
Parallel-Ping -hosts $hosts -timeout 500 -MaxConcurrent 50 -showprogress | ogv
I would like to mention another function in my script.
My goal is it, to add the function that i can see the actual logged in user to the pinged machines?
Unfortunately i dont know where to add.

multithread with runspaces instead of foreach cycle

I have a script with a foreach cycle. It has about a dozen functions, each collecting information from remote machines' C$ share (cutting text files, checking file version, etc.)
This is however taking some time, since each machine's data collected after one by one. (sometimes it runs with 500+ input)
Wish to put this into runspaces with parallel execution, but so far no examples worked. I am quite new to the concept.
Current script's outline
$inputfile = c:\temp\computerlist.txt
function 1
function 2
function 3, etc
foreach cycle
function 1
function 2
function 3
All results written to screen with write-host for now.
This example pings a number of server in parallel, so you easily can modify it for your demands:
Add-Type -AssemblyName System.Collections
$GH = [hashtable]::Synchronized(#{})
[System.Collections.Generic.List[PSObject]]$GH.results = #()
[System.Collections.Generic.List[string]]$GH.servers = #('server1','server2','server3');
[System.Collections.Generic.List[string]]$GH.functions = #('Check-Server');
[System.Collections.Generic.List[PSObject]]$jobs = #()
#-----------------------------------------------------------------
function Check-Server {
#-----------------------------------------------------------------
# a function which runs parallel
param(
[string]$server
)
$result = Test-Connection $server -Count 1 -Quiet
$GH.results.Add( [PSObject]#{ 'Server' = $server; 'Result' = $result } )
}
#-----------------------------------------------------------------
function Create-InitialSessionState {
#-----------------------------------------------------------------
param(
[System.Collections.Generic.List[string]]$functionNameList
)
# Setting up an initial session state object
$initialSessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
foreach( $functionName in $functionNameList ) {
# Getting the function definition for the functions to add
$functionDefinition = Get-Content function:\$functionName
$functionEntry = New-Object System.Management.Automation.Runspaces.SessionStateFunctionEntry -ArgumentList $functionName, $functionDefinition
# And add it to the iss object
[void]$initialSessionState.Commands.Add($functionEntry)
}
return $initialSessionState
}
#-----------------------------------------------------------------
function Create-RunspacePool {
#-----------------------------------------------------------------
param(
[InitialSessionState]$initialSessionState
)
$runspacePool = [RunspaceFactory]::CreateRunspacePool(1, ([int]$env:NUMBER_OF_PROCESSORS + 1), $initialSessionState, $Host)
$runspacePool.ApartmentState = 'MTA'
$runspacePool.ThreadOptions = "ReuseThread"
[void]$runspacePool.Open()
return $runspacePool
}
#-----------------------------------------------------------------
function Release-Runspaces {
#-----------------------------------------------------------------
$runspaces = Get-Runspace | Where { $_.Id -gt 1 }
foreach( $runspace in $runspaces ) {
try{
[void]$runspace.Close()
[void]$runspace.Dispose()
}
catch {
}
}
}
$initialSessionState = Create-InitialSessionState -functionNameList $GH.functions
$runspacePool = Create-RunspacePool -initialSessionState $initialSessionState
foreach ($server in $GH.servers)
{
Write-Host $server
$job = [System.Management.Automation.PowerShell]::Create($initialSessionState)
$job.RunspacePool = $runspacePool
$scriptBlock = { param ( [hashtable]$GH, [string]$server ); Check-Server -server $server }
[void]$job.AddScript( $scriptBlock ).AddArgument( $GH ).AddArgument( $server )
$jobs += New-Object PSObject -Property #{
RunNum = $jobCounter++
JobObj = $job
Result = $job.BeginInvoke() }
do {
Sleep -Seconds 1
} while( $runspacePool.GetAvailableRunspaces() -lt 1 )
}
Do {
Sleep -Seconds 1
} While( $jobs.Result.IsCompleted -contains $false)
$GH.results
Release-Runspaces | Out-Null
[void]$runspacePool.Close()
[void]$runspacePool.Dispose()
This would be concurrent and run in about 10 seconds total. The computer could be localhost three times if you got it working.
invoke-command comp1,comp2,comp3 { sleep 10; 'done' }
Simple attempt at api (threads):
$a = [PowerShell]::Create().AddScript{sleep 5;'a done'}
$b = [PowerShell]::Create().AddScript{sleep 5;'b done'}
$c = [PowerShell]::Create().AddScript{sleep 5;'c done'}
$r1 = $a.BeginInvoke(); $r2 = $b.BeginInvoke() $r3 = $c.BeginInvoke()
$a.EndInvoke($r1); $b.EndInvoke($r2); $c.EndInvoke($r3)
a done
b done
c done

What can I change in this script to prevent an infinite loop

Here is my Powershell code that when i execute it, it will continually loop while waking up my internal sites
$urlFile = "C:\Users\lfouche\Desktop\url\url.txt"
foreach($site in Get-Content $urlFile){
function WakeUp([string] $url)
{
Write-Host "Waking up $url ..." -NoNewLine
$client = new-object system.net.WebClient
$client.UseDefaultCredentials = $true
$null = $client.OpenRead($url)
$client.Dispose()
Write-Host " Ok"
}
#(
Get-Content $urlFile
) | % { WakeUp $_ }
}
Well, your script can generally be improved, however I think the error is that you already iterate over Get-Content $urlFile within your foreach condition and also in the loop. Try this:
# define the wake up function once
function WakeUp
{
Param
(
[string] $url
)
Write-Host "Waking up $url ..." -NoNewLine
$client = new-object system.net.WebClient
$client.UseDefaultCredentials = $true
$null = $client.OpenRead($url)
$client.Dispose()
Write-Host " Ok"
}
$urlFile = "C:\Users\lfouche\Desktop\url\url.txt"
$sites = Get-Content $urlFile
foreach($site in $sites)
{
# call wake up for each site
WakeUp $site
}

Start() Error In Powershell (Starting a Shortcut)

I have created a Powershell script to do a timed restart on my server. This is a snippet of my code and more specifically the startServers Function.
I have added the global variables into the function in an effort to have an example.
Basically each server name is the name of a shortcut that starts an exe in a separate directory. The names are exactly that and have no .lnk after (I have tried both ways).
The aff array is for processor affinity.
This script should:
For Each Server
Create a New Sysem Diag Process with start info
Start the process
Set affinity
This is not the case: I get an error revolving around the "arg0" and it not being able to start the process. I suppose if anyone has some further explanation of what is going on with args0 and what it is would be helpful.
Code Snippet:
function startServers
{
#Added For Example Purposes
$servers = #( "public", "private" )
$aff = #( 196, 56 )
#End
$i = 0
foreach($s in $servers)
{
$app_name = "$s"
$a = $aff[$i]
$app_arguments = "arg0"
$pinfo = New-Object System.Diagnostics.ProcessStartInfo
$pinfo.Arguments = $app_arguments
$pinfo.FileName = $app_name
$p = New-Object System.Diagnostics.Process
$p.StartInfo = $pinfo
$p.Start()
$p.ProcessorAffinity=$a
$i = $i + 1
#Start "$s"
Write-Output "Starting $s"
}
}
function startServers {
$i = 0
$global:pids = #( )
foreach($s in $servers)
{
$a = $aff[$i]
$Starter = Start-Process -FilePath $s -WindowStyle Minimized -PassThru
$Process = Get-Process -Id $Starter.ID
$global:pids += $Process.ID
$Process.ProcessorAffinity = $a
$CurrentTime = updateTime
Write-Output "Starting $s at $CurrentTime"
$i = $i + 1
Start-Sleep -s 10
}
return
}
This is how I ended up solving the problem. I switched over to tracking the process by PID.
I used -PassThru on my Start-Process and I made it a variable.
I then made a variable that would hold the PID after it was started called Process.
Then you do the basic $foo.ProcessorAffinity = foo and you get your result.