Scriptblock works as Invoke-Command, but not as Start-Job - powershell

I have written a ScriptBlock that POSTs file uploads to a custom http server and saves the response to disk. It is intended to be used for testing said server and in addition to checking the correct response I'd like to run it in parallel to load the server.
The code in the ScriptBlock was built as separate ps1 script and works. I have then transplanted it into a framework-script using Start-Job for managing the jobs and eventually presenting overall results.
Unfortunately it does not work after that transplantation.
The symptom is that using Start-Job the job never finishes. For testing I have switched to Invoke-Command. A simple Invoke-Command works, InvokeCommand -AsJob does not.
Works:
Invoke-Command -Scriptblock $ScriptBlock -ArgumentList $Name,"C:\Projects\DCA\CCIT\ServiceBroker4\Java\eclipse-workspace\XFAWorker\testdata","P7-T"
Do not work:
Invoke-Command -AsJob -Computer localhost -Scriptblock $ScriptBlock -ArgumentList $Name,"C:\Projects\DCA\CCIT\ServiceBroker4\Java\eclipse-workspace\XFAWorker\testdata","P7-T"
Start-Job -Name $Name -ScriptBlock $Scriptblock -ArgumentList $Name,"C:\Projects\DCA\CCIT\ServiceBroker4\Java\eclipse-workspace\XFAWorker\testdata","P7-T"
The ScriptBlock being used is rather longish. It starts off by declaring Parameters:
$Scriptblock = {
Param (
[string]$JobName,
[string]$path,
[string]$BASEJOBNAME
)
And further down uses HttpWebRequest and a few other .NET classes:
$url = "http://localhost:8081/fillandflatten"
[System.Net.HttpWebRequest] $req = [System.Net.WebRequest]::create($url)
...
$xfabuffer = [System.IO.File]::ReadAllBytes("$path\$BASEJOBNAME.xml")
...
$header = "--$boundary`r`nContent-Disposition: form-data; name=`"xfa`"; filename=`"xfa`"`r`nContent-Type: text/xml`r`n`r`n"
$buffer = [Text.Encoding]::ascii.getbytes($header)
...
[System.Net.httpWebResponse] $res = $req.getResponse()
As described when using Start-Job or Invoke-Command -AsJob the child job started simply remains in Running state forever.
What can cause this behaviour? What can be used to debug it? If there is some error can I force the child-job to terminate and tell me what it does not like?
I am using PowerShell 2.0 on Windows XP.
In the framework I came up with I do the Start-Job (currently just one of them, but I plan to ramp up that number for stress-testing). Then I have a loop waiting for them all to terminate:
do {
$Jobs = #(Get-Job | Where { $_.State -eq "Running" -and $_.Name.StartsWith($BASEJOBNAME) });
$n = $Jobs.Count
if ($n -gt 0)
{
Log -message "Waiting for $n jobs to finish..."
Get-Job | Where { $_.Name.StartsWith($BASEJOBNAME) } | Format-Table -AutoSize -Property Id,Name,State,HasMoreData,Location
start-Sleep -Seconds 3
}
} until ($n -eq 0)
This produces output of the form:
2014-08-01 18:58:52 - Waiting for 1 jobs to finish...
Id Name State HasMoreData Location
-- ---- ----- ----------- --------
2 XFA001 Running True localhost
Forever.
A full minimal test-case is:
# Code for the Jobs:
$Scriptblock = {
[System.Net.HttpWebRequest] $req = [System.Net.WebRequest]::create("http://locahost/index.html")
return "done"
}
# Start a job. Three variants, The Job-based ones hang.
# Invoke-Command -Scriptblock $ScriptBlock
#Invoke-Command -AsJob -Computer localhost -Scriptblock $ScriptBlock
$Job = Start-Job -Name $Name -ScriptBlock $Scriptblock
## The rest of the code is only applicable for the Start-Job-Variant
# Wait for all Jobs to finish
do {
$Jobs = #(Get-Job | Where { $_.State -eq "Running" });
$n = $Jobs.Count
if ($n -gt 0)
{
Write-Host "Waiting for $n jobs to finish..."
Get-Job | Format-Table -AutoSize -Property Id,Name,State,HasMoreData
Start-Sleep -Seconds 3
}
} until ($n -eq 0)
# Get output from all jobs
$Data = ForEach ($Job in (#(Get-Job | Where { $_.Name.StartsWith($BASEJOBNAME) } ))) {
Receive-Job $Job
}
# Clean out all jobs
ForEach ($Job in (#(Get-Job | Where { $_.Name.StartsWith($BASEJOBNAME) } ))) {
Remove-Job $Job
}
# Dump output
Write-Host "Output data:"
$Data | Format-Table
Write-Host ""
This hangs for me. If I comment out the line creating the WebRequest object it works.
Thank you.

When you run Get-Job , does the job's "HasMoreData" properties is "True" ?
If yes, check the output of the job :
Receive-Job <JobName or JobID> -Keep

Related

Adobe flash player powershell remote install problem

I'm attempting to develop a script with PowerShell to remotely install/update flash player for multiple machines. No matter what I do, I can't get the install to work properly at all. I'm very limited with my tools so I have to use PowerShell, and the MSI install of Flashplayer. I'll post my script below, any help at all would be greatly appreciated.
$Computers = Get-Content C:\Users\name\Desktop\flash.txt
(tried these 3 methods to install none work)
$install = #("/a","/i", "\\$Computer\c$\temp\flash\install_flash_player_32_plugin.msi", "/qn","/norestart")
Invoke-Command -ComputerName $Computer -ScriptBlock {Start-Process "Msiexec" -arg "$using:install" -Wait -PassThru} -Filepath msiexec.exe
#This returns with "invoke-command: parameter set cannot be resolved using the specified named parameters"
Invoke-Command -ComputerName $computer -ScriptBlock {Start-Process -Filepath msiexec.exe "$using:install" -Wait -PassThru} -Filepath msiexec.exe
#this returns the same error.
Invoke-Command -ComputerName $Computer -ScriptBlock {start-process msiexec -argumentlist #('/a','/i','"\\$Computer\c$\temp\flash\install_flash_player_32_plugin.msi"','/qn')}
#this seemingly skips the install entirely.
I've used similar scripts for other programs and had no problems installing them, but none of the methods I use or have researched are working properly.
This should do the trick, I'll explain why it wasn't working bellow:
$Computers = Get-Content C:\Users\name\Desktop\flash.txt
$params = '/i <path to AcroPro.msi> LANG_LIST=en_US TRANSFORMS="1033.mst" /qb'
$Computers | % {
Invoke-Command -ScriptBlock {
Param(
[Parameter(Mandatory=$true,Position=0)]
[String]$arguments
)
return Start-Process msiexec.exe -ArgumentList $arguments -Wait -PassThru
} -ComputerName $_ -ArgumentList $params
}
So, it wasn't working because the ScriptBlock on Invoke-Command cant see variables that you've declared on your powershell session, think of it like you are walking to that remote computer and inputting that code by hand, you wont have the value (metaphor).
I did a few more changes:
I moved all params into 1 single string, no need to have them in array.
Added $Computers | to iterate through computer names.
Removed FilePath as this is meant to be used differently, documentation(Example #1).
Set $MinutesToWait to whatever amount of minutes you want.
No need to try to pass msiexec, as it comes with windows the default path is "C:\WINDOWS\system32\msiexec.exe"
Added a return even though its never necessary, but to make it more readable and to show you intent to return the output of the msiexec process.
Replaced \\$Computer\c$ with C:\ as there's no need to use a network connection if you are pointing to the host you are running the command in/
Hope it helps, good luck.
EDIT:
So, as you mentioned the pipeline execution gets stuck, I had this issue in the past when creating the computer preparation script for my department, what I did was use jobs to create parallel executions of the installation so if there's a computer that for some reason is slower or is just flat out stuck and never ends you can identify it, try the following as is to see how it works and then do the replaces:
#region ######## SetUp #######
$bannerInProgress = #"
#######################
#Jobs are still running
#######################
"#
$bannerDone = #"
##################################################
#DONE see results of finished installations bellow
##################################################
"#
#VARS TO SET
$MinutesToWait = 1
$computers = 1..10 | % {"qwerty"*$_} #REPLACE THIS WITH YOUR COMPUTER VALUES (Get-Content C:\Users\name\Desktop\flash.txt)
#endregion
#region ######## Main #######
#Start Jobs (REPLACE SCRIPTBLOCK OF JOB WITH YOUR INVOKE-COMMAND)
$jobs = [System.Collections.ArrayList]::new()
foreach($computer in $computers){
$jobs.Add(
(Start-Job -Name $computer -ScriptBlock {
Param(
[Parameter(Mandatory=$true, Position=0)]
[String]$computer
)
Sleep -s (Get-Random -Minimum 5 -Maximum 200)
$computer
} -ArgumentList $computer)
) | Out-Null
}
$timer = [System.Diagnostics.Stopwatch]::new()
$timer.Start()
$acceptedWait = $MinutesToWait * 60 * 1000 # mins -> sec -> millis
$running = $true
do {
cls
$jobsRunning = $jobs | Where-Object State -EQ 'Running'
if ($jobsRunning) {
Write-Host $bannerInProgress
foreach ($job in $jobsRunning) {
Write-Host "The job `"$($job.Name)`" is still running. It started at $($job.PSBeginTime)"
}
Sleep -s 3
} else {
$running = $false
}
if($timer.ElapsedMilliseconds -ge $acceptedWait){
$timer.Stop()
Write-Host "Accepted time was reached, stopping all jobs still pending." -BackgroundColor Red
$failed = #()
foreach($job in $jobsRunning){
$output = $job | Receive-Job
$failed += [PsCustomObject]#{
"ComputerName" = $job.Name;
"Output" = $output;
}
$job | Remove-Job -Force
$jobs.Remove($job)
}
$failed | Export-Csv .\pendingInstallations.csv -NoTypeInformation -Force
$running = $false
}
}while($running)
Write-host $bannerDone
$results = #()
foreach($job in $jobs){
$output = $job | Receive-Job
$results += [PsCustomObject]#{
"ComputerName" = $job.Name;
"Output" = $output;
}
}
$results | Export-Csv .\install.csv -NoTypeInformation -Force
#endregion
This script will trigger 10 jobs that only wait and return its names, then the jobs that got completed in the time that you set are consider correct and the ones that didn't are consider as pending, both groups get exported to a CSVfor review. You will need to replace the following to work as you intended:
Add $params = '/i <path to AcroPro.msi> LANG_LIST=en_US TRANSFORMS="1033.mst" /qb' in the SetUp region
Replace the declaration of $computers with $computers = Get-Content C:\Users\name\Desktop\flash.txt
Replace the body of Start-Job scriptblock with Invoke-command from thew first snippet of code in this answer.
you should end-up with something like:
.
.code
.
$params = '/i <path to AcroPro.msi> LANG_LIST=en_US TRANSFORMS="1033.mst" /qb'
#VARS TO SET
$MinutesToWait = 1
$computers = Get-Content C:\Users\name\Desktop\flash.txt
#endregion
#region ######## Main #######
#Start Jobs
$jobs = [System.Collections.ArrayList]::new()
foreach($computer in $computers){
$jobs.Add(
(Start-Job -Name $computer -ScriptBlock {
Param(
[Parameter(Mandatory=$true, Position=0)]
[String]$computer
)
Invoke-Command -ScriptBlock {
Param(
[Parameter(Mandatory=$true,Position=0)]
[String]$arguments
)
return Start-Process msiexec.exe -ArgumentList $arguments -Wait -PassThru
} -ComputerName $computer -ArgumentList $params
} -ArgumentList $computer)
) | Out-Null
}
.
. code
.
I know it looks like a complete mess, but it works.
Hope it helps.

Using Powershell To Distribute Script Level Jobs On Remote Servers

More of a theory question...
I have a powershell script that exists on three servers. In this example the three servers are:
server1
server2
server3
I am using another machine, server4, to call script C:\ExampleScript.ps1 remotely using Invoke-Command while specifying the remote machine via the ComputerName parameter. The ultimate goal of the script is to detect whether powershell is running, if it is not, then the computer is "not busy" and can open up the script being called remotely. If the computer is "busy", move onto the next server and continue on through the three machines until all the parameter values have been exhausted. If all machines are busy, it would be ideal if there was a way to periodically check the processes and see if they are still open. In this way, execution of the script can be balanced across the various machines, in an albeit primitive fashion.
Consider the following code:
$servers = "server1","server2","server3"
$data = "param1", "param2", "param3", "param4", "param5", "param6"
#somehow loop through the different servers/data using the above arrays
$job = Invoke-Command $servers[0] {
$ProcessActive = Get-Process powershell -ErrorAction SilentlyContinue
if($ProcessActive -eq $null)
{
"Running"
Invoke-Command -ComputerName $env:computername -FilePath C:\ExampleScript.ps1 -ArgumentList $data[0]
}
else
{
"Busy go to next machine"
}
} -AsJob
Wait-Job $job
$r = Receive-Job $job
$r
The expected result trying to be achieved is attempting to load balance the script across the machines based on whether there is an active powershell process, if not move onto the next machine and perform the same test and subsequent possible execution. The script should go through all the values as specified in the $data array (or whatever).
I found this question interesting, so I wanted to give it a try.
$servers = "server1","server2","server3"
$data = New-Object System.Collections.ArrayList
$data.AddRange(#("param1", "param2", "param3", "param4", "param5", "param6"))
$jobs = New-Object System.Collections.ArrayList
do
{
Write-Host "Checking job states." -ForegroundColor Yellow
$toremove = #()
foreach ($job in $jobs)
{
if ($job.State -ne "Running")
{
$result = Receive-Job $job
if ($result -ne "ScriptRan")
{
Write-Host " Adding data back to que >> $($job.InData)" -ForegroundColor Green
$data.Add($job.InData) | Out-Null
}
$toremove += $job
}
}
Write-Host "Removing completed/failed jobs" -ForegroundColor Yellow
foreach ($job in $toremove)
{
Write-Host " Removing job >> $($job.Location)" -ForegroundColor Green
$jobs.Remove($job) | Out-Null
}
# Check if there is room to start another job
if ($jobs.Count -lt $servers.Count -and $data.Count -gt 0)
{
Write-Host "Checking servers if they can start a new job." -ForegroundColor Yellow
foreach ($server in $servers)
{
$job = $jobs | ? Location -eq $server
if ($job -eq $null)
{
Write-Host " Adding job for $server >> $($data[0])" -ForegroundColor Green
# No active job was found for the server, so add new job
$job = Invoke-Command $server -ScriptBlock {
param($data, $hostname)
$ProcessActive = Get-Process powershell -ErrorAction SilentlyContinue
if($ProcessActive -eq $null)
{
# This will block the thread on the server, so the JobState will not change till it's done or fails.
Invoke-Command -ComputerName $hostname -FilePath C:\ExampleScript.ps1 -ArgumentList $data
Write-Output "ScriptRan"
}
} -ArgumentList $data[0], $env:computername -AsJob
$job | Add-Member -MemberType NoteProperty -Name InData -Value $data[0]
$jobs.Add($job) | Out-Null
$data.Remove($data[0])
}
}
}
# Just a manual check of $jobs
Write-Output $jobs
# Wait a bit before checking again
Start-Sleep -Seconds 10
} while ($data.Count -gt 0)
Basically I create an array, and keep it constantly populated with one job for each server.
Data is removed from the list when a new job starts, and is added back if a job fails. This is to avoid servers running the script with the same data/params.
I lack a proper environment to test this properly at the moment, but will give it a whirl at work tomorrow and update my answer with any changes if needed.

Multi-threading with PowerShell

I have done lots of reading about multi-threading in PowwerShell with Get-Job and Wait-Job but still cant seem to work it out.
Eventually, I will have this as a GUI based script to run and don't want my GUI to freeze up while its doing its task.
The script is looking for Event Logs of my Domain Controllers and then getting the details I want, then outputting them, it works like I need it to.
I can start a job using Invoke-Command {#script goes here} -ComputerName ($_) -AsJob -JobName $_ and the jobs run.
Script below:
Clear-Host
Get-Job | Remove-Job
(Get-ADDomainController -Filter *).Name | ForEach-Object {
Invoke-Command -ScriptBlock {
$StartTime = (Get-Date).AddDays(-4)
Try{
Get-WinEvent -FilterHashtable #{logname='Security'; id=4740;StartTime=$StartTime} -ErrorAction Stop `
| Select-Object * | ForEach-Object {
$Username = $_.Properties[0].Value
$lockedFrom = $_.Properties[1].Value
$DC = $_.Properties[4].Value
$Time = $_.TimeCreated
Write-Host "---------------------------------------------"
Write-Host $Username
Write-Host $lockedFrom
Write-Host $DC
Write-Host $Time
Write-Host "---------------------------------------------"
}#ForEach-Object
}catch [Exception] {
If ($_.Exception -match "No events were found that match the specified selection criteria") {
Write-Host "No events for locked out accounts." -BackgroundColor Red
}#If
}#Try Catch
} -ComputerName ($_) -AsJob -JobName $_ | Out-Null # Invoke-Command
}#ForEach-Object
Currently I have a While loop to tell me its waiting then to show me the result:
(Get-ADDomainController -Filter *).Name | ForEach-Object {
Write-Host "Waiting for: $_."
While ($(Get-Job -Name $_).State -ne 'Completed') {
#no doing anything here
}#While
Receive-Job -Name $_ -Keep
}#ForEach-Object
#clean up the jobs
Get-Job | Remove-Job
Thinking of my GUI (to be created), I will have a column for each Domain Controller and showing results under each heading, how do make it not freeze my GUI and show the results when they arrive?
I know its been asked a few times, but the examples I cant work out.
I would avoid Start-Job for threading - for efficiency try a runspace factory.
This is a basic setup which could be useful (I also have PS 4.0), and open to suggestions/improvements.
$MaxThreads = 2
$ScriptBlock = {
Param ($ComputerName)
Write-Output $ComputerName
#your processing here...
}
$runspacePool = [RunspaceFactory]::CreateRunspacePool(1, $MaxThreads)
$runspacePool.Open()
$jobs = #()
#queue up jobs:
$computers = (Get-ADDomainController -Filter *).Name
$computers | % {
$job = [Powershell]::Create().AddScript($ScriptBlock).AddParameter("ComputerName",$_)
$job.RunspacePool = $runspacePool
$jobs += New-Object PSObject -Property #{
Computer = $_
Pipe = $job
Result = $job.BeginInvoke()
}
}
# wait for jobs to finish:
While ((Get-Job -State Running).Count -gt 0) {
Get-Job | Wait-Job -Any | Out-Null
}
# get output of jobs
$jobs | % {
$_.Pipe.EndInvoke($_.Result)
}

Use PowerShell to run virus scan on multiple servers

I'm trying to run a virus scan on a list of servers in our environment. There are hundreds of machines, so we'd like to run the scan (using a command line prompt that we already have) around 10 at a time. We're totally new to PowerShell so any help would be really appreciated. We have a general idea of what commands we need to use -- here's how we think it might work for now:
$server = Get-Content "serverlist.txt"
$server | % {
$VirusScan = { Scan32.exe }
Invoke-Command -ScriptBlock { $VirusScan } -computerName $server -ThrottleLimit 10 -Authentication domain/admin
}
Does anyone have any suggestions on how we might orchestrate this?
I'm using something like this for running tasks in parallel on remote hosts:
$maxSlots = 10
$hosts = "foo", "bar", "baz", ...
$job = {
Invoke-Command -ScriptBlock { Scan32.exe } -Computer $ARGV[0] -ThrottleLimit 10 -Authentication domain/admin
}
$queue = [System.Collections.Queue]::Synchronized((New-Object System.Collections.Queue))
$hosts | ForEach-Object { $queue.Enqueue($_) }
while ( $queue.Count -gt 0 -or #(Get-Job -State Running).Count -gt 0 ) {
$freeSlots = $maxSlots - #(Get-Job -State Running).Count
for ( $i = $freeSlots; $i -gt 0 -and $queue.Count -gt 0; $i-- ) {
Start-Job -ScriptBlock $job -ArgumentList $queue.Dequeue() | Out-Null
}
Get-Job -State Completed | ForEach-Object {
Receive-Job -Id $_.Id
Remove-Job -Id $_.Id
}
Sleep -Milliseconds 100
}
# Remove all remaining jobs.
Get-Job | ForEach-Object {
Receive-Job -Id $_.Id
Remove-Job -Id $_.Id
}

Variables in Start-Job do not get evaluated

My Powershell code doesn't evaluate the $agent variable:
foreach ($agent in $agentcomputers) {
Write-Output 'Starting agent on '$agent
# psexc to start the agent
Start-Job -ScriptBlock {& psexec $agent c:\grinder\examples\startAgent.cmd}
}
This link is similar to my problem, except I'm not calling an external Powershell script.
I tried adding that in, using $args[0] for $agent, and adding the -ArgumentList parameters, but that didn't work.
Edits/Replies
$agentcomputers is just a list of computer names - each on its own line:
$agentcomputers = Get-Content c:\grinder-dist\agent-computers.txt
I have also tried this - and $args[0] doesn't evaluate:
Start-Job -ScriptBlock {& psexec $args[0] c:\grinder\examples\startAgent.cmd} -ArgumentList #($agent)
Here are 3 different ways I would do it.
First, all aligned and pretty.
$agents = Get-Content c:\grinder-dist\agent-computers.txt
$jobs = {
Param($agent)
write-host "Starting agent on" $agent
& psexec \\$agent c:\grinder\examples\startAgent.cmd
}
foreach($agent in $agents) {
Start-Job -ScriptBlock $jobs -argumentlist $agent | Out-Null
}
Get-Job | Wait-Job | Receive-Job
Or you could just put it all on one line without creating any variables.
(Get-Content c:\grinder-dist\agent-computers.txt) | %{ Start-Job -ScriptBlock { param($_) write-host "Starting agent on" $_; & psexec \\$_ c:\grinder\examples\startAgent.cmd } -argumentlist $_ | Out-Null }
Get-Job | Wait-Job | Receive-Job
And in this final example, you could manage how many threads are run concurrently by doing it this way.
$MaxThreads = 5
$agents = Get-Content c:\grinder-dist\agent-computers.txt
$jobs = {
Param($agent)
write-host "Starting agent on" $agent
& psexec \\$agent c:\grinder\examples\startAgent.cmd
}
foreach($agent in $agents) {
Start-Job -ScriptBlock $jobs -argumentlist $agent | Out-Null
While($(Get-Job -State 'Running').Count -ge $MaxThreads) {
sleep 10
}
Get-Job | Wait-Job | Receive-Job
}
Here is the solution. As Andy said, I needed to use $args array with the -ArgumentList parameter. This other thread was helpful: Powershell: passing parameters to a job
foreach($agent in $agentcomputers){
$agentslash = "\\"+$agent
$args = ($agentslash,"c:\grinder\examples\startAgent.cmd")
Write-Output 'Starting agent on '$agent
#psexc to start the agent
$ScriptBlock = {& 'psexec' #args }
Start-Job -ScriptBlock $ScriptBlock -ArgumentList $args
}