I need to output several logfiles - while they are written - to the shell.
In the unix version of my script this is achieved by tail -f output\worker*.log. Note the wildcard.
In Powershell I tried Get-Content -Path "output\worker*.log" -Wait, but this only prints the first logfile it can find to the shell.
For completion, here is my code where i call my program:
foreach ($worker in $workers)
{
Write-Host " Start $worker in background"
$block = {& $args[0] $args[1] $args[2] $args[3] $args[4] $args[5] $args[6] 2> $args[7] > $args[8]}
start-job -name $worker -scriptblock $block -argumentlist `
"$strPath\worker\bin\win32\php.exe", `
"-q", `
"-c", `
"$strPath\worker\conf\php_win32.ini", `
"$strPath\worker\bin\os-independant\logfilefilter\logfilefilter.php", `
"-f", `
"$strPath\worker\$worker\conf\logfilefilter-$worker.xml", `
"$strPath\output\$worker-error.log", `
"$strPath\output\$worker.log"
}
Get-Content -Path "output\worker*.log" -Wait
In my test case there are 8 workers and logfiles( output\worker01.log, output\worker02.log, output\worker03.log, output\worker04.log, output\worker05.log, output\worker06.log, output\worker07.log, output\worker08.log )
Is there a workaround to output all these logfiles? Or is it possible to duplicate the stdout stream from the background process to print it in the shell?
You can duplicate the the output streams by reading them directly from the output buffers of the child jobs.
Here's a demo script. Note that for the Verbose output, I'm removing output from the buffer as it's read, so that it doesn't get re-displayed on subsequent reads. This doesn't appear to have any affect on the Receive Job buffers for the job. If you do a Receive Job on it after the script completes, you'll still get the Verbose output all over again.
$sb = {
$VerbosePreference = 'Continue'
for ($i = 1; $i -le 100; $i++ )
{
start-sleep -Milliseconds 150
Write-Verbose "$(get-date)"
write-progress -activity "Search in Progress" -status "$i% Complete:" -percentcomplete $i;}
}
$job = start-job -ScriptBlock $sb
$verbose = ($job.ChildJobs[0].Verbose)
While ($job.State -ne 'Completed')
{
$job.ChildJobs |
foreach {
Start-Sleep -seconds 1
$Pct_Complete = $_.Progress | select -last 1 | select -ExpandProperty PercentComplete
Write-Host "`rBackground job $($_.ID) is $Pct_Complete percent completed." -ForegroundColor Cyan
While ($verbose.count){
Write-Host $verbose[0] -ForegroundColor Gray
$verbose.removeat(0)}
}
}
write-host "`nDone!"
Related
I have a small script that runs through all share paths within a csv file and invokes a quick Get-ChildItem to count the number of files within the folder/subfolders etc. This is working but it is taking an age, especially if a folder have more than 500k files.
I would like to add a timeout within the loop, so that it will just write-host and then output into another column 'TIMED OUT' but i can't seem to get it to work.
I have tried a few different ways but somehow my loop breaks and runs the top line in my csv over and over.
This is the code i have so far:
...
#Import middle to filter unique
$sharesMiddle = Import-Csv -Path './middle.csv' | Sort-Object NFTSPath -Unique
$result = foreach ($share in $sharesMiddle) {
#Replace the colon for $ within the path
$nftsPath = join-path \\ $share.AssetName $share.Path.Replace(':', '$')
$path = $share.Path
$count = Invoke-Command -computername $share.AssetName -ScriptBlock { param($path) (Get-ChildItem $path -File -Recurse).count } -Credential $Cred -Verbose -ArgumentList $path
Write-Host $nftsPath : $count
$share | Select-Object *, #{n = "Files"; e = { $count } }
}
$result | Export-CSV '.\newcsvfile.csv' -NoTypeInformation
Any help on this would be super!
Thanks.
I tested the following which should be suitable for you:
# define the time limit in seconds
$TimeoutLimitSeconds = 2
# Define the job, start job
$job = Start-Job -ScriptBlock {420+69 ; start-sleep -Seconds 3}
# Await job
$job | Wait-Job -Timeout ( $TimeoutLimitSeconds ) | Out-Null
# If job doesn't finish before the time limit is reached
if ($job.state -eq 'Running') {
# Job timed out
$job | Stop-Job | Remove-Job
$job = $null
$output = "TIMED OUT"
# If job managed to finalize in time
}Else{
# Finished on time
$job | Stop-Job
$output = Receive-Job $job
$job | Remove-Job
}
# Write the output to host
Write-Host "output is: $output"
output is: TIMED OUT
# define the time limit in seconds
$TimeoutLimitSeconds = 4
# Define the job, start job
$job = Start-Job -ScriptBlock {420+69 ; start-sleep -Seconds 3}
# Await job
$job | Wait-Job -Timeout ( $TimeoutLimitSeconds ) | Out-Null
# If job doesn't finish before the time limit is reached
if ($job.state -eq 'Running') {
# Job timed out
$job | Stop-Job | Remove-Job
$job = $null
$output = "TIMED OUT"
# If job managed to finalize in time
}Else{
# Finished on time
$job | Stop-Job
$output = Receive-Job $job
$job | Remove-Job
}
# Write the output to host
Write-Host "output is: $output"
output is: 489
Using a script in PowerShell to recursivly pass through all folders on multiple NAS boxes to display every folder with its full path in an Out-File.
Using the Get-FolderEntry script I found here.
Since I have multiple NAS boxes with more then 260 chars in the filename/pathlength I figured I'd use multithreading to speed the process up.
Code:
. C:\Users\mdevogea\Downloads\Get-FolderEntry.ps1
# list with the servers
$Computers = Get-Content C:\Users\mdevogea\Desktop\servers.txt
# scriptblock calling on get-FolderEntry
$sb = {
param ($Computer, $fname)
C:\Users\mdevogea\Downloads\Get-FolderEntry.ps1 -Path $Computer |
fl | Out-File -Append -Width 1000 -FilePath $fname
}
foreach($Computer in $Computers)
{
$name = $Computer.Replace("\", "")
$fname = $("C:\Users\mdevogea\Desktop\" + $name + ".txt")
#Get-FolderEntry -Path $Computer | fl | Out-File -Append -Width 1000 $fname
$res = Start-Job $sb -ArgumentList $Computer, $fname
}
# Wait for all jobs
Get-Job
while(Get-Job -State "Running")
{
Write-Host "Running..."
Start-Sleep 2
}
# Get all job results
Get-Job | Receive-Job | Out-GridView
So far:
I either get empty files with the correct naming of the file.
I get the correct named file with the code of Get-FolderEntry in it.
I get errors depend on what I pass along to the scriptblock.
In short, it's probably stupid but don't see it.
Found it eventually myself after some trial and error:
. C:\Users\mdevogea\Downloads\Get-FolderEntry.ps1
# list with the servers
$Computers = Get-Content C:\Users\mdevogea\Desktop\servers.txt
# scriptblock calling on get-FolderEntry
$sb = {
Param ($Computer, $fname)
. C:\Users\mdevogea\Downloads\Get-FolderEntry.ps1
(Get-FolderEntry -Path $Computer | fl | Out-File -Append -Width 1000 -FilePath $fname)
}
foreach ($Computer in $Computers)
{
$name = $Computer.Replace("\", "")
$fname = $("C:\Users\mdevogea\Desktop\" + $name + ".txt")
$res = Start-Job $sb -ArgumentList $Computer, $fname
}
# Wait for all jobs
Get-Job
while (Get-Job -State "Running")
{
Write-Host "Running..."
Start-Sleep 2
}
# Get all job results
Get-Job | Receive-Job | Out-GridView
Thanks a lot Ansgar for pointing my in the right direction!
More of a theory question...
I have a powershell script that exists on three servers. In this example the three servers are:
server1
server2
server3
I am using another machine, server4, to call script C:\ExampleScript.ps1 remotely using Invoke-Command while specifying the remote machine via the ComputerName parameter. The ultimate goal of the script is to detect whether powershell is running, if it is not, then the computer is "not busy" and can open up the script being called remotely. If the computer is "busy", move onto the next server and continue on through the three machines until all the parameter values have been exhausted. If all machines are busy, it would be ideal if there was a way to periodically check the processes and see if they are still open. In this way, execution of the script can be balanced across the various machines, in an albeit primitive fashion.
Consider the following code:
$servers = "server1","server2","server3"
$data = "param1", "param2", "param3", "param4", "param5", "param6"
#somehow loop through the different servers/data using the above arrays
$job = Invoke-Command $servers[0] {
$ProcessActive = Get-Process powershell -ErrorAction SilentlyContinue
if($ProcessActive -eq $null)
{
"Running"
Invoke-Command -ComputerName $env:computername -FilePath C:\ExampleScript.ps1 -ArgumentList $data[0]
}
else
{
"Busy go to next machine"
}
} -AsJob
Wait-Job $job
$r = Receive-Job $job
$r
The expected result trying to be achieved is attempting to load balance the script across the machines based on whether there is an active powershell process, if not move onto the next machine and perform the same test and subsequent possible execution. The script should go through all the values as specified in the $data array (or whatever).
I found this question interesting, so I wanted to give it a try.
$servers = "server1","server2","server3"
$data = New-Object System.Collections.ArrayList
$data.AddRange(#("param1", "param2", "param3", "param4", "param5", "param6"))
$jobs = New-Object System.Collections.ArrayList
do
{
Write-Host "Checking job states." -ForegroundColor Yellow
$toremove = #()
foreach ($job in $jobs)
{
if ($job.State -ne "Running")
{
$result = Receive-Job $job
if ($result -ne "ScriptRan")
{
Write-Host " Adding data back to que >> $($job.InData)" -ForegroundColor Green
$data.Add($job.InData) | Out-Null
}
$toremove += $job
}
}
Write-Host "Removing completed/failed jobs" -ForegroundColor Yellow
foreach ($job in $toremove)
{
Write-Host " Removing job >> $($job.Location)" -ForegroundColor Green
$jobs.Remove($job) | Out-Null
}
# Check if there is room to start another job
if ($jobs.Count -lt $servers.Count -and $data.Count -gt 0)
{
Write-Host "Checking servers if they can start a new job." -ForegroundColor Yellow
foreach ($server in $servers)
{
$job = $jobs | ? Location -eq $server
if ($job -eq $null)
{
Write-Host " Adding job for $server >> $($data[0])" -ForegroundColor Green
# No active job was found for the server, so add new job
$job = Invoke-Command $server -ScriptBlock {
param($data, $hostname)
$ProcessActive = Get-Process powershell -ErrorAction SilentlyContinue
if($ProcessActive -eq $null)
{
# This will block the thread on the server, so the JobState will not change till it's done or fails.
Invoke-Command -ComputerName $hostname -FilePath C:\ExampleScript.ps1 -ArgumentList $data
Write-Output "ScriptRan"
}
} -ArgumentList $data[0], $env:computername -AsJob
$job | Add-Member -MemberType NoteProperty -Name InData -Value $data[0]
$jobs.Add($job) | Out-Null
$data.Remove($data[0])
}
}
}
# Just a manual check of $jobs
Write-Output $jobs
# Wait a bit before checking again
Start-Sleep -Seconds 10
} while ($data.Count -gt 0)
Basically I create an array, and keep it constantly populated with one job for each server.
Data is removed from the list when a new job starts, and is added back if a job fails. This is to avoid servers running the script with the same data/params.
I lack a proper environment to test this properly at the moment, but will give it a whirl at work tomorrow and update my answer with any changes if needed.
I am trying to be a good a powerscript user and use Write-Verbose as per best practices, but I have no way to get the Verbose stream from a running Job.
$Job = Start-Job -Name "Scanning Work Item" -ScriptBlock{
Write-Verbose "Write-Verbose"
Write-Host "Write-Host"
}
while ($Job.HasMoreData -or $Job.State -eq "Running") {
Receive-Job -Job $Job -Verbose
Start-Sleep -Seconds 1
}
The output for this is
Write-Host
Please only answer with tested code as I have spent hours trying various permutations of Powershell script.
First of all, you're not getting any verbose ouput because you haven't changed the default VerbosePreference for the session.
As for reading Verbose ouput while the job is running, you can read each of the output stream buffers from the associated child job individually, without doing a Receive-job, and without affecting later output when you do the Receive-Job,
$Job = Start-Job -Name "Scanning Work Item" -ScriptBlock{
$VerbosePreference = 'Continue'
Write-Verbose "Write-Verbose"
Write-Host "Write-Host"
Start-Sleep -Seconds 10
}
Start-sleep -Seconds 2
$Verbose = $Job.ChildJobs[0].verbose.readall()
$verbose
while ($Job.HasMoreData -or $Job.State -eq "Running") {
Receive-Job -Job $Job
Start-Sleep -Seconds 1
}
Write-Verbose
VERBOSE: Write-Verbose
Write-Host
I wrote a little powershell function that executes Get-EventLog against remote servers. On some servers this seems to just hang and never times out. Can I timeout a powershell function call? I see how to do this against a different process, but i want to do this for a power shell function.
thanks
#######################
function Get-Alert4
{
param($computer)
$ret = Get-EventLog application -after (get-date).addHours(-2) -computer $computer | select-string -inputobject{$_.message} -pattern "Some Error String" | select-object List
return $ret
} #
You can implement timeouts by using a background job like so:
function Get-Alert4($computer, $timeout = 30)
{
$time = (Get-Date).AddHours(-2)
$job = Start-Job { param($c) Get-EventLog Application -CN $c -After $time |
Select-String "Some err string" -inputobject{$_.message} |
Select-Object List } -ArgumentList $computer
Wait-Job $job -Timeout $timeout
Stop-Job $job
Receive-Job $job
Remove-Job $job
}
FYI - your argumentlist is only good for one parameter.
If you want to pass more than one argument to the job, you have to pass them as an array:
$job = Start-Job { param($c, $t) Get-EventLog Application -CN $c -After $t |
Select-String "Some err string" -inputobject{$_.message} |
Select-Object List } -ArgumentList #($computer, $time)
This is a one liner (due to semi-colons) that will display a count down while pausing similar (but not the same) as the timeout cmd command
$NumOfSecs = 15; $n = $NumOfSecs; write-host "Starting in " -NoNewLine; do {if($n -lt $NumOfSecs) {write-host ", " -NoNewLine}; write-host $n -NoNewLine; $n = $n - 1; Start-Sleep 1} while ($n -gt 0)