I am having to run a Microsoft cmdlet, and the important bit of information is written to console using a Write-Host line within the cmdlet.
It is NOT returned, so I cannot do $result = Commandlet ...
A different value is returned that is not of use to me, what I actually need is printed to console within the commandlet is there anyway I can 'sniff' or 'scrape' the console to get the information I want?
$result = Test-Cluser
Test-Cluster will print stuff like: 'HadUnselectedTests', 'ClusterConditionallyApproved', etc.
But the value it returns in the path to the .htm report file.
And the .htm report file does not contain one of those status codes unfortunately so I cannot just parse the .htm file for it either.
Any suggestions?
Note: As for why you should never use Write-Host to output data, see this answer.
In PSv5+:
$result = Test-Cluster 6>&1
Since version 5, Write-Host writes to the newly introduced information stream, whose number is 6.
6>&1 redirects that stream to the success output stream (number 1), so that it too can be captured in $result.
Caveat: The related Out-Host cmdlet does not write to the information stream; its output cannot be captured - see this answer for the differences between Write-Host and Out-Host.
In PSv4-:
There is no way to capture Write-Host output in-session.
The only workaround is to launch another instance of Powershell with the target command specified as a string.
Caveats:
Such an invocation is slow,
prevents passing of arguments with their original data type
invariably only returns string data (lines of text)
returns output from all output streams, including error output
for a list of all output streams, see Get-Help about_Redirection
$result = powershell -noprofile -command 'Test-Cluster'
Note that using a script block to pass the command (-command { Test-Cluster }) would not work, because PowerShell then uses serialization and deserialization to emulate the in-session behavior.
Optional reading: output streams in PowerShell and how to redirect them:
Get-Help about_Redirection discusses a list of all output streams, which can be targeted by their numbers; since PSv5, these are:
1 ... success output stream (implicit output and Write-Output output)
2 ... error output stream (Write-Error and unhandled errors)
3 ... warnings (Write-Warning)
4 ... verbose output (Write-Verbose)
5 ... debug output (Write-Debug)
6 ... (v5+) Write-Information and Write-Host output
Note that some streams are silent by default and require opt-in to produce output, either via a preference variable (e.g., $VerbosePreference) or a common parameter (e.g., -Verbose)
{n}> allows redirecting the number {n} stream; if {n} is omitted, 1 is implied:
to a file (e.g., 3> c:/tmp/warnings.txt
to "nowhere", i.e suppressing the output (e.g., 3> $null)
to the success output stream (e.g., 3>&1); note: only stream 1 can be targeted this way.
*> targets all output streams.
Note: Unlike in POSIX-like shells (e.g., bash), the order of multiple redirection expression does not matter.
Therefore, the following POSIX-like shell idiom - which redirects error output to the success stream and silences only the original success output - does NOT work:
... 2>&1 1>$null # !! NO output in PowerShell
To achieve this in PowerShell, you mustn't redirect 1 and instead filter the objects in the success by their stream of origin.
Case in point: In the end, the OP wanted the following: capture only warning output, without the regular (success) output:
Test-Cluster 3>&1 | Where-Object { $_ -is [System.Management.Automation.WarningRecord] }
Objects that came from the warning stream have type [System.Management.Automation.WarningRecord], which is what enables the filtering above.
I use *> instead of > to redirect all outputs from console to a file.
Example of redirecting text and a variable to a file:
"The "+$set_groups+"ADGroup set!" | Out-File -FilePath $log_path -Append
Related
We have a bunch of different powershell tools that we use for build/deploy and other development and admin activity in my team. Mostly these are calling other Powershell scripts and Cmdlets but there are some x86 command line apps also (e.g. msbuild)
They largely all are setup to output verbose output. I do that so that I can troubleshoot retrospectively when something goes wrong in the team
However the team have asked to have less noisy output to the console. I still want the verbose output to be available retrospectively
So it feels like I need something like a continuous loop of the last 100k rows of verbose activity. Including any console input and output written to a file - regardless of the -verbose setting that the developer applied
Is there anything like this available in Powershell?
I know about redirection but I'm not sure how it can solve this problem as it seems that you have to match the stdout with the redirect even if you use Tee-Object - also it wouldn't capture inputs.
Eager to learn some hidden secrets or elegant creative solutions! :)
UPDATE: as mentioned redirect is not a practical solution. I've created a request https://github.com/PowerShell/PowerShell/issues/17482
An article that addresses this topic is about_Redirection. Each output stream has a numeric identifier - the range is 1 to 6. The referenced article has a table showing the mapping between the ID's and streams.
Write-Host's ID is 6, so if you wanted everything to go to a log file except that, you can try redirecting all streams except 6:
PS:> Start-VeryNoiseyOperation -Debug 1>C:\mylogs\noisey.log 2>&1 3>&1 4>&1 5>&1
This example is redirecting all streams generated by the Write-____ cmdlets, except Write-Host and Write-Information. To send all streams to the log file, you can use *>C:\mylogs\noisey.log.
If you want to experiment, you can run this function with different combos of redirection to see the effect.
PS:> function Foo {
[CmdletBinding()]param()
Write-Output #{message = "my printed object"} # 1
Write-Error "This is an error message." # 2
Write-Warning "This is a warning message." # 3
Write-Verbose "This is a verbose message." # 4
Write-Debug "This is a debug message." # 5
Write-Host "This is a host message." # 6
}
PS:> $log = New-TemporaryFile
PS:> $logPath = $log.FullName
PS:>
PS:> Foo -Debug -Verbose # Print everything.
:
PS:> Foo -Debug -Verbose 1>$logPath # Send object stream to file.
:
PS:> Get-Content $logPath # Object should print to console.
:
PS:> Foo -Debug -Verbose 1>$logPath 2>&1 # Try different combos.
: # see effect on console output #
PS:> Get-Content $logPath
: # see log content #
This works perfectly on Windows 10 using powershell but fails badly on Win Server 2012 R2.
I have tried many forms using redirect and -outfile but at best end-up with an empty file or broken script.
Can any one help me send the one liner below to a txt file on win server 2012 R2, I'm just not getting it
$((Get-WmiObject -Query "select Name, PercentProcessorTime from Win32_PerfFormattedData_PerfOS_Processor") | foreach-object { write-host "$($_.Name) : $($_.PercentProcessorTime)"}) *>&1 > output.txt
Write-Host, as the name implies, writes to the host, which in a console window is that window's display (screen), bypassing PowerShell's (success) output stream, the latter being what the pipeline operator (|) and the redirection operator > operate on.
In PowerShell v5+ only, Write-Host writes to the information output stream (stream number 6 - see the about_Redirection help topic; by default, that output still goes to the host) and can therefore be redirected - either via 6> or via *> - so the catch-all redirection *>&1, which redirects all streams to the success output stream (1), can indeed be used to redirect Write-Host to the success output stream, but not in earlier PowerShell versions - and Windows Server 2012 R2 shipped with PowerShell version 4.
However, in your case there is no good reason to use Write-Host to begin with: either use Write-Output - the cmdlet whose purpose is to write to the success output stream (1) - or, preferably, use PowerShell's implicit output feature, where any output (return value) not captured in a variable, piped or redirected is implicitly written to the success output stream:
# Note how the use of "$($_.Name) : $($_.PercentProcessorTime)"
# *by itself* implicitly causes it to be *output* (written to the pipeline).
Get-WmiObject -Query "select Name, PercentProcessorTime from Win32_PerfFormattedData_PerfOS_Processor" |
Foreach-Object { "$($_.Name) : $($_.PercentProcessorTime)" } > output.txt
Note the absence of $(...), the subexpression operator in the command, which isn't needed.
If the specific spacing between the columns isn't important, you can more simply write (since only 2 properties are being select, implicit Format-Table formatting is applied):
Get-WmiObject -Query "select Name, PercentProcessorTime from Win32_PerfFormattedData_PerfOS_Processor" |
Select-Object Name, PercentProcessorTime > output.txt
Note: The CIM cmdlets (e.g., Get-CimInstance) superseded the WMI cmdlets (e.g., Get-WmiObject) in PowerShell v3 (released in September 2012). Therefore, the WMI cmdlets should be avoided, not least because PowerShell [Core] (version 6 and above), where all future effort will go, doesn't even have them anymore. For more information, see this answer.
Try this... note that it is appending to the file, so if you run it twice, you will have two runs' worth of data
$((Get-WmiObject -Query "select Name, PercentProcessorTime from Win32_PerfFormattedData_PerfOS_Processor") | foreach-object { "$($_.Name) : $($_.PercentProcessorTime)" >> output.txt})
EDIT
OP accepted this as the answer, but it's not necessarily the best solution. For a thorough explanation and another solution, I recommend a different answer submitted for this question.
I am trying to get the job details without outputting the data to the screen. However, regardless of what option I try, the job logs always get sent to the console. Any ideas on how to save the logs in a variable or file without outputting that data to console?
Receive-Job -Id $id -Keep -ErrorAction Continue > C:\Temp\Transcript-$VM.txt
$info = Receive-Job -Id $id -Keep -ErrorAction Continue
You state that your job uses Write-Host output and that you're running Windows PowerShell v5.1.
In order to also capture Write-Host output - which in v5+ is sent to the information stream (stream number 6) - use redirection 6>&1:
# Capture both success output and information-stream output
# (Write-Host) output in $info.
$info = Receive-Job -Id $id -Keep -ErrorAction Continue 6>&1
Unfortunately, due to a known bug, you'll still get console output as well (bug is still present in PowerShell Core 7.0.0-preview.5).
Catch-all redirection *>&1 normally routes all streams through the success output stream.
Unfortunately, due to the bug linked to above, the following streams cannot be captured or redirected at all when using background jobs or remoting:
verbose messages (4)
debug messages (5)
The only workaround is to capture the streams inside the job and save them to a file from there, and then access the files from the caller later.
Of course, this requires that you have control over how the jobs are created.
A simplified example:
# Redirect all output streams *inside* the job to a file...
Start-Job {
& {
# The job's commands go here.
# Note that for any *verbose* output to be captured,
# verbose output must explicitly turned on, such as with
# the -Verbose common parameter here.
# You can also set $VerbosePreference = 'Continue', which
# cmdlets (including advanced functions/scripts) will honor.
'success'; write-verbose -Verbose 'verbose'; write-host 'host'
} *> $HOME/out.txt
} | Receive-Job -Wait -AutoRemove
# ... then read the resulting file.
Get-Content $HOME/out.txt
Note that I've used a full path as the redirection target, because, unfortunately, in v6- versions of PowerShell script blocks executed in background jobs do not inherit the caller's current location. This will change in PowerShell Core v7.0.
Try placing it in a pipeline, and see if that works:
Receive-Job -Id $id -Keep -ErrorAction Continue | Set-Content 'C:\Temp\Transcript-$VM.txt'
Please have a look at this test script and the conclusions I've made about how 'Receive-Job' works in detail.
I have still issues to figure out, how exaclty 'Receive-Job' pulls the streams from the code block.
<# .SYNOPSIS Test the console output and variable capturing of Write- cmdlet calls in a code block used by 'Start-Job'
.NOTES
.NET Version 4.7.2
PSVersion 5.1.16299.431
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.16299.431
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
#>
Set-StrictMode -Version latest
if ($host.Name -inotmatch 'consolehost') { Clear-Host }
$errorBuffer = $null
$warningBuffer = $null
$outBuffer = $null
$infoBuffer = $null
# Start the job
$job = Start-Job -ScriptBlock {
Set-StrictMode -Version latest
PowerShell starts this script block in its own process, like it would start an external executable.
Therfore PowerShell can only map stdout/success and stderr/error from the codeblock to the PowerShell's success (1) and error (2) streams in the script's process.
Those two streams will be passed by Receive-Job and can be redirected in the Receive-Job line as expected.
And those two streams can be stored into variables by Receive-Job on request. (-OutVariable -ErrorVariable)
Additionally, Receive-Job can caputure the PowerShell streams info (stream 6) and warning (stream 3) and can store them in variables, too. (-WarningVariable -InformationVariable)
But storing those streams in the variables is no redirection.
Every call of a Write- cmdlet can display a message on the console, independent to the -variable swiches.
A visible message on the console depends only on the Write- cmdlet's own preferences and possible redirection in the Write- cmdlet call.
# This will, by default, output to the console over stream 6 (info), and always get captured in $infoBuffer.
Write-Host "***WRITE_HOST***" # 6> $null # Supresses the output to the console.
# This will not output to the console over stream 6 (info) by default, but always get captured in $infoBuffer.
$InformationPreference = 'Continue' # Outputs to the console, default is 'SilentlyContinue'.
Write-Information "***INFO***" # 6> $null # Supresses the output to the console for preference 'Continue'.
$InformationPreference = "SilentlyContinue"
# This will not output to the console over stream 5 (debug) by default, and can't get captured in a variable.
$DebugPreference = 'Continue' # Outputs to the console, default is 'SilentlyContinue'.
Write-Debug "***DEBUG***" # 5> $null # Supresses the output to the console for preference 'Continue'.
$DebugPreference = "SilentlyContinue"
# This will not output to the console over stream 4 (verbose), by default, and can't get captured in a variable.
$VerbosePreference = 'Continue' # Outputs to the console, default is 'SilentlyContinue'.
Write-Verbose "***Verbose***" # 4> $null # Supresses the output to the console for preference 'Continue'.
$VerbosePreference = 'SilentlyContinue'
# This will, by default, output to the console over stream 3 (warning), but get captured in $warningBuffer only for
# preference 'Continue'.
#$WarningPreference = 'SilentlyContinue' # Supresses console output AND variable capturing, default is 'Continue'.
Write-Warning "***WARNING***" # 3> $null # Supresses the warning output to the console for preference
#$WarningPreference = 'Continue' # 'Continue'.
# This will output to the console over stream 2 (error), and always get captured in $errorBuffer, if not redirected
# in the code block.
# For 'Receive-Job -ErrorAction Stop' it would raise an execption, the content in $errorBuffer is quite useless then.
Write-Error '***ERROR***' # 2> $null # Supresses the output AND variable capturing, but you can supress/redirect
# this stream in the 'Receive-Job' line without breaking the variable
# capturing: 'Receive-Job ... -ErrorVariable errorBuffer 2> $null'
# These will output to the console over stream 1 (success), and always get captured in $result and $outBuffer, if
# not redirected in the code block.
Write-Output '***OUTPUT***' # 1> $null # Supresses the output AND variable capturing, but you can supress/redirect
Write-Output '***NEXT_OUTPUT***' # this stream in the 'Receive-Job' line without breaking the variable
"***DIRECT_OUT***" # capturing: '$result = Receive-Job ... -OutVariable outBuffer 1> $null'
}
# Wait for the job to finish
Wait-Job -Job $job
try
{
# Working only outside the code block, this is a workaround for catching ALL output.
#$oldOut = [Console]::Out
#$stringWriter = New-Object IO.StringWriter
#[Console]::SetOut($stringWriter)
# Pull the buffers from the code block
$result = Receive-Job <#-ErrorAction Stop#> `
-Job $job `
-ErrorVariable errorBuffer `
-WarningVariable warningBuffer `
-OutVariable outBuffer `
-InformationVariable infoBuffer `
# 1> $null #2> $null # Only the success and error streams can be redirected here, other
# streams are not available.
# Restore the console
#[Console]::SetOut($oldOut)
# Get all catched output
#$outputOfAllWriteFunctions = $stringWriter.ToString()
}
catch
{
Write-Host "EXCEPTION: $_" -ForegroundColor Red
}
finally
{
Write-Host "error: $errorBuffer"
Write-Host "warning: $warningBuffer"
Write-Host "out: $outBuffer"
Write-Host "info: $infoBuffer"
Write-Host "result: $result"
#Write-Host "`noutputOfAllWriteFunctions:`n";Write-Host "$outputOfAllWriteFunctions" -ForegroundColor Cyan
Remove-Job -Job $job
}
My final conclusions:
Because the code block of Start-Job runs in its own process, it can't write to the scripts process console directly.
The code block is wrapped by a capture mechanism, which captures all 6 PS streams in buffers.
A call of Receive-Job uses inter process communication to get all those streams.
Receive-Job passes through stream 1 and 2 and makes them to its own output and therefore avaiable for redirection.
Receive-Job uses Write-Error to write stream 2 to the console, and therfore Receive-Job will raise an exception for parameter -ErrorAction Stop.
Then Write-Error uses Console.Out.WriteLine() to write to the console in red.
Then Receive-Job checks for variable storing and stores stream 1 (success), 2 (error), 3 (warning) and 6 (info).
Finally Receive-Job uses Console.Out.WriteLine() to write stream 1, 3, 4, 5 and 6 with different ForegroundColors to the console.
That's why you can capture ALL those 6 stream outputs with Console.SetOut(), even the error stream output, for which I had expected Console.SetError() would be needed.
But there is an issue in those conclusions:
The output of Write-Host is written to the console by default and its output is added to the information variable.
So Write-Host maybe just write into stream 6.
But the output of Write-Information is not visible on the console by default, but is also added to the information variable.
So Write-Information can't just share the same IPC pipe with Write-Host.
And Write-Warning can write to the console and the variable independently, so only one stream/pipe couldn't be used here, too.
Have a look at my diagram for that issue.
Receive-Job output transport diagram:
You can verify the diagram by redirecting stream 1-6 in the code block and stream 1 or 2 in the script.
|<-------- code block process -------->|<-- IPC -->|<-------------------- script process ------------------->|
Method Preference Stream Stream/Variable Console output
Write-Out * --> 1 --> PIPE 1 --> 1 --> Console.Out.Write(gray)
PIPE 1 --> Out Variable
Write-Error * --> 2 --> PIPE 2 --> 2 --> Console.Out.Write(red)
PIPE 2 --> Error Variable
Write-Warning Continue ----??????---> PIPE 3 --> Warning Variable
Write-Warning Continue --> 3 --> PIPE 4 --> Console.Out.Write(yellow)
Write-Verbose Continue --> 4 --> PIPE 4 --> Console.Out.Write(yellow)
Write-Debug Continue --> 5 --> PIPE 4 --> Console.Out.Write(yellow)
Write-Information Continue --> 6 --> PIPE 6 --> Console.Out.Write(gray)
Write-Information * ----??????---> PIPE 5 --> Information Variable
Write-Host * ----??????---> PIPE 5 --> Information Variable
Write-Host * --> 6 --> PIPE 6 --> Console.Out.Write(gray)
IPC : Inter Process Communication
* : always, independent from Preference or has no own Preference
There is no redirection you can add after Write-Information or Write-Warning to prevent storing in their variables.
If you'd redirect 3 and 6 after the methods, then it would only affect the console output, not the variable storing.
Only when $InformationPreference (not default) or $WarningPreference (default) are set to Continue, they write into stream 6 or 3, whose are always written in gray or yellow color to the console of the script process.
And only Write-Warning needs preference Continue to store in its variable, Write-Informations always writes to its variable.
Question:
How can 'Write-Warning' and 'Write-Information' pass their output to their assigned variables in the script process ?
(They can't use stream 7,8,9, since they don't exists in windows.)
Best practice:
After the call of Job-Start you should Start-Sleep 1-3 seconds to give the code block time to start or fail.
Then use Receive-Job the first time to get the current progress, start debug info, warning or errors.
You should not use Wait-Job, but use your own loop to check for the job's running state and check a timeout by yourself.
In that own wait loop, you call Receive-Job every X seconds to get progress, debug and error information from the code block process.
When the job's state is finished or failed, you call Receive-Job a last time to get the remaining data of all the buffers.
To redirect/capture stream 1 (success) and 2 (error) you can use normal redirection in the Receive-Job line or storing to the variables.
To capture stream 3 (warning) and 6 (info & Write-Host) you have to use the variable storing.
You can't redirect or capture stream 4 (verbose) or 5 (debug) directly, but you could redirect (4>&1 or 5>&1) those streams in the code block to stream 1 (success) to add them to the output variable.
To supress console output of Write-Output or Write-Error, you can just redirect stream 1 or 2 in the Receive-Job line.
You don't have to supress console output of Write-Information, Write-Verbose or Write-Debug, since they don't write to console with their default preferences.
If you want to capture the output of Write-Information in the assigned variable without console output, you have to redirect stream 6: Write-Information <message> 6>$null.
To supress console output of Write-Warning or Write-Host, you have to redirect stream 3 or 6 in their call lines: Write-Warning <message> 3>$null and Write-Host <message> 6>$null.
Be aware:
If you redirect stream success (1) or error (2) in the code block, they will not be tranfered to the script process, not written to the console and not be stored in the output or error variable.
You are a bit hard to follow with your terminology use but I will do my best with my limited experience.
The output of Write-Host is written to the console by default and its output is added to the information variable.
So Write-Host maybe just write into stream 6.
But the output of Write-Information is not visible on the console by default, but is also added to the information variable.
So Write-Information can't just share the same IPC pipe with Write-Host.
First of all, I read somewhere (do not remember so cannot link, sorry) and confirmed for myself that Write-Host and Write-Information do, in fact, use the same stream. However, Write-Host is, essentially, a special case of Write-Information whereby it ignores the preference variable and always writes. So I would expect Write-Information to show up in its respective variable when the respective preference variable is set properly.
And Write-Warning can write to the console and the variable independently, so only one stream/pipe couldn't be used here, too.
This observation is likely a design choice. (I am guessing here) I expect it works similar to the Tee-Object cmdlet so it can, indeed, write to the console and variable despite only being one stream.
$result = 'some string' | Tee-Object -Variable var
Write-Host $result
Write-Host $var
# same string in both variables
I have a script that pass over files and I want to redirect any errors to a variable and print them in the end of the script.
Here is example for a problem I am getting:
Get-ChildItem C:\Windows\appcompat -Recurse | ForEach-Object {
# do stuff ...
}
Sometimes there are folders that I don't have access to them and it throws exception:
I know how to ignore these errors and continue using the switch -ErrorAction, but I wanted to collects all the folders that I don't have access and print them in the end of the script.
With redirection it is possible to use 2> which will redirect the errors to a file:
Get-ChildItem C:\Windows\appcompat -Recurse 2> errors.txt | ForEach-Object {
# do stuff ...
}
Is it possible to redirect, only the errors, to a variable and then I will print them in the end of the script ?
You can use the common -ErrorVariable parameter to collect a cmdlet's errors in a variable.
Since this collecting happens in addition to errors getting sent to the error stream, as usual, you must explicitly silence the error-stream output with 2>$null if you want the collecting to be silent.
Therefore, in order to silently collect errors in, say, variable $errs, use the following:
# Shorter equivalent of `-ErrorVariable errs`: `-ev errs`
Get-ChildItem C:\Windows\appcompat -Recurse -ErrorVariable errs 2>$null | ForEach-Object {
# do stuff ...
}
Note:
The short alias name for -ErrorVariable is -ev
Be sure not to prefix the target variable name with $ - you're passing its name, not its value.
Do not use the name Errors, because $Errors is the automatic (built-in) variable in which all errors that have occurred in the session are being collected.
The target variable receives a collection of type [System.Collections.ArrayList] containing [System.Management.Automation.ErrorRecord] instances.
Note you get a collection even in the case of only a single error having occurred, which may be surprising, given that PowerShell usually unwraps single-element collections; this surprising behavior is discussed in this GitHub issue.
Unfortunately, this convenient error-collecting mechanism is not available when calling external programs (e.g., when you call git), because such calls do not support common parameters; however, there is a suggestion on GitHub to introduce an alternative syntax.