We have a bunch of different powershell tools that we use for build/deploy and other development and admin activity in my team. Mostly these are calling other Powershell scripts and Cmdlets but there are some x86 command line apps also (e.g. msbuild)
They largely all are setup to output verbose output. I do that so that I can troubleshoot retrospectively when something goes wrong in the team
However the team have asked to have less noisy output to the console. I still want the verbose output to be available retrospectively
So it feels like I need something like a continuous loop of the last 100k rows of verbose activity. Including any console input and output written to a file - regardless of the -verbose setting that the developer applied
Is there anything like this available in Powershell?
I know about redirection but I'm not sure how it can solve this problem as it seems that you have to match the stdout with the redirect even if you use Tee-Object - also it wouldn't capture inputs.
Eager to learn some hidden secrets or elegant creative solutions! :)
UPDATE: as mentioned redirect is not a practical solution. I've created a request https://github.com/PowerShell/PowerShell/issues/17482
An article that addresses this topic is about_Redirection. Each output stream has a numeric identifier - the range is 1 to 6. The referenced article has a table showing the mapping between the ID's and streams.
Write-Host's ID is 6, so if you wanted everything to go to a log file except that, you can try redirecting all streams except 6:
PS:> Start-VeryNoiseyOperation -Debug 1>C:\mylogs\noisey.log 2>&1 3>&1 4>&1 5>&1
This example is redirecting all streams generated by the Write-____ cmdlets, except Write-Host and Write-Information. To send all streams to the log file, you can use *>C:\mylogs\noisey.log.
If you want to experiment, you can run this function with different combos of redirection to see the effect.
PS:> function Foo {
[CmdletBinding()]param()
Write-Output #{message = "my printed object"} # 1
Write-Error "This is an error message." # 2
Write-Warning "This is a warning message." # 3
Write-Verbose "This is a verbose message." # 4
Write-Debug "This is a debug message." # 5
Write-Host "This is a host message." # 6
}
PS:> $log = New-TemporaryFile
PS:> $logPath = $log.FullName
PS:>
PS:> Foo -Debug -Verbose # Print everything.
:
PS:> Foo -Debug -Verbose 1>$logPath # Send object stream to file.
:
PS:> Get-Content $logPath # Object should print to console.
:
PS:> Foo -Debug -Verbose 1>$logPath 2>&1 # Try different combos.
: # see effect on console output #
PS:> Get-Content $logPath
: # see log content #
This one is confusing me and wondered if anyone could shed some light on this, maybe also answer 2 parts with this one example?
I have a Powershell script I run like a batch file and I have it output values it has captured from a separate file out to a log. This particular part is not outputting in the transcript where I get the DB version of a database. I have tried different placements of the ", using $DBVersion on it's own and this is a simple way to show what I have trouble with. e.g.:
## Read the DBs Extended properties
Function Get-DB_Version {
Param (
$SQLInstance,
$DBName
)
Invoke-Sqlcmd -ServerInstance $SQLInstance -Database $DBName -Query "SELECT value FROM fn_listextendedproperty(default, default, default, default, default, default, default) WHERE name = 'version'"
}
**Other Variables used are also pre-set above here **
## Get DB version
$DBVersion = Get-DB_Version -SQLInstance $SQLInstance -DBName $DBName
Start-Transcript -Path "$LogOutput\$LogName" -IncludeInvocationHeader -Append
Write-Information "
**** DEBUG INFO ****
Write-Host "Debug:"$DBVersion.value
Write-Information "Debug:"$DBVersion.value
Read-Host -Prompt "PAUSE" # This is so I can visually see the console seeing only the write-host is there as expected.
Stop-Transcript
In my log file I get the output:
Debug: 2.16.51443.5147
INFO: Debug:
This shows me that the variable contains a value as the write-host outputs it, however when use Write-Information it does not show anything in the log, All other variables I use do show, why would $DBVersion.value or $DBVersion not show anything please?
Also the second part is, why do I have to use:
$DBVersion.value
Outside of the write-host "" quotes?
Many thank in Advance
As #abraham said in the comments. All I had to do, to have the variable inside of the quotes (my question 2) was use the sub-expression operator $() to expand the value inside the quotes: Write-Host "Debug: $($DBVersion.value)". The same goes for your Write-Information.
Doing this alone also resolved my original question of why Write-Information didn't output anything into the transaction logs and I did NOT need to change the $InformationPreference.
I am trying to get the job details without outputting the data to the screen. However, regardless of what option I try, the job logs always get sent to the console. Any ideas on how to save the logs in a variable or file without outputting that data to console?
Receive-Job -Id $id -Keep -ErrorAction Continue > C:\Temp\Transcript-$VM.txt
$info = Receive-Job -Id $id -Keep -ErrorAction Continue
You state that your job uses Write-Host output and that you're running Windows PowerShell v5.1.
In order to also capture Write-Host output - which in v5+ is sent to the information stream (stream number 6) - use redirection 6>&1:
# Capture both success output and information-stream output
# (Write-Host) output in $info.
$info = Receive-Job -Id $id -Keep -ErrorAction Continue 6>&1
Unfortunately, due to a known bug, you'll still get console output as well (bug is still present in PowerShell Core 7.0.0-preview.5).
Catch-all redirection *>&1 normally routes all streams through the success output stream.
Unfortunately, due to the bug linked to above, the following streams cannot be captured or redirected at all when using background jobs or remoting:
verbose messages (4)
debug messages (5)
The only workaround is to capture the streams inside the job and save them to a file from there, and then access the files from the caller later.
Of course, this requires that you have control over how the jobs are created.
A simplified example:
# Redirect all output streams *inside* the job to a file...
Start-Job {
& {
# The job's commands go here.
# Note that for any *verbose* output to be captured,
# verbose output must explicitly turned on, such as with
# the -Verbose common parameter here.
# You can also set $VerbosePreference = 'Continue', which
# cmdlets (including advanced functions/scripts) will honor.
'success'; write-verbose -Verbose 'verbose'; write-host 'host'
} *> $HOME/out.txt
} | Receive-Job -Wait -AutoRemove
# ... then read the resulting file.
Get-Content $HOME/out.txt
Note that I've used a full path as the redirection target, because, unfortunately, in v6- versions of PowerShell script blocks executed in background jobs do not inherit the caller's current location. This will change in PowerShell Core v7.0.
Try placing it in a pipeline, and see if that works:
Receive-Job -Id $id -Keep -ErrorAction Continue | Set-Content 'C:\Temp\Transcript-$VM.txt'
I am having to run a Microsoft cmdlet, and the important bit of information is written to console using a Write-Host line within the cmdlet.
It is NOT returned, so I cannot do $result = Commandlet ...
A different value is returned that is not of use to me, what I actually need is printed to console within the commandlet is there anyway I can 'sniff' or 'scrape' the console to get the information I want?
$result = Test-Cluser
Test-Cluster will print stuff like: 'HadUnselectedTests', 'ClusterConditionallyApproved', etc.
But the value it returns in the path to the .htm report file.
And the .htm report file does not contain one of those status codes unfortunately so I cannot just parse the .htm file for it either.
Any suggestions?
Note: As for why you should never use Write-Host to output data, see this answer.
In PSv5+:
$result = Test-Cluster 6>&1
Since version 5, Write-Host writes to the newly introduced information stream, whose number is 6.
6>&1 redirects that stream to the success output stream (number 1), so that it too can be captured in $result.
Caveat: The related Out-Host cmdlet does not write to the information stream; its output cannot be captured - see this answer for the differences between Write-Host and Out-Host.
In PSv4-:
There is no way to capture Write-Host output in-session.
The only workaround is to launch another instance of Powershell with the target command specified as a string.
Caveats:
Such an invocation is slow,
prevents passing of arguments with their original data type
invariably only returns string data (lines of text)
returns output from all output streams, including error output
for a list of all output streams, see Get-Help about_Redirection
$result = powershell -noprofile -command 'Test-Cluster'
Note that using a script block to pass the command (-command { Test-Cluster }) would not work, because PowerShell then uses serialization and deserialization to emulate the in-session behavior.
Optional reading: output streams in PowerShell and how to redirect them:
Get-Help about_Redirection discusses a list of all output streams, which can be targeted by their numbers; since PSv5, these are:
1 ... success output stream (implicit output and Write-Output output)
2 ... error output stream (Write-Error and unhandled errors)
3 ... warnings (Write-Warning)
4 ... verbose output (Write-Verbose)
5 ... debug output (Write-Debug)
6 ... (v5+) Write-Information and Write-Host output
Note that some streams are silent by default and require opt-in to produce output, either via a preference variable (e.g., $VerbosePreference) or a common parameter (e.g., -Verbose)
{n}> allows redirecting the number {n} stream; if {n} is omitted, 1 is implied:
to a file (e.g., 3> c:/tmp/warnings.txt
to "nowhere", i.e suppressing the output (e.g., 3> $null)
to the success output stream (e.g., 3>&1); note: only stream 1 can be targeted this way.
*> targets all output streams.
Note: Unlike in POSIX-like shells (e.g., bash), the order of multiple redirection expression does not matter.
Therefore, the following POSIX-like shell idiom - which redirects error output to the success stream and silences only the original success output - does NOT work:
... 2>&1 1>$null # !! NO output in PowerShell
To achieve this in PowerShell, you mustn't redirect 1 and instead filter the objects in the success by their stream of origin.
Case in point: In the end, the OP wanted the following: capture only warning output, without the regular (success) output:
Test-Cluster 3>&1 | Where-Object { $_ -is [System.Management.Automation.WarningRecord] }
Objects that came from the warning stream have type [System.Management.Automation.WarningRecord], which is what enables the filtering above.
I use *> instead of > to redirect all outputs from console to a file.
Example of redirecting text and a variable to a file:
"The "+$set_groups+"ADGroup set!" | Out-File -FilePath $log_path -Append
I have a PowerShell script that I am debugging and would like to redirect all Write-Host statements to a file. Is there an easy way to do that?
Until PowerShell 4.0, Write-Host sends the objects to the host. It does not return any objects.
Beginning with PowerShell 5.0 and newer, Write-Host is a wrapper for Write-Information, which allows to output to the information stream and redirect it with 6>> file_name.
http://technet.microsoft.com/en-us/library/hh849877.aspx
However, if you have a lot of Write-Host statements, replace them all with Write-Log, which lets you decide whether output to console, file or event log, or all three.
Check also:
Add-Content
redirection operators like >, >>, 2>, 2>, 2>&1
Write-Log
Tee-Object
Start-Transcript.
You can create a proxy function for Write-Host which sends objects to the standard output stream instead of merely printing them. I wrote the below cmdlet for just this purpose. It will create a proxy on the fly which lasts only for the duration of the current pipeline.
A full writeup is on my blog here, but I've included the code below. Use the -Quiet switch to suppress the console write.
Usage:
PS> .\SomeScriptWithWriteHost.ps1 | Select-WriteHost | out-file .\data.log # Pipeline usage
PS> Select-WriteHost { .\SomeScriptWithWriteHost.ps1 } | out-file .\data.log # Scriptblock usage (safer)
function Select-WriteHost
{
[CmdletBinding(DefaultParameterSetName = 'FromPipeline')]
param(
[Parameter(ValueFromPipeline = $true, ParameterSetName = 'FromPipeline')]
[object] $InputObject,
[Parameter(Mandatory = $true, ParameterSetName = 'FromScriptblock', Position = 0)]
[ScriptBlock] $ScriptBlock,
[switch] $Quiet
)
begin
{
function Cleanup
{
# Clear out our proxy version of write-host
remove-item function:\write-host -ea 0
}
function ReplaceWriteHost([switch] $Quiet, [string] $Scope)
{
# Create a proxy for write-host
$metaData = New-Object System.Management.Automation.CommandMetaData (Get-Command 'Microsoft.PowerShell.Utility\Write-Host')
$proxy = [System.Management.Automation.ProxyCommand]::create($metaData)
# Change its behavior
$content = if($quiet)
{
# In quiet mode, whack the entire function body,
# simply pass input directly to the pipeline
$proxy -replace '(?s)\bbegin\b.+', '$Object'
}
else
{
# In noisy mode, pass input to the pipeline, but allow
# real Write-Host to process as well
$proxy -replace '(\$steppablePipeline\.Process)', '$Object; $1'
}
# Load our version into the specified scope
Invoke-Expression "function ${scope}:Write-Host { $content }"
}
Cleanup
# If we are running at the end of a pipeline, we need
# to immediately inject our version into global
# scope, so that everybody else in the pipeline
# uses it. This works great, but it is dangerous
# if we don't clean up properly.
if($pscmdlet.ParameterSetName -eq 'FromPipeline')
{
ReplaceWriteHost -Quiet:$quiet -Scope 'global'
}
}
process
{
# If a scriptblock was passed to us, then we can declare
# our version as local scope and let the runtime take
# it out of scope for us. It is much safer, but it
# won't work in the pipeline scenario.
#
# The scriptblock will inherit our version automatically
# as it's in a child scope.
if($pscmdlet.ParameterSetName -eq 'FromScriptBlock')
{
. ReplaceWriteHost -Quiet:$quiet -Scope 'local'
& $scriptblock
}
else
{
# In a pipeline scenario, just pass input along
$InputObject
}
}
end
{
Cleanup
}
}
You can run your script in a secondary PowerShell shell and capture the output like this:
powershell -File 'Your-Script.ps1' > output.log
That worked for me.
Using redirection will cause Write-Host to hang. This is because Write-Host deals with various formatting issues that are specific to the current terminal being used. If you just want your script to have flexibility to output as normal (default to shell, with capability for >, 2>, etc.), use Write-Output.
Otherwise, if you really want to capture the peculiarities of the current terminal, Start-Transcript is a good place to start. Otherwise you'll have to hand-test or write some complicated test suites.
Try adding a asterisk * before the angle bracket > to redirect all streams:
powershell -File Your-Script.ps1 *> output.log
When stream redirection is requested, if no specific stream is indicated then by default only the Success Stream(1>) is redirected. Write-Host is an alias for Write-Information which writes to the Information Stream (6>). To redirect all streams use *>.
Powershell-7.1 supports redirection of multiple output streams:
Success Stream (#1): PowerShell 2.0 Write-Output
Error Stream (#2): PowerShell 2.0 Write-Error
Warning Stream (#3): PowerShell 3.0 Write-Warning
Verbose Stream (#4): PowerShell 3.0 Write-Verbose
Debug Stream (#5): PowerShell 3.0 Write-Debug
Information Stream (#6): PowerShell 5.0 Write-Information
All Streams (*): PowerShell 3.0
This worked for me in my first PowerShell script that I wrote few days back:
function logMsg($msg)
{
Write-Output $msg
Write-Host $msg
}
Usage in a script:
logMsg("My error message")
logMsg("My info message")
PowerShell script execution call:
ps> .\myFirstScript.ps1 >> testOutputFile.txt
It's not exactly answer to this question, but it might help someone trying to achieve both logging to the console and output to some log file, doing what I reached here :)
Define a function called Write-Host. Have it write to a file. You may have some trouble if some invocations use a weird set of arguments. Also, this will only work for invocations that are not Snapin qualified.
If you have just a few Write-Host statements, you can use the "6>>" redirector operator to a file:
Write-Host "Your message." 6>> file_path_or_file_name
This is the "Example 5: Suppress output from Write-Host" provided by Microsoft, modified accordingly to about_Operators.
I just added Start-Transcript at the top of the script and Stop-Transcript at the bottom.
The output file was intended to be named <folder where script resides>-<datestamp>.rtf, but for some reason the trace file was being put where I did not expect it — the desktop!
You should not use Write-Host if you wish to have the messages in a file. It is for writing to the host only.
Instead you should use a logging module, or Set/Add-Content.
I have found the best way to handle this is to have a logging function that will detect if there is a host UI and act accordingly. When the script is executed in interactive mode it will show the details in the host UI, but when it is run via WinRM or in a non-interactive mode it will fall back on the Write-Output so that you can capture it using the > or *> redirection operators
function Log-Info ($msg, $color = "Blue") {
if($host.UI.RawUI.ForegroundColor -ne $null) {
Write-Host "`n[$([datetime]::Now.ToLongTimeString())] $msg" -ForegroundColor $color -BackgroundColor "Gray"
} else {
Write-Output "`r`n[$([datetime]::Now.ToLongTimeString())] $msg"
}
}
In cases where you want to capture the full output with the Write-Host coloring, you can use the Get-ConsoleAsHtml.ps1 script to export the host's scrolling buffer to an HTML or RTF file.
Use Write-Output instead of Write-Host, and redirect it to a file like this:
Deploy.ps1 > mylog.log or Write-Output "Hello World!" > mylog.log
Try using Write-Output instead of Write-Host.
The output goes down the pipeline, but if this is the end of the pipe, it goes to the console.
> Write-Output "test"
test
> Write-Output "test" > foo.txt
> Get-Content foo.txt
test