As I didn't find a solution by searching the forum and spent some time for finding out how to do it properly, I'm placing here the issue along with the working solution.
Scenario: in Powershell, need to remotely execute a script block stored in a variable and capture its output for further processing. No output should appear on the screen unless the script generates it on purpose. The script block can contain Write-Warning commands.
Note that the behaviors of interest apply generally to PowerShell commands, not just in the context of Invoke-Command and the - generally to be avoided - Invoke-Expression; in your case, it is only needed to work around a bug.[1]
Your own answer shows how to redirect a single, specific output streams to the success output stream; e.g, 3>&1 redirects (>&) the warning stream (3) to the success (output) stream (1).
The & indicates that the redirection target is a stream, as opposed to a file; for more information about PowerShell's output stream, see about_Redirection.
If you want to redirect all output streams to the success output stream, use redirection *>&1
By redirecting all streams to the output stream, their combined output can be captured in a variable, redirected to a file, or sent through the pipeline, whereas by default only the success output stream (1) is captured.
Separately, you can use the common parameters named -*Variable parameters to capture individual stream output in variables for some streams, namely:
Stream 1 (success): -OutVariable
Stream 2 (error): -ErrorVariable
Stream 3 (warning): -WarningVariable
Stream 6 (information): -InformationVariable
Be sure to specify the target variable by name only, without the $ prefix; e.g., to capture warnings in variable $warnings, use
-WarningVariable warnings, such as in the following example:
Write-Warning hi -WarningVariable warnings; "warnings: $warnings"
Note that with -*Variable, the stream output is collected in the variable whether or not you silence or even ignore that stream otherwise, with the notable exception of -ErrorAction Ignore, in which case an -ErrorVariable variable is not populated (and the error is also not recorded in the automatic $Error variable that otherwise records all errors that occur in the session).
Generally, -{StreamName}Action SilentlyIgnore seems to be equivalent to {StreamNumber}>$null.
Note the absence of the verbose (4) and the debug (5) streams above; you can only capture them indirectly, via 4>&1 and 5>&1 (or *>&1), which then requires you to extract the output of interest from the combined stream, via filtering by output-object type:
Important:
The verbose (4) and debug (5) streams are the only two streams that are silent at the source by default; that is, unless these streams are explicitly turned on via -Verbose / -Debug or their preference-variable equivalents, $VerbosePreference = 'Continue' / $DebugPreference = 'Continue', nothing is emitted and nothing can be captured.
The information stream (5) is silent only on output by default; that is, writing to the information stream (with Write-Information) always writes objects to the stream, but they're not displayed by default (they're only displayed with -InformationAction Continue / $InformationPreference = 'Continue')
Since v5, Write-Host now too writes to the information stream, though its output does print by default, but can be suppressed with 6>$null or -InformationAction Ignore (but not -InformationAction SilentlyContinue).
# Sample function that produces success and verbose output.
# Note that -Verbose is required for the message to actually be emitted.
function foo { Write-Output 1; Write-Verbose -Verbose 4 }
# Get combined output, via 4>&1
$combinedOut = foo 4>&1
# Extract the verbose-stream output records (objects).
# For the debug output stream (5), the object type is
# [System.Management.Automation.DebugRecord]
$verboseOut = $combinedOut.Where({ $_ -is [System.Management.Automation.VerboseRecord] })
[1] Stream-capturing bug, as of PowerShell v7.0:
In a nutshell: In the context of remoting (such as Invoke-Command -Session here), background jobs, and so-called minishells (passing a script block to the PowerShell CLI to execute commands in a child process), only the success (1) and error (2) streams can be captured as expected; all other are unexpectedly passed through to the host (display) - see this GitHub issue.
Your command should - but currently doesn't - work as follows, which would obviate the need for Invoke-Expression:
# !! 3>&1 redirection is BROKEN as of PowerShell 7.0, if *remoting* is involved
# !! (parameters -Session or -ComputerName).
$RemoteOutput =
Invoke-Command -Session $Session $Commands 3>&1 -ErrorVariable RemoteError 2>$null
That is, in principle you should be able to pass a $Commands variable that contains a script block directly as the (implied) -ScriptBlock argument to Invoke-Command.
Script block is contained in $Commands variable. $Session is an already established Powershell remoting session.
The task is resolved by the below command:
$RemoteOutput =
Invoke-Command -Session $Session {
Invoke-Expression $Using:Commands 3>&1
} -ErrorVariable RemoteError 2>$null
After the command is executed all output of the script block is contained in $RemoteOutput. Errors generated during remote code execution are placed in $RemoteError.
Additional clarifications. Write-Warning in Invoke-Expression code block generates its own output stream that is not captured by Invoke-Command. The only way to capture it in a variable is to redirect that stream to the standard stream of Invoke-Expression by using 3>&1. Commands in the code block writing to other output streams (verbose, debug) seems not to be captured even by adding 4>&1 and 5>&1 parameters to Invoke-Expression. However, stream #2 (errors) is properly captured by Invoke-Command in the way shown above.
I'd like to run an external process and capture it's command output to a variable in PowerShell. I'm currently using this:
$params = "/verify $pc /domain:hosp.uhhg.org"
start-process "netdom.exe" $params -WindowStyle Hidden -Wait
I've confirmed the command is executing but I need to capture the output into a variable. This means I can't use the -RedirectOutput because this only redirects to a file.
Note: The command in the question uses Start-Process, which prevents direct capturing of the target program's output. Generally, do not use Start-Process to execute console applications synchronously - just invoke them directly, as in any shell. Doing so keeps the application's output streams connected to PowerShell's streams, allowing their output to be captured by simple assignment $output = netdom ... (and with 2> for stderr output), as detailed below.
Fundamentally, capturing output from external programs works the same as with PowerShell-native commands (you may want a refresher on how to execute external programs; <command> is a placeholder for any valid command below):
# IMPORTANT:
# <command> is a *placeholder* for any valid command; e.g.:
# $cmdOutput = Get-Date
# $cmdOutput = attrib.exe +R readonly.txt
$cmdOutput = <command> # captures the command's success stream / stdout output
Note that $cmdOutput receives an array of objects if <command> produces more than 1 output object, which in the case of an external program means a string[1] array containing the program's output lines.
If you want to make sure that the result is always an array - even if only one object is output, type-constrain the variable as an array ([object[]]), or enclose the command in #(...), the array-subexpression operator:[2]
[array] $cmdOutput = <command>
$cmdOutput = #(<command>) # alternative
By contrast, if you want $cmdOutput to always receive a single - potentially multi-line - string, use Out-String, though note that a trailing newline is invariably added (GitHub issue #14444 discusses this problematic behavior):
# Note: Adds a trailing newline.
$cmdOutput = <command> | Out-String
With calls to external programs - which by definition only ever return strings in PowerShell[1] - you can avoid that by using the -join operator instead:
# NO trailing newline.
$cmdOutput = (<command>) -join "`n"
Note: For simplicity, the above uses "`n" to create Unix-style LF-only newlines, which PowerShell happily accepts on all platforms; if you need platform-appropriate newlines (CRLF on Windows, LF on Unix), use [Environment]::NewLine instead.
To capture output in a variable and print to the screen:
<command> | Tee-Object -Variable cmdOutput # Note how the var name is NOT $-prefixed
Or, if <command> is a cmdlet or advanced function, you can use common parameter
-OutVariable / -ov:
<command> -OutVariable cmdOutput # cmdlets and advanced functions only
Note that with -OutVariable, unlike in the other scenarios, $cmdOutput is always a collection, even if only one object is output. Specifically, an instance of the array-like [System.Collections.ArrayList] type is returned.
See this GitHub issue for a discussion of this discrepancy.
To capture the output from multiple commands, use either a subexpression ($(...)) or call a script block ({ ... }) with & or .:
$cmdOutput = $(<command>; ...) # subexpression
$cmdOutput = & {<command>; ...} # script block with & - creates child scope for vars.
$cmdOutput = . {<command>; ...} # script block with . - no child scope
Note that the general need to prefix with & (the call operator) an individual command whose name/path is quoted - e.g., $cmdOutput = & 'netdom.exe' ... - is not related to external programs per se (it equally applies to PowerShell scripts), but is a syntax requirement: PowerShell parses a statement that starts with a quoted string in expression mode by default, whereas argument mode is needed to invoke commands (cmdlets, external programs, functions, aliases), which is what & ensures.
The key difference between $(...) and & { ... } / . { ... } is that the former collects all input in memory before returning it as a whole, whereas the latter stream the output, suitable for one-by-one pipeline processing.
Redirections also work the same, fundamentally (but see caveats below):
$cmdOutput = <command> 2>&1 # redirect error stream (2) to success stream (1)
However, for external commands the following is more likely to work as expected:
$cmdOutput = cmd /c <command> '2>&1' # Let cmd.exe handle redirection - see below.
Considerations specific to external programs:
External programs, because they operate outside PowerShell's type system, only ever return strings via their success stream (stdout); similarly, PowerShell only ever sends strings to external programs via the pipeline.[1]
Character-encoding issues can therefore come into play:
On sending data via the pipeline to external programs, PowerShell uses the encoding stored in the $OutVariable preference variable; which in Windows PowerShell defaults to ASCII(!) and in PowerShell [Core] to UTF-8.
On receiving data from an external program, PowerShell uses the encoding stored in [Console]::OutputEncoding to decode the data, which in both PowerShell editions defaults to the system's active OEM code page.
See this answer for more information; this answer discusses the still-in-beta (as of this writing) Windows 10 feature that allows you to set UTF-8 as both the ANSI and the OEM code page system-wide.
If the output contains more than 1 line, PowerShell by default splits it into an array of strings. More accurately, the output lines are streamed one by one, and, when captured, stored in an array of type [System.Object[]] whose elements are strings ([System.String]).
If you want the output to be a single, potentially multi-line string, use the -join operator (you can alternatively pipe to Out-String, but that invariably adds a trailing newline):
$cmdOutput = (<command>) -join [Environment]::NewLine
Merging stderr into stdout with 2>&1, so as to also capture it as part of the success stream, comes with caveats:
To do this at the source, let cmd.exe handle the redirection, using the following idioms (works analogously with sh on Unix-like platforms):
$cmdOutput = cmd /c <command> '2>&1' # *array* of strings (typically)
$cmdOutput = (cmd /c <command> '2>&1') -join "`r`n" # single string
cmd /c invokes cmd.exe with command <command> and exits after <command> has finished.
Note the single quotes around 2>&1, which ensures that the redirection is passed to cmd.exe rather than being interpreted by PowerShell.
Note that involving cmd.exe means that its rules for escaping characters and expanding environment variables come into play, by default in addition to PowerShell's own requirements; in PS v3+ you can use special parameter --% (the so-called stop-parsing symbol) to turn off interpretation of the remaining parameters by PowerShell, except for cmd.exe-style environment-variable references such as %PATH%.
Note that since you're merging stdout and stderr at the source with this approach, you won't be able to distinguish between stdout-originated and stderr-originated lines in PowerShell; if you do need this distinction, use PowerShell's own 2>&1 redirection - see below.
Use PowerShell's 2>&1 redirection to know which lines came from what stream:
Stderr output is captured as error records ([System.Management.Automation.ErrorRecord]), not strings, so the output array may contain a mix of strings (each string representing a stdout line) and error records (each record representing a stderr line). Note that, as requested by 2>&1, both the strings and the error records are received through PowerShell's success output stream).
Note: The following only applies to Windows PowerShell - these problems have been corrected in PowerShell [Core] v6+, though the filtering technique by object type shown below ($_ -is [System.Management.Automation.ErrorRecord]) can also be useful there.
In the console, the error records print in red, and the 1st one by default produces multi-line display, in the same format that a cmdlet's non-terminating error would display; subsequent error records print in red as well, but only print their error message, on a single line.
When outputting to the console, the strings typically come first in the output array, followed by the error records (at least among a batch of stdout/stderr lines output "at the same time"), but, fortunately, when you capture the output, it is properly interleaved, using the same output order you would get without 2>&1; in other words: when outputting to the console, the captured output does NOT reflect the order in which stdout and stderr lines were generated by the external command.
If you capture the entire output in a single string with Out-String, PowerShell will add extra lines, because the string representation of an error record contains extra information such as location (At line:...) and category (+ CategoryInfo ...); curiously, this only applies to the first error record.
To work around this problem, apply the .ToString() method to each output object instead of piping to Out-String:
$cmdOutput = <command> 2>&1 | % { $_.ToString() };
in PS v3+ you can simplify to:
$cmdOutput = <command> 2>&1 | % ToString
(As a bonus, if the output isn't captured, this produces properly interleaved output even when printing to the console.)
Alternatively, filter the error records out and send them to PowerShell's error stream with Write-Error (as a bonus, if the output isn't captured, this produces properly interleaved output even when printing to the console):
$cmdOutput = <command> 2>&1 | ForEach-Object {
if ($_ -is [System.Management.Automation.ErrorRecord]) {
Write-Error $_
} else {
$_
}
}
An aside re argument-passing, as of PowerShell 7.2.x:
Passing arguments to external programs is broken with respect to empty-string arguments and arguments that contain embedded " characters.
Additionally, the (nonstandard) quoting needs of executables such as msiexec.exe and batch files aren't accommodated.
For the former problem only, a fix may be coming (though the fix would be complete on Unix-like platforms), as discussed in this answer, which also details all the current problems and workarounds.
If installing a third-party module is an option, the ie function from the Native module (Install-Module Native) offers a comprehensive solution.
[1] As of PowerShell 7.1, PowerShell knows only strings when communicating with external programs. There is generally no concept of raw byte data in a PowerShell pipeline. If you want raw byte data returned from an external program, you must shell out to cmd.exe /c (Windows) or sh -c (Unix), save to a file there, then read that file in PowerShell. See this answer for more information.
[2] There are subtle differences between the two approaches (which you may combine), though they usually won't matter: If the command has no output, the [array] type-constraint approach results in $null getting stored in the target variable, whereas it is an empty ([object[]) array in the case of #(...). Additionally, the [array] type constraint means that future (non-empty) assignments to the same variable are coerced to an array too.
Have you tried:
$OutputVariable = (Shell command) | Out-String
If you want to redirect the error output as well, you have to do:
$cmdOutput = command 2>&1
Or, if the program name has spaces in it:
$cmdOutput = & "command with spaces" 2>&1
Or try this. It will capture output into variable $scriptOutput:
& "netdom.exe" $params | Tee-Object -Variable scriptOutput | Out-Null
$scriptOutput
Another real-life example:
$result = & "$env:cust_tls_store\Tools\WDK\x64\devcon.exe" enable $strHwid 2>&1 | Out-String
Notice that this example includes a path (which begins with an environment variable). Notice that the quotes must surround the path and the EXE file, but not the parameters!
Note: Don't forget the & character in front of the command, but outside of the quotes.
The error output is also collected.
It took me a while to get this combination working, so I thought that I would share it.
I tried the answers, but in my case I did not get the raw output. Instead it was converted to a PowerShell exception.
The raw result I got with:
$rawOutput = (cmd /c <command> 2`>`&1)
I got the following to work:
$Command1="C:\\ProgramData\Amazon\Tools\ebsnvme-id.exe"
$result = & invoke-Expression $Command1 | Out-String
$result gives you the needful
I use the following:
Function GetProgramOutput([string]$exe, [string]$arguments)
{
$process = New-Object -TypeName System.Diagnostics.Process
$process.StartInfo.FileName = $exe
$process.StartInfo.Arguments = $arguments
$process.StartInfo.UseShellExecute = $false
$process.StartInfo.RedirectStandardOutput = $true
$process.StartInfo.RedirectStandardError = $true
$process.Start()
$output = $process.StandardOutput.ReadToEnd()
$err = $process.StandardError.ReadToEnd()
$process.WaitForExit()
$output
$err
}
$exe = "C:\Program Files\7-Zip\7z.exe"
$arguments = "i"
$runResult = (GetProgramOutput $exe $arguments)
$stdout = $runResult[-2]
$stderr = $runResult[-1]
[System.Console]::WriteLine("Standard out: " + $stdout)
[System.Console]::WriteLine("Standard error: " + $stderr)
This thing worked for me:
$scriptOutput = (cmd /s /c $FilePath $ArgumentList)
If all you are trying to do is capture the output from a command, then this will work well.
I use it for changing system time, as [timezoneinfo]::local always produces the same information, even after you have made changes to the system. This is the only way I can validate and log the change in time zone:
$NewTime = (powershell.exe -command [timezoneinfo]::local)
$NewTime | Tee-Object -FilePath $strLFpath\$strLFName -Append
Meaning that I have to open a new PowerShell session to reload the system variables.
What did the trick for me, and would work when using external commands and also when both standard error and standard output streams could be the result of running the command (or a mix of them), was the following:
$output = (command 2>&1)
I have a long script in powershell which calls an even longer function located in a separate .ps1 file. The function runs some svn update commands and some compiled executables which produce standard output. When I run these directly from the script the output gets redirected to the debug console in Powershell ISE. When I run them through the function I can tell they are running but I get no standard output in the console.
How do I redirect standard output from my function back to the powershell debug console where I can see it?
THanks.
EDIT
I am importing the function as follows:
. "D:\common.ps1"
and calling it as follows:
$MedianValue = Run-Comparison $LastRev $FirstRev $ScriptPath $SolutionPath $DevenvPath $TestPath $refFile $SVNPAth
Within the function, one of the calls is as follows
svn update $FirstRev
Start-Process ExecutableName Argument
It is for the above two statements that I cannot see the standard output for when I call their containing function.
If you're capturing a script's / function's output and that script / function contains a mix of PowerShell-native output statements and external-program calls producing stdout output, both types of output are sent to PowerShell's regular success output streams.
Therefore, unless you redirect at the source, you cannot selectively pass stdout from external programs through to the the host (e.g., a regular console window or the console pane in the ISE), because you won't be able to tell which output objects (lines) come from where.
To redirect at the source - if you have control over the callee's source code - you have several options, the simplest being Write-Host, as the following example demonstrates:
function Run-Comparison {
'PS success output'
cmd /c 'echo external stdout output' | Write-Host
}
# Captures 'PS success output', but passes the cmd.exe output through to the console.
$MedianValue = Run-Comparison
The above selectively sends the cmd.exe command's output to the host.
In PSv5+, where Write-Host writes to the newly introduced information stream (number 6), you can optionally suppress the to-host output with 6>$null on invocation.
To reverse the logic, use Write-Information instead of Write-Host (PSv5+ only), which is silent by default and allows you to turn on output with $InformationPreference = 'Continue'.
If you want silent-by-default behavior in PSv4-, use Write-Verbose or Write-Debug, but note that such output will be a different color, with each line having a prefix (VERBOSE: and DEBUG:, respectively).
I'm very new to powershell and I'm looking to pipe all lines to an output file for logging with it's time, but I would rather not explicitly pipe all cmdlets to the log file to avoid visual clutter and to make the code easily extensible.
I'm not sure whether existing methods do what I'm asking and I just can't recognize it or what but any pointers or help would be appreciated.
If you want to capture all of the output generated in a PowerShell session (including other streams such as verbose or error messages) you can use Start-Transcript at the start of your script and Stop-Transcript at the end.
If you want to output specific log entries, consider using a Write-Log function as described in one of the answers here: Create Log File in Powershell
Or here's a simple version to get your started:
function Write-Log {
Param(
$Message,
$Path = "$env:USERPROFILE\log.txt"
)
function TS {Get-Date -Format 'hh:mm:ss'}
"[$(TS)]$Message" | Tee-Object -FilePath $Path -Append | Write-Verbose
}
Write-Log 'Some message'
This adds a timestamp to the start of the message and directs it to the Verbose output stream as well as a file via Tee-Object. You can then see the Verbose messages if you enable them via $VerbosePreference (or by supporting -Verbose in your script).
Alternatively, if you want to redirect all output from your script that isn't the other message streams (e.g just things sent to the standard pipeline) you could save the script, run it from within PowerShell and then pipe its output to another command such as Out-File, e.g:
.\yourscript.ps1 | Out-File output.txt
I am running a command
hg st
and then checking it's $LASTEXITCODE to check for availability of mercurial in the current directory. I do not care about its output and do not want to show it to my users.
How do I suppress ALL output, success or error?
Since mercurial isn't a PowerShell commandlet hg st | Out-Null does not work.
Out-Null works just fine with non-PowerShell commands. However, it doesn't suppress output on STDERR, only on STDOUT. If you want to suppress output on STDERR as well you have to redirect that file descriptor to STDOUT before piping the output into Out-Null:
hg st 2>&1 | Out-Null
2> redirects all output from STDERR (file descriptor #2). &1 merges the redirected output with the output from STDOUT (file descriptor #1). The combined output is then printed to STDOUT from where the pipe can feed it into STDIN of the next command in the pipline (in this case Out-Null). See Get-Help about_Redirection for further information.
A fun thing you can do is to pipe the output to Write-Verbose, then you can still see it if you need it by running your script with the -Verbose switch.
ping -n 2 $APP 2>&1 | Write-Verbose
Can also do this
hg st *> $null
Powershell suppress console output