I have a PowerShell script that runs via automation, so I need to capture the output of the script to a file, but I'd also like to capture the commands that were run, to give the output some context (I'd use set -x in a Linux shell script). I can't figure out how to capture those commands into the output file in Windows though; any help is much appreciated!
# script.ps1
Set-PSDebug -Trace 1
Write-Host "Hello, World!"
How I'm calling the script (from cmd):
$ powershell -command "./script.ps1 *>&1 | Tee-Object -FilePath ./output.txt"
Terminal output:
DEBUG: 2+ >>>> Write-Host "Hello, World!"
Hello, World!
output.txt file contents:
Hello, World!
You can see that the output file is missing the debug line that was shown in the terminal. Ideally, I'd like the debug lines to only be in the output file (not the terminal output), but having the debug lines in both places would be fine too.
Note that PowerShell 5.1 is the version I have installed.
Unfortunately, PowerShell's tracing output (activated via Set-PSDebug) operates outside PowerShell's system of output streams, so such output cannot be captured from inside a PowerShell session.[1]
However, an outside caller of PowerShell's CLI[2], such as cmd.exe in your case, does receive tracing output via stdout (the standard output stream)[3], so you can use cmd.exe's redirection to capture everything in a file and print it to the console afterwards.
A simplified example (from cmd.exe):
C:\>powershell -c "Set-PSDebug -Trace 1; Write-Host 'Hello, World!'" > output.txt & type output.txt
DEBUG: 1+ Set-PSDebug -Trace 1; >>>> Write-Host 'Hello, World!'
Hello, World!
[1] Perhaps surprisingly, despite the word "Debug" in the cmdlet name, tracing output does not go through stream number 5, the debugging stream, the way that Write-Debug output does.
[2] Note that you can use this technique of calling the CLI, invariably as a child process, from inside an existing PowerShell session as well, in which case that existing session acts as the outside caller. However, note that passing the command to execute via a script block ({ ... }), which is normally preferable, would not work in this case, because a script block-based invocation uses PowerShell's remoting infrastructure behind the scenes, which preserves the PowerShell-specific output streams (and also type fidelity, with limitations), so that the tracing output again goes straight to the console.
[3] Perhaps surprisingly, all of PowerShell's output streams - as well as to-console-only output such as produced by tracing - are by default mapped to an outside caller's stdout stream - including errors. While you can at least separate out errors on demand via a stderr redirection - 2> - on the caller side, all remaining streams are invariably sent to stdout - see the bottom section of this answer and GitHub issue #7989 for more information.
Related
I'm using pwsh on linux to run some specific powershell modules.
I want to output the data received to file descriptor 3. I want data on file descriptor 3 as Powershell doesn't respect the convention that stdout is data and stderr is logging. I want file descriptor 3 to serve as our "data" file descriptor.
End goal is to be able to do something like this as we wrap this powershell call in Python and we'll redirect the file descriptor 3 data ourselves
pwsh -f script.ps1 3>data
PowerShell has no built-in way to output to streams other than stdout (1) and stderr (2) (more on that below.
Conceivably, you can roll your own output behavior with .NET API and/or P/Invoke calls, but that would be cumbersome.
However, this may not be required:
While it is true that - unfortunately - PowerShell by default sends output from all its output streams to stdout - see GitHub issue #7989 - you can redirect PowerShell error-stream to stderr, if you apply a 2> redirection on the caller's side.
The following call, e.g. from bash, demonstrates this:
# Prints just 'hi', because the error-stream output was
# redirected to stderr thanks to 2>, and due to targeting /dev/null, suppressed.
pwsh -noprofile -c '"hi"; Write-Error no!' 2>/dev/null
The downside is that if you want to print the stderr output too, you must capture it in a file and print it afterwards, which means that it (a) won't appear at the time it is being produced and (b) therefore won't be properly interleaved with any stdout output.
As an aside:
The current behavior of the PowerShell CLI (as of v7.2.x) is unfortunate not only with respect to how output streams are mapped, but also because it loads profile files by default.
There was talk about providing a separate CLI to address the latter problem, in the context of which the stream-mapping behavior could be fixed too, but nothing has happened so far: see GitHub issue #8072.
When I execute the following hello.ps1 in PowerShell, I get an output like I would expect, a Hello World! with a newline.
#powershell script hello.ps1
"Hello World!"
Output of run
PS C:\test>.\hello.ps1
Hello World!
PS C:\test>
But when I call the script from a bat file, I get two (2) newlines output in Windows console.
:: Windows CMD script name: mytest.bat
powershell -ExecutionPolicy Bypass hello.ps1
Now the output has two newlines, note the empty line after Hello World!.
C:\test>mytest.bat
Hello World!
C:\test
How can I avoid this second newline?
Mandragor -
Firstly, that is not two lines, it's just one.
Secondly, this is not a PowerShell specific thing.
You are calling one shell from another, and both must provide you a response and return to interactive mode. Calling an external command for one shell to another must be processed by the called shell first and results returned from the called shell to the calling shell, then exit the called process, and that calling shell must complete its process and any stdout stuff, and exit that process.
Double-clicking a bat/cmd file, starts cmd.exe, task manager shows
that process is running
In your bat/cmd you are calling another standalone exe, task manager
shows that the process is running. It must execute and return stdout
results, then close/exit.
cmd.exe the completes, and returns its stdout stuff, if any, and places the cursor at the next line for more interactive work.
Hence, the two responses, only if you ask for cmd.exe stdout stuff will you see it, vs just null.
You'll always have one more line return to get you back to the calling process.
Also, point of note:
Write-Host except in specific circumstances, and what you are doing
is not one of them.
You only need that bypass policy setting if you are in a restricted
environment. RemoteSigned is now the PowerShell default, meaning all local scripts will run, and remote scripts must be signed.
Simple strings should use single quotes in most cases. Double quotes
are for expanding variable content and other formatting scenarios.
Try the same thing calling any other shell, you'll get the same results. Heck, even calling another bat/cmd from a bat/cmd.
Example:
Running mytest.bat in cmd.exe that only contains these two lines, to show that only 1 CRLF is actually returned, per process, then you get bat the interactive shell.
powershell -File D:\Temp\hello.ps1
call d:\temp\hello.bat
# Results
Hello World!
D:\Temp>call d:\temp\hello.bat
D:\Temp>echo 'Hello World!'
'Hello World!'
D:\Temp>
You can change your powershell script like this:
#powershell script hello.ps1
if ($args[0] -eq "cmd") {
write-host "Hello World!" -nonewline
}
else {
"Hello World!"
}
Then change the caller batch file:
#echo off
powershell -executionpolicy Bypass TheNameOfPowershellScript.ps1 "cmd"
Now when run from cmd use the caller batch file and from powershell directly use the ps1 script. It will give the expected result(Tested).
What's PowerShell equivalent of exec redirection in bash like this?
exec 2>&1
I wanted to make a PowerShell script to redirect its stderr to stdout by itself, without any function or script block calls like this, or recursive script invocation.
The redirection should occur in the beginning of the script, then all stderr output from the rest of the script and sub-process should go to stdout.
You can't: Unlike Bash, PowerShell has no feature for script-wide redirection of streams (as of PowerShell v7).
You'll have to do one of the very things you're trying to avoid:
Enclose all code in your script in . { ... } 2>&1
Recursively invoke your script via such an enclosure (which requires some mechanism, such as a conceptually private parameter, to prevent infinite recursion).
Also note that while stdout and stderr are the conceptual equivalents of PowerShell's success output stream and error stream, respectively:
PowerShell has 6 output streams in total - see about_Redirection (which your question already links to).
Regrettably, all of these streams are by default mapped to stdout when PowerShell is called from the outside, via its CLI.
However, you can selectively redirect the error stream using 2> from the outside - but none of the other streams, aside from the success stream.
There's no "exec" in powershell (overlaying a process). But redirecting stderr to stdout is the same:
yourscript.ps1 2>&1
If you were running pwsh from some other unix shell, or powershell from cmd in windows, everything would go to stdout anyway:
pwsh yourscript.ps1 > yourscript.log
Don't worry about marking me down. Other people will mark me up later. I'm used to it.
From a powershell script (with nuget installed and on the path), I'm trying to execute an executable (.net, if that matters) ... but for some reason, I can't get the STDOUT to display in the command window.
nuget install mdoc -OutputDirectory packages -ExcludeVersion
start-process "packages/mdoc/tools/mdoc.exe" "--version"
echo "done"
This should output mdoc 5.7.2 (at the time of this post, current version). But you'll see the nuget output, then done.
Any thoughts on why this is not showing up?
As Ansgar's comment implies: On Windows, Start-Process runs console programs in a new console window by default, asynchronously.
If that program completes quickly, you may see the new console window flash only briefly, as it opens and closes soon thereafter, or you may miss the flash altogether - either way, its output will not show in the caller's console window.
Adding -Wait to the Start-Process call would make the invocation synchronous, and adding -NoNewWindow would make it run in the same console, yet the calling PowerShell session wouldn't be able to capture or redirect the invoked program's output - see below.
Taking a step back: Do NOT use Start-Process[1] if you want to run a console program synchronously, with its standard streams connected to PowerShell's streams - just invoke such a program directly:
packages/mdoc/tools/mdoc.exe --version
If the external program's path / name must be quoted (because its path contains spaces) and/or it is stored in a variable, simply use &, the call operator, to invoke it:
# Via a quoted string:
& "packages/mdoc/tools/mdoc.exe" --version
# Via a variable:
$exePath = "packages/mdoc/tools/mdoc.exe"
& $exePath --version
Using the direct-invocation approach gives you:
synchronous execution by default
the ability to capture and/or redirect the invoked program's stdout and stderr streams.
access to the program's process exit code is via PowerShell's automatic $LASTEXITCODE variable.
To put it all together (based on your later comments):
nuget install mdoc -OutputDirectory packages -ExcludeVersion
$exePath = "packages/mdoc/tools/mdoc.exe"
& $exePath --version
"done"
This prints the version number - mdoc 5.7.2 as of this writing - just before printing done (verified on Windows PowerShell v5.1.17134.48 on Microsoft Windows 10 Pro (64-bit; Version 1709, OS Build: 16299.371)).
Optional reading: capturing stdout / stderr output from external programs:
To capture stdout output, simply assign the call to a variable:
$version = & $exePath --version # $version receives stdout output as an *array of lines*
$version receives either a string scalar (single string) if there was only 1 line of output, or an array of strings representing the output lines.
To also capture stderr output, use redirection 2>&1:
[string[]] $allOutput = & $exePath --version 2>&1
Note the cast to [string[]], which ensures that the stderr lines are captured as strings too.
By default, they are captured as [System.Management.Automation.ErrorRecord] instances, which in Windows PowerShell will somewhat confusingly print them as if they were PowerShell errors - this problem has been fixed in PowerShell Core.
Conversely, however, if you don't convert the type of the elements of the array that is returned to strings, you can examine each element with -is [System.Management.Automation.ErrorRecord] to determine whether it originated from stdout or stderr.
This answer uses this approach to separate stdout lines from stderr lines after merging with 2>&1, so that they can be saved to separate files (which with > in Windows PowerShell requires explicitly stringifying the [System.Management.Automation.ErrorRecord] instances).
Note: When PowerShell communicates with external programs, character-encoding issues come into play: see this answer for details.
[1] Or the underlying .NET API, System.Diagnostics.Process.
For guidance on when Start-Process is or isn't appropriate, see GitHub docs issue #6239
Here is a piece of function I'm trying to create to make my testing faster:
function t++ {
param($Source, $Input, $Output)
...
g++ $Source -o test;
.\test.exe
Write-Output "===End-Of-Output==="
}
But the output in my Windows PowerShell ISE comes as:
Hello, World!
===End-Of-Output===
\n
(the last line is blank)
I can't understand why is it outputting that extra newline after the End-Of-Output.
When output is written to the standard output stream in PowerShell, it's picked up by the Host application - and the host application may format and manipulate the output.
In case of powershell.exe that would be System.Console, rendering string output as is, but a PowerShell Host application is not necessarily Console-based.
PowerShell ISE for example, doesn't use System.Console, since the output pane is also your debugger - I assume the PowerShell development team found it easier to implement a new output mechanism rather than extending Console.