Why does the tree command look different on Powershell than what it does on Cmd.
The problem only occurs in the PowerShell ISE, where output from external programs such as tree.com isn't passed straight through to the console.
Consider migrating away from the PowerShell ISE, given that it is no longer actively developed and there are reasons not to use it (bottom section), notably the inability to run PowerShell [Core] 6+, where all future effort will go. The editor that offers the best PowerShell development experience, across platforms, is Visual Studio Code, combined with its PowerShell extension, which is under active development.
To fix it, you must (temporarily) change [Console]::OutputEncoding to match your system's active OEM legacy code page, because that is the character encoding tree.com uses:
# Switch the encoding that PowerShell expects external programs to use
# to the active OEM code page.
[Console]::OutputEncoding = [System.Text.Encoding]::GetEncoding(
[cultureinfo]::CurrentCulture.TextInfo.OEMCodePage
)
# tree.com now produces the expected output in the ISE
tree
Note:
Even in a regular console window / Windows Terminal window and in Visual Studio Code [Console]::OutputEncoding can come into play, but only when you capture or redirect an external program's output (e.g., $treeOutput = tree or tree | ...).
As indicated above, this is not necessary for direct-to-display output - except in the ISE, where [Console]::OutputEncoding matters even then, which default to the system's active ANSI legacy code page.
For PowerShell to correctly interpret an external program's output when capturing or redirecting it, [Console]::OutputEncoding must match the actual encoding used by that program.
Therefore, if you want to capture or redirect tree.coms output, you may have to set [Console]::OutputEncoding:
In short, unless [Console]::OutputEncoding.CodePage matches the code-page number reported by [cultureinfo]::CurrentCulture.TextInfo.OEMCodePage - e.g., 437 on US-English systems - setting [Console]::OutputEncoding is needed.
In console windows and Windows Terminal windows this is generally not necessary (as of PowerShell 7.0) - it is only necessary if you've explicitly changed the active code page to UTF-8, for instance - see this answer; note that UTF-8 may become the default over time.
In Visual Studio Code, however, it is necessary by default, because in the PowerShell Integrated Console [Console]::OutputEncoding defaults to UTF-8 (code page 65001).
I believe it's because it's one of those commands that PowerShell runs cmd for in the background then pipes the output to Write-Host, similar to ping or ipconfig
Related
I am experiencing a rather puzzling error while trying to perform a diff on two files using Visual Studio Code from the command line. I have a text file in the cloud where I save some work related notes. I need to resolve conflicts with other clients editing the file. Usually this only happens during a loss of connection though somehow I find myself having to resolve a lot of them so between this and other uses of diff I will use the usual syntax. It looks something like this:
code --diff "R&D (cloud conflict 2-5-23).txt" "R&D.txt"
My filename happens to have a '&' in it and this command launches the usual 2-way diff in VS Code and reads through the first file name with no problem but doesn't read past the second '&' and the resulting diff tab in VS Code looks something like:
R&D (cloud conflict 2-25-23).txt <-> R
Where the right side "R" doesn't exist. So it would seem '&' needs to be processed literally.
No problem, let's see if backslash \ is an accepted escape parameter...
code --diff "R\&D (cloud conflict 2-5-23).txt" "R\&D.txt"
Nope. Same problem. 🤔 In fact this outputs something even stranger:
Code diff tab:
&D (cloud conflict 2-25-23).txt <-> R
with shell output:
'D.txt' is not recognized as an internal or external command, operable program or batch file.
I also tried the carrot symbol '^' as an escape parameter to a similar effect. I just includes it in the first file and the editor still thinks the second file name is just "R".
The help file for the VS Code command line integration didn't have a lot to say about the --diff parameter other than a short description and I was hoping to get something about processing strings literally or escape characters. Perhaps another parameter that I need or maybe this has more to do with the shell in general.
I find it really strange that it can read the first full file name but breaks at the second '&'. Weirder still that if a supposed escape character is included in the second file name, it will omit that as well. 😵
For now all I can do is rename the file which is a bummer. 🤷♂️ I have VS Code version 1.75.0 on Windows 10 Home latest version/build and I'm using PowerShell version 5.1.19041.2364.
Edit: The issue definitely appears to be PowerShell related as it turns out. I was finally able to run this command successfully in a regular command prompt. (Simply typing "cmd" and Enter into the PowerShell window before running the diff command). Unfortunately, I happen to be running this command as part of PowerShell script. I may have to figure out how to run a CMD command from inside my PowerShell script if that is at all possible. I'm not sure. 🤔 If not, I need to figure out what exactly PowerShell is doing to my command when it reaches the '&' character.
tl;dr
You need a workaround:
cmd /c 'code --diff "R&D (cloud conflict 2-5-23).txt" "R&D.txt"'
Alternatively, using --%, the stop-parsing token:
code --diff "R&D (cloud conflict 2-5-23).txt" --% "R&D.txt"
Note: --% comes with fundamental limitations, notably the inability to reference PowerShell variables - see this answer.
Background information:
The root cause is that code is implemented as a batch file (code.cmd) and that cmd.exe, the interpreter that executes batch file inappropriately parses its list of arguments as if they had been submitted from INSIDE a cmd.exe session.
PowerShell, which - of necessity - has to rebuild the process command line behind the scenes on Windows after having performed argument parsing based on its rules, and - justifiably - places "R&D.txt" as verbatim R&D.txt on the process command line, given that the argument value contains no spaces.
The result is that cmd.exe interprets the unquoted R&D.txt argument on its command line as containing metacharacter &, which is its command-sequencing operator, causing the call to break.
Given that cmd.exe, the legacy Windows shell, is unlikely to receive fixes, the actively maintained PowerShell (Core) 7+ edition could as a courtesy compensate for cmd.exe's inappropriate behavior.
Doing so has been proposed in GitHub issue #15143, but, alas, it looks like these accommodations will not be implemented.
I use WinSCP within a Powershell script. It suddenly stopped working. After a while I could figure out that the problem appeared from a more recent version of PowerShell:
Reduced code:
& winscp `
/log `
/command `
'echo Connecting...' `
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=`"`"ssh-ed25519 includes spaces`"`""
Error message using v7.2.7
Host "lkjhlk.com" does not exist.
Errror message using v7.3.0
Too many parameters for command 'open'.
As you can see with v7.3.0 WinSCP receives different input depending on the version of PS. I found out that the difference has something to do with the spaces in the hostkey. If they are omitted v7.3.0 outputs the same error.
What change to PowerShell caused this, and how can I fix it?
(How can I debug such issues? I played a bit around with escaping, but the strings look the same no matter the version, no obvious breaking change that could be responsible)
Version 7.3.0 of PowerShell (Core) introduced a breaking change with respect to how arguments with embedded " characters are passed to external programs, such as winscp:
While this change is mostly beneficial, because it fixes behavior that was fundamentally broken since v1 (this answer discusses the old, broken behavior), it also invariably breaks existing workarounds that build on the broken behavior, except those for calls to batch files and the WSH CLIs (wscript.exe and cscript.exe) and their associated script files (with file-name extensions such as .vbs and .js).
CAVEAT:
GitHub issue #18694 implies that some version after 7.3.1 and all later versions will make this breaking change opt-in, but on Windows only, for the sake of backward compatibility; that is:
On Windows:
Old workarounds will continue to work by default.
Getting the new, correct behavior requires (temporarily) setting
$PSNativeCommandArgumentPassing = 'Standard'
On Unix-like platforms:
The new, correct behavior ($PSNativeCommandArgumentPassing = 'Standard') will remain the default
Old workarounds will require (temporarily) setting $PSNativeCommandArgumentPassing = 'Legacy' in order to continue to work, as is already the case in 7.3.0
To make existing workarounds continue to work, set the $PSNativeCommandArgumentPassing preference variable (temporarily) to 'Legacy':
# Note: Enclosing the call in & { ... } makes it execute in a *child scope*
# limiting the change to $PSNativeCommandArgumentPassing to that scope.
& {
$PSNativeCommandArgumentPassing = 'Legacy'
& winscp `
/log `
/command `
'echo Connecting...' `
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=`"`"ssh-ed25519 includes spaces`"`""
}
Unfortunately, because winscp.exe only accepts
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces""" on its process command line (i.e., embedded " escaped as ""), and not also the most widely used form
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=\"ssh-ed25519 includes spaces\"" (embedded " escaped as \"), which the fixed behavior now employs, for winscp.exe, specifically, a workaround will continue to be required.
If you don't want to rely on having to modify $PSNativeCommandArgumentPassing for the workaround, here are workarounds that function in both v7.2- and v7.3+ :
Use --%, the stop-parsing token, which, however, comes with pitfalls and severe limitations, notably the inability to (directly) use PowerShell variables or subexpressions in the arguments that follow it - see this answer for details; however, you can bypass these limitations if you use --% as part of an array that you construct and assign to a variable first and then pass via splatting:
# Note: Must be single-line; note the --% and the
# unescaped use of "" in the argument that follows it.
# Only "..." quoting must be used after --%
# and the only variables that can be used are cmd-style
# *environment variables* such as %OS%.
winscp /log /command 'echo Connecting...' --% "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
# Superior alternative, using splatting:
$argList = '/log', '/command', 'echo Connecting...',
'--%', "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
winscp #argList
Alternatively, call via cmd /c:
# Note: Pass-through command must be single-line,
# Only "..." quoting supported,
# and the embedded command must obey cmd.exe's syntax rules.
cmd /c #"
winscp /log /command "echo Connecting..." "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
"#
Note: You don't strictly need to use a here-string (#"<newline>...<newline>"# or #'<newline>...<newline>'#), but it helps readability and simplifies using embedded quoting.
Both workarounds allow you to pass arguments directly as quoted, but unfortunately also require formulating the entire (pass-through) command on a single line - except if --% is combined with splatting.
Background information:
The v7.3 default $PSNativeCommandArgumentPassing value on Windows, 'Windows':
regrettably retains the old, broken behavior for calls to batch files and the WSH CLIs (wscript.exe and cscript.exe) and their associated script files (with file-name extensions such as .vbs and .js).
While, for these programs only, this allows existing workarounds to continue to function, future code that only needs to run in v7.3+ will continue to be burdened by the need for these obscure workarounds, which build on broken behavior.
The alternative, which was not implemented, would have been to build accommodations for these programs as well as some program-agnostic accommodations into PowerShell, so that in the vast majority of case there won't even be a need for workarounds in the future: see GitHub issue #15143.
There are also troublesome signs that this list of exceptions will be appended to, piecemeal, which all but guarantees confusion for a given PowerShell version as to which programs require workarounds and which don't.
commendably, for all other programs, makes PowerShell encode the arguments when it - of necessity - rebuilds the command line behind the scenes as follows with respect to ":
It encodes the arguments for programs that follow the C++ command-line parsing rules (as used by C / C++ / .NET applications) / the parsing rules of the CommandLineToArgv WinAPI function, which are the most widely observed convention for parsing a process' command line.
In a nutshell, this means that embedded " characters embedded in an argument, to be seen as a verbatim part of it by the target program, are escaped as \", with \ itself requiring escaping only (as \\) if it precedes a " but is meant to be interpreted verbatim.
Note that if you set $PSNativeCommandArgumentPassing value to 'Standard' (which is the default on Unix-like platforms, where this mode fixes all problems and makes v7.3+ code never require workarounds), this behavior applies to all external programs, i.e. the above exceptions no longer apply).
For a summary of the impact of the breaking v7.3 change, see this comment on GitHub.
If you have / need to write cross-edition, cross-version PowerShell code: The Native module (Install-Module Native; authored by me), has an ie function (short for: Invoke Executable), which is a polyfill that provides workaround-free cross-edition (v3+), cross-platform, and cross-version behavior in the vast majority of cases - simply prepend ie to your external-program calls.
Caveat: In the specific case at hand it will not work, because it isn't aware that winscp.exe requires ""-escaping.
My windows Powershell gives the expected results when interpreting commands such as "date" and "echo", while problems occur when it interprets such commands as "which" and "tail".
I thought it might be that I haven't add the address of these commands to the target directories, but where can I probably find these commands.
What happens when I apply "which", with the same thing happening when applying "tail"
which and tail are the names of external programs, which are unrelated to PowerShell. These programs can only be expected to be present on Unix-like platforms, not (natively) on Windows. (However, they would be present in Unix-like subsystems on Windows, such as WSL.)
By contrast, date and echo are (potentially) commands that are built into PowerShell, irrespective of what platforms it runs on:
echo is an alias of PowerShell's Write-Output cmdlet on all platforms.
date - unless an external program by that name is present in a directory listed in $env:PATH on a given platform - refers to the built in Get-Date cmdlet.
Note: This relies on PowerShell's ill-conceived default verb feature, which falls back on prefixing a command name with Get- if no command form is found by the given name. This shouldn't be relied upon, both in the interest of conceptual clarity and to avoid unnecessary overhead. Also, this fallback is (unexpectedly) not reported by the Get-Command discussed below, and also not by Get-Help - see GitHub issue #3987.
Use Get-Command to determine what command form, if any, a given name refers to (add -All to see if potentially multiple forms exist, with the effective one being listed first).
I wrote a program which prints a string, which contains ANSI escape sequences to make the text colored. But it doesn't work as expected in the default Windows 10 console, as you can see in the screenshot.
The program output appears with the escape sequences as printed characters.
If I feed that string to PowerShell via a variable or piping, the output appears as intended (red text).
How can I achieve that the program prints colored text without any workarounds?
This is my program source (Haskell) - but the language is not relevant, just so you can see how the escape sequences are written.
main = do
let red = "\ESC[31m"
let reset = "\ESC[39m"
putStrLn $ red ++ "RED" ++ reset
Note:
The following applies to regular (legacy) console windows on Windows (provided by conhost.exe), which are used by default, including when a console application is launched from a GUI application.
By contrast, the console windows (terminals) provided by Windows Terminal as well as Visual Studio Code's integrated terminal provide support for VT / ANSI escape sequences by default, for all console applications.
While console windows in Windows 10 do support VT (Virtual Terminal) / ANSI escape sequences in principle, support is turned OFF by default.
You have three options:
(a) Activate support globally by default, persistently, via the registry, as detailed in this SU answer.
In short: In registry key [HKEY_CURRENT_USER\Console], create or set the VirtualTerminalLevel DWORD value to 1
From PowerShell, you can do this programmatically as follows:
Set-ItemProperty HKCU:\Console VirtualTerminalLevel -Type DWORD 1
From cmd.exe (also works from PowerShell):
reg add HKCU\Console /v VirtualTerminalLevel /t REG_DWORD /d 1
Open a new console window for changes to take effect.
See caveats below.
(b) Activate support from inside your program, for that program (process) only, with a call to the SetConsoleMode() Windows API function.
See details below.
(c) Ad-hoc workaround, from PowerShell:
PowerShell (Core) 7+: Enclose external-program calls in (...) (invariably collects all output first before printing):
(.\test.exe)
Streaming Windows PowerShell-only alternative: Pipe output from external programs to Write-Host
.\test.exe | Out-Host
See details below.
Re (a):
The registry-based approach invariably activates VT support globally, i.e., for all console windows, irrespective of what shell / program runs in them:
Individual executables / shells can still deactivate support for themselves, if desired, using method (b).
Conversely, however, this means that the output of any program that doesn't explicitly control VT support will be subject to interpretation of VT sequences; while this is generally desirable, hypothetically this could lead to misinterpretation of output from programs that accidentally produce output with VT-like sequences.
Note:
While there is a mechanism that allows console-window settings to be scoped by startup executable / window title, via subkeys of [HKEY_CURRENT_USR\Console], the VirtualTerminalLevel value seems not to be supported there.
Even if it were, however, it wouldn't be a robust solution, because opening a console window via a shortcut file (*.lnk) (e.g. from the Start Menu or Task Bar) wouldn't respect these settings, because *.lnk files have settings built into them; while you can modify these built-in settings via the Properties GUI dialog, as of this writing the VirtualTerminalLevel setting is not surfaced in that GUI.
Re (b):
Calling the SetConsoleMode() Windows API function from inside the program (process), as shown here, is cumbersome even in C# (due to requiring P/Invoke declarations), and may not be an option:
for programs written in languages from which calling the Windows API is not supported.
if you have a preexisting executable that you cannot modify.
In that event, option (c) (from PowerShell), discussed next, may work for you.
Re (c):
PowerShell automatically activates VT (virtual terminal) support for itself when it starts (in recent releases of Windows 10 this applies to both Windows PowerShell and PowerShell (Core) 7+) - but that does not extend to external programs called from PowerShell, in either edition, as of v7.3.2.
Separately, in v7.2+ there is the $PSStyle.OutputRendering preference variable, which controls whether PowerShell commands produce colored output via the formatting system, such as the colored headers of Get-ChildItem output. However, this setting has no effect on (direct) output from external programs. $PSStyle.OutputRendering defaults to Host, meaning that only formatted output that prints to the terminal (console) is colored. $PSStyle.OutputRendering = 'PlainText' disables coloring, and $PSStyle.OutputRendering = 'Ansi' makes it unconditional; see this answer for more information.
However, as a workaround you can relay an external program's (stdout) output via PowerShell's (success) output stream, in which case VT sequences are recognized:
As of PowerShell (Core) 7.3.2, this only works either by enclosing the call in (...) or by using Out-String, but note that all output is invariably collected first before it is printed.[1]
(.\test.exe)
In Windows PowerShell, in addition to the above, streaming the relayed output is possible too, by piping to Write-Host (Out-Host, Write-Output or Out-String -Stream would work too)
.\test.exe | Write-Host
Note: You need these techniques only if you want to print to the console. If, by contrast, you want to capture the external program's output (including the escape sequences), use $capturedOutput = .\test.exe
Character-encoding caveat: Windows PowerShell by default expects output from external programs to use the OEM code page, as defined by the legacy system locale (e.g., 437 on US-English systems) and as reflected in [console]::OutputEncoding.
.NET console programs respect that setting automatically, but for non-.NET programs (e.g., Python scripts) that use a different encoding (and produce not just pure ASCII output (in the 7-bit range)), you must (at least temporarily) specify that encoding by assigning to [console]::OutputEncoding; e.g., for UTF-8:
[console]::OutputEncoding = [Text.Encoding]::Utf8.
Note that this is not only necessary for the VT-sequences workaround, but generally necessary for PowerShell to interpret non-ASCII characters correctly.
PowerShell Core (v6+), unfortunately, as of v7.3.2, still defaults to the OEM code page too, but that should be considered a bug (see GitHub issue #7233), given that it otherwise defaults to UTF-8 without BOM.
[1] Using Out-String -Stream or its built-in wrapper function, oss, is tempting in order to achieve streaming output, but this no longer works as of PowerShell 7.3.2, possibly due to the optimization implemented in GitHub PR #16612.
Thanks to user mklement0 for making me aware that VT support is not enabled automatically. This made me look in the right direction and I found this helpful post.
So to answer my question: Add or set the registry key
HKCU:\Console - [DWORD] VirtualTerminalLevel = 1
Restart the console and it works.
For Haskell specifically, you need to call the hSupportsANSIWithoutEmulation function to enable ANSI escape sequences in your program on Windows 10.
I am able to use Set-Culture (Powershell as Admin) to set the Current Culture to "en-DE" which is English (Germany). However, when I run the different PS commands to view the Current Culture, I am still getting en-US. I checked my Region (Format) and Location as well.
Do I have to change the system locale as well to Germany (German) ?
This is causing an error in an application, because the datetime format is different from en-DE to en-US and causing the date to be read incorrectly.
When I Set-Culture to de-DE, everything appears to be in working order.
I make sure to run Powershell Console as Administrator, Set-Culture, close console. Open Powershell and run Get-Culture, [CultureInfo]::CurrentCulture, [CultureInfo]::CurrentUICulture and a few more to check and and still getting en-US
Note: Use of en-DE as a culture identifier - i.e., mixing language en (English) with normally unrelated region/country DE (Germany) - requires Windows 10 with release channel 1607 or later or Windows Server 2016, according to Microsoft.
However, there's a bug that prevents use of such mixed cultures, observed on Windows 10 Pro (64-bit; Version 1709, OS Build: 16299.371)
While you can successfully set such mixed-culture values with Set-Culture, subsequent sessions do not recognize it and fall back to en-US (as reflected in $PSCulture, Get-Culture and [cultureinfo]::currentCulture)
This problem has been reported on UserVoice.
Note that PowerShell Core is not affected (but it doesn't support Set-Culture).
The rest of this answer discusses persistently setting the current user's culture in general, irrespective of the bug.
Set-Culture - via the registry - sets the culture for future PowerShell sessions (only), not (also) for the current session.
Get-Culture, by contrast, only ever reports the current session's culture at session-startup time. That is, if you change the culture during a session (see below), it will not be reflected in Get-Culture.
In order to also apply the newly set culture to the current session, run the following in addition to the Set-Culture call:
[cultureinfo]::CurrentCulture = 'de-DE'
Caveat re interactive (command-line) use:
In Windows PowerShell (still as of v5.1), the active culture is reset after every command submitted; e.g.,
[cultureinfo]::CurrentCulture = 'de-DE'; Get-Date works as expected, because it is part of the same command line, but when executing just Get-Date as the next command, the current culture has reverted to the one that was current at session-startup time.
This problem has been fixed in PowerShell Core.
This perhaps surprising asymmetry - Set-Culture only applying to future sessions, but Get-Culture reporting the current session's (startup) culture - is something that may change in future PowerShell Core versions.