My windows Powershell gives the expected results when interpreting commands such as "date" and "echo", while problems occur when it interprets such commands as "which" and "tail".
I thought it might be that I haven't add the address of these commands to the target directories, but where can I probably find these commands.
What happens when I apply "which", with the same thing happening when applying "tail"
which and tail are the names of external programs, which are unrelated to PowerShell. These programs can only be expected to be present on Unix-like platforms, not (natively) on Windows. (However, they would be present in Unix-like subsystems on Windows, such as WSL.)
By contrast, date and echo are (potentially) commands that are built into PowerShell, irrespective of what platforms it runs on:
echo is an alias of PowerShell's Write-Output cmdlet on all platforms.
date - unless an external program by that name is present in a directory listed in $env:PATH on a given platform - refers to the built in Get-Date cmdlet.
Note: This relies on PowerShell's ill-conceived default verb feature, which falls back on prefixing a command name with Get- if no command form is found by the given name. This shouldn't be relied upon, both in the interest of conceptual clarity and to avoid unnecessary overhead. Also, this fallback is (unexpectedly) not reported by the Get-Command discussed below, and also not by Get-Help - see GitHub issue #3987.
Use Get-Command to determine what command form, if any, a given name refers to (add -All to see if potentially multiple forms exist, with the effective one being listed first).
Related
I am experiencing a rather puzzling error while trying to perform a diff on two files using Visual Studio Code from the command line. I have a text file in the cloud where I save some work related notes. I need to resolve conflicts with other clients editing the file. Usually this only happens during a loss of connection though somehow I find myself having to resolve a lot of them so between this and other uses of diff I will use the usual syntax. It looks something like this:
code --diff "R&D (cloud conflict 2-5-23).txt" "R&D.txt"
My filename happens to have a '&' in it and this command launches the usual 2-way diff in VS Code and reads through the first file name with no problem but doesn't read past the second '&' and the resulting diff tab in VS Code looks something like:
R&D (cloud conflict 2-25-23).txt <-> R
Where the right side "R" doesn't exist. So it would seem '&' needs to be processed literally.
No problem, let's see if backslash \ is an accepted escape parameter...
code --diff "R\&D (cloud conflict 2-5-23).txt" "R\&D.txt"
Nope. Same problem. 🤔 In fact this outputs something even stranger:
Code diff tab:
&D (cloud conflict 2-25-23).txt <-> R
with shell output:
'D.txt' is not recognized as an internal or external command, operable program or batch file.
I also tried the carrot symbol '^' as an escape parameter to a similar effect. I just includes it in the first file and the editor still thinks the second file name is just "R".
The help file for the VS Code command line integration didn't have a lot to say about the --diff parameter other than a short description and I was hoping to get something about processing strings literally or escape characters. Perhaps another parameter that I need or maybe this has more to do with the shell in general.
I find it really strange that it can read the first full file name but breaks at the second '&'. Weirder still that if a supposed escape character is included in the second file name, it will omit that as well. 😵
For now all I can do is rename the file which is a bummer. 🤷♂️ I have VS Code version 1.75.0 on Windows 10 Home latest version/build and I'm using PowerShell version 5.1.19041.2364.
Edit: The issue definitely appears to be PowerShell related as it turns out. I was finally able to run this command successfully in a regular command prompt. (Simply typing "cmd" and Enter into the PowerShell window before running the diff command). Unfortunately, I happen to be running this command as part of PowerShell script. I may have to figure out how to run a CMD command from inside my PowerShell script if that is at all possible. I'm not sure. 🤔 If not, I need to figure out what exactly PowerShell is doing to my command when it reaches the '&' character.
tl;dr
You need a workaround:
cmd /c 'code --diff "R&D (cloud conflict 2-5-23).txt" "R&D.txt"'
Alternatively, using --%, the stop-parsing token:
code --diff "R&D (cloud conflict 2-5-23).txt" --% "R&D.txt"
Note: --% comes with fundamental limitations, notably the inability to reference PowerShell variables - see this answer.
Background information:
The root cause is that code is implemented as a batch file (code.cmd) and that cmd.exe, the interpreter that executes batch file inappropriately parses its list of arguments as if they had been submitted from INSIDE a cmd.exe session.
PowerShell, which - of necessity - has to rebuild the process command line behind the scenes on Windows after having performed argument parsing based on its rules, and - justifiably - places "R&D.txt" as verbatim R&D.txt on the process command line, given that the argument value contains no spaces.
The result is that cmd.exe interprets the unquoted R&D.txt argument on its command line as containing metacharacter &, which is its command-sequencing operator, causing the call to break.
Given that cmd.exe, the legacy Windows shell, is unlikely to receive fixes, the actively maintained PowerShell (Core) 7+ edition could as a courtesy compensate for cmd.exe's inappropriate behavior.
Doing so has been proposed in GitHub issue #15143, but, alas, it looks like these accommodations will not be implemented.
I use WinSCP within a Powershell script. It suddenly stopped working. After a while I could figure out that the problem appeared from a more recent version of PowerShell:
Reduced code:
& winscp `
/log `
/command `
'echo Connecting...' `
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=`"`"ssh-ed25519 includes spaces`"`""
Error message using v7.2.7
Host "lkjhlk.com" does not exist.
Errror message using v7.3.0
Too many parameters for command 'open'.
As you can see with v7.3.0 WinSCP receives different input depending on the version of PS. I found out that the difference has something to do with the spaces in the hostkey. If they are omitted v7.3.0 outputs the same error.
What change to PowerShell caused this, and how can I fix it?
(How can I debug such issues? I played a bit around with escaping, but the strings look the same no matter the version, no obvious breaking change that could be responsible)
Version 7.3.0 of PowerShell (Core) introduced a breaking change with respect to how arguments with embedded " characters are passed to external programs, such as winscp:
While this change is mostly beneficial, because it fixes behavior that was fundamentally broken since v1 (this answer discusses the old, broken behavior), it also invariably breaks existing workarounds that build on the broken behavior, except those for calls to batch files and the WSH CLIs (wscript.exe and cscript.exe) and their associated script files (with file-name extensions such as .vbs and .js).
CAVEAT:
GitHub issue #18694 implies that some version after 7.3.1 and all later versions will make this breaking change opt-in, but on Windows only, for the sake of backward compatibility; that is:
On Windows:
Old workarounds will continue to work by default.
Getting the new, correct behavior requires (temporarily) setting
$PSNativeCommandArgumentPassing = 'Standard'
On Unix-like platforms:
The new, correct behavior ($PSNativeCommandArgumentPassing = 'Standard') will remain the default
Old workarounds will require (temporarily) setting $PSNativeCommandArgumentPassing = 'Legacy' in order to continue to work, as is already the case in 7.3.0
To make existing workarounds continue to work, set the $PSNativeCommandArgumentPassing preference variable (temporarily) to 'Legacy':
# Note: Enclosing the call in & { ... } makes it execute in a *child scope*
# limiting the change to $PSNativeCommandArgumentPassing to that scope.
& {
$PSNativeCommandArgumentPassing = 'Legacy'
& winscp `
/log `
/command `
'echo Connecting...' `
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=`"`"ssh-ed25519 includes spaces`"`""
}
Unfortunately, because winscp.exe only accepts
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces""" on its process command line (i.e., embedded " escaped as ""), and not also the most widely used form
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=\"ssh-ed25519 includes spaces\"" (embedded " escaped as \"), which the fixed behavior now employs, for winscp.exe, specifically, a workaround will continue to be required.
If you don't want to rely on having to modify $PSNativeCommandArgumentPassing for the workaround, here are workarounds that function in both v7.2- and v7.3+ :
Use --%, the stop-parsing token, which, however, comes with pitfalls and severe limitations, notably the inability to (directly) use PowerShell variables or subexpressions in the arguments that follow it - see this answer for details; however, you can bypass these limitations if you use --% as part of an array that you construct and assign to a variable first and then pass via splatting:
# Note: Must be single-line; note the --% and the
# unescaped use of "" in the argument that follows it.
# Only "..." quoting must be used after --%
# and the only variables that can be used are cmd-style
# *environment variables* such as %OS%.
winscp /log /command 'echo Connecting...' --% "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
# Superior alternative, using splatting:
$argList = '/log', '/command', 'echo Connecting...',
'--%', "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
winscp #argList
Alternatively, call via cmd /c:
# Note: Pass-through command must be single-line,
# Only "..." quoting supported,
# and the embedded command must obey cmd.exe's syntax rules.
cmd /c #"
winscp /log /command "echo Connecting..." "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
"#
Note: You don't strictly need to use a here-string (#"<newline>...<newline>"# or #'<newline>...<newline>'#), but it helps readability and simplifies using embedded quoting.
Both workarounds allow you to pass arguments directly as quoted, but unfortunately also require formulating the entire (pass-through) command on a single line - except if --% is combined with splatting.
Background information:
The v7.3 default $PSNativeCommandArgumentPassing value on Windows, 'Windows':
regrettably retains the old, broken behavior for calls to batch files and the WSH CLIs (wscript.exe and cscript.exe) and their associated script files (with file-name extensions such as .vbs and .js).
While, for these programs only, this allows existing workarounds to continue to function, future code that only needs to run in v7.3+ will continue to be burdened by the need for these obscure workarounds, which build on broken behavior.
The alternative, which was not implemented, would have been to build accommodations for these programs as well as some program-agnostic accommodations into PowerShell, so that in the vast majority of case there won't even be a need for workarounds in the future: see GitHub issue #15143.
There are also troublesome signs that this list of exceptions will be appended to, piecemeal, which all but guarantees confusion for a given PowerShell version as to which programs require workarounds and which don't.
commendably, for all other programs, makes PowerShell encode the arguments when it - of necessity - rebuilds the command line behind the scenes as follows with respect to ":
It encodes the arguments for programs that follow the C++ command-line parsing rules (as used by C / C++ / .NET applications) / the parsing rules of the CommandLineToArgv WinAPI function, which are the most widely observed convention for parsing a process' command line.
In a nutshell, this means that embedded " characters embedded in an argument, to be seen as a verbatim part of it by the target program, are escaped as \", with \ itself requiring escaping only (as \\) if it precedes a " but is meant to be interpreted verbatim.
Note that if you set $PSNativeCommandArgumentPassing value to 'Standard' (which is the default on Unix-like platforms, where this mode fixes all problems and makes v7.3+ code never require workarounds), this behavior applies to all external programs, i.e. the above exceptions no longer apply).
For a summary of the impact of the breaking v7.3 change, see this comment on GitHub.
If you have / need to write cross-edition, cross-version PowerShell code: The Native module (Install-Module Native; authored by me), has an ie function (short for: Invoke Executable), which is a polyfill that provides workaround-free cross-edition (v3+), cross-platform, and cross-version behavior in the vast majority of cases - simply prepend ie to your external-program calls.
Caveat: In the specific case at hand it will not work, because it isn't aware that winscp.exe requires ""-escaping.
I wrote a program which prints a string, which contains ANSI escape sequences to make the text colored. But it doesn't work as expected in the default Windows 10 console, as you can see in the screenshot.
The program output appears with the escape sequences as printed characters.
If I feed that string to PowerShell via a variable or piping, the output appears as intended (red text).
How can I achieve that the program prints colored text without any workarounds?
This is my program source (Haskell) - but the language is not relevant, just so you can see how the escape sequences are written.
main = do
let red = "\ESC[31m"
let reset = "\ESC[39m"
putStrLn $ red ++ "RED" ++ reset
Note:
The following applies to regular (legacy) console windows on Windows (provided by conhost.exe), which are used by default, including when a console application is launched from a GUI application.
By contrast, the console windows (terminals) provided by Windows Terminal as well as Visual Studio Code's integrated terminal provide support for VT / ANSI escape sequences by default, for all console applications.
While console windows in Windows 10 do support VT (Virtual Terminal) / ANSI escape sequences in principle, support is turned OFF by default.
You have three options:
(a) Activate support globally by default, persistently, via the registry, as detailed in this SU answer.
In short: In registry key [HKEY_CURRENT_USER\Console], create or set the VirtualTerminalLevel DWORD value to 1
From PowerShell, you can do this programmatically as follows:
Set-ItemProperty HKCU:\Console VirtualTerminalLevel -Type DWORD 1
From cmd.exe (also works from PowerShell):
reg add HKCU\Console /v VirtualTerminalLevel /t REG_DWORD /d 1
Open a new console window for changes to take effect.
See caveats below.
(b) Activate support from inside your program, for that program (process) only, with a call to the SetConsoleMode() Windows API function.
See details below.
(c) Ad-hoc workaround, from PowerShell:
PowerShell (Core) 7+: Enclose external-program calls in (...) (invariably collects all output first before printing):
(.\test.exe)
Streaming Windows PowerShell-only alternative: Pipe output from external programs to Write-Host
.\test.exe | Out-Host
See details below.
Re (a):
The registry-based approach invariably activates VT support globally, i.e., for all console windows, irrespective of what shell / program runs in them:
Individual executables / shells can still deactivate support for themselves, if desired, using method (b).
Conversely, however, this means that the output of any program that doesn't explicitly control VT support will be subject to interpretation of VT sequences; while this is generally desirable, hypothetically this could lead to misinterpretation of output from programs that accidentally produce output with VT-like sequences.
Note:
While there is a mechanism that allows console-window settings to be scoped by startup executable / window title, via subkeys of [HKEY_CURRENT_USR\Console], the VirtualTerminalLevel value seems not to be supported there.
Even if it were, however, it wouldn't be a robust solution, because opening a console window via a shortcut file (*.lnk) (e.g. from the Start Menu or Task Bar) wouldn't respect these settings, because *.lnk files have settings built into them; while you can modify these built-in settings via the Properties GUI dialog, as of this writing the VirtualTerminalLevel setting is not surfaced in that GUI.
Re (b):
Calling the SetConsoleMode() Windows API function from inside the program (process), as shown here, is cumbersome even in C# (due to requiring P/Invoke declarations), and may not be an option:
for programs written in languages from which calling the Windows API is not supported.
if you have a preexisting executable that you cannot modify.
In that event, option (c) (from PowerShell), discussed next, may work for you.
Re (c):
PowerShell automatically activates VT (virtual terminal) support for itself when it starts (in recent releases of Windows 10 this applies to both Windows PowerShell and PowerShell (Core) 7+) - but that does not extend to external programs called from PowerShell, in either edition, as of v7.3.2.
Separately, in v7.2+ there is the $PSStyle.OutputRendering preference variable, which controls whether PowerShell commands produce colored output via the formatting system, such as the colored headers of Get-ChildItem output. However, this setting has no effect on (direct) output from external programs. $PSStyle.OutputRendering defaults to Host, meaning that only formatted output that prints to the terminal (console) is colored. $PSStyle.OutputRendering = 'PlainText' disables coloring, and $PSStyle.OutputRendering = 'Ansi' makes it unconditional; see this answer for more information.
However, as a workaround you can relay an external program's (stdout) output via PowerShell's (success) output stream, in which case VT sequences are recognized:
As of PowerShell (Core) 7.3.2, this only works either by enclosing the call in (...) or by using Out-String, but note that all output is invariably collected first before it is printed.[1]
(.\test.exe)
In Windows PowerShell, in addition to the above, streaming the relayed output is possible too, by piping to Write-Host (Out-Host, Write-Output or Out-String -Stream would work too)
.\test.exe | Write-Host
Note: You need these techniques only if you want to print to the console. If, by contrast, you want to capture the external program's output (including the escape sequences), use $capturedOutput = .\test.exe
Character-encoding caveat: Windows PowerShell by default expects output from external programs to use the OEM code page, as defined by the legacy system locale (e.g., 437 on US-English systems) and as reflected in [console]::OutputEncoding.
.NET console programs respect that setting automatically, but for non-.NET programs (e.g., Python scripts) that use a different encoding (and produce not just pure ASCII output (in the 7-bit range)), you must (at least temporarily) specify that encoding by assigning to [console]::OutputEncoding; e.g., for UTF-8:
[console]::OutputEncoding = [Text.Encoding]::Utf8.
Note that this is not only necessary for the VT-sequences workaround, but generally necessary for PowerShell to interpret non-ASCII characters correctly.
PowerShell Core (v6+), unfortunately, as of v7.3.2, still defaults to the OEM code page too, but that should be considered a bug (see GitHub issue #7233), given that it otherwise defaults to UTF-8 without BOM.
[1] Using Out-String -Stream or its built-in wrapper function, oss, is tempting in order to achieve streaming output, but this no longer works as of PowerShell 7.3.2, possibly due to the optimization implemented in GitHub PR #16612.
Thanks to user mklement0 for making me aware that VT support is not enabled automatically. This made me look in the right direction and I found this helpful post.
So to answer my question: Add or set the registry key
HKCU:\Console - [DWORD] VirtualTerminalLevel = 1
Restart the console and it works.
For Haskell specifically, you need to call the hSupportsANSIWithoutEmulation function to enable ANSI escape sequences in your program on Windows 10.
When I used the command below [1] to set my configuration variable MONGODB_URI, it gives an error [2].
I am using Windows PowerShell.
[1] >> heroku config:set MONGODB_URI='mongodb+srv://myprojectname:<mypassword>#cluster0.rkitj.mongodb.net/<myusername>?retryWrites=true&w=majority'
[2] The system cannot find the file specified.
'w' is not recognized as an internal or external command,
operable program or batch file.
Note: myprojectname, mypassword and myusername are placeholders for the actual value.
It looks like the heroku CLI entry point is a batch file, as implied by the wording of the error messages, which are cmd.exe's, not PowerShell's.
PowerShell doesn't take the special parsing needs of batch files (cmd.exe) into account when it synthesizes the actual command line to use behind the scenes, which involves re-quoting, using double quotes only, and only when PowerShell thinks quoting is needed.
In this case PowerShell does not double-quote (because the value contains no spaces), which breaks the batch-file invocation.
You have the following options:
You can use embedded quoting so as to ensure that the value part of your MONGODB_URI=... key-value pair is passed in double quotes; note the '"..."' quoting:
heroku config:set MONGODB_URI='"mongodb+srv://myprojectname:<mypassword>#cluster0.rkitj.mongodb.net/<myusername>?retryWrites=true&w=majority"'
Caveat: This shouldn't work, and currently only works because PowerShell's passing of arguments to external program is fundamentally broken as of PowerShell 7.1 - see this answer. Should this ever get fixed, the above will break.
If your command line doesn't involve any PowerShell variables and expressions, you can use --%, the stop-parsing symbol, which, however, in general, has many limitations (see this answer); essentially, everything after --% is copied verbatim to the target command line, except for expanding cmd.exe-style environment-variable references (e.g., %USERNAME%):
heroku config:set --% MONGODB_URI="mongodb+srv://myprojectname:<mypassword>#cluster0.rkitj.mongodb.net/<myusername>?retryWrites=true&w=majority"
If you're willing to install a module, you can use the ie function from the PSv3+ Native module (install with Install-Module Native from the PowerShell Gallery in PSv5+), which internally compensates for all of PowerShell's argument-passing and cmd.exe's argument-parsing quirks (it is implemented in a forward-compatible manner so that should PowerShell itself ever get fixed, the function will simply defer to PowerShell); that way, you can simply focus on meeting PowerShell's syntax requirements, and let ie handle the rest:
# 'ie' prepended to an invocation that uses only PowerShell syntax
ie heroku config:set MONGODB_URI='mongodb+srv://myprojectname:<mypassword>#cluster0.rkitj.mongodb.net/<myusername>?retryWrites=true&w=majority'
Is there a syntax checker for conditionals in PowerShell?
This code will skip the conditional
if ($templList -ne $null -and $templList.items.Length > 0) {
$templID=$templList.items[0].id
write-host $templList.items
}
because the '-gt' is substituted with '>'.
Strictly speaking, you're looking for a semantics checker, given that your code is syntactically correct.
The problem here is that while the code was formally correct, it doesn't do what you intended it to do.
PSScriptAnalyzer (PSSA) is a linter for PowerShell, and it is integrated into the PowerShell extension for Visual Studio Code.
It would catch your problem, emitting the following message:
Did you mean to use the redirection operator '>'?
The comparison operators in PowerShell are '-gt' (greater than) or '-ge' (greater or equal).
Use in Visual Studio Code:
Install the PowerShell extension.
Once installed, with your script opened for editing, you'll see the > 0 part underlined, and hovering over it shows the message as a tooltip.
You can see all messages for the current file in the Problems view (Ctrl-Shift-M or, via the menus, View > Problems), which incidentally, will show you that PSSA found additional potential problems with your code snippet.
To configure what rules to apply (which potential problems to warn about), use >PowerShell: Select PSScriptAnalyzer Rules from the command palette.
The rule names, such as PSAvoidGlobalVars, are mostly self-explanatory; the rules documentation describes them (without the PS prefix); and there's also a best practices topic.
Important:
After checking / unchecking (or pressing Enter on) the rules of interest, be sure to press Enter on the Confirm menu item at the top, otherwise your changes won't take effect.
As of v2019.11.0, these selections aren't persisted (are remembered in the current session only) - see this GitHub issue
Note that PSSA also offers automatic (re)formatting of PowerShell code (Alt-Shift-F or, via the command palette, >Format Document).
Stand-alone use:
Install the PSScriptAnalyzer module from the PowerShell Gallery; e.g.:
Install-Module -Scope CurrentUser PSScriptAnalyzer
The module comes with the following cmdlets:
Invoke-ScriptAnalyzer ... lints script files.
Invoke-Formatter ... reformats code passed as a string.
Get-ScriptAnalyzerRule ... lists standard and optionally custom analyzer rules.
Pass your script's path to the Invoke-ScriptAnalyzer cmdlet to perform linting.
With your code, you'd see the following output (having been given a script named pg.ps1):
RuleName Severity ScriptName Line Message
-------- -------- ---------- ---- -------
PSPossibleIncorrectUsageOfRedirecti Warning pg.ps1 1 Did you mean to use the redirection operator '>'? The
onOperator comparison operators in PowerShell are '-gt' (greater than)
or '-ge' (greater or equal).
PSPossibleIncorrectComparisonWithNu Warning pg.ps1 1 $null should be on the left side of equality comparisons.
ll
PSUseDeclaredVarsMoreThanAssignment Warning pg.ps1 2 The variable 'templID' is assigned but never used.
s
PSAvoidUsingWriteHost Warning pg.ps1 3 File 'pg.ps1' uses Write-Host. Avoid using Write-Host
because it might not work in all hosts, does not work when
there is no host, and (prior to PS 5.0) cannot be
suppressed, captured, or redirected. Instead, use
Write-Output, Write-Verbose, or Write-Information.