I am experiencing a rather puzzling error while trying to perform a diff on two files using Visual Studio Code from the command line. I have a text file in the cloud where I save some work related notes. I need to resolve conflicts with other clients editing the file. Usually this only happens during a loss of connection though somehow I find myself having to resolve a lot of them so between this and other uses of diff I will use the usual syntax. It looks something like this:
code --diff "R&D (cloud conflict 2-5-23).txt" "R&D.txt"
My filename happens to have a '&' in it and this command launches the usual 2-way diff in VS Code and reads through the first file name with no problem but doesn't read past the second '&' and the resulting diff tab in VS Code looks something like:
R&D (cloud conflict 2-25-23).txt <-> R
Where the right side "R" doesn't exist. So it would seem '&' needs to be processed literally.
No problem, let's see if backslash \ is an accepted escape parameter...
code --diff "R\&D (cloud conflict 2-5-23).txt" "R\&D.txt"
Nope. Same problem. 🤔 In fact this outputs something even stranger:
Code diff tab:
&D (cloud conflict 2-25-23).txt <-> R
with shell output:
'D.txt' is not recognized as an internal or external command, operable program or batch file.
I also tried the carrot symbol '^' as an escape parameter to a similar effect. I just includes it in the first file and the editor still thinks the second file name is just "R".
The help file for the VS Code command line integration didn't have a lot to say about the --diff parameter other than a short description and I was hoping to get something about processing strings literally or escape characters. Perhaps another parameter that I need or maybe this has more to do with the shell in general.
I find it really strange that it can read the first full file name but breaks at the second '&'. Weirder still that if a supposed escape character is included in the second file name, it will omit that as well. 😵
For now all I can do is rename the file which is a bummer. 🤷♂️ I have VS Code version 1.75.0 on Windows 10 Home latest version/build and I'm using PowerShell version 5.1.19041.2364.
Edit: The issue definitely appears to be PowerShell related as it turns out. I was finally able to run this command successfully in a regular command prompt. (Simply typing "cmd" and Enter into the PowerShell window before running the diff command). Unfortunately, I happen to be running this command as part of PowerShell script. I may have to figure out how to run a CMD command from inside my PowerShell script if that is at all possible. I'm not sure. 🤔 If not, I need to figure out what exactly PowerShell is doing to my command when it reaches the '&' character.
tl;dr
You need a workaround:
cmd /c 'code --diff "R&D (cloud conflict 2-5-23).txt" "R&D.txt"'
Alternatively, using --%, the stop-parsing token:
code --diff "R&D (cloud conflict 2-5-23).txt" --% "R&D.txt"
Note: --% comes with fundamental limitations, notably the inability to reference PowerShell variables - see this answer.
Background information:
The root cause is that code is implemented as a batch file (code.cmd) and that cmd.exe, the interpreter that executes batch file inappropriately parses its list of arguments as if they had been submitted from INSIDE a cmd.exe session.
PowerShell, which - of necessity - has to rebuild the process command line behind the scenes on Windows after having performed argument parsing based on its rules, and - justifiably - places "R&D.txt" as verbatim R&D.txt on the process command line, given that the argument value contains no spaces.
The result is that cmd.exe interprets the unquoted R&D.txt argument on its command line as containing metacharacter &, which is its command-sequencing operator, causing the call to break.
Given that cmd.exe, the legacy Windows shell, is unlikely to receive fixes, the actively maintained PowerShell (Core) 7+ edition could as a courtesy compensate for cmd.exe's inappropriate behavior.
Doing so has been proposed in GitHub issue #15143, but, alas, it looks like these accommodations will not be implemented.
Related
I use WinSCP within a Powershell script. It suddenly stopped working. After a while I could figure out that the problem appeared from a more recent version of PowerShell:
Reduced code:
& winscp `
/log `
/command `
'echo Connecting...' `
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=`"`"ssh-ed25519 includes spaces`"`""
Error message using v7.2.7
Host "lkjhlk.com" does not exist.
Errror message using v7.3.0
Too many parameters for command 'open'.
As you can see with v7.3.0 WinSCP receives different input depending on the version of PS. I found out that the difference has something to do with the spaces in the hostkey. If they are omitted v7.3.0 outputs the same error.
What change to PowerShell caused this, and how can I fix it?
(How can I debug such issues? I played a bit around with escaping, but the strings look the same no matter the version, no obvious breaking change that could be responsible)
Version 7.3.0 of PowerShell (Core) introduced a breaking change with respect to how arguments with embedded " characters are passed to external programs, such as winscp:
While this change is mostly beneficial, because it fixes behavior that was fundamentally broken since v1 (this answer discusses the old, broken behavior), it also invariably breaks existing workarounds that build on the broken behavior, except those for calls to batch files and the WSH CLIs (wscript.exe and cscript.exe) and their associated script files (with file-name extensions such as .vbs and .js).
CAVEAT:
GitHub issue #18694 implies that some version after 7.3.1 and all later versions will make this breaking change opt-in, but on Windows only, for the sake of backward compatibility; that is:
On Windows:
Old workarounds will continue to work by default.
Getting the new, correct behavior requires (temporarily) setting
$PSNativeCommandArgumentPassing = 'Standard'
On Unix-like platforms:
The new, correct behavior ($PSNativeCommandArgumentPassing = 'Standard') will remain the default
Old workarounds will require (temporarily) setting $PSNativeCommandArgumentPassing = 'Legacy' in order to continue to work, as is already the case in 7.3.0
To make existing workarounds continue to work, set the $PSNativeCommandArgumentPassing preference variable (temporarily) to 'Legacy':
# Note: Enclosing the call in & { ... } makes it execute in a *child scope*
# limiting the change to $PSNativeCommandArgumentPassing to that scope.
& {
$PSNativeCommandArgumentPassing = 'Legacy'
& winscp `
/log `
/command `
'echo Connecting...' `
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=`"`"ssh-ed25519 includes spaces`"`""
}
Unfortunately, because winscp.exe only accepts
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces""" on its process command line (i.e., embedded " escaped as ""), and not also the most widely used form
"open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=\"ssh-ed25519 includes spaces\"" (embedded " escaped as \"), which the fixed behavior now employs, for winscp.exe, specifically, a workaround will continue to be required.
If you don't want to rely on having to modify $PSNativeCommandArgumentPassing for the workaround, here are workarounds that function in both v7.2- and v7.3+ :
Use --%, the stop-parsing token, which, however, comes with pitfalls and severe limitations, notably the inability to (directly) use PowerShell variables or subexpressions in the arguments that follow it - see this answer for details; however, you can bypass these limitations if you use --% as part of an array that you construct and assign to a variable first and then pass via splatting:
# Note: Must be single-line; note the --% and the
# unescaped use of "" in the argument that follows it.
# Only "..." quoting must be used after --%
# and the only variables that can be used are cmd-style
# *environment variables* such as %OS%.
winscp /log /command 'echo Connecting...' --% "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
# Superior alternative, using splatting:
$argList = '/log', '/command', 'echo Connecting...',
'--%', "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
winscp #argList
Alternatively, call via cmd /c:
# Note: Pass-through command must be single-line,
# Only "..." quoting supported,
# and the embedded command must obey cmd.exe's syntax rules.
cmd /c #"
winscp /log /command "echo Connecting..." "open sftp://kjhgk:jkgh#lkjhlk.com/ -hostkey=""ssh-ed25519 includes spaces"""
"#
Note: You don't strictly need to use a here-string (#"<newline>...<newline>"# or #'<newline>...<newline>'#), but it helps readability and simplifies using embedded quoting.
Both workarounds allow you to pass arguments directly as quoted, but unfortunately also require formulating the entire (pass-through) command on a single line - except if --% is combined with splatting.
Background information:
The v7.3 default $PSNativeCommandArgumentPassing value on Windows, 'Windows':
regrettably retains the old, broken behavior for calls to batch files and the WSH CLIs (wscript.exe and cscript.exe) and their associated script files (with file-name extensions such as .vbs and .js).
While, for these programs only, this allows existing workarounds to continue to function, future code that only needs to run in v7.3+ will continue to be burdened by the need for these obscure workarounds, which build on broken behavior.
The alternative, which was not implemented, would have been to build accommodations for these programs as well as some program-agnostic accommodations into PowerShell, so that in the vast majority of case there won't even be a need for workarounds in the future: see GitHub issue #15143.
There are also troublesome signs that this list of exceptions will be appended to, piecemeal, which all but guarantees confusion for a given PowerShell version as to which programs require workarounds and which don't.
commendably, for all other programs, makes PowerShell encode the arguments when it - of necessity - rebuilds the command line behind the scenes as follows with respect to ":
It encodes the arguments for programs that follow the C++ command-line parsing rules (as used by C / C++ / .NET applications) / the parsing rules of the CommandLineToArgv WinAPI function, which are the most widely observed convention for parsing a process' command line.
In a nutshell, this means that embedded " characters embedded in an argument, to be seen as a verbatim part of it by the target program, are escaped as \", with \ itself requiring escaping only (as \\) if it precedes a " but is meant to be interpreted verbatim.
Note that if you set $PSNativeCommandArgumentPassing value to 'Standard' (which is the default on Unix-like platforms, where this mode fixes all problems and makes v7.3+ code never require workarounds), this behavior applies to all external programs, i.e. the above exceptions no longer apply).
For a summary of the impact of the breaking v7.3 change, see this comment on GitHub.
If you have / need to write cross-edition, cross-version PowerShell code: The Native module (Install-Module Native; authored by me), has an ie function (short for: Invoke Executable), which is a polyfill that provides workaround-free cross-edition (v3+), cross-platform, and cross-version behavior in the vast majority of cases - simply prepend ie to your external-program calls.
Caveat: In the specific case at hand it will not work, because it isn't aware that winscp.exe requires ""-escaping.
yesterday I had SumatraPDF and VisualStudioCode with latex-workshop working with forward and reverse-search. Today the reverse-search didn't work any more. With a simple bat file I tried to show the commandline arguments.
echo %*
pause
They seemed correct and when I copied the command and paste it into a new cmd it works. To do further testing I tried to direct command and run it.
"C:\....\Code.exe" -g "%1:%2"
pause
Visual Studio Code responds: bad option -g
In SumatraPDF I set cmd as command.
Now the behavior is that I have two cmd windows. With the cmd directly opened the VSC open's the file. With the cmd launched indirect I got the error message.
I have tried resetting the environment variables, changing the current working directory, and checking the code page currently in use.
How can it be that cmd behaves differently with seemingly the same environment? And what can I do to make a cmd started from an application work like a cmd started by windows?
Update: It appears there were a couple of recent security changes in the way VSCODE.exe is allowed to interact with the command line (especially affecting LaTeX-workshop users) so for recent changes twice this year see the discussion at https://forum.sumatrapdfreader.org/t/inverse-search-not-performed-for-vs-code-exe/4486/27
Within SumatraPDF the reverse syntex command for %1 is replaced by %f for file and (l)L for line
It is triggered by a double click near the line of interest and if the synctex index file was compiled correctly by PdfLaTeX (or similar) it will include the tex %f(ilename) and the nearest %l(ine) reference to the point where double clicked.
Thus your tex syctex enhanced "reverse search" call out of SumatraPDF should historically be
"C:\...path to...\Code.exe" -g "%f:%l"
that's Lower L not 1
Avoid using any depreciated -inverse-search parameter from a LaTeX editor just add it once into SumatraPDF-settings.txt and then it's not disturbed by repeated assignments when running your -forward-search.
It will NOT work if the file.synctex or file.synctex.gz is corrupt by a bad PDF compilation.
HOWEVER It seem Microsoft have added the requirement to add a CLI.js handler and requires another switch setting after that ! (see link to discussion in Update above)
For a small test file download https://github.com/GitHubRulesOK/MyNotes/raw/master/AppNotes/SumatraPDF/LATeX%20and%20Reverse-Search.zip unpack and open sync.pdf in SumatraPDF to test that double click on page opens sync.tex in the editor
If the message is cannot start ... then the command line is not configured correctly. A rough test for a bad synctex is to see what happens if the call is changed by adding cmd /k echo to the start, since that will confirm the reverse command. Here I wrote "wrong" as the path to code.exe, once corrected I can remove cmd /k echo.
For some other systems where the reverse might change
see https://github.com/sumatrapdfreader/sumatrapdf/issues/1197#
However there should be no interference in a valid VsCode call.
When I used the command below [1] to set my configuration variable MONGODB_URI, it gives an error [2].
I am using Windows PowerShell.
[1] >> heroku config:set MONGODB_URI='mongodb+srv://myprojectname:<mypassword>#cluster0.rkitj.mongodb.net/<myusername>?retryWrites=true&w=majority'
[2] The system cannot find the file specified.
'w' is not recognized as an internal or external command,
operable program or batch file.
Note: myprojectname, mypassword and myusername are placeholders for the actual value.
It looks like the heroku CLI entry point is a batch file, as implied by the wording of the error messages, which are cmd.exe's, not PowerShell's.
PowerShell doesn't take the special parsing needs of batch files (cmd.exe) into account when it synthesizes the actual command line to use behind the scenes, which involves re-quoting, using double quotes only, and only when PowerShell thinks quoting is needed.
In this case PowerShell does not double-quote (because the value contains no spaces), which breaks the batch-file invocation.
You have the following options:
You can use embedded quoting so as to ensure that the value part of your MONGODB_URI=... key-value pair is passed in double quotes; note the '"..."' quoting:
heroku config:set MONGODB_URI='"mongodb+srv://myprojectname:<mypassword>#cluster0.rkitj.mongodb.net/<myusername>?retryWrites=true&w=majority"'
Caveat: This shouldn't work, and currently only works because PowerShell's passing of arguments to external program is fundamentally broken as of PowerShell 7.1 - see this answer. Should this ever get fixed, the above will break.
If your command line doesn't involve any PowerShell variables and expressions, you can use --%, the stop-parsing symbol, which, however, in general, has many limitations (see this answer); essentially, everything after --% is copied verbatim to the target command line, except for expanding cmd.exe-style environment-variable references (e.g., %USERNAME%):
heroku config:set --% MONGODB_URI="mongodb+srv://myprojectname:<mypassword>#cluster0.rkitj.mongodb.net/<myusername>?retryWrites=true&w=majority"
If you're willing to install a module, you can use the ie function from the PSv3+ Native module (install with Install-Module Native from the PowerShell Gallery in PSv5+), which internally compensates for all of PowerShell's argument-passing and cmd.exe's argument-parsing quirks (it is implemented in a forward-compatible manner so that should PowerShell itself ever get fixed, the function will simply defer to PowerShell); that way, you can simply focus on meeting PowerShell's syntax requirements, and let ie handle the rest:
# 'ie' prepended to an invocation that uses only PowerShell syntax
ie heroku config:set MONGODB_URI='mongodb+srv://myprojectname:<mypassword>#cluster0.rkitj.mongodb.net/<myusername>?retryWrites=true&w=majority'
I am running into a bit of a problem when attempting to use Powershell with NRPE. Now I ran this command,
command[alias_check_commerce_log]=cmd /c echo C:\Program Files\nrpe\plugins\file_checker.ps1; exit($LastExitCode) | powershell.exe -command -
in cmd and it went through without a hitch. Just when I call it via Icinga, for some odd reason, it spits out 'powershell.exeA' is not recognized as an internal or external command. Now I know the A is not supposed to be sitting with powershell.exe, so how would I keep this from happening? Keep in mind that the command is the same in the config as posted here. Checked all text in hopes of it just being a simple Typo. I just can't seem to wrap my head around this one and figure out how to keep this from happening. Any and all help is appreciated, thank you.
Probably file encoding. You likely copy/pasted the command line from a web site or something, and there is a special character after .exe.
Open the file where that command is defined in an editor that supports different encodings and hopefully one that can show whitespace and "special" characters, and inspect the file.
Or Start over and manually type the commands.
I am running this command in powershell:
sqlplus system/passwd#mydb #my_sql
I have tried it with and without backticks and various other versions I found via Google. I keep getting an error when the command is passed off to sqlplus and have been unsucessful in finding the fix. Hopefully someone here can help out?
The error I get is:
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SP2-0042: unknown command " ■#" - rest of line ignored.
So I am sucessfully connecting to the database but there is an extra character being passed to sqlplus in front of the '#'. " ■#" in notepad++ looks like " ¦#"
If you created your SQL command file using a redirect (> or >>) in powershell - like:
myProgram > mySQL.out
and then run it like:
&sqlplus mypw/myuser#mydb.xyz.com "#mySQL.out"
Powershell may have saved the output file in UTF-16 format, which Sqlplus does not like.
(You can confirm by creating the exact same file by hand and then comparing it - byte count will be off and in KDiff you'll get message to the effect that the text is equal, but the files are not binary equal).
To fix - you need to do two things: :
Add some blank lines to the top of your SQL commands - Powershell will still write a BOM (Byte Order Mark) there and it looks like it's pretty hard to get it to avoid that - but sqlplus will just go by it, albeit giving an error - but will move on to the rest of your code OK.
And then run this command in powershell before creating your file: $PSDefaultParameterValues['Out-File:Encoding'] = 'utf8'
See Changing PowerShell's default output encoding to UTF-8
I received this error:
SP2-0042: unknown command " ■S" - rest of line ignored.
and this fixed that - or at least I was able to run it. You can also just cut and past it from one window into another using Notepad++ and that will solve the BOM and encoding issue.
Update Problem Solved. This turned out being "not seeing the trees through the forest". I have been using these sql scripts for several years without issue called from a bash script. When I tried converting the bash script to powershell and ran into issues I blamed it on powershell. However; it turned out there was something corrupt in the sql file itself. There were no obvious errors when looking at the file in notepad++ even with show all symbols clicked and it was ANSI format. I determined it was the sql file itself when I manually ran sqlplus from a cmd window I still had the same error I was getting with powershell. I rewrote the script and saved it and the problem was fixed. I should have manually ran the script on day one and I probably could have resolved sooner.
I had the same problem. My issue was caused because the script file was saved as unicode. I don't know if this will help you or not, but here is how I fixed it:
Edit the script with notepad. Click File -> Save As. Change type from Unicode (or whatever) to ANSI, and save.
A couple of suggestions
Try the invoke operator:
&sqlplus system/passwd#mydb #my_sql
Try start-process:
start-process -NoNewWindow -FilePath sqlplus -ArgumentList #"
system/passwd#mydb #my_sql
"#
I had typical problem. The message was:
unknown command "and" - rest of line ignored.
The reason was an empty string in code.
e.g.
select ...
from ...
where ...
[empty string]
and ... < here was an error message
use as following
sqlplus -s system/passwd#mydb "#my_sql";