Why Unicode codepoints are needed to use AHK hotstrings in WSL - unicode

AutoHotKey cannot insert numbers on the WSL unless I use codepoints
I would like to use python3 every time I use pipenv. For that, I need to insert: pip --python /usr/bin/python3 etc.. However, I don't want to type --python /usr/bin/python3 every time, and therefore I have automated that part with an AutoHotKey script.
The following line works fine when I type the hotstring in any place such as a Notepad file or a path bar:
::pipenv::pipenv --python /usr/bin/python3
When I type pipenv in, say, Notepad, the string pipenv --python /usr/bin/python3 is inserted.
However, if I type it on the WSL (Windows Subsytem for Linux), then I get:
pipenv 66python LusrLbinLpython·
I get the same result in CMD, but surprisingly not in PowerShell (where I get the expected output).
To make it work on the command line, I had to re-write my script using hexadecimal codepoints (or entity references, not sure what the proper name is) instead of the characters themselves:
::pipenv::pipenv {U+2D}{U+2D}python {U+2F}usr{U+2F}bin{U+2F}python{U+33}
That way the output is the expected pipenv --python /usr/bin/python3 both on the command line and in any other environment. I'm surprised that this is the case, because those characters (-, / and 3) are normal ASCII characters, so I don't understand why using codepoints is necessary.
The AutoHotKey script contains UTF-8 Unicode (with BOM) text, with CRLF line terminators.
In a nutshell, I have managed to make the script work. The reason for opening this question is to understand why this happens. Thanks.

Related

code --diff fails when filename contains an ampersand '&'

I am experiencing a rather puzzling error while trying to perform a diff on two files using Visual Studio Code from the command line. I have a text file in the cloud where I save some work related notes. I need to resolve conflicts with other clients editing the file. Usually this only happens during a loss of connection though somehow I find myself having to resolve a lot of them so between this and other uses of diff I will use the usual syntax. It looks something like this:
code --diff "R&D (cloud conflict 2-5-23).txt" "R&D.txt"
My filename happens to have a '&' in it and this command launches the usual 2-way diff in VS Code and reads through the first file name with no problem but doesn't read past the second '&' and the resulting diff tab in VS Code looks something like:
R&D (cloud conflict 2-25-23).txt <-> R
Where the right side "R" doesn't exist. So it would seem '&' needs to be processed literally.
No problem, let's see if backslash \ is an accepted escape parameter...
code --diff "R\&D (cloud conflict 2-5-23).txt" "R\&D.txt"
Nope. Same problem. 🤔 In fact this outputs something even stranger:
Code diff tab:
&D (cloud conflict 2-25-23).txt <-> R
with shell output:
'D.txt' is not recognized as an internal or external command, operable program or batch file.
I also tried the carrot symbol '^' as an escape parameter to a similar effect. I just includes it in the first file and the editor still thinks the second file name is just "R".
The help file for the VS Code command line integration didn't have a lot to say about the --diff parameter other than a short description and I was hoping to get something about processing strings literally or escape characters. Perhaps another parameter that I need or maybe this has more to do with the shell in general.
I find it really strange that it can read the first full file name but breaks at the second '&'. Weirder still that if a supposed escape character is included in the second file name, it will omit that as well. 😵
For now all I can do is rename the file which is a bummer. 🤷‍♂️ I have VS Code version 1.75.0 on Windows 10 Home latest version/build and I'm using PowerShell version 5.1.19041.2364.
Edit: The issue definitely appears to be PowerShell related as it turns out. I was finally able to run this command successfully in a regular command prompt. (Simply typing "cmd" and Enter into the PowerShell window before running the diff command). Unfortunately, I happen to be running this command as part of PowerShell script. I may have to figure out how to run a CMD command from inside my PowerShell script if that is at all possible. I'm not sure. 🤔 If not, I need to figure out what exactly PowerShell is doing to my command when it reaches the '&' character.
tl;dr
You need a workaround:
cmd /c 'code --diff "R&D (cloud conflict 2-5-23).txt" "R&D.txt"'
Alternatively, using --%, the stop-parsing token:
code --diff "R&D (cloud conflict 2-5-23).txt" --% "R&D.txt"
Note: --% comes with fundamental limitations, notably the inability to reference PowerShell variables - see this answer.
Background information:
The root cause is that code is implemented as a batch file (code.cmd) and that cmd.exe, the interpreter that executes batch file inappropriately parses its list of arguments as if they had been submitted from INSIDE a cmd.exe session.
PowerShell, which - of necessity - has to rebuild the process command line behind the scenes on Windows after having performed argument parsing based on its rules, and - justifiably - places "R&D.txt" as verbatim R&D.txt on the process command line, given that the argument value contains no spaces.
The result is that cmd.exe interprets the unquoted R&D.txt argument on its command line as containing metacharacter &, which is its command-sequencing operator, causing the call to break.
Given that cmd.exe, the legacy Windows shell, is unlikely to receive fixes, the actively maintained PowerShell (Core) 7+ edition could as a courtesy compensate for cmd.exe's inappropriate behavior.
Doing so has been proposed in GitHub issue #15143, but, alas, it looks like these accommodations will not be implemented.

/usr/local/bin/perl5: bad interpreter: Permission denied

I have a unix command
(script) which has a nested perl script in it.
when i run this unix command from command line it works fine.
If I am running same command from a tcl file using exec, i am getting following error:
'sh: /cmdpath/cmd.pl: /usr/local/bin/perl5: bad interpreter: Permission denied'
Any Idea what could be causing this. My tcl code is trying to execute this command several times ( more than 100 times).
Thanks
Ruchi
Almost certainly your Perl script is encoded in DOS/Windows line-ending format, which uses \r\n to terminate lines. Since Unix terminates lines with \n only, the \r is interpreted as belonging to the executable name, so that the kernel tries to run a program named perl5\r and fails.
Deleting the trailing \r on this line should fix the problem.
Alternatively, it may be that the perl5 executable either does not exist at the given path, or exists but lacks the execute permission bit. If you have this executable living somewhere else in the filesystem, update the path on the first line of the script to point to it. To fix the latter problem, run
chmod +x /usr/local/bin/perl5
You will need to be root to do this.
Given the output you are showing, you are likely executing "sh cmd.pl". In turn, sh is trying to execute the perl interpreter.
Why not spawn "/usr/local/bin/perl5 cmd.pl" directly, this will be more efficient, especially if you are doing that hundreds of time.

Passing arguments to script, getting: "Use of uninitialized value $ARGV[0]"

Problem: Windows XP is not passing command line arguments to perl scripts.
Symptom: a simple command like:
say "Argument 1 (\$ARGV[0]) is: $ARGV[0], argument 2 (\$ARGV[1]) is: $ARGV[1].";
Resulted in:
Use of uninitialized value $ARGV[0] in concatenation (.) or string at...
Solution:
The root problem is in Windows XP. The default method for starting perl passes only the first variable, which is the script name. Result is that $ARGV[0] is uninitialized.
The fix is to edit the Windows registry at:
\HKEY_CLASSES_ROOT\Perl\shell\Open\command
And make the entry:
"C:\Perl\bin\perl.exe" %*
Result is:
C:\whatever>perl argtest.pl 1 2
Argument 1 ($ARGV[0]) is: 1, argument 2 ($ARGV[1]) is: 2.
Thanks especially to David W who pointed me in the right direction.
Note that #ARGV in Perl is not quite like argv in C.
C Perl
Name of the program argv[0] $0
1st argument argv[1] $ARGV[0]
2nd argument argv[2] $ARGV[1]
n-th argument argv[n] $ARGV[n-1]
So if you provide one command line arguments to a Perl script, it will be found in $ARGV[0]. $ARGV[1] will be uninitialized.
Download Cygwin and test your code in the Cygwin environment. I bet this is a Windows issue. (If you're a Unix head, you'll like Cygwin because it gives you the Unix/Linux like environment on your Windows machine. I don't use Windows without it.)
Windows uses suffixes to determine what program opens what files. Is your Perl script called argtest, argtest.bat or argtest.pl?
On Windows, make sure all of your Perl scripts use the *.pl suffix, so Windows will use whatever Perl parameter to execute them. Windows doesn't use the shebang.
Another possible issue: On Windows XP, I had a problem with Perl scripts with parameters because Windows had this as an execution string:
perl %1
which would execute the Perl program with my script, but ignore the parameters. I had to change this to:
perl %*
Unfortunately, Windows Vista through Windows 8 changed the way this is set. However, I have Windows 7 and don't have this issue. I did make sure to install Perl under C:\Perl and not C:\Program Files\Perl because of the space in the directory name. I also have Strawberry Perl installed.
There is a special Windows environment variable called PATHEXT. This allows you to type in foo instead of foo.pl. If Windows can't see how to execute your file, Windows goes through %PATHEXT and tries appending various suffixes until it finds one that works. You might want to append .PL to that environment variable, so you can type in foo instead of foo.pl all the time.
There are two ways that Windows knows it is supposed to use Perl to execute a program.
The command line begins with the perl executable and the name of the script to be run is provided as a command line argument. This is how it works on Unix and other environments, too.
Your system associates one or more extensions, such as .pl, .pm, and/or .cgi with the Perl application, and Windows will launch Perl when you type a filename with one of those extensions or click on a file with one of those extensions in the Windows explorer.
You have invoked your script simply as
argtest 1 2
and not one of
perl argtest 1 2
argtest.pl 1 2
That makes me think that Perl is not the first application that gets to look at the file referred to by argtest. Perhaps there is a file called argtest.bat or argtest.exe which has the task of getting Perl to run your Perl code. For some reason this intermediate program is not passing the command line arguments that you provide on to the Perl application.
Provide the code for this intermediate file and we can help some more.
UPDATE: David W proposes a 3rd way -- setting the PATHEXT environment variable to inclue .pl files and invoking argtest from the command-line -- see his answer.
Then if Window's file association with the .pl extension was messed up, say, set as just C:\Dwimperl\perl\bin\perl" and not "C:\Dwimperl\perl\binperl %*" then the OP would get the behavior that he describes.
In perl file name is not part of the array of arguments. At least in the test I did you must remove ARGV[2] or pass three arguments, like argtest 1 2 3

replace lf with cr-lf in text file Cygwin

I've got a Cygwin script that outputs stuff to a file. My only problem is that it always uses the \n "lf" line endings instead of \r\n "cr-lf". I want to either replace them or just get cygwin to always output \r\n for line endings.
I've tried the following two commands to no avail within the script.
unix2dos d:/temp.txt
sed -i -e 's/\n/\r\n/g' d:/temp.txt
Your unix2dos call should work. Are you getting some kind of error?
The Cygwin installer has an option for selecting the default line ending convention (DOS or Unix), although it might only show it during a new installation -- I don't remember offhand.
You're specifying the path to unix2dos using a bizarre mix of Windows and Linux file endings.
If you're calling it from a Windows command shell, use this:
unix2dos d:\temp.txt
If you're calling it from within Cygwin, or a Cygwin shell script or similar, use this:
unix2dos /cygdrive/d/temp.txt

Script showing bad interpreter and will not run

I was sent a perl script over mail and asked to run it .I placed it on my local drive as is but when I tried to run the script it shows me
/usr/bin/perl^M: bad interpreter: No such file or directory
I checked and usr/bin/ does have perl in there .I am not sure what is wrong.I checked a bit and looks like I am missing some spaces or something ..I tried adding those at the end of
usr/bin/perl and at the end of the file but that didnt help either.
I even tried to use dos2unix
dos2unix oldfile newfile
'dos2unix' not found.This is on MacOSX.
Might I also mention that I am sshing into my mac using my windows machine at home.
You're on the right track. Your script has DOS style newlines at the end, which is not supported by your kernel.
The solution is to use something to convert the DOS newlines to Unix style. dos2unix would presumably work if you had it, so use something else equivalent.
In the absence of dos2unix, you can use tr (on Mac OS X) to strip the DOS / Windows new-lines:
tr -d '\r' < old.pl > new.pl
This will solve the "bad interpreter" issue.
"Can't locate Gpu.pm in #INC" is a different issue. Either you don't have Gpu.pm installed on your Mac (or whichever computer on which you are running this, I'm confused by your comments) or it's not in your include path. I don't who what that script is or what it does. A quick look on http://search.cpan.org/ revealed nothing.
If you can get that Perl module (presumably from whoever supplied oldfile), you'll have to ensure it is in #INC.
Do this in vim:
:%s/^M//g
save the file and try running it again
execute: vim
when vim opens go to command mode by hitting the escape key .... at the command prompt (:) type: %s/^M//g. This will remove all "^M" characters from the file.
dos2unix in Perl:
perl -pi -e 'tr/\r//d' file.txt