My python file:
print('Amanhã')
I am using the integrated terminal in VSCode 1.28.1, on Windows 10 Pro.
When I activate a Python 3.6-based virtual environment then run this script, it executes as expected and I see Amanhã in the terminal.
But when I activate a Python 3.5-based virtual environment then run this script, it fails with a UnicodeEncodeError:
UnicodeEncodeError: 'charmap' codec can't encode character '\xe3' in position 5: character maps to <undefined>
If I run set PYTHONIOENCODING=utf8 in the 3.5-based environment, then execute the script, the Unicode error is gone but the output is not exactly as expected: Amanhã
How can I see Amanhã in the 3.5-based venv?
(I replicated this in the normal Windows terminal (cmd.exe), not inside VSCode -- exact same result. I also will note that sys.getdefaultencoding() returns utf-8 both before and after the set PYTHONIOENCODING=utf8 command)
Based on the incorrect output, your terminal is using cp437, which doesn't support the character ã.
Pre-Python 3.6, Python encodes Unicode to the encoding of the terminal on Windows. As of Python 3.6, Python uses Unicode Win32 APIs when writing to the terminal and, as you have found, works much better.
If you must use Python 3.5, check out win-unicode-console.
Related
I want to build an interactive story with Python to be printable and runable in both the windows terminal and the Spyder control. To adjust and justify the text I need the dimensions (mainly the width) of the console in character lines.
I have tried using os.get_terminal_size() but that only works for the windows terminal, not the Spyder console.
Syper version = 5.4.0
Python version = 3.9
Powershell ignores tab characters in copied-and-pasted code snippets. This is annoying because I like to write my Python code with single tabs instead of quadruple spaces, and in testing I frequently copy and paste snippets in to test them.
So far I've been using other terminals like Cygwin that properly address pasted tabs, but it would be more convenient for me to use Powershell. How can it be made to address these pasted tabs correctly?
The MWE is to copy and paste anything with a tab into Powershell. For example,
test
appears in Powershell as
>>> test
with no indentation. I've copied and pasted from both Notepad and Notepad++ with the same effect; I'm not using any formatted word processors or anything like that.
Caveat: The next section only applies to PowerShell's own interactive prompt. Different rules may apply to external programs launched from PowerShell that present their own interactive prompt, such as python and, preferably, ipython - see the bottom section for considerations specific to these programs.
Preserving tabs on pasting directly into PowerShell's interactive prompt:
Note: Strictly speaking, when it does work, tabs are converted to 4 spaces each on pasting.
Only works when all of the following prerequisites are met:
When running PowerShell (Core) 7+ (as opposed to Windows PowerShell)
When pasting via Ctrl-V rather than via right-click (the latter emulates typing and therefore triggers tab completion).
When running in one of the following console environments:
Regular console window (conhost.exe)
Windows Terminal
Note:
Does not work in Visual Studio Code's integrated terminal (which seemingly always emulates typing in PowerShell).
Haven't tried third-party consoles such as Cygwin and ConEmu.
Notably, this categorically excludes Windows PowerShell (where a tab ends up as ^I).
Preserving tabs on pasting into the interactive python / ipython REPL, from PowerShell or cmd.exe:
Note:
ipython provides a superior REPL experience compared to python.
Unlike the above, the following also applies to launching from Windows PowerShell, as well as from cmd.exe.
When it works, pasted tabs are converted to 4 spaces, except where noted otherwise.
ipython
Regular console windows (conhost.exe):
Works, with Ctrl-V only.
Windows Terminal:
Works, with Ctrl-V only.
Visual Studio Code's integrated terminal:
Does NOT work (strips tabs).
python
Regular console windows (conhost.exe):
Works, with both Ctrl-V and right-click, as long as Filter clipboard contents on paste is turned OFF on the Options tab of the console window's Properties dialog; pastes actual tabs.
Windows Terminal:
Works, with right-click only(!); pastes actual tabs.
Visual Studio Code's integrated terminal:
Works.
The only way I was able to make this work was by using Windows Terminal. Which is a new terminal application built by Microsoft that supports a bunch of new features.
I don't know the technical reasons why, but it appears it supports pasting both tabs and spaces appropriately, without loss or conversion into the Python CLI REPL.
I tested this using Windows Terminal while running Windows PowerShell, PowerShell Core 7+ and Command Prompt, while running the Python CLI and all supported pasting tabs.
You can install Windows Terminal through a number of sources...Windows Store, Binary download, WinGet, Choco, etc.
Here's the github repo with instructions on various ways to install.
https://github.com/microsoft/terminal
In general, control-v is better for pasting characters than right-click. Note control-v has to be bound to psreadline's paste function (so no emacs mode). Here's an example with an a and an em dash (0x2013). Right click will only paste the a.
a–
Windows 10 Jetbrains Clion 2018.2.1
MinGW-W64 Encoding
input[1] :
std::cout << "가나다라 abc" << std::endl;
output[1] :
媛?섎떎??abc
Settings > Editor > File Encodings
I can solve with Path setting: EUC-KR, but only configured file only. I have to configure Encoding setting every each project, every each file.
I found when using CMD, chcp 65001 works, but clion's cpp output cannot manage like that.
VM option :
-Dconsole.encoding=EUC_KR
-Dconsole.encoding=EUC-KR
-Dconsole.encoding=UTF8
-Dconsole.encoding=UTF-8
-Dfile.encoding=EUC_KR
-Dfile.encoding=EUC-KR
-Dfile.encoding=UTF8
-Dfile.encoding=UTF-8
I tried them all, each by each, but they does not work.
cygwin basically works well, but mingw-w64 doesn't work well. I searched for hours, but I couldn't find any answer.. :(
Thank you for reading..
Unfortunately, the Windows console only supports standard characters. If you try to print a special character, it will appear abnormally.
Windows cmd uses a simple ASCII table, while Windows uses an extended ANSI table. The 128 first characters are identical, so you can use them only.
In Eclipse Luna (4.0.4) / Python 3.4.1 I can't get the PyDev 3.6.0 console to work with Unicode despite having tried several recommended corrective steps.
I attempted this example in Dive Into Python 3: http://www.diveintopython3.net/files.html
If I do the example's a_file.read(), the PyDev console displays:
'Dive Into Python \u662f\u4e3a\u6709\u7ecf\u9a8c......' instead of 'Dive Into Python 是为有经验的程序员编写的一本...'
If I paste the Chinese characters into the PyDev console ala cstring = '是为有经验的程序员编写的一本', then in the console type "cstring ENTER', I again get '\u662f\u4e3a\u6709\u7ecf\u9a8c...'
If I then try print(cstring), I get the following error:
Traceback (most recent call last):
File "", line 1, in
File "C:\Python34\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-13: character maps to
I don't know what do do with that information. The cp1252.py bit is at the heart of my confusion because I've told Eclipse and PyDev to use UTF-8 in every place I can find to do that, beginning with the information in the following:
Printing Unicode in eclipse Pydev console and in Idle
However, I cannot modify "site.py" as described in those steps because [EclipseInstallDir]/lib/site.py does not contain "encoding = ".
I can't figure out what to do next.
For regular runs:
The problem there is that the PYTHONIOENCODING is overridden by PyDev based on the encoding you used in the run configuration (which is the encoding used by the console view too -- so, it'd make no sense having a PYTHONIOENCODING with a value and the allocated console in the PyDev side with another value).
You can change the encoding you're using for a run configuration in:
Run > Run configurations > common > encoding.
By default it'll use the encoding of the file being launched (you can change the default encoding for the workspace in general > workspace > text file encoding).
For interactive console runs:
For interactive console runs there's currently no API to change the encoding in the java side, so, one has to start Eclipse itself in a VM that uses UTF-8 by default... In practice, this means that you have to add:
-Dfile.encoding=UTF-8
to the vmargs of Eclipse in eclipse.ini (and in this case it's also recommended to set the PYTHONIOENCODING environment variable in the related interpreter and set it to UTF-8).
Found it! (At least for Python 3.4.1, Eclipse Luna 4.0.4, PyDev 3.6.0.)
In Eclipse, in Preferences —> PyDev —> Interpreters —> Python Interpreter, in the Environment tab, I added the environment variable PYTHONIOENCODING and specified its value as utf-8.
The PyDev Interactive Console now properly displays the aforementioned Chinese characters (and Thai characters, too).
I have been trying to force Python 2.7.4 to run in 32-bit mode on OS X 10.8.3 by default. I can get Python to run in 32-bit mode by running python32. But when I try to configure Eclipse+PyDev to run Python, it ends up running in 64-bit mode.
I have also tried
export VERSIONER_PYTHON_PREFER_32_BIT=yes
and
defaults write com.apple.versioner.python Prefer-32-Bit -bool yes
but neither seem to have any effect on my Python, and it still runs in 64-bit.
I was wondering if anyone has any idea how to permanently run Python 2.7 in 32-bit mode. Alternately, I am open to any solution that makes Eclipse/PyDev use the 32-bit Python.
My Python has been installed in /Library/Frameworks/Python.framework/Versions/2.7 and sys.executable returns /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python on my computer.
Thanks.
This made python go to 32-bit:
arch -i386 /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
import ctypes
print ctypes.sizeof(ctypes.c_voidp)
output -> 4
but I had the same issue as you that couldn't setup a permanent solution with environment flags as before.