Can't make Ant write proper version info with unicode (c) character - unicode

After upgrading ant from 1.6 to 1.8.3 version info resources of Windows .dlls that are built with Ant became corrupted.
Previously this value was properly saved to the version-info resource:
product.copyright=\u00a9 Copyright 20xx-20xx yyyyyyyyyy \u2122 (so (c) and TM symbols were properly displayed).
After upgrading Ant default encoding was changed to UTF-8 which is expected, but currently Copyright string looks like this:
© Copyright 20xx-20xx yyyyyy ™
This is not a console issue - I checked with hex editor and File Properties dialog - both display it incorrectly.
Looking at file's hexdump I see that following (obviously incorrect) mapping occurs
\u00a9 -> 0x00c2 0x00a9
\u2122 -> 0x00e2 0x201e 0x00a2
The problem here is that Ant encodes UTF-8 bytes (not Unicode string) into 16-bit characters and writes it to version-info.
Although this looks like a bug in ant, I would ask if anyone managed to find any workarounds for this or similar problems.
Here are some snippets from the script:
Project properties file:
...
product.copyright=(c) Copyright 2005-2012 Clarabridge
....
Files included into build.xml:
<versioninfo id="current-version" if="is-windows"
fileversion="${product.version}"
productversion="${product.version}"
compatibilityversion="1"
legalcopyright="${product.copyright}"
companyname="${product.company}"
filedescription="${ant.project.name}"
productname="${ant.project.name}"
/>
...
<cc objdir="${target.dir}/${target.platform}/obj"
outfile="${target.dir}/${target.platform}/${ant.project.name}"
subsystem="other"
failonerror="true"
incremental="false"
outtype="shared"
runtime="dynamic"
>
<versioninfo refid="current-version" />
<compiler refid="compiler-shared-${target.platform}" />
<compiler refid="rc-compiler" />
<linker extends="linker-${target.platform}">
<libset dir="${target.dir}/${target.platform}/lib" libs="${lib.list}" />
</linker>
<fileset dir="${src.dir}" casesensitive="false">
<include name="*.cpp"/>
</fileset>
</cc>

Your bug is that something is misinterpreting the UTF-8 characters as 8-bit ones!!!
BTW, Java doesn’t use 16-bit characters; that would be UCS-2. Java uses UTF-16, which is just as much a variable-width encoding as UTF-8 is. Distressing how many Java programmers screw this up!
UTF-8 has 8-bit code units where UTF-16 has 16-bit code units; neither one supports an “8-bit character” or a “16-bit character”. If you catch yourself writing code that thinks they do, you’ve just written buggy code.
Your output is the result of erroneously displaying UTF-8 as though it were in Latin1, which does use 8-bit characters. You, however, do not.

Related

Ant Jar task corrupts manifest encoding

Per Jar specification manifest encoding has to be UTF-8.
In some scenario (eg merge), manifests produced by ant's jar task got corrupted and special chars would be double encoded.
Original manifest (utf-8):
...
Application-Name: spécial
...
Final manifest (utf-8) after beeing processed by ant's jar task:
...
Application-Name: spécial
...
Jar tasks beeing able to process file-sets allows the developper to specify the original manifest character encoding.
Unfortunately, although the mandatory (final) encoding is utf-8 there is no default in ant's jar task and then the original manifest processing is relying on the platform default... Windows-1252 in my case where the original manifest (coming from another jar) is truly in utf-8
Solution : specify the encoding in the task attribute
<jar destfile="final.jar" filesetmanifest="merge" manifestencoding="UTF-8">
<zipfileset src="original.jar">
[...]
</zipfileset>
</jar>
I've just found my old bugreport about this for NetBeans.
As a workaround, I added manifestEncoding="${source.encoding}"
attribute to the copylibs tag in build-impl.xml

Character encoding annoyance

I am suddenly getting a character encoding error when trying to run ant build-all. I have been in the properties for my project, and choose UTF-8 under resource. Still I'm getting the following error (actually there are more than 100 encoding errors) when trying to build-all:
error: unmappable character for encoding UTF8
[javac] // nedenfor inden f�rste angreb, s� total = 41 = tur 21
I cannot commit my project because of this error. Any idea how to fix this? And it just all of a sudden started complaining about encoding
Sounds like you need to specify the encoding argument on your Ant javac task:
<javac encoding="UTF-8" ...
Ant tasks do not know about Eclipse project properties.

Unicode characters in Javadoc [duplicate]

I am trying to generate Java documentation in Eclipse. The source files are UTF-8 encoded and contain some umlauts. The resulting HTML files do not specify an encoding and do not use HTML entities, so the umlauts aren't displayed correctly in any browser.
What can I do to change this?
Modified from Eclipse javadoc in utf-8:
Project -> Generate Javadoc -> Next -> on the last page, in Extra Javadoc options write:
-encoding UTF-8 -charset UTF-8 -docencoding UTF-8
See the -charset, -encoding and -docencoding flags for the javadoc command.
-encoding specifies the input encoding
-docencoding specifies the output encoding
-charset makes javadoc include a meta tag with encoding info
If you generate your javadoc with an ant task and you use UTF-8 you can do:
<javadoc encoding="UTF-8" charset="UTF-8" docencoding="UTF-8" sourcepath="yoursources" destdir="yourdocdir" />
When generating the javadoc with Gradle add the following to your build.gradle file:
javadoc {
options.encoding = 'UTF-8'
options.docEncoding = 'UTF-8'
options.charSet = 'UTF-8'
}

Unable to gather python interpreter info for pydev

OS: Windows 7 x64
Python: 2.7.3
Pydev: 2.7.1.2012100913
Eclipse: Version: Juno Service Release 1 Build id: 20121004-1855
I am unable to configure a python interpreter in Pydev. If I try using the auto-configuration or the manual configuration I get the same type of error output:
Unable to gather the needed info from the system.
This usually means that your interpreter is not in the system PATH. No
output was in the standard output when trying to create the
interpreter info. The error output contains:>><<
or
See error log for details. No output was in the standard output when
trying to create the interpreter info. The error output contains:>><<
Python is in my PATH. Here is the output of interpreterInfo.py (stdout and stderr):
<xml>
<version>2.7</version>
<executable>C:\Python27\python.exe</executable>
<lib path="out">D:\eclipse\plugins\org.python.pydev_2.7.1.2012100913\pysrc</lib>
<lib path="ins">C:\Python27\lib\site-packages\setuptools-0.6c11-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\logilab_common-0.58.0-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\logilab_astng-0.23.1-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\pip-1.1-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\pylint-0.25.1-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\rope-0.9.3-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\sphinx-1.1.3-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\docutils-0.9.1-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\jinja2-2.6-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\pygments-1.5-py2.7.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\lxml-2.2.8-py2.7-win32.egg</lib>
<lib path="ins">C:\Python27\lib\site-packages\spyder-2.0.10-py2.7.egg</lib>
<lib path="out">C:\Windows\system32\python27.zip</lib>
<lib path="ins">C:\Python27\DLLs</lib>
<lib path="ins">C:\Python27\lib</lib>
<lib path="ins">C:\Python27\lib\plat-win</lib>
<lib path="ins">C:\Python27\lib\lib-tk</lib>
<lib path="ins">C:\Python27</lib>
<lib path="ins">C:\Python27\lib\site-packages</lib>
<lib path="ins">C:\Python27\lib\site-packages\PIL</lib>
<lib path="ins">C:\Python27\lib\site-packages\win32</lib>
<lib path="ins">C:\Python27\lib\site-packages\win32\lib</lib>
<lib path="ins">C:\Python27\lib\site-packages\Pythonwin</lib>
<lib path="ins">C:\Python27\lib\site-packages\wx-2.8-msw-ansi</lib>
<forced_lib>__builtin__</forced_lib>
<forced_lib>__main__</forced_lib>
<forced_lib>_ast</forced_lib>
<forced_lib>_bisect</forced_lib>
<forced_lib>_codecs</forced_lib>
<forced_lib>_codecs_cn</forced_lib>
<forced_lib>_codecs_hk</forced_lib>
<forced_lib>_codecs_iso2022</forced_lib>
<forced_lib>_codecs_jp</forced_lib>
<forced_lib>_codecs_kr</forced_lib>
<forced_lib>_codecs_tw</forced_lib>
<forced_lib>_collections</forced_lib>
<forced_lib>_csv</forced_lib>
<forced_lib>_functools</forced_lib>
<forced_lib>_heapq</forced_lib>
<forced_lib>_hotshot</forced_lib>
<forced_lib>_io</forced_lib>
<forced_lib>_json</forced_lib>
<forced_lib>_locale</forced_lib>
<forced_lib>_lsprof</forced_lib>
<forced_lib>_md5</forced_lib>
<forced_lib>_multibytecodec</forced_lib>
<forced_lib>_random</forced_lib>
<forced_lib>_sha</forced_lib>
<forced_lib>_sha256</forced_lib>
<forced_lib>_sha512</forced_lib>
<forced_lib>_sre</forced_lib>
<forced_lib>_struct</forced_lib>
<forced_lib>_subprocess</forced_lib>
<forced_lib>_symtable</forced_lib>
<forced_lib>_warnings</forced_lib>
<forced_lib>_weakref</forced_lib>
<forced_lib>_winreg</forced_lib>
<forced_lib>array</forced_lib>
<forced_lib>audioop</forced_lib>
<forced_lib>binascii</forced_lib>
<forced_lib>cPickle</forced_lib>
<forced_lib>cStringIO</forced_lib>
<forced_lib>cmath</forced_lib>
<forced_lib>datetime</forced_lib>
<forced_lib>errno</forced_lib>
<forced_lib>exceptions</forced_lib>
<forced_lib>future_builtins</forced_lib>
<forced_lib>gc</forced_lib>
<forced_lib>imageop</forced_lib>
<forced_lib>imp</forced_lib>
<forced_lib>itertools</forced_lib>
<forced_lib>marshal</forced_lib>
<forced_lib>math</forced_lib>
<forced_lib>mmap</forced_lib>
<forced_lib>msvcrt</forced_lib>
<forced_lib>nt</forced_lib>
<forced_lib>operator</forced_lib>
<forced_lib>parser</forced_lib>
<forced_lib>signal</forced_lib>
<forced_lib>strop</forced_lib>
<forced_lib>sys</forced_lib>
<forced_lib>thread</forced_lib>
<forced_lib>time</forced_lib>
<forced_lib>xxsubtype</forced_lib>
<forced_lib>zipimport</forced_lib>
<forced_lib>zlib</forced_lib>
</xml>Traceback (most recent call last):
File "D:\eclipse\plugins\org.python.pydev_2.7.1.2012100913\pysrc\interpreterInfo.py", line 199, in <module>
raise RuntimeError('Ok, this is so that it shows the output (ugly hack for some platforms, so that it releases the output).')
RuntimeError: Ok, this is so that it shows the output (ugly hack for some platforms, so that it releases the output).
Any help would be great, thanks!
I did not expect such a problem: (PyDev worked on 27 64-bit PCs under Indigo; I tested The Same PyDev plus Python under Juno on a 32-bit Pc with success; Then by the first attempt on a 64-bit PC came the error ...
I have not used "autoconfig" at all. I was going the line
"File>New>PyDevProject> ...PrjectName... click_here_to_configure_an_interpreter_not_listed" when it happened."
After short googleing the luck came:-)
http://pydev.org/manual_101_interpreter.html
helped me to solve the problem ,
using just an alternative path to the same thing.
{1. Go to: window > preferences > PyDev > Interpreter - (Python/Jython/IronPython). ...}
I had a similar issue. I could not manually or automatically configure. I worked around the problem by re-installing python 2.7. After re-install auto-configuration worked.
I am running Python 3, so I had to make these
changes to get python.exe as my interpreter.
File:
C:\EasyEclipse-for-LAMP-1.2.2.2\extensions
\pydev-1.3.3\eclipse\plugins \org.python.pydev_1.3.3\PySrc
\interpreterInfo.py
Edit all occurrences of “print …” to “
print(…)
” (the … is whatever is being printed), so wrap it in quotes.
Comment out line 16, since True and False are
considered keywords in Python 3.
Save, and you should be able to successfully
set C:\Python32\python.exe as the interpreter.
I kept the default folders checked.
I encountered this problem on Windows after directly copying my python 2.7 (64bits) directory to a new computer. Neither automatic not manual configuration would work, and I had the same errors as you did.
I came back to the original installation, went to C:\Windows\System32 (you might also check SysWOW64, but I'm not sure Python even uses it) and fetched :
python27.dll
pythoncom27.dll
pythoncomloader27.dll
pywintypes27.dll
I copied them into the new computer's System32 and pydev's autoconfig was able to find my python interpreter from the environment variables.
Please Make sure you have installed proper version [32 or 64 bit] of python corresponding to your system OS version
I have a similar issue. Try to run "python --version" to see that python executes in your system
Install OS compatible python executable in your machine.
During installation remember to check "Add Python 3.8 to PATH. (as shown in this image)
Once Python installed successfully in your machine In Eclipse open in the Window -> Preference -> Pydev -> Interpreter -> Python Interpreters menu. (Assuming you already installed PyDev plugin in Eclipse).
Next select Config first in PATH option. (as shown in this image)
You can now see the Python interpreter info in eclipse. (as shown in this image)

ANT Javac and special characters

I have an ANT task defined like so:
<javac source="1.5" target="1.5" srcdir="${src.dir}" destdir="${classes.dir}" deprecation="on" debug="on" classpathref="classpath" fork="true" memoryMaximumSize="512m" encoding="UTF-8">
<include name="${app.directory}/**/*.java"/>
</javac>
This works fine, but when I have classes with special characters in their names it gives me the following error:
[iosession] Compiling 131 source files to /C24/PUB/io-stds/trunk/standards/GSIT/build/test/deployment/build/classes
[iosession] javac: file not found: /C24/PUB/io-stds/trunk/standards/GSIT/build/test/deployment/src/java/biz/c24/io/minos/AléaChiffréClass.java
[iosession] Usage: javac <options> <source files>
[iosession] use -help for a list of possible options
[iosession] Target compile finished
[iosession]
[iosession] Building unsuccessful 2 seconds
When I remove the "fork=true" it works, but then it ignores the "memoryMaximumSize" setting. I also tried the nested approach, but to no avail.
Any ideas?
It's perhaps not the answer you expect but my advice would be to remove all non-ascii letters from the names of methods and classes. I'm French-speaking too, and I've never seen any company, even in France and using French as its development language, accept accented letters in class names and methods. It's just not good practice, simply because it would be very hard for a non French developer, without accents on his keyboard, to use these classes and methods.
If you use a good IDE, it should allow you to refactor your code easily.
Apache did confirm that the encoding attribute only applies to the file contents and not file names. I reverted back to using fork only when needed and kept encoding="UTF-8".