Does any language do Unicode and cross-platform properly and fully? - unicode

I've used a good few programming languages over the years and I'm an armchair linguist and contributor to Wiktionary. I've been making some of my own tools to look up Wiktionary from the commandline but I've run into a surprising problem.
Neither Perl nor Python can output Unicode to the console natively under both *nix and Windows (though there are various workarounds). The main reason is that *nix OSes like their Unicode in UTF-8 and Windows likes its Unicode in UTF-16. But it also seems that Windows makes it very difficult to use wide characters with the console even though both the console and wprintf are wide character native.
So the question is, is the situation any better if I look beyond these languages into Java, C#, Scala, etc. Or are there any scripting languages which started out on Windows and were then ported to *nix?
Here is some ideal pseudocode:
function main()
{
print( L"hello, 世界" );
}

Does any language do Unicode and cross-platform properly and fully?
C# supports Unicode very extensively. Its standard library (.NET Framework) also has outstanding support for Unicode. Cross-platform is reasonable, but not perfect: it's achieved via Mono, and on mobile platforms via Xamarin.
Command-line programs are pretty portable but can get screwed by ancient relics, like SSH terminals that haven't been updated for a decade or more.
Here is some ideal pseudocode:
C# gets pretty close:
using System;
class Program
{
static void Main(string[] args)
{
Console.OutputEncoding = System.Text.Encoding.UTF8;
Console.WriteLine("tést, тест, τεστ, ←↑→↓∏∑√∞①②③④, Bài viết chọn lọc");
}
}
Screenshot of the output (use Consolas or another font that has all the above characters):
Of course C# is not a scripting language; it is quite different in its approach to pretty much everything.

AFAIK almost all scripting languages started in the Unix world and were then ported to Windows. I don't know any example of a (scripting) language that started on Windows... One scripting language that seems to do pretty fine with Unicode these days is Ruby.

Eight and a half years have passed and things are improving.
NodeJS was the first language to "just work" out of the box with Unicode on *nix, Mac, and Windows using Unicode in their terminals/consoles without regard to whether the OS preferred UTF-8 or UTF-16.
At the time I asked this question, this did not work for Perl, Python, or Ruby. I'm not sure about PHP. But at least the Python developers eventually took the relevant bug report / feature request seriously and put some work into it. Python has now worked with cross-platform in-terminal Unicode for some years.
I just started looking at Rust, and thought to check this. I was very pleasantly surprised that they also took this issue seriously and Rust is the first low-level / non-scripting language to just work out of the box cross-platform with Unicode in the console on Mac, Windows, and *nix.

Perhaps this is one of the workarounds you hinted at, but: You can chcp 65001 in a 'DOS box' with a non-raster font select and view UTF-8 output from scripts (or programs) that run unchanged under Unix or Windows. The price to pay is that .bat/.cmd files won't execute.

Related

Which commands and functions are not supported in TCL based on the version

SO far I've been using TCL 8.6 , but recently I had to move back to 8.0 because the program I am using only supports 8.0
The issue is that I can't find a documentation or a comparison between what new commands and functions are compatible and what are not
Are there any websites that contain such info
//please note that I checked the tcl.tk website and it only features what is new per version but not so specific
This page, and the pages it links to, have fairly detailed information on command and syntax changes. You may have to search backwards to find the first version that supports the feature you're wondering about.
http://www.tcl.tk/man/ has links to documentation for Tcl commands in various versions from 7.5 to 8.6, though it skips 8.1 for some reason.
Only supporting 8.0? That's rather old. (I've still got some systems like that, but they're really just on life support.)
The second-most definitive source of what has changed between versions is the changes file in any Tcl source code distribution. (The first most accurate source is comparing the source code, but that's a bit annoying prior to 8.1 for various unimportant reasons.) You can use the current version for this, since those old parts of that file are effectively static; here's the equivalent for Tk. The file includes a lot more information than you probably need, including mentioning many of the more significant bugfixes as well as some cases where functionality flapped a bit during betas (we try to avoid doing that, but it happens occasionally).
However, the big things that far back are fairly simple, so we can probably narrow the things to watch out for to these big-ticket items:
Expansion syntax ({*}...) was introduced in Tcl 8.5. Before that, you use eval a lot more.
There's a large bunch of commands in modern Tcl that simply weren't there back then. There are workarounds for some of these; easier to just try and see what's actually missing that you expect and then ask here again than to try to anticipate them all. Some known exceptions that are known to be entirely unavailable for 8.0:
dictionaries
ensembles
coroutines
the TclOO object system
the Ttk widget set
Tcl 8.0 assumed that characters were always 8-bit quantities and made no attempt to understand encodings. It just slings bytes around. (This also means that the encoding command was absent, and so too was the -encoding option to fconfigure.)
Make sure you only use ASCII in your scripts themselves. Saves pain.
The RE engine in 8.0 was different to the one used from 8.1 onwards. It's much simpler, and supports only a fraction of what the newer engine does.
Threading support in 8.0 and before is not for the faint of heart. Assume your code is single-threaded and avoid a lot of difficult debugging.

What module to use for perl GUI on linux? [duplicate]

I'd like to write some interactive GUIs in Perl. I've used TclTk but it looks dated. I've written QT code for C++, but the PerlTk module hasn't had a release in several years. Googling around I see other possible options.
What are good packages for this, including basic windowing, menus, drawing canvas, scrollbars, and so on.
Gtk2 has glade2 which can write out an XML file usable by Gtk2::GladeXML. Here is an example of how to bundle the XML with the app in the same file.
I misread the question at first. I thought you wanted a GUI editor for making GUIs (which is what glade2 is). You can also create GUIs using Gtk2 without glade2:
#!/usr/bin/perl
use strict;
use warnings;
use Gtk2;
Gtk2->init;
my $window = Gtk2::Window->new;
my $vbox = Gtk2::VBox->new;
my $label = Gtk2::Label->new("Hello World");
my $button = Gtk2::Button->new("Press me");
$window->add($vbox);
$vbox->add($label);
$vbox->add($button);
$window->set_default_size(200, 200);
$window->signal_connect(
destroy => sub {
Gtk2->main_quit;
}
);
my $i = 0;
$button->signal_connect(
clicked => sub {
$label->set_text("button pressed " . ++$i . " times");
}
);
$window->show_all;
Gtk2->main;
Try wxPerl!
From the web site:
wxPerl is an extension module allowing
the creation of GUI (Graphical User
Interface) from Perl; it is built as a
wrapper for the awesome wxWidgets C++
GUI toolkit.
Echoing Chas Owens, glade is quite usable with Gtk2 in Perl. In addition, Gtk2 also supports GtkBuilder files (which you can create the latest glade too).
The main problem with wxPerl (and wxWidgets itself) is that it doesn't let you install file event watchers into its main loop (it only has GUI, Socket and Timer events), unlike Tk and Gtk.
The Qt, Tk and Gtk2 event loops can be used in Perl with AnyEvent, and Gtk2 can be hooked into applications running the Event or Ev mainloop with Glib::Event and Glib::EV modules.
A GUI builder for WxPerl would be wxGlade or wxFormBuilder, both open-source.
I would probably use Gtk2, but Tk is definitely not dead. The lack of releases is due to the bugs being ironed out over the years. A lot of people still use it, and it generally works fine. (Tk's event loop is somewhat silly, but that is a detail that probably shouldn't concern you.) The only disadvantage is that your GUI looks like it is from 1996, but who cares?
(The gitk tool included with git showed that Tk GUIs look fine as long as they're useful.)
Recently I was made aware of another GUI toolkit for Perl called Prima. Its claim to fame is that it was written in C especially for Perl, not a port from TCL or bindings to something else. Assuming you have some rather standard libraries on your system, it will install directly from CPAN and its cross-platform!
David Mertens has been writing a new plotting library for PDL using it, and a REPL too!
Regarding quality of widgets, all three gtk2, wx and qt4 are equally good -- gtk2 is fastest, qt4 has best resolution, whereas wx is somewhere between the two in terms of both the parameters.
But tk is behind all the three both in terms of speed and resolution, but remains firstly because it is the original widget for all unix based interpreted languages, and secondly because it is native-widget of freebsd.
As on today in case of perl, documentation of perl-gtk2 is slightly more developed than both perl-qt4 which is in turn is more developed than wxperl. In fact, if you browse the web, you will find that tutorials for perl-gtk2 is more extensively available than even pygtk (which is also very well established) and ruby-gtk2. In case of python, wxpython documentation leads over PyQt and pygtk, whereas in case of ruby, ruby-qt4 leads over ruby-gtk2 and wxruby. However, others are developing very fast (e.g. ruby-gtk2) and after some time all nine will be equal in all aspects.
The perl-qt4 specific problem is: In case of non-kde desktops, perl-qt4 may not run. I tried running all in ubuntu and sabayon and found that perl-gtk2 and even wxperl (and wxruby also) runs fine whereas I'm not able to run even a single perl-qt4 programs in gnome-desktop where I have installed complete qt4 support without kde.
perl-qt4 asks for installation of not just qt4-libraries, but also kde-desktop libraries. This problem is uniquely perl-qt4-specific, does not exist in case of PyQt and rubyqt4 since they both run fine with qt4 runtime under gnome.
Hence, in this way eight (plus three tk) out of nine options did well (I havent tested PySide yet).
In case of perl-tk, widget is well developed and established and unlike tkinter, pmw and rubytk, its documentation is the best among all of these mentioned above.
But maybe after some time all the documentations become developed and any would become equally good.
After taking a balance between speed, clarity and cross-platform into consideration choose either perl-gtk2, or perl-qt4 or wxperl.
perl-tk, perl-gtk2 and perl-qt4 syntaxes nicely go well with perl programming language syntax, whereas wxperl syntax look more like proper c++ blended into perl which is different from the respective language syntax (just like in case of wxpython and wxruby).
But if you are already keeping in mind that wx code is uniquely different then you can straightaway begin with wxperl.
Otherwise, best way out is: begin with perl-tk, then switch over to perl-gtk2 then perl-qt4 or wxperl alike but make sure that you learn all four instead of only one (the same sequence should also apply to python and ruby too despite the fact that documentation support is reverse, but in case of perl even for this aspect it is same).
I don't have a real answer for you, but often you have to consider your deployment targets. Some of the GUI libraries are very nice, but only if you can get them to work on your operating system. I don't necessarily think that all frameworks need to be cross-platform compatible, a very laudable goal to work toward in a perfect world, as long as the one you choose doesn't lock out a significant portion of your users because the foundation GUIs libraries are hard to install or support on a particular platform.
I just want to let everyone know PerlQt is great and this site has a lot of tutorials on how to use it and lots of examples to download.
PerlQt Wiki tutorial
PerlQt is a very fast and easy way to create great looking GUI programs using the Qt drag and drop form designer and Perl. PerlQt is powerful enough for advanced developers and easy for beginners as well.
The solution I'm going with is Gtk-Perl. At first I had difficulty installing it, but in the end I found Camelbox Perl+Gtk2 in one package.
I have used Tk, but in the last few days I have been playing around with Continuity (CPAN entry), a stateful (persistent state per user) web application/server, and it could easily be a way to add "GUI" features into an application, while making it multi-user and portable at the same time.
Edit: At the time I wrote this, I was most interested in the fact that Continuity had its own built-in webserver. This is now true of other webframeworks for Perl as well. I have been using Mojolicious though I hear good things about Dancer too.

What should I use to create a Windows GUI?

I'm an experienced UNIX programmer.
Now I want to develop a simple Windows application; the programming part would be very easy with almost any tool (it might be C, Perl, or something else).
However, I wonder what tool to use in order to have some simple GUI around it?
I've read a little about Perl/TK, but understood that it's too old, and
Visual Studio requirement seems a little over-kill.
WxPerl is presently the widely used GUI toolkit in the Perl community.
I'd suggest having another look at Perl/Tk. Tk is old but nothing else have managed to do what it does best: being a simple, easy to use but powerful UI library (note its strengths in that order, note also I didn't mention speed).
Unix programmers both love (simple!) and hate (ugly!) Tk. But Tk is only really ugly on Unixen because it defaults to a Motif theme (modern Tk is/should be replaced by TTk which is themeable). Under Windows, Tk uses native widgets so look and feel is not so much of an issue like it is on Unix.
As for the other suggested libraries, QT feels alien under Windows (from your user's perspective) reflecting its Linux heritage but WxWidgets looks right at home reflecting its mostly Windows heritage.
I suggest you use QT (C/C++), more info here: http://qt.nokia.com/products/platform/qt-for-windows/
If you want good portablity between operating systems, os native (or other) styling, and installation without C/C++ compilation, you can take a look at the module I've been writing, XUL::Gui on CPAN. It uses Firefox as its rendering engine. Here's a quick example:
use XUL::Gui;
display Window title=>'my window',
H2('events!'),
Button(
label => 'click me',
oncommand => sub {
my ($self, $event) = #_;
$self->label = 'ouch!';
}
);
There are more detailed examples on CPAN. Let me know if you have any questions.
For a windows GUI use Win32::GUI and possibly Win32::GUI::XMLBuilder

Is Perl a good option for writing platform independent desktop applications?

Is Perl a good option for writing (possibly and partially) platform independent desktop applications? I know there are interesting widget libraries like GTK2 Perl and wxWidgets but I'm not familiar with development on Windows.
Is it possible to write a good application with Perl and those tools, maybe embedding everything needed to avoid asking the user to install external (and probably non conventional for many) libraries? Are there examples of this kind of apps around?
Sort of. You can certainly write Perl scripts that provide a Tk GUI that will work on any platform, and you can even package the libraries they need to work with PAR to put together an application that will run anywhere with an appropriate version of perl installed without requiring the end use to install a bunch of stuff from CPAN to get it to work.
However, an application that will run anywhere without requiring an existing perl installation (and it's probably not safe to assume the average Windows user will have perl) isn't really viable. One solution would be to distribute an "everywhere but Windows" version that simply uses PAR to include the necessary libraries, and a PAR::Packer-built version for Windows, which would be a fully-functional .exe including the perl interpreter and the libraries.
EDIT: following daotoad's response, it does appear that ActiveState's PerlApp can build cross-platform binaries for for Windows, Mac OS X, Linux, Solaris, and AIX; I haven't tried this but if it works as advertised it would seem to meet your requirements.
My employer uses ActiveState's PerlApp to produce executable versions of our apps with great success.
We tested PAR and PerlApp before selecting our packaging method. At that time, PerlApp had faster load times. Several versions of each tool have come and gone since then, so I would recommend testing with each before selecting a tool.
Executable packaging has been very effective for us.
There are a few portability issues issues with Perl, but if you pay attention to perlport, it's easy to avoid most issues. Our biggest problems are always with Win32. From time to time, some simple thing will require stupid, bizarre work-arounds or digging into Win32 API for a platform specific hack.
We have used both Wx and Tk guis.
Frozen Bubble is a well known and widely available app you can look at.
Check out PAR (available via CPAN) with respect to your bundling requirement.
Consider having a look at Padre for an example of a complex application written in perl using Wx.
So yes. All of your requirements are achievable, including portability.
I am working on the module XUL::Gui on CPAN, which uses Firefox to display cross platform gui apps from Perl. It is under development, but stable, and may be complete enough for your needs. You can build your gui in HTML and/or XUL (the Mozilla gui language that Firefox itself is written in), and then style everything with CSS. Let me know if you have any feature requests.

What are good interactive GUI builder packages for Perl?

I'd like to write some interactive GUIs in Perl. I've used TclTk but it looks dated. I've written QT code for C++, but the PerlTk module hasn't had a release in several years. Googling around I see other possible options.
What are good packages for this, including basic windowing, menus, drawing canvas, scrollbars, and so on.
Gtk2 has glade2 which can write out an XML file usable by Gtk2::GladeXML. Here is an example of how to bundle the XML with the app in the same file.
I misread the question at first. I thought you wanted a GUI editor for making GUIs (which is what glade2 is). You can also create GUIs using Gtk2 without glade2:
#!/usr/bin/perl
use strict;
use warnings;
use Gtk2;
Gtk2->init;
my $window = Gtk2::Window->new;
my $vbox = Gtk2::VBox->new;
my $label = Gtk2::Label->new("Hello World");
my $button = Gtk2::Button->new("Press me");
$window->add($vbox);
$vbox->add($label);
$vbox->add($button);
$window->set_default_size(200, 200);
$window->signal_connect(
destroy => sub {
Gtk2->main_quit;
}
);
my $i = 0;
$button->signal_connect(
clicked => sub {
$label->set_text("button pressed " . ++$i . " times");
}
);
$window->show_all;
Gtk2->main;
Try wxPerl!
From the web site:
wxPerl is an extension module allowing
the creation of GUI (Graphical User
Interface) from Perl; it is built as a
wrapper for the awesome wxWidgets C++
GUI toolkit.
Echoing Chas Owens, glade is quite usable with Gtk2 in Perl. In addition, Gtk2 also supports GtkBuilder files (which you can create the latest glade too).
The main problem with wxPerl (and wxWidgets itself) is that it doesn't let you install file event watchers into its main loop (it only has GUI, Socket and Timer events), unlike Tk and Gtk.
The Qt, Tk and Gtk2 event loops can be used in Perl with AnyEvent, and Gtk2 can be hooked into applications running the Event or Ev mainloop with Glib::Event and Glib::EV modules.
A GUI builder for WxPerl would be wxGlade or wxFormBuilder, both open-source.
I would probably use Gtk2, but Tk is definitely not dead. The lack of releases is due to the bugs being ironed out over the years. A lot of people still use it, and it generally works fine. (Tk's event loop is somewhat silly, but that is a detail that probably shouldn't concern you.) The only disadvantage is that your GUI looks like it is from 1996, but who cares?
(The gitk tool included with git showed that Tk GUIs look fine as long as they're useful.)
Recently I was made aware of another GUI toolkit for Perl called Prima. Its claim to fame is that it was written in C especially for Perl, not a port from TCL or bindings to something else. Assuming you have some rather standard libraries on your system, it will install directly from CPAN and its cross-platform!
David Mertens has been writing a new plotting library for PDL using it, and a REPL too!
Regarding quality of widgets, all three gtk2, wx and qt4 are equally good -- gtk2 is fastest, qt4 has best resolution, whereas wx is somewhere between the two in terms of both the parameters.
But tk is behind all the three both in terms of speed and resolution, but remains firstly because it is the original widget for all unix based interpreted languages, and secondly because it is native-widget of freebsd.
As on today in case of perl, documentation of perl-gtk2 is slightly more developed than both perl-qt4 which is in turn is more developed than wxperl. In fact, if you browse the web, you will find that tutorials for perl-gtk2 is more extensively available than even pygtk (which is also very well established) and ruby-gtk2. In case of python, wxpython documentation leads over PyQt and pygtk, whereas in case of ruby, ruby-qt4 leads over ruby-gtk2 and wxruby. However, others are developing very fast (e.g. ruby-gtk2) and after some time all nine will be equal in all aspects.
The perl-qt4 specific problem is: In case of non-kde desktops, perl-qt4 may not run. I tried running all in ubuntu and sabayon and found that perl-gtk2 and even wxperl (and wxruby also) runs fine whereas I'm not able to run even a single perl-qt4 programs in gnome-desktop where I have installed complete qt4 support without kde.
perl-qt4 asks for installation of not just qt4-libraries, but also kde-desktop libraries. This problem is uniquely perl-qt4-specific, does not exist in case of PyQt and rubyqt4 since they both run fine with qt4 runtime under gnome.
Hence, in this way eight (plus three tk) out of nine options did well (I havent tested PySide yet).
In case of perl-tk, widget is well developed and established and unlike tkinter, pmw and rubytk, its documentation is the best among all of these mentioned above.
But maybe after some time all the documentations become developed and any would become equally good.
After taking a balance between speed, clarity and cross-platform into consideration choose either perl-gtk2, or perl-qt4 or wxperl.
perl-tk, perl-gtk2 and perl-qt4 syntaxes nicely go well with perl programming language syntax, whereas wxperl syntax look more like proper c++ blended into perl which is different from the respective language syntax (just like in case of wxpython and wxruby).
But if you are already keeping in mind that wx code is uniquely different then you can straightaway begin with wxperl.
Otherwise, best way out is: begin with perl-tk, then switch over to perl-gtk2 then perl-qt4 or wxperl alike but make sure that you learn all four instead of only one (the same sequence should also apply to python and ruby too despite the fact that documentation support is reverse, but in case of perl even for this aspect it is same).
I don't have a real answer for you, but often you have to consider your deployment targets. Some of the GUI libraries are very nice, but only if you can get them to work on your operating system. I don't necessarily think that all frameworks need to be cross-platform compatible, a very laudable goal to work toward in a perfect world, as long as the one you choose doesn't lock out a significant portion of your users because the foundation GUIs libraries are hard to install or support on a particular platform.
I just want to let everyone know PerlQt is great and this site has a lot of tutorials on how to use it and lots of examples to download.
PerlQt Wiki tutorial
PerlQt is a very fast and easy way to create great looking GUI programs using the Qt drag and drop form designer and Perl. PerlQt is powerful enough for advanced developers and easy for beginners as well.
The solution I'm going with is Gtk-Perl. At first I had difficulty installing it, but in the end I found Camelbox Perl+Gtk2 in one package.
I have used Tk, but in the last few days I have been playing around with Continuity (CPAN entry), a stateful (persistent state per user) web application/server, and it could easily be a way to add "GUI" features into an application, while making it multi-user and portable at the same time.
Edit: At the time I wrote this, I was most interested in the fact that Continuity had its own built-in webserver. This is now true of other webframeworks for Perl as well. I have been using Mojolicious though I hear good things about Dancer too.