How to make Tcl_WriteChars support Unicode? - unicode

Question

Is there any initial setup needed to make Tcl_WriteChars output UTF-8 characters correctly? e.g.
#include <tcl.h>
int main()
{
Tcl_Interp *tcl = Tcl_CreateInterp();
Tcl_Channel channel = Tcl_GetStdChannel(TCL_STDOUT);
Tcl_WriteChars(channel, "hello\n", -1);
Tcl_WriteChars(channel, "你好\n", -1);
Tcl_WriteRaw(channel, "你好\n", -1);
Tcl_Close(tcl, channel);
Tcl_DeleteInterp(tcl);
return 0;
}
Source code is saved in UTF-8 encoding, and following output is from UTF-8 locale Linux:
hello
??
你好

Answer 1

You need to configure the encoding to be UTF-8 (and the host you're running on appears to be using something else for its default). Do this before you write to the channel.
Tcl_SetChannelOption(interp, channel, "-encoding", "utf-8");
Properly, you should check for the return code of that (as below) but all channels have that option and the utf-8 encoding is baked directly into Tcl, so it won't fail.
if (Tcl_SetChannelOption(interp, channel, "-encoding", "utf-8") != TCL_OK) {
return TCL_ERROR;
}
[EDIT]: Having re-read the code a little more carefully (and found out that the system's default encoding is really UTF-8 in the first place), the actual problem is that you're not calling Tcl_FindExecutable(). That routine is a bit mis-named, as what it actually does (apart from making info nameofexecutable work inside scripts) is let Tcl initialise its internal library. In particular, it initialises the encoding management subsystem, and that's the point where it works out what the system encoding really is (otherwise it falls back to iso8859-1, which is the least problematic ordinary encoding to recover from).
Your code should read:
#include <tcl.h>
int main(int argc, char *argv[]) /// <<<< CHANGED HERE
{
Tcl_FindExecutable(argv[0]); /// <<<< CHANGED HERE
Tcl_Interp *tcl = Tcl_CreateInterp();
Tcl_Channel channel = Tcl_GetStdChannel(TCL_STDOUT);
Tcl_WriteChars(channel, "hello\n", -1);
Tcl_WriteChars(channel, "你好\n", -1);
Tcl_WriteRaw(channel, "你好\n", -1);
Tcl_Close(tcl, channel);
Tcl_DeleteInterp(tcl);
return 0;
}
I'm assuming you're using a compiler that is happy with putting declarations after statements. That's a widely-implemented C99 feature (and is also in C++) so I expect it will be fine.