How to manually convert between Latin-5 and Unicode code points? - unicode

Question

I have socket connection and I can get data from my socket. The data encoded as Latin-5.
Example: // will get data from socket like:
_xCustomerName = “879255:_:NÝYAZÝ TOROS”;
_xCustomerName.replaceAll(new RegExp(r'Ý'), 'İ');
An seems it doesnt replace the 'Ý' to ‘İ’.
How to manually convert between Latin-5 and Unicode code points?

Answer 1

The Latin-5 characters are specified, e.g., here. That looks like Cyrillic to me, so what you need is probably codepage 1254 Windows Latin-5 or ISO-8859-9/Latin-9. They both have Turkish characters with capital dotted I at position 0xDD. The six-character change you mention in your follow-up post matches Latin-9.
From the character table, you can build a translation table with one character per byte in the 0..255 range:
import "dart:typed_data";
// The Unicode characters of ISO-8859-9.
const _latin9Table =
"\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
"\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f"
" !\"#\$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_"
"`abcdefghijklmnopqrstuvwxyz{|}~\x7f"
"\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f"
"\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f"
"\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf"
"\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf"
"\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf"
"\u011e\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\u0130\u015e\xdf"
"\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef"
"\u011f\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\u0131\u015f\xff";
String decodeLatin9(List<int> bytes) {
Uint16List charCodes = new Uint16List(bytes.length);
for (int i = 0; i < bytes.length; i++) {
charCodes[i] = _latin9Table.codeUnitAt(bytes[i]);
}
return String.fromCharCodes(charCodes);
}
That allows decoding of Latin-9. If you want to combine it into that dart:convert framework, you may need to create a Converter around that, or even an Encoding. Sadly, the package:convert doesn't have an easy way to create a code-page convereter. I think it should.

Answer 2

I know the differences Latin-5 (Turkish) character codes.
With simple switch statements I can turn “879255::NÝYAZÝ TOROS” to “879255::NİYAZİ TOROS”. I wish I had a better solution (some sort of Dart plugin) but this is best I came so far.
_xCustomerName.runes.forEach((int rune) {
var character = new String.fromCharCode(rune);
switch (character) {
case 'Ð':
character = 'Ğ';
break;
case 'Ý':
character = 'İ';
break;
case 'Þ':
character = 'Ş';
break;
case 'ð':
character = 'ğ';
break;
case 'ý':
character = 'ı';
break;
case 'þ':
character = 'ş';
break;
}
_xCustomerNameTurkish = _xCustomerNameTurkish + character;
});
print(_xCustomerNameTurkish);