how to convert from gbk encoding to utf-8 encoding in Perl - perl

Question

I have a simple question which I do not know how to solve in Perl. I know how to convert from utf-8 to GBK, for example, from e4b8ad to d6d0. But I am not sure how to go backward, i.e. given d6d0, how do I know e4b8ad.
Please enlighten me! Many thanks.

Answer 1

When you have hex digits, pack is your friend. Following is a REPL session. Notes:
To reverse the direction, pack the hex digits into octets, decode from GB octets to character string, encode character string to UTF-8 octets, unpack octets into hex digits.
GBK is superseded. Use of GB18030 (provided by Encode::HanExtra in Perl) has been mandatory for five years already.
$ use Encode qw(decode encode); use Encode::HanExtra; use Devel::Peek qw(Dump);
$ 'e4b8ad'
e4b8ad # hex digits
$ pack('H*', 'e4b8ad')
中
$ Dump(pack('H*', 'e4b8ad'))
SV = PV(0x3657680) at 0x36b7188
REFCNT = 1
FLAGS = (PADTMP,POK,pPOK)
PV = 0x36c0768 "\344\270\255"\0 # octets of UTF-8 encoded data
CUR = 3
LEN = 8
$ decode('UTF-8', pack('H*', 'e4b8ad'))
中
$ Dump(decode('UTF-8', pack('H*', 'e4b8ad')))
SV = PV(0x326c3a0) at 0x36a50c8
REFCNT = 1
FLAGS = (TEMP,POK,pPOK,UTF8)
PV = 0x3698a48 "\344\270\255"\0 [UTF8 "\x{4e2d}"] # character string
CUR = 3
LEN = 8
$ encode('GB18030', decode('UTF-8', pack('H*', 'e4b8ad')))
"\xd6\xd0"
$ Dump(encode('GB18030', decode('UTF-8', pack('H*', 'e4b8ad'))))
SV = PV(0x36a2da0) at 0x36b6d98
REFCNT = 1
FLAGS = (TEMP,POK,pPOK)
PV = 0x36db3e8 "\326\320"\0 # octets of GB18030 encoded data
CUR = 2
LEN = 8
$ unpack('H*', encode('GB18030', decode('UTF-8', pack('H*', 'e4b8ad'))))
d6d0 # hex digits

Answer 2

The answer to the question asked:
use Encode qw( from_to );
my $gbk = "\xD6\xD0";
from_to(my $utf8 = $gbk, 'GB18030', 'UTF-8'); # E4 B8 AD
or
use Encode qw( decode encode );
my $gbk = "\xD6\xD0";
my $utf8 = encode('UTF-8', decode('GB18030', $gbk)); # E4 B8 AD
However, a more normal flow looks like the following:
open(my $fh_in, '<:encoding(GB18030)', ...) or die ...;
open(my $fh_out, '>:encoding(UTF-8)', ...) or die ...;
while (<$fh_in>) {
...
print $fh_out ...;
...
}
Encode::HanExtra must be installed for Encode to find the encoding.

Answer 3

use Encode qw/encode decode/;
$utf8 = decode("euc-cn", $euc_cn); # ditto
You can also normally specify the encoding when you open or close a FD and it will perform necessary conversions.
Works like a charm:
perl -e 'open(X,">","/tmp/x"); print X chr(0xd6).chr(0xd0);close(X)'
perl -mEncode -e 'open(X,"<","/tmp/x"); $x=<X>; print Encode::decode("euc-cn",$x);' > /tmp/xx