Showing glyphs with Unicodes higher than decimal 256 - unicode

Question

I am looking for help in printing the club symbol from the Arial font in postscript.
It has a Unicode of 9827 (2663 hexadecimal).
The ampersand is Unicode 38 (26 hexadecimal)
This postscript code snippet
!PS
/ArialMT findfont
12 scalefont setfont
72 72 moveto
<26> show
showpage
produces the ampersand system when I run it through Adobe Distiller. It appears that postscript understands Unicodes with UTF-8 encoding by default.
I am unable to do the same with the club symbol.
My research indicates that I have to use Character Encoding and this where I am lost.
Could some kind soul show me (hopefully fairly short) how to show the club symbol using Character Encoding?
Alternatively if you could point me to a simple tutorial it would be greatly appreciated.
Reading the reference manual leaves my head spinning.

Answer 1

PostScript does not understand Unicode, not at all, or at least not as standard, though there are ways to deal with it.
Section 5.3 of the PostScript Language Reference Manual contains complete information on Character Encoding. You really need to read this in detail, what you are asking is a deceptively simple question, with no simple answer.
The way this works for PostScript fonts is that the characters in the document have a character code which lies between 0 and 255. When processing text, the interpreter takes the character code and looks it up in the Encoding attached to the font. If you didn't supply an Encoding to the font, then it will normally have a pre-defined StandardEncoding.
StandardEncoding has some congruence with UTF-8, for character codes 0x7F and below, but it's not exactly the same I don't think.
The Encoding maps the character code to a glyph name, For example 0x41 in StandardEncoding maps to /A (that's a name in PostScript). Note that is not UTF-8 or anything else, it's a mapping. It's entirely possible, and common practice for subset fonts, to map the first character used to character code 1, the second to character code 2 and so on.
So if we applied that scheme to 'Hello World' we would use an Encoding which maps
0x01->/H
0x02->/e
0x03->/l
0x04->/o
0x05->/space
0x06->/W
0x07->/r
0x08->/d
and then we might draw the text by :
<0102030304050604070308> show
Which, as you can see, bears no relation to UTF-8 at all.
Anyway, having retrieved the glyph name the interpreter then looks at the CharStrings dictionary in the font and locates the key associated with the character code. So for StandardEncoding we would map the 0x41 to /A and we'd then look in the CharStrings dictionary for the /A key. We then take the value associated with that key, which will be a PostScript glyph program and run it.
Your problem is that you are trying to use a TrueType font. PostScript does not support TrueType fonts in that way, it does support them when they are defined as Type42 fonts, because a Type42 font carries around some additional information which allows the PostScript interpreter to treat them, broadly speaking, the same way as PostScript fonts.
Many modern PostScript interpreters will load a TrueType font and create a Type42 font from it for you, but this involves guessing at the additional information, and there's no real way to tell in advance how any given interpreter will deal with this. I suspect that Adobe Distiller will behave similarly to Ghostscript and attempt to map the type42 to a StandardEncoding.
Essentially the Encoding maps the character code to a key in the CharStrings dictionary and the value associated with that key is the GID. The GID is used to index the GLYF table in the TrueType font, the TrueType rasteriser then reads that glyph program.
So in order to create a type42 font with an Encoding which will map a character code to a club symbol, you would need to know what the GID of the club symbol in the font actually is. This can be derived from one of the CMAP subtables in the TrueType font, which is how PostScript interpreters such as Ghostscript build the required Encoding when they load a TrueType font as a Type 42. You would then need to alter the CharStrings dictionary in the type42 font so that it maps to the correct GID. You would also need to alter the Encoding; choose a character code that you want to use, map the character code to the key in the CharStrings dictionary.
You would have to determine what kind of keys the Encoding and CharStrings dictionary is using. It might be names or it might be integers or anything else. You could figure that out of course by looking at the content of the Encoding array.
In all honesty unless you know a lot about TrueType fonts I think it would be hard for you to reverse-engineer the font to retrieve the correct GID and then re-encode the font that gets loaded by the interpreter. You would also need to examine the contents of the font dictionary returned by findfont to see what the existing mapping is. And crucially you may need to modify the CharStrings dictionary to map the key to the GID. It may be that Distiller returns a dictionary which is defined as no-access which will prevent you looking inside or (or at least, inside parts of it). You might be able to get away with looking at the Encoding in the font dictionary and modifying that, if the CharStrings dictionary already contains a key for every glyph in the font, which it may well do.
I could probably guide you through doing this with Ghostscript, but I have no idea how Adobe Distiller defines TrueType fonts loaded from disk.
You could use a CIDFont instead. These are defined in section 5.11.1 and it may be that if you were to use something like the pre-defined Identity-H or UCS2 CMaps you could create a CID-Keyed instance of ArialMT with TrueType outlines which would work for your Unicode code point.
But that would mean defining the font yourself, so you would need to include the whole TrueType font as part of your PostScript program. Again this isn't simple.

Answer 2

There is some good information here: Show Unicode characters in PostScript
I also have the ArialMT.ttf and made the ArialMT.ttf.t42 just to look inside. I found the /club glyph with GID 389 as described by KenS and tried this as described in the linked post with good results:
%!
100 300 moveto
/ArialMT.ttf 46 selectfont (ArialMT) show
100 200 moveto /club glyphshow
showpage
Note: I use ArialMT.ttf because the TT font wasn't installed in the ghostscript Fontmap just in the current directory so used gs -P for that reason. The normal /ArialMT findfont should work when the TT font is already installed in the search path. This is my first attempt with these glyphs and was just using trial and error.

Answer 3

There is a comprehensive Adobe list of glyphs that map many of the Unicode characters: https://github.com/adobe-type-tools/agl-aglfn/blob/master/glyphlist.txt.
If the desired Unicode character is in that list, say club;2663 or clubsuitblack;2663 or clubsuitwhite;2667, all one needs to say is
/club glyphshow and most modern fonts will know what to do. But #KenS says this "can cause problems".
Instead the preferred scheme that emerges from the recommended references is to:
create a composite font in the preamble, one for each of the fonts
you are using;
include the lower 256 characters as Font0;
add whatever Unicode characters you are planning to use, in chunks of
256 characters, as Font1, Font2 etc.;
remap the Unicode of the special characters onto a two-character
sequence, of the sub-font index within the composite font, followed
by the byte that is the index of the character within that sub-font.
The following is a complete example of both methods.
I use http://www.acumentraining.com/Acumen_Journal/AcumenJournal_May2002.zip, but with Font1 is a custom-remapping of a portion of the same font as Font0, re-using some of the well known ascii character(s).
This a complete file.eps:
%!PS-Adobe-3.0 EPSF-3.0
%%BoundingBox: 0 0 792 612
%%LanguageLevel: 2
%%EndComments
%%BeginProlog
userdict begin
%%EndProlog
%%BeginSetup
% The following encodes a few useful Unicode glyphs, if only a few are needed.
% Based on https://stackoverflow.com/questions/54840594/show-unicode-characters-in-postscript
% Usage: /Times-Roman /Times-Roman-Uni UniVec new-font-encoding
/new-font-encoding { <<>> begin
/newcodesandnames exch def
/newfontname exch def
/basefontname exch def
/basefontdict basefontname findfont def % Get the font dictionary on which to base the re-encoded version.
/newfont basefontdict maxlength dict def % Create a dictionary to hold the description for the re-encoded font.
basefontdict
{ exch dup /FID ne % Copy all the entries in the base font dictionary to the new dictionary except for the FID field.
{ dup /Encoding eq
{ exch dup length array copy % Make a copy of the Encoding field.
newfont 3 1 roll put }
{ exch newfont 3 1 roll put }
ifelse
}
{ pop pop } % Ignore the FID pair.
ifelse
} forall
newfont /FontName newfontname put % Install the new name.
newcodesandnames aload pop % Modify the encoding vector. First load the new encoding and name pairs onto the operand stack.
newcodesandnames length 2 idiv
{ newfont /Encoding get 3 1 roll put}
repeat % For each pair on the stack, put the new name into the designated position in the encoding vector.
newfontname newfont definefont pop % Now make the re-encoded font description into a POSTSCRIPT font.
% Ignore the modified dictionary returned on the operand stack by the definefont operator.
end} def
/Helvetica /Helvetica-Uni [
16#43 /club % ASCII 43 = C = /club
] new-font-encoding
/Helv
<<
/FontType 0
/FontMatrix [ 1 0 0 1 0 0 ]
/FDepVector [
/Helvetica findfont % this is Font0
/Helvetica-Uni findfont % this is Font1
]
/Encoding [ 0 1 ]
/FMapType 3
>> definefont pop
%%EndSetup
%%BeginScript
/Helv 20 selectfont
72 300 moveto
(The club character is \377\001C\377\000 a part of the string.) show
/Helvetica findfont 20 scalefont setfont
263 340 moveto
/club glyphshow
showpage
%%EOF
Which produces this
Obviously, this can be extended to more characters, but only 256 per sub-font. I am not aware of a "standard" convention for such re-encoding, although I would imagine a set of Greek letters alpha,beta,gamma... would map pretty obviously onto a,b,c... Perhaps somebody else is aware of such an implementation for all of the Unicode characters from the Adobe glyph list using multiple custom sub-fonts, and can provide a pointer..