gnuplot/pngcairo not rendering UTF-8 character correctly - unicode

I have two CentOS 7.4.1708 machines, running gnuplot 4.6 patchlevel 2,
currently behaving differently, and I can't work out why.
Install fonts:
sudo yum install dejavu-sans-mono-fonts
Then create the following GNUplot script:
cat << EOF > test.gnuplot
set terminal pngcairo enhanced font "DejaVuSansMono,10"
set encoding utf8
set title "同"
plot sin(x)
EOF
Finally, pipe it into the application:
cat test.gnuplot | gnuplot > test.png
On one machine I get this:
But on another I get this:
I can't work out the cause of the discrepancy. The desired character is U+540C so it's not like the second machine is interpreting the input bytes any differently; it's just not rendering the glyph.
What differences in system configuration should I be looking for?
More broadly, how can I "fix" the output in the second case? I don't even vastly care if some characters end up getting replaced by placeholders like this (after all, I must recognise that not all fonts implement all glyphs), but those placeholders being rendered at super-size is a problem.

This post is rather a collection of observations than a complete answer, but perhaps it might be useful as well (I tried your example on an almost fresh install of CentOS and it does reproduce the second plot in your post):
judging from the charset table printed by the command
fc-match -v DejaVuSansMono
it seems that 540C is indeed not supported. Perhaps the first machine has some additional fonts installed which are used as a fallback for this particular glyph? How would the output of fc-list differ?
Hard to say if it is complete, but the list of fonts supporting this glyph seems to be rather limited. Nonetheless, for example Google Droid is available via yum, so if I do
sudo yum install google-droid-sans-fonts google-droid-sans-mono-fonts
and rerun the Gnuplot script, the plot renders in an acceptable way.
as for the size of the "fallback" box, I first noticed that its size is directly proportional to the specified font size, i.e., it also doubles if one doubles the font size. From src/wxterminal/gp_cairo.c, it seems that Gnuplot uses by default an "oversampling" strategy to render the text, i.e., it renders everything in plot->oversampling_scale times larger resolution and then scales it back (via the transformation matrix defined in void gp_cairo_initialize_context(plot_struct*)).
For example, when rendering the text, it calls
pango_font_description_set_size(desc, \
(int) (plot->fontsize*PANGO_SCALE*plot->oversampling_scale));
However, for some reason, the "fallback" box is not scaled back and thus is plot->oversampling_scale times larger than the specified font size. The default value of plot->oversampling_scale is set to GP_CAIRO_SCALE which is defined to be 20 in src/wxterminal/gp_cairo.h.
I downloaded the source of Gnuplot 4.6.2 and replaced plot->oversampling = TRUE; with plot->oversampling = FALSE; in void gp_cairo_initialize_plot(plot_struct*) in src/wxterminal/gp_cairo.c. After recompilation, the "fallback" box is rendered with the same size as the rest of text. Unfortunately, I haven't found a way how to change this behavior directly from Gnuplot.

Related

How to configure Perl Tidy not wrap the lines?

I am using Per Tidy plugin in Padre IDE. By default, Tidy wrap my long lines into multiple lines which I don't like. How can I tell Tidy never wrap my lines?
Testing via perltidy on the command-line (referring to the man page), I found that I preferred to change the default perltidy options to always keep newlines (line breaks) by enabling "freeze newlines" (-fnl), and and disabling all whitespace modifications (-fws, freeze whitespace), via options: perltidy -fws -fhl -b
For your specific case, to ignore all line breaks, all you would needs is perltidy -fnl -b (the -b creates a backup file, and modifies in-place.)
There's also a second reasonable option of just setting the line length much longer (for example to 120 chars) than the default of 80 using: -l=120 (or: --maximum-line-length=120 or an even longer "really large number" is enabled by setting it to zero (0).
Note: as an aside, I'm testing on bluefish html editor, which uses either html tidy or perltidy, depending. For some reason, I had to change the rc file directly and couldn't change the setting in the GUI.
You can also change the default settings of perltidy. First find out where it is installed:
perl -MPerl::Tidy -e 'print $INC{"Perl/Tidy.pm"}'
Then edit the file where maximum-line-length is defined (or any other parameters you want). You might need sudo if perltidy is installed gobally. I usually use VS code to format my code and that in turn uses perltidy but I can't define paramters. So I found doing this particularly useful.

command line drawing unicode boxes for a graph

I don't know what the correct keywords are for what I am trying to achieve.
I have rainbarf for tmux which AFAIK is written in Perl.
I am wondering if there is some library, extension or functionality in node.js which would enable me to draw bars like the ones rainbarf draws, in order to write tmux monitors (wifi, GPU usage, etc).
Edit
I think that the correct term is unicode boxes as I have finally found a page that documents them.
They seem to be unicode geometric shapes So I am guessing that correctly printing them to screen (assuming the terminal supports them) should achieve the same result?
They appear to be small pixel-sized boxes constrained by the size of a font.
rainbarf is inspired on spark. There are several implementations of spark in JavaScript (and also other languages), check here: https://github.com/holman/spark/wiki/Alternative-Implementations
I found the answer to my question.
This is not related to Node.JS and is in fact simple unicode character printing.
The above picture simply prints a sequence of those characters in a sequence:
'\u2581','\u2582','\u2583','\u2584',
'\u2585','\u2586','\u2587','\u2588'
This in effect produces the "unicode" graph.
It is not language specific, but the colors are.

ImageMagick Command-Line Option Order (and Categories of Command-Line Parameters)

My supervisor has asked me to convert the parts of our Perl scripts that use PerlMagick to instead pipe and use the command line version of ImageMagick (for various unrelated reasons).
Using the our existing interface (crop, scale, save, etc) I'm building up a list of operations the user wants to perform on an image, constructing the statement to pipe and then executing it.
What I would like to know is:
Are convert operations performed from left to right? ie the order I pass them
What happens if I pass the same option twice? Are they performed separately?
Obviously the order in which operations are performed on an image is vital, so I'm trying to work out if I can perform all of the operations in one go (possibly gaining efficiency?) or if it I'm going to have to perform each operation separately.
Thanks
Unfortunately, the accepted answer to this question is not yet complete... :-)
Three (major) classes of parameters
Assuming, your ImageMagick version is a recent one, here is an important amendment to it:
you should differentiate between 3 major classes of command line parameters:
Image Settings
Image Operators
Image Sequence Operators
These three classes do behave differently:
Image Settings
An image setting persists as it appears on the command line.
It may affect all subsequent processing (but not previous processing):
processing such as reading an image or more images later in the command line;
processing done by a following image operator;
processing conducted by writing an image as output.
An image setting stays in effect...
...either until it is reset or replaced by a different setting of the same type,
...or until the command line terminates.
Image Operators
An image operator is applied to a (single) image and forgotten.
It differs from an image setting because it affects the image immediately as it appears on the command line.
(Remember: an image setting which persists until the command line terminates, or until it is reset.)
If you need to apply the same image operator to a second image in the same commandline, you have to repeat that exact operator on the commandline.
Strictly speaking, in compliance with the new architecture of ImageMagick command lines, all image operators should be written after the loading of the image it is meant for.
However, the IM developers compromised:
in the interest of backward compatibility, image operators can still appear before loading an image -- they will then be applied to the first image that is available to them.
Image Sequence Operators
An image sequence operator is applied to all currently loaded images (and then forgotten).
It differs from a simple image operator in that it does not only affect a single image.
(Some operators only make sense if their operation has multiple images for consumption: think of -append, -combine, -composite, -morph...)
From above principles you can already conclude: the order of the command line parameters is significant in most cases. (If you know what they do, you also know which order you need to use applying them.)
(For completeness' sake I should add: there is another class of miscellanious, or other parameters, which do not fall into any of the above listed categories. Think -debug, -verbose or -version.)
Unfortunately, the clear distinction between the 3 classes of IM command line paramaters is not yet common knowledge among (otherwise quite savvy) IM users. So it merits to get much more exposure.
This clear differentiation was introduced with ImageMagick major version 6. Before, it was more confusing: some settings' semantics changed with context and also with the order they were given. Results from complex commands were not always predictable and sometimes surprising and illogical. (Now they may be surprising too sometimes, but when you closely look at them, understanding the above, they are always quite logical.)
Which is which ?!?
Whenever you are not sure, which class one particular parameter belongs to, run
convert -help | less
Search for your parameter. Once found, scroll back: you should then find the "heading" under which it appears. Now you can be sure which type it is: an Image Setting, an Image Operator, or an Image Sequence Operator -- and take into account what I've said about them above.
Some more advice
If your job is to port your ImageMagick interface from PerlMagick to CLI, you should be aware of one other trick: You can insert
+write output-destination
anywhere on the command line (even multiple times). This will then write out the currently loaded image (or the currently loaded image sequence) in its currently processed state to the given output destination. (Think of it as something similar as the tee-command for shell/terminal usage, which re-directs a copy of <stdout> into a file.) Output destination can be a file, or show: or whatever else is valid for IM outputs. After writing to the output, processing of the complete command will continue.
Of course, it only makes sense to insert +write after the first (or any other) image operator -- otherwise the current image list will not have changed.
Should there by multiple output images (because the current image list consists of more than one image), then ImageMagick will automatically assign index numbers to the respective filename.
This is a great help with debugging (or optimizing, streamlining, simplifying...) complex command setups.
Are convert operations performed from left to right? ie the order I
pass them
Yes. If I take the following two examples, which are identical except for the operations order, I can expect different results based on the left to right.
convert rose: -sample 300% -wave 5x10 rose_post_wave.png
convert rose: -wave 5x10 -sample 300% rose_pre_wave.png
You can see the effects of the wave operation impact the image after, or before the sampling of the image.
What happens if I pass the same option twice? Are they performed
separately?
The will be executed twice. No special locking, or automatic operation counting exists.
convert rose: -blur 0.5x0.5 -scale 300% rose_blur1.png
convert rose: -blur 0.5x0.5 -blur 0.5x0.5 -scale 300% rose_blur2.png

How to programmatically determine character dimensions of my terminal window?

I am writing a script which will display a stock chart as ASCII art in a terminal emulator window. I normally use OSX/Terminal.app but a Linux-based solution would be an acceptable alternative.
My script currently has command-line args for screen width and height (again, as measured in CHARACTERS, not pixels), with defaults determined by environment variables of my own invention. I would like these scripts to determine the current window's size (IN CHARACTERS), and use THAT as the default. A typical size for a big window on my 17-inch Macbook Pro might be 200 x 68.
This is a perl script, but if you know a solution in some other language, do tell!
TIA.
Ken
The usual way to do this is tput lines and tput cols; this queries, in order:
$LINES and $COLUMNS environment variables;
termios settings, which are set by terminal emulators when you resize their windows;
the terminfo description identified by $TERM.
From C, you'd use the TIOCGWINSZ option to an ioctl system call on /dev/tty.
This is exposed by the Term::ReadKey module - from man perlfaq8:
How do I get the screen size?
If you have Term::ReadKey module installed from CPAN, you can use it to fetch the width and height in characters and in pixels:
use Term::ReadKey;
($wchar, $hchar, $wpixels, $hpixels) = GetTerminalSize();
When run interactively, bash defines $LINES and $COLUMNS. Otherwise, use curses/ncurses to retrieve the terminal dimensions.

Get current terminal color pair in Perl

I'm trying to learn about color text in a terminal window. (In case it matters I'm using Terminal.app on OS X.) I'd like to get the terminal's current foreground and background color pair. It looks like I should be able to get this info in a perl script using the Term::Cap library, but the solution eludes me.
In a perl script how would I query the terminal's current foreground and background color pair value?
The feature is outside the scope of terminfo and termcap, because it deals with terminal responses, while terminfo/termcap describe these capabilities:
how to tell the terminal to do some commonly-implemented feature (such as clearing the screen), or
what sequence of characters might some special key (such as Home) send from the keyboard.
While in principle, there is no limitation on what could be part of a terminal description, there was little commonality across terminals back in the 1980s for responses. A few terminals could report specific features, most of those were constant (e.g., version information). Most of the variable responses came after terminfo/termcap had more or less solidified in X/Open Curses. ncurses extends that, but again, most of the extensions are either features or special keys.
Terminal.app implements the most commonly-used features of xterm, but (like other imitators) omits most of the terminal responses. Among other things, xterm provides terminal responses which can tell an application what the window's colors are currently. There are a couple of command-line utilities (xtermset and xtermcontrol) which have been written to use this information (and again, they cover only a part of the repertoire). Using xtermcontrol demonstrates that Terminal.app is lacking in this area — see screenshot:
I don't think most terminals support reporting this -- and it doesn't look like termcap or terminfo have any entries for it. You're just expected to set the color pair as necessary, not to ask the terminal what it's set to right now. In the ECMA-48 standard (better known as "ANSI" after ANSI X3.64, where it used to live), the only command that makes reference to color is SGR "Set Graphic Rendition", which is purely write-only.
Dunno about perl or Terminal.app, but xterm etc will write foreground/background color control sequences to stdin if you output "\033]10;?\07" or "\033]11;?\07" respectively. Check out http://invisible-island.net/xterm/ctlseqs/ctlseqs.html, http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h2-Operating-System-Controls in particular.