Emacs multi-term not displaying special characters correctly - emacs

This is odd. I have defined the following prompt in zsh:
local user_host='%{$terminfo[bold]$fg[green]%}%n # %m%{$reset_color%}'
local current_dir='%{$terminfo[bold]$fg[blue]%} %~%{$reset_color%}'
local git_branch='$(git_prompt_info)%{$reset_color%}'
local return_code="%(?..%{$fg[red]%}%? ↵%{$reset_color%})"
PROMPT="╭─${user_host} %D{[%a, %b %d %I:%M:%S]} ${current_dir} ${git_branch}
╰─%B$%b "
RPS1="${return_code}"
It works great on gnome-terminal as well as in an ansi-term terminal in Emacs (M-x ansi-term) - see the example below:
However, it does not work well under multi-term in Emacs as you can see below:
I thought multi-term would be capable of interpreting the same set of escape characters that a terminal like gnome-terminal or ansi-term does. Why is it not interpreting the escape characters returned by git-prompt_info and others correctly?
I have also tried:
M-x set-terminal-coding-system and setting it to utf-8-unix
TERM=eterm-color within the multi-term terminal, or before calling Emacs, etc.
TERM= within the multi-term terminal, or before calling Emacs, etc.
Removing any export TERM from my .zshrc
Update (January 29, 2014):
The best solution so far seems to be to do the following:
TERM=xterm-256color
but causes another problem that I have reported here: Passing escape sequences to shells within ansi-term in Emacs.

Why is it not interpreting the escape characters returned by
git-prompt_info and others correctly?
The answer is most likely that multi-term just isn't prepared to accept those escape sequences, in that format, for whatever reason. This may be a configuration issue, bug, or intentional. Setting the mode to accept colors, i.e. TERM=xterm-256color, improves the situation because it then accepts color escape sequences similar to the standard format used among terminal emulators, e.g.:
RED='\033[0;31m'
NC='\033[0m' # No Color
echo "I ${RED}love${NC} Stack Overflow\n" # this output IS NOT interpreting the escapes
echo -e "I ${RED}love${NC} Stack Overflow\n" # this output IS interpreting them
code borrowed from here
the salient part is the [0;31m for color, which is referenced in the linked thread in "Update 2" of your other question, asking why lines are printed out that begin with 4m which is part of this color escape sequence.
Here is more info, with a relevant excerpt:
Thus it is your terminal emulator that understands colors. Your
terminal emulator understands that \033[0;36m is cyan, but another
terminal emulator might use an entirely different set of characters
for cyan (though no sane terminal emulator would flaunt the standard
and do this). This is the reason for tput. When you run tput setaf
6, tput is going to look up your terminal's escape codes for the
color 6 (cyan), and output that escape code.
I suspect, in your other question, the reason that alt+b and alt+f aren't working in your other question is due to the terminal width count being off because of improper interpretation of these escape sequences which are supposed to be non-printing or zero-width. I haven't messed with multi-term a lot but the solution may involve using tput or similar to allow it to properly understand the escape sequences.
Possible relevant troubleshooting info

Related

How to fix some tmux things (screen instead of xterm; function keys)?

Why does tmux change the terminal from xterm to screen, and how can I fix the resulting text color change in emacs? I think the easiest way would be to prevent it from changing to screen in the first place.
I can use TERM=xterm emacs file.ext to do it temporarily, but that's just a workaround that doesn't solve the root of the problem.
Furthermore, the function keys no longer work in emacs when using tmux. Instead of F3 and F4 being macro shortcuts, they just print a tilde as they would in the shell. This seems unrelated to xterm/screen mentioned above. What is happening here, and how I can fix this?
tmux sets TERM to screen because that terminal description is limited to things that tmux knows how to work with. Like screen, tmux translates features from the outer terminal description to the inner.
If a special key (function-key, cursor-key, etc) does not have an exact match in the terminal description, tmux will ignore it.
The default configuration for PuTTY sends different escape sequences for F1-F4. The sequences which PuTTY sends are not in the terminal description for xterm.
Here's a comparison of the two (as a CSV file, but readable enough):
NAME,putty,xterm
kf1,\E[11~,\EOP
kf2,\E[12~,\EOQ
kf3,\E[13~,\EOR
kf4,\E[14~,\EOS
kf5,\E[15~,\E[15~
kf6,\E[17~,\E[17~
kf7,\E[18~,\E[18~
kf8,\E[19~,\E[19~
kf9,\E[20~,\E[20~
kf10,\E[21~,\E[21~
kf11,\E[23~,\E[23~
kf12,\E[24~,\E[24~
kf13,\E[25~,\E[1;2P
kf14,\E[26~,\E[1;2Q
kf15,\E[28~,\E[1;2R
kf16,\E[29~,\E[1;2S
kf17,\E[31~,\E[15;2~
kf18,\E[32~,\E[17;2~
kf19,\E[33~,\E[18;2~
kf20,\E[34~,\E[19;2~
You'd have trouble getting PuTTY to send F13-F20, but will certainly run into trouble using PuTTY and tmux with TERM=xterm.
Regarding colors, the same issue applies. The screen terminal description tells applications that the terminal can support 8 colors, and tells how to display those eight colors. If your external terminal can do more, then tmux and screen hide that.
The conversion is not perfect. GNU screen has a feature where it looks for a corresponding screen.$TERM terminal description (i.e., concatenating the outer TERM value to screen). tmux does not do that: it makes assumptions regarding xterm. But PuTTY is not xterm...
ncurses has several of those concatenated terminal-names for terminal descriptions, but no one has suggested a way for tmux to use them automatically.

Emacs: How to read command help just as one would read Man pages?

It's nice to run M-x man foo for command foo within Emacs. It's then possible to navigate easily within the man page and to find details quickly.
When the help of a command (such as git) produces limited output, one can just use a terminal instead of emacs.
But occasionally, a command (such as aws help—run in a terminal) produces extensive output. Yet the output is not compatible with the emacs Man mode. An option is to use M-x shell within emacs, but that will not display the page at once. It will report "WARNING: terminal is not fully functional" and require pressing a key endlessly until the complete help appears, or, for Emacs 25, "Could not find executable named 'groff'".
What is a good way to read long manual pages produced by commands within emacs?
I just ran into this exact problem a few days ago.
Type escape + ! then type (for example) “aws ec2 help”. This brings the help text into a new buffer called Shell Command Output, with all of the control-h characters, etc.
Switch to the new buffer with control-x then lowercase ‘o’ (for other buffer)
Type escape + lowercase ‘x’ to run an emacs function, then type ‘man’ and hit Enter. It will prompt for man page entry and default to EC2, just hit Enter. In my case, it displays an error in the status line, “error in process sentinel: Can’t find the EC2 manpage”.
However, the “man page” functions are now available, so now (in that buffer)you can type escape + x and run the function Man-fontify-manpage. The page should now look like a nice man page with bold, etc.
You can then rename the buffer (escape + x then something like ec2) so the buffer isn’t replaced if you run another shell command.
I you just want the output in a buffer, you can simply use:
M-! aws help RET
If you want the output in a shell buffer, you can do git help commit | cat (so no more "terminal is not fully functional").
Probably you can do M-! aws help | cat RET also. I do not have aws, but hopefully the piping will remove the escape characters if aws output formatting is done right. You should try also TERM=dumb aws help. Any command should know better than using fancy output when TERM is set to dumb. If aws is dumb that way itself, you could pipe its output to something that filters out control characters -- try this
For forcing man mode in an arbitrary buffer, M-x Man-mode (yes, uppercase). I am not sure if that will play nice with aws's output.
By the way, for git I suppose you know you can do man git-commit (or man git-any_git_command, in general), so you have a nice alternative to git help when using emacs (output of help and man page is the same).

Ansi-coloured file editing

Context
Have some transcript files from terminal interaction, obtained using traditional Unix command "script".
Those transcripts contain lots of control character (like backspace when editing shell commands), and lots of color code sequences as result of running various commands. Occasionally, even colorful full-terminal (ncurses-based) applications like "emacs -nw" or "aptitude" were run.
At program runtime, TERM environment variable was set as "xterm".
Need 1: read (more or less solved)
I need to read those files again and sometimes copy-paste some small parts.
The trouble is : while one color change here and there is not so much of a problem, their actual density makes the output barely readable. Worse, edited command lines (with cursor jumps and edited words) are completely unreadable.
"Okay" solution
Browse through files using e.g. "less -r". Paging forward in the same terminal setup reproduces the various color and character style.
But many other features come out more or less broken, e.g. search backwards produces jumbled terminal output, often have to pres "Ctrl-L" to clean thing up.
Need 2: editing
My preferred editor is emacs. Some people have had a similar situation when running the shell inside emacs, e.g. Something wrong with Emacs shell.
Here is not the same situation. Examples of differences: here we don't have to run an actual shell, but we need to move cursor freely like in usual editing.
Editing here means easily open such a transcript file in editor and then:
at all times through editing, see character changes (color, attributes) as conveyed by the terminal codes
(optional) some character that are neighbour on the terminal grid but are separated in file by some control characters would have a visual hint about this
ability to insert some text,
delete sections,
use all editor features like search/replace etc.
copy & paste to & from file (including to external programs, which would receive just plain text)
in my wildest dreams, some kind of "flatten" action, like select a sequence with a heavily edited command line and replace it with a simple series of characters as if it was typed in one run. "Visual hints" mentioned above would disappear.
Type Alt-: to evaluate something in the minibuffer. Evaluate (ansi-color-apply-on-region (point-min) (point-max)) and it will convert ansi color codes to be font colors.

Terminal emacs colors only work with TERM=xterm-256color

I've found that terminal emacs does not render the correct colors unless I explicitly set TERM=xterm-256color. I use gnome-terminal, and from what I understand, TERM should be set to gnome-256color. Similarly, I tend to use tmux a lot, which advises against any TERM setting other than screen-256color. Unfortunately, both of those settings (within their respective context - gnome-terminal or tmux) result in emacs having wrong colors, whereas vim displays colors correctly. However, if I export TERM=xterm-256color, the colors work just fine in emacs.
Can anyone explain what's going on, or offer a solution?
Update
Here's what I'm dealing with:
I can get the colors to look correct in the terminal by adding the following to my init.el:
(defun terminal-init-gnome ()
"Terminal initialization function for gnome-terminal."
;; This is a dirty hack that I accidentally stumbled across:
;; initializing "rxvt" first and _then_ "xterm" seems
;; to make the colors work... although I have no idea why.
(tty-run-terminal-initialization (selected-frame) "rxvt")
(tty-run-terminal-initialization (selected-frame) "xterm"))
This feels really, really wrong though. There has to be a logical explanation for this...
P.S.
I have very little knowledge of terminfo and the precise role that $TERM plays in the process of color terminal behavior. If it's safe to always use xterm-256color (even when $TERM "should" be gnome-256color or screen-256color), I'll go with that.
Maybe I'm not understanding something, buy why don't you run emacs like this:
TERM=xterm-256color emacs -nw
This way Emacs has its own TERM setting that you know works. You can also make an alias or wrap this in shell-script.
Terminals are a special type of device. When a process sends special byte sequences (called control sequences) to the terminal, it performs some action (like cursor positioning, change colors, etc).
You can read the ANSI terminal codes to find more detail about control sequences.
But terminals come from 70s, when hardware was limited in its capabilities, and a terminal cannot provide info about its capabilities (ie. which sequences it supports).
$TERM was used to resolve this issue - it allows programs to know what to send to the terminal to get the job done. termcap and terminfo are databases that store info about terminal capabilities for many $TERM names. If your $TERM is not in the db, you must ask an administrator to add it.
All terminal emulators inherit these limitations from old hardware terminals. So they need a properly set $TERM, and the terminfo/termcap DB MUST have data for this terminal. When a virtual terminal starts it sets the $TERM variable for you (and inside programs like bash). If $TERM is not in the terminfo/termcap you can quickly define an alias from $TERM to xterm-256color (you can find examples in the termcap file on how to do that).
This behavior has to do with the logic EMACS uses to determine whether the terminal background is dark or light. Run M-x list-colors-display with TERM set to either xterm-256color or screen-256color and you'll see that the exact same colors are listed. As you pointed out in the comments, the difference in color schemes that you've observed is due to the frame background mode. To see this, with your TERM set to screen-256color, compare the colors in
emacs -Q -nw --eval "(setq frame-background-mode 'light)"
and
emacs -Q -nw --eval "(setq frame-background-mode 'dark)"
The function frame-set-background-mode (in frame.el) checks to see whether the terminal type matches "^\\(xterm\\|\\rxvt\\|dtterm\\|eterm\\)" if it can't deduce the background color otherwise.
Within a running session, you can change the color scheme to 'light by evaluating
(let ((frame-background-mode 'light)) (frame-set-background-mode nil))
I am not that familiar with how emacs handles different terminals exactly. But looking at lisp/term directory in emacs sources, I found out that the existence of a function terminal-init-xxx allows you to add support for different terminals. For example, I've got:
(defun terminal-init-screen ()
"Terminal initialization function for screen."
;; Use the xterm color initialization code.
(xterm-register-default-colors)
(tty-set-up-initial-frame-faces))
in my .emacs, which adds support for screen-256color. You may try defining a similar function for gnome by renaming the above function to terminal-init-gnome.
NOTE: If you are interested, you can try to track down the calls from tty-run-terminal-initialization code. It first gets the terminal type using tty-type function, then looks at certain locations to load a relevant terminal file, then tries to locate the matching terminal-init-xxx function, and finally calls it. It may help you figure out the correct name for gnome-terminal.
It looks like unless your TERM indicates that your terminal has 256 colors, emacs will only use 8. Changing TERM to gnome-256color allowed the color registration functions to work.
There is a way to cheat, after all. When I run gnome-terminal, my terminal is set to xterm by default. Instead of changing TERM variable, it is possible to redirect xterm to another terminal, say, gnome-256color. Simply create the directory $(HOME)/.terminfo/x, then run ln -s /usr/share/terminfo/g/gnome-256color ~/.terminfo/x/xterm. I think this is better than setting TERM manually in .bashrc, because it only redirects a particular terminal to something else. A console login would still leave TERM as linux, and not xterm-256color.
Add this to your ~/.emacs:
(add-to-list 'term-file-aliases
'("st-256color" . "xterm-256color"))
It tells emacs that if it sees TERM=st-256color then it should initialize the terminal as if it had seen TERM=xterm-256color.
Longer answer:
Emacs is showing strange colors because it thinks your terminal can only support 8 colors. In Emacs, run M-x list-colors-display to see the colors it thinks are available. The correct number of colors is detected during terminal-specific initialization. It says, in part:
Each terminal type can have its own Lisp library that Emacs loads when run on that type of terminal.
On my machine, the terminal-specific initialization files are in /usr/local/share/emacs/25.*/lisp/term. It has files for xterm, rxvt, screen, etc. but nothing for st. We need to help Emacs find the right initialization file. The documentation further says:
If there is an entry matching TERM in the term-file-aliases association list, Emacs uses the associated value in place of TERM
So that association list is a recommended way to handle unknown terminals. It works without you having to manually override the TERM environment variable.
On ubuntu 10.04 I too had noticed that running emacs -nw inside byobu/tmux/screen was using different colours from emacs -nw in the regular gnome-terminal.
I found that this is because byobu was setting TERM to screen-bce. Then setting TERM to xterm (for me, in the normal gnome-terminal TERM=xterm) gave me the same syntax highlighting when not running through byobu/screen.
So still not sure what the proper solution is.
See also this post:
Emacs Python-mode syntax highlighting

gnu screen - changing the default escape command key to ALT-X?

In GNU screen, I want to change the default command binding to Alt-s (by tweaking .screenrc) instead of the default C-a, the reason is I use emacs hence GNU screen binds the C-a key, sending "C-a" to the emacs becomes tedious (as #Nils said, to send "C-a" I should type "C-a a"), as well as "C-a" in bash shell, and I could change the escape to C- but some of them are already mapped in emacs and other combinations are not as easy as ALT-s . If anyone has already done a ALT key mapping, please do let me know.
It is possible to work around :escape command limitations using registers and :bindkey command. Just put this in .screenrc:
# reset escape key to the default
escape ^Aa
# auxiliary register
register S ^A
# Alt + x produces ^A and acts as an escape key
bindkey "^[x" process S
## Alt + space produces ^A and acts as an escape key
# bindkey "^[ " process S
See http://adb.cba.pl/gnu-screen-tips-page-my.html#howto-alt-key-as-escape
From my reading of man screen it seems like the only meta character that screen can use for the command binding is CTRL:
escape xy
Set the command character to x and the character generating a literal
command character (by triggering the "meta" command) to y (similar to
the -e option). Each argument is either a single character, a two-character
sequence of the form "^x" (meaning "C-x"), a backslash followed by an octal
number (specifying the ASCII code of the character), or a backslash followed
by a second character, such as "\^" or "\\". The default is "^Aa".
If there is some mapping that you don't use in emacs, even if it's inconvenient, like C-|, then you could use your terminal input manager to remap ALT-X to that, letting you use the ALT binding instead. That would be a little hackish though.
I'm an Emacs and screen user as well. Although I rarely use Emacs in a terminal -- and as such in a screen session -- I didn't want to give up C-a for the shell either (which uses Emacs key bindings). My solution was to use C-j as the prefix key for screen, which I was willing to sacrifice. In Emacs programming modes it is bound to (newline-and-indent) which I bound to RET as well, so I really don't miss it.
By the way: I know this is an advise rather than an answer, but I felt this would be valuable enough to post nevertheless.
To make Alt+X the default prefix for commands and free C-a, add the following lines to .screenrc:
escape ^||
bindkey "^[x" command
As a side effect C-| will be command prefix too. If you need this keys to be free too, then fix "escape ^||" accordingly.
Screen doesn't have any shorthand syntax for alt bindings, but you can give it the octal code directly. For instance on my machine, Alt-x has the hex code F8, or 370 octal, so putting
escape \370x
in my screenrc changed the escape code to alt-X
Tested and works with screen 4.00.03 on Linux.
You may have to change the escape, since I think this may depend on things like your language and codeset, etc: how I found out what my escape code was was to type
$ echo -n ^QM-x | perl -ne 'printf "%lo\n", ord($_)'
^Q is the quoted-insert command for readline (it inserts what you type directly without trying to interpret it) and M-x was a literal Alt-X.
Fellow emacs user here.
The best solution I've found is a ~/.screenrc file with the following:
# C-a :source .screenrc
escape ^gg
Live updated here: https://gist.github.com/1058111
See also: http://ubuntuforums.org/showthread.php?t=498675
Something I have had for years in my .screenrc:
escape ^Zz
which is now hardwired in muscle memory for me.
Somehow I ended up having to share a screen with someone else's config, and now I keep stopping processes all the time (bash ^Z)... Not funny...