ipython: %paste over ssh connection - ipython

In ipython >=0.11, the %paste command is required to paste indented commands. However, if I run an ipython shell in a remote terminal, the buffer %paste refers to is on the remote machine rather than the local machine. Is there any way around this?

I think this is exactly what %cpaste is for (I am always forgetting about all the things IPython does). %cpaste enters a state allowing you to paste already formatted or indented code, and it will strip leading indentation and prompts, so you can copy/paste indented code from files, or even from an interactive Python session including leading >>> or In [1] which will be stripped.

Not a brilliant solution, but I think this will work:
Use %autoindent to turn off autoindenting, type if True: and press enter. Then paste your indented code.
I think it should be possible to write an IPython extension to handle this better. I'll try to get round to looking into it. I've made an issue.
If you use IPython a lot, you may want to get the new kernel/client architecture working - it should be possible to tunnel the connections over SSH, so you can use the Qt console on your local machine, talking to a kernel on a server. But that might take a bit of fiddling to get in place.

Related

tmux 2.1 in ssh session: mouse events & making copy-paste local (ubuntu --> ubuntu)

I ssh from my ubuntu machine to a remote (also ubuntu) machine and run tmux 2.1 there. I would like to use mouse events to change/resize panes while still being able to select/copy/paste from the remote to the local machine. If I set -g mouse off, my local select/copy/paste works, with the caveat that it does not recognize pane boundaries. I tried:
set -g mouse on
unbind MouseDrag1Pane
This allows for the pane selection/resize, but the mouse drag still triggers tmux's copy mode. (Why?) So:
(1) is there some way to have mouse selection/resize of pane on, without interfering with local selection & copy?
(2) Even better, is there some way to select/copy on the remote machine and have that passed through to the local machine so that pane boundaries will be recognized? Note that I can not ssh from remote to local, as with this answer, which might otherwise work using linux's equivalents to pbcopy.
Possible Approaches to (2)? Not an expert at this, but a couple of thoughts:
Perhaps I could output the remote clipboard into a remote file, which I could pole from the local end and, when it changes, copy it to the local clipboard?
Or, perhaps when I ssh to the remote, I might also set up a reverse tunnel which could be used to shuttle something back to the local machine?
Also note that when experimenting I found that changing mouse options in my .tmux.conf and running: source-file ~/.tmux.conf does not change the mouse on/off option. I have to terminate and restart tmux to see what's happening.
Here are two easy things to try:
As of tmux 1.8, pane zoom works out of the box: [leader]z. This lets you zoom a pane to full screen. This makes copies possible even when you have vertically split panes because the others are temporarily hidden. If you are stuck with an older tmux, there are ways to simulate this zoom with key bindings.
Try holding down the shift key, then click and drag to select text. Right click selection and copy. This will copy to your local clipboard.

Sending ipython to the background with control-z

I frequently need to access the shell directly. In the past few weeks, every time I've used Ctrl+Z to send an ipython task to the background and tried to resuscitate it with fg, it has failed. I suspect this has to do with some recent change in ipython. Is there any way to make ipython foregroundable again?
EDIT: ipython version 2.2.0, and the failure is that the terminal hangs and never recovers. All I can do after fg is send ipython to the background again with Ctrl+Z and then kill it.
EDIT: Apparently this is a problem with anaconda: if I try to start ipython, it insists on wrapping it in a shell and then a python.app:
37463 ttys008 0:00.00 /bin/bash /Users/adam/anaconda/envs/astropy27/bin/python.app /Users/adam/anaconda/envs/astropy27/bin/ipython
37464 ttys008 0:00.40 /Users/adam/anaconda/envs/astropy27/python.app/Contents/MacOS/python /Users/adam/anaconda/envs/astropy27/bin/ipython
so it appears that Ctrl+Z is sending only one process to the background, perhaps? Is there a reason for this behavior, and can I disable it?
A workaround has been posted here:
https://github.com/ContinuumIO/anaconda-issues/issues/191
You can either kill -CONT {pid} explicitly before using fg, or (better), use fg %{jobid}. The former required that I also reset the terminal, while the latter just worked.
The workaround at https://github.com/ContinuumIO/anaconda-issues/issues/191
did not work for me, but I managed to workaround by below alias:
alias ipython='python -m IPython'

synchronizing history when ipython notebook and console are connected to the same kernel

I have ipython notebook running on a remote server, i.e.
ipython notebook --profile=nbserver
which I access from my local machine. Further, I ssh to the remote server from my machine, and start ipython console (terminal) on that server. I have found following command to work well:
ipython console --existing \
~/.config/ipython/profile_nbserver/security/kernel-*.json
Now I am connected to the same remote kernel from two different clients (lets call them browser and terminal). Everything works well, except one annoying detail:
1) in browser, I type a=1
2) in terminal, I type b=2
3) in both clients I can see both commands using %history. But when I want to cycle through the history (in terminal) using Up, it only shows the commands which have been typed in the terminal, (i.e b=2). Similarly, I am unable to use a + PageDown in the terminal, to go back in history and find the command starting with a.
From what I understand, my two clients are using two separate history files history.sqlite. But why does %history show all commands ?
Question:
Is there any way to configure using one history.sqlite for both clients ?
I find, having easy access to history is absolutely crucial. Moreover, I see using both terminal and browser as complementary, they both have tradeoffs and are best used combined.
You can set where the history gets loaded either by setting it at the terminal:
ipython --HistoryManager.hist_file=$HOME/ipython_hist.sqlite
or within the ipython config files:
import os
c.HistoryManager.hist_file=os.path.expanduser("~/ipython_hist.sqlite")

Starting mongo then leaving shell without shutting down the server

Hi Guys I start Mongo remotely on Putty SSH and after mongod, it says "listening for ports" but I can't then leave without shutting down the server. how do I do that?
On linux, look up the Screen module. There are other alternatives that do the same thing as well. It basically makes a saved session that you can reattach to later -- I use it extensively to run long tests/services that I can reattach to quickly.
The Starting and Stopping Mongo documentation article explains several options, one of which is using the --fork command line parameter to start mongod as a background process. Additionally, you can look into using service controls provided by your operating system (e.g. Windows services, init.d, upstart).
You can make a quick Mongo service script in either Windows or Linux.
You can also press CTRL+Z which will basically switch it to a background process. This basically gives you back temrinal (or at least does for me).
I also like "Screen". It is quite powerful and easy to navigate so I would personally give that a try.

How to do remote development with Emacs?

I started using emacs as my main editor a few days ago, and I'm in the process of gathering all customizations I need.
The main usage I'm giving it is for development of a C project on a remote Linux machine (RHEL 5). So far I was using plain vim (only as a text editor) and a bunch of shell scripts, and one day I decided I would try emacs. So far I like it.
But as a drawback I see that setting up the environment to run emacs is a long way to go, and instead of using it directly on the development machine (and set up the environment on every machine I use) I would try to just use it from my laptop and do the remote development.
I know about TRAMP but that only edits remote files as far as I know. The big obstacle is running shell commands on the remote machine with a single keystroke (I bound F5 to run a custom compile command on the remote machine's emacs).
How do you run shell commands on a remote machine with a single emacs keystroke?
Do you know any other tips & tricks to do full remote development?
TRAMP supports compilation and debugging on the remote machines (on which the files reside). This should be automatic and relatively seamless. Check out the documentation for remote processes.
Don't worry. You can run anything in Emacs with a single keystroke. Others have noted ways you can do this elegantly with TRAMP, but if that doesn't suit (for whatever reason) you could also just cook up your own function using:
(defun arbitrary-remote-command ()
(interactive)
(shell-command
"ssh user#remotehost arbitrary-command"))
(global-set-key (kbd "<f5>") 'arbitrary-remote-command)
When doing remote development, I use screen which provides multiple virtual remote terminals available at the press of a couple of keys. This is indispensable for me. Not only does it provide virtual terminals, but your session is preserved on the server if your connection drops for any reason.
See the GNU Screen Survival Guide question for more information about getting started with screen.
I usually have one terminal open for source control operations, one for editing, one for compiling (I just have to switch to it and hit UpArrow+Enter), and one for debugging. Depending on the environment and application I'm working on, I may have more or fewer than this, but that's the general layout.
Nice answer from Trey Jackson, but the answer from Matthew Flaschen actually helped me more.
I couldn't find a way to get TRAMP do the compiling, but accidentally I found the command remote-compile.
Now I just add these lines to my .emacs:
(setq remote-compile-user "root")
(setq remote-compile-host "localhost -p 3000") ;This is just when tunneling
(setq compile-command " cd /usr/src/nexus/traffic && ./builder.sh compile ; cat builder.log")
(global-set-key (kbd "<f5>") 'remote-compile)
And before doing anything, set up the ssh connection using ControlMaster and I don't have to type passwords.
Summarizing:
Editing remote files: TRAMP
Multiple terminals saving states:
SCREEN
Define emacs keybindings to execute
remote commands: See Matthew Flaschen
response
Remote compiling: Command
remote-compile with ControlMaster
I think that covers some important actions to do remote-development.
I've tried TRAMP before for a situation like yours, but I too found it failing in strange ways, and slow in general.
I am now using Emacs to edit files mounted over an sshfs mount. This works very well for me.
To mount the remote file system (assuming you have the host set up correctly in .ssh/config and set up public keys to connect without a password)
$ sshfs remote:dir localdir
and edit as you like from then on!
As for compilation, see the other answers, they cover this in full.
I would just ssh into the remote machine and run emacs in terminal mode instead of using TRAMP. TRAMP is really slow when you don't expect slowness, at least in my experience. I have also seen it fail in weird ways, and it is not always obvious how to "fix" it. If you ssh in, you won't be able to use emacs UI stuff, like the menus etc, but since you are used to vi that should not be an issue for you. In fact, you can just turn them off so that your ssh emacs experience is very similar to your local experience (if local != terminal mode) (1).
Using screen may be useful if you care about preserving your session in case the connection drops - apart from that you get similar functionality as virtual terminals with emacs buffers. For example, you can open many shell buffers and run various shell commands in emacs buffers. I use this to run many instances of sqlplus (use rename-buffer to give them all nice, friendly names).
Since you are ssh'd into the remote machine, you don't need to worry about running "remote" shell commands.
On the other hand, I am not sure what you mean by "I see that setting up the environment to run emacs is a long way to go". If you have source control, this should be trivial...(I bet you have a good reason).
(1) turn off menu, tool bar and scroll bars:
(if (fboundp 'menu-bar-mode) (menu-bar-mode -1))
(if (fboundp 'tool-bar-mode) (tool-bar-mode -1))
(if (fboundp 'scroll-bar-mode) (scroll-bar-mode -1))
For me, there is a problem with TRAMP, that is the remote toolchain is too old, and often it's not under my control.
So I wrote a package (ppcompile) to help me develop locally and compiling remotely, and then fixes compilation errors locally with the help of next-error .
Assuming you don't want to run emacs in terminal mode on your remote
machine then it's fairly easy. As you know you can use TRAMP to edit
remote files. I just set compile-command to 'ssh me#remote "cd
/path/to/root && make"' and be done with it.
About tramp it may depends on tramp version you are using.
Personnally I was encoutering lot of problem with default tramp package and plink as backend to connect to remote hosts, so I upgraded tramp to a more recent version and solve numerous issues.
Actually I'm on a windows machine accessing to a linux (RH5) for remote compiling and browsing directories, reading/writing some files, compiling on remote machine works fine.
One thing no one has mentioned is running emacs off the remote machine, with a valid $DISPLAY variable corresponding to your current display.
Obviously you must run an X Server for your local display, but that's not a big deal even on windows [use the X Server built into Cygwin].
This generally requires network latency < 50ms. If you ping that remote box and get something > 50ms, you will be frustrated with this approach.
Below that, works great.
You will need to have SSH map the X port, there are plenty of docs out there on that.
What I've used since about 1994:
cat <<-EOF >> ~/.emacs
(define-key global-map "\e`" 'compile)
(define-key global-map "\e~" 'remote-compile)
(require 'compile)
(setq remote-shell-program "ssh")
EOF
Now ESC-` runs compile, and ESC-~ runs /remote/ compile. It'll ask you for the hostname, and then pre-populate the command. Both compilations share the same history buffer.
It's soooo easy. It leverages a feature that's worked the same for decades, and it doesn't require any other lifting to work -- and it's just text over an ssh, so it's going to be very responsive and efficient over long or slow or congested links. I've found this prevents Fists of Network Rage a bit more.
I have the same needs: virtual machine over Windows running Debian 10 connecting to RHEL7 development machines. and here is my solution.
First of all, I use sshfs to mount my remote home in a way that preserves paths in my local machine : shfs -o allow_other <remote-user-name>#<remote-host>:/home/<remote-host-home> /home/<remote-host-home>
Second, I configure ssh so that I don't need to type password again and again: I can follow the instructions in this link.
Finally, I can launch compilations through M-x compile, I edit the command and enter something like : ssh -C <remote-user-name>#<remote-host> "make -C <build-folder>"
The great thing with it is that you can click on the links when it founds compilation errors and the source files are opened automatically. It is very convenient.
Two cents: if bandwidth is not a big concern, I would give a crack at VNC.