I use IPython as my interactive shell and I almost always run two commands at the start - something like:
%run helper_functions.py
data = import_data('file_with_data')
I would like a notation to run these two at the same time (so I could recall them with an "up" arrow from history) akin to what echo 1 ; echo 2 does in Bash.
I am aware that I can start IPython running with a script as per How can I start IPython running a script?.
Semicolon is a valid statement separator in Python, the same as the newline, so you can use it with regular commands:
a=5; a=a+1; print "This is a: %s" % a
Anyway, as pointed out by tobias_k, this is not working with "run" which is a special IPython macro (see the IPython built-in magic commands) and changes the command line interpretation. Instead of %run helper_functions.py; data = import_data('file_with_data'), you could use something almost equivalent:
__builtin__.execfile('helper_functions.py') ; data = import_data('file_with_data')
You don't have the trace in IPython, but still your script is executed and the variables are in your interactive namespace.
Related
Question is in the title.
I know that % usually denotes a "magic variable" in IPython. That's not a concept I'm terribly familiar with yet, but I have read about it.
However, today I saw a tutorial where someone was using it to run a shell command. Normally I have seen and used !.
Is there a difference? Both seem to be doing the same thing when I try them.
The difference is this:
When you run a command with !, it directly executes a bash command in a subshell.
When you run a command with %, it executes one of the magic commands defined in IPython.
Some of the magic commands defined by IPython deliberately mirror bash commands, but they differ in the implementation details.
For example, running the !cd bash command does not persistently change your directory, because it runs in a temporary subshell. However, running the %cd magic command will persistently change your directory:
!pwd
# /content
!cd sample_data/
!pwd
# /content
%cd sample_data/
!pwd
# /content/sample_data
Read more in IPython: Built-in Magic Commands.
In matlab I can change to another shell by the bang (!) Notation.
Example:
I enter a conda Environment in MATLAB by the following command:
!cmd '"%windir%\System32\cmd.exe" /K ""C:\Program Files\Anaconda3\Scripts\activate_<conda-env-name>.bat" "C:\Program Files\Anaconda3""'
My MATLAB Command window then Displays following:
(<conda-env-name>) U:\some_starting_path>
Now, is there a way to send commands to this newly entered shell in a programmatic way, so that that command is evaluated in that very shell's syntax and not as a MATLAB-command?
For example, how can I write code that will execute a Python command without needing to enter it manually into the command line?
Not using the ! command or system(). Those are "one and done" functions.
But you can use Java's java.lang.Process API from within Matlab to control and interact with an ongoing process.
function control_another_process
pb = java.lang.ProcessBuilder(["myCommand", "myArg1", "myArg2"]);
proc = pb.start; % now proc is a java.lang.Process object
stdin = proc.getOutputStream; % Here's how you send commands to the process
stdout = proc.getInputStream; % And here's how you get its output
stderr = proc.getErrorStream;
% ... now do stuff with the process ...
end
You can use this with a shell, with python, or any other command.
Here's a Matlab class that wraps up the Java code to make it convenient to work with in Matlab: https://github.com/apjanke/janklab/blob/master/Mcode/classes/%2Bjl/%2Butil/Process.m
How can I put a comment at the end of an IPython magic command?
Eg, trying to use %cd:
%cd "dir" # comment
I want to change to directory dir but instead I get:
[Errno 2] No such file or directory: 'dir # comment'
It's not possible in general because IPython magic commands parse their own command lines.*
There are some magic commands where comments seem to work, but actually they're being handled in another way. For example:
%ls # comment works properly, but %ls is an alias to a shell command, so the shell is what's parsing the comment.
%automagic 0 # comment behaves like %automagic, toggling the setting instead of exclusively disabling it like %automagic 0 would do. It's as if the 0 # comment is all one string, which it ignores.
%run $filename # comment treats the comment as arguments, so ['#', 'comment'] ends up in sys.argv for $filename.
Workaround
You could call magic commands using exclusively Python syntax.
First get the running IPython instance with get_ipython():
ipython = get_ipython()
Then:
ipython.run_line_magic('cd', 'dir') # comment
Docs: InteractiveShell.run_line_magic()
Or for cell magics, you could go from this (which doesn't work):
%%script bash # comment
echo hello
to this:
ipython.run_cell_magic('script', 'bash', 'echo hello') # comment
Docs: InteractiveShell.run_cell_magic()
Another option, I suppose, is to run an entire cell, although, suppressing the output since it returns an ExecutionResult object:
ipython.run_cell('%cd dir'); # comment
Docs: InteractiveShell.run_cell()
* I'm not sure if this is documented explicitly, but it is documented implicitly under defining custom magics, and I found the clues in this GitHub issue: Allow comments after arguments in magic_arguments.
TL;DR: You cannot as things stand. This might or might not be an error in the implementation of %cd.
Until this is fixed, one way to avoid this is to use:
import os
os.chdir("dir") #some comment here
A second alternative is using bash commands.
However % is a magic command, not equivalent to a bash command. This is on purpose,
as it can change the current directory of the notebook.
This is not the same as e.g.
!cd dir #some comment here
Which will spawn a shell and execute the command there thus not changing the current directory. (You can verify using pwd before/after each command)
Note that if your goal is not to change the current jupyter notebook directory, you can issue multiple commands in one cell with the magic
%sh:
%%sh
cd dir #some comment here
ls #some more commands here
....
This command will spawn a shell and all bash commands will be executed there, so the current jupyter directory will not change.
I want to run IPython from the command line. However, I get a syntax error on the first line, importing pylab with the magic function %pylab is giving a syntax error on the %. The command I am using is simply ipython -i script.py.
Any ideas how to solve this?
You need to name your file script.ipy. When it ends in .ipy it can contain ipython syntax.
From ipython --help:
Usage
ipython [subcommand] [options] [files]
If invoked with no options, it executes all the files listed in sequence
and exits, use -i to enter interactive mode after running the files. Files
ending in .py will be treated as normal Python, but files ending in .ipy
can contain special IPython syntax (magic commands, shell expansions, etc.)
I want to accomplish a loop over R code within an IPython notebook. What is the best way to do this?
l = []
for i in range(10):
# execute R script
%%R -i i -o result #some arbitrary R code
# and use the output
l.append(result)
Can this be done inside a notebook (Looping over next cell)?
Have you looked into rmagic and rpy2 module?
If you have R scripts, then you can call them and assign their output to a variable using the shell command notation:
var=!R_script arguments....
The above does not need you need you to install rpy2 since ! shell command execution is basic in ipython. You can pass values of variables from ipython notebook by using $var in the arg list.