I'm interested in implementing a behavior in IPython that would be like a combination of ! and !!. I'm trying to use an IPython terminal as an adjunct to my (Windows) shell. For a long running command (e.g., a build script) I would like to be able to watch the output as it streams by as ! does. I would also like to capture the output of the command into the output history as !! does, but this defers printing anything until all output is available.
Does anyone have any suggestions as to how to implement something like this? I'm guessing that a IPython.utils.io.Tee() object would be useful here, but I don't know enough about IPython to hook this up properly.
Here is a snippet of code I just tried in iPython notebook v2.3, which seems to do what was requested:
import sys
import IPython.utils.io
outputstream = IPython.utils.io.Tee("outputfile.log", "w", channel="stdout")
outputstream.write("Hello worlds!\n")
outputstream.close()
logstream=open("outputfile.log", "r")
sys.stdout.write("Read back from log file:\n")
sys.stdout.write(logstream.read())
The log file is created in the same directory as the iPython notebook file, and the output from running this cell is displayed thus:
Hello worlds!
Read back from log file:
Hello worlds!
I haven't tried this in the iPython terminal, but see no reason it wouldn't work as well there.
(Researched and answered as part of the Oxford participation in http://aaronswartzhackathon.org)
Related
I'm a Python/programming novice and this is probably an novice mistake, but I've scoured the internet for answers and have found none! I'm learning on a Rosalind module that's about opening data files. http://rosalind.info/problems/ini5/ I'm pretty sure I understand everything clearly, so I'm frustrated about my inability to do this simple task.
I'm using Python 3.6.2 and IDLE. The assignment is to simply open a .txt file and read a few lines.
I downloaded the .txt file to my working directory. Then, I opened up IDLE Shell and made sure I was in the right working directory (using ls & cd). I then opened a new IDLE .py file and wrote a script:
f = open('filename.txt', 'r')
f.readlines()[2]
I saved the script as p5.py. Then, I tried to run the script by calling F5. In the Shell, I got this message:
================ RESTART: /Users/liv/Desktop/Rosalind/p5.py =================
Is that an error? I think it's just a message from IDLE that IDLE has opened p5.py. Therein lies my problem, because now I have the wrong file open.
I started realizing that when I used the Shell and called it to print, and it came back with an empty string.
What am I doing wrong?? How do I get IDLE to open the filename.txt file? ...not the .py file.
The RESTART line means that IDLE has sent p5.py to Python to be run, which is exactly what you want and what you asked. Python should have then opened the text file, read it, retrieved the 3rd line, and stopped. Since p5.py has no output statements and does not raise any exceptions, you will not see anything. If you change the 2nd line to
print('line is ', f.readlines()[2])
then you should see something.
I know about nbconvert and use it to generate static html or ipynb files with the results output. However, I want to be able to generate a notebook that stays attached to a kernel that I already have running, so that I can do further data exploration after all of the template cells have been run. Is there a way to do that?
Apparently, you can do this through the Python API. I didn't try it myself, but for someone who will be looking for a solution, this PR has an example in the comments:
from nbconvert.preprocessors.execute import executenb, ExecutePreprocessor
from nbformat import read as nbread
from jupyter_client.manager import start_new_kernel
nb = nbread('parsee.ipynb', as_version=4)
kernel_name = nb.metadata.get('kernelspec', {}).get('name', 'python')
km, kc = start_new_kernel(kernel_name=kernel_name)
executenb(nb, kernel=(km, kc))
kc.execute_interactive('a') # a is a variable defined in parsee.ipynb with 'a = 1'
Not quite sure about your purpose. But my general solutions are,
to execute the notebook in command line and see the execution at the same time,
jupyter nbconvert --debug --allow-errors --stdout --execute test.ipynb
this will show the execute through all cells in debug mode even exception happens. but I can't see the result until the end of the execution.
to output the result to a html file, and then open the html file to see the results. I found this is more convenient.
jupyter nbconvert --execute --allow-errors --stdout test.ipynb >> result.html 2>&1
if you open result.html, it will be,
and all the errors and results will be shown on the page.
I would like to learn other answers/solutions from you all. thank you.
If I understood correctly you wish to open a Python console, and connect Jupyter notebook to that kernel instance?
Perhaps your solution would be to edit jupyter scripts itself and run the server in separate thread/background task implementing some sort of connection between threads and work in the jupyter console? Currently it's impossible because main thread is running the server.
This would require some work and I don't have any solution as-is, but I will look into that and maybe edit this answer if I can make it work.
But it seems that the easiest solution is to simply add another field in the notebook and do whatever you wish to do there. Is there a reason for not doing that?
I'm relatively new to NumPy/SciPy and IPython.
To execute a python script in the python interactive mode, we may use the following commands.
>>> import os
>>> os.system('executable.py')
Then the print outputs can be seen from the python prompt.
But the same idea doesn't work with IPython notebook.
In [64]:
import os
os.system('executable.py')
Out[64]:
0
In this case, I cannot see any print outputs. The notebook only tells weather execution was successful or not. Are there any ways to see the outputs when I use IPython notebook?
Use the magic function %run:
%run executable.py
This properly redirects stdout to the browser and you will see the output from the program in the notebook.
It gives you both, the typical features of running from command line plus Python tracebacks if there is exception.
Parameters after the filename are passed as command-line arguments to
the program (put in sys.argv). Then, control returns to IPython's
prompt.
This is similar to running at a system prompt python file args,
but with the advantage of giving you IPython's tracebacks, and of
loading all variables into your interactive namespace for further use
(unless -p is used, see below).
The option -t times your script. With -d it runs in the debugger pdb. More nice options to explore.
I am writing my script interactively with IPython. This is what I currently do:
write a chunk of code,
run in ipython with "run -i file_name.py".
make changes and repeat 2 until I think it is OK .
comment out the entire previous chunk.
write new chunk of code that is based on the previous one.
go back to step 2.
......
Is there more efficient way? Can I start a script from a specific line while using all the variables in current namespace?
Use ipdb ("pip install ipdb" on the command line to install it).
Suppose you want to run script "foo.py" from line 18 to 23.
You'll want to start like this:
ipdb foo.py
Now, let's jump to line 18 (i.e., ignore all the lines before the 18th):
ipdb> j 18
Next, we set a breakpoint at line 23 (we don't want to go further):
ipdb> b 23
Finally, let's execute:
ipdb> c
Job done :)
I'd personally also use the ipython notebook, but you call also use you favorite text editor and always copy out the chunk of code you want to run and use the magic command %paste to run that chunk in the ipython shell. It will take care of indentation for you.
Use the magic of %edit stuff.py (first use) and %ed -p (after the first use) and it will invoke your $EDITOR from inside of ipython. Upon exiting from the editor ipython will run the script (unless you called %ed -x). That is by far the fastest way I found to work in CLI-ipython. The notebooks are nice, but I like having a real editor for code.
(Based on lev's answer)
From the interactive shell:
%run -i -d foo.py
should then enter the debugger, and proceed with:
j <line_number>
c
etc.
EDIT: unfortunately, this seems to sort of break ipython's magic %debug command.
An IPython Notebook allows you to interactively run scripts line by line. It comes with IPython, just run:
ipython notebook
from the terminal to launch it. Its a web interface to IPython, where you can save the notebooks to *.py files by clicking save as in the settings.
Here's some more info from this video.
For something fast as well as flexible use http://qtconsole.readthedocs.io/en/stable/
It is similar to the Jupyter notebook based on your browsers (as pointed out by #agonti and #magellan88, but presumably much faster. It also has emacs style keybindings.
I use ipdb, ipython, comupled with tmux and vim and get almost IDE like features and much faster.
Is it possible to interact with an IPython interactive session (or with a kernel) from a Bash script? Ideally, I'd like to do something like this, within a shell script (I'm aware that the send subcommand probably doesn't exist like this):
# do stuff in Bash ...
# start a kernel and get its Id
KERNEL=`ipython init --command="print(__KERNELID__)"`
# do something inside the kernel
ipython send --kernel=KERNELID --command="mylist = [0,1,2]"
Then, ideally, the command
ipython send --kernel=KERNELID --command="print(mylist)"
would output
[0, 1, 2]
In the end, I would need to destroy the kernel somehow:
ipython --kernel=KERNELID --command="sys.exit()"
Probably, there is already a mechanism to do what I'd like,
right? Unfortunately, I wasn't able to find it ...
There are quite a number of ways around this problem. Since you are having to use python you might as well use python for the whole thing. Python programs can take command line arguments like mylist and do whatever you want with them.
Since you are sending commands to be evaluated make sure you are the one controlling the inputs. Don't let someone start typing "import os" and "os.unlink([your hard drive here])" for example.
For other options: Check out expect for your interactive needs http://expect.sourceforge.net/
or just the python version check out the pexpect module http://pexpect.sourceforge.net/pexpect.html