I have some data analysis steps combined in a Jupyter notebook.
As the data change, I want to be able to
Re-run all the cells (to take the new data into account)
Convert to html for viewing
I know I can do #2 through jupyter nbconvert, but how do I do #1 without manually interacting with the notebook web interface?
nbconvert can do that as well, with the --execute argument.
https://nbconvert.readthedocs.io/en/latest/execute_api.html#executing-notebooks-from-the-command-line
Related
I have quite a large jupyter notebook I'm working on in VScode, while using the high performance computing facility (connected through SSH).
I am trying to create some internal hyperlinks so I can jump from my table of contents to a specific section in the notebook. However, this does not work.
Code I'm using to create the link:
[Summary](#Summary)
some other cells
## Summary
It does create a link, but clicking on it does not work in VScode. This one keeps me right at the top of the notebook, other links to sections further down bring me to seemingly random cells down my file. Also, when opening the notebook in Github, clicking the link brings me back to the folder where my notebook is located.
The same construction does work both in a local jupyter instance, and in vscode, when not connected to the HPC.
One difference I can think of is that I use miniconda on the HPC, and the full anaconda installation locally.
Are the links not supported in miniconda? Or is it something else?
Using jupyter-lab
%run otherNotebook.ipynb
gives the following error message
Error: file not found otherNotebook.ipynb.py
How can I use the magic method and prevent it from adding .py to the file
As described here %run is for running a named file inside IPython as a program. Jupyter notebooks are not Python programs.
Notebooks can be converted to Python programs/scripts using Jupytext. Following that conversion you could then use %run.
Alternatively, you can use nbconvert to execute a notebook or use Papermill to execute a notebook. Papermill allows you to easily pass in parameters at the time of run. I have an example of both commented out in code under 'Step #5' here and 'Step#2' here.
If you are actually trying to bring the code into your present notebook, then you may want to explore importing Jupyter notebooks as modules. importnb is recommended here for making importing notebooks more convenient. Or, I just came across the subnotebook project that let's you run a notebook as you would call a Python function, pass parameters and get results back, including output contents.
I have a notebook script that I run on different datasets. I want to save the script, INCLUDING the output cells, in the data folder each time I run it.
I have the following command placed at the end of my script that I run in jupyter. I intend to save pretty much what I can see on the screen to a HTML file.
"here is my notebook script with inputs and ouput including graphs"
cmd='jupyter nbconvert --to html odnp_postprocessing.ipynb --output-dir '+dataFolder
os.system(cmd)
However, nbconvert does not export the actual cells. It will print out only the input cell without ouput the first time I run, but if I re-run, it will finally export both input and output. However then if I change something in the script, it will always export the first version. Then the only way around I found is to restart the kernel and re-run (twice) the new script with the modification.
Basically, it looks like nbconvert exports some kind of buffer that is not necesseraly the actual input and ouput cells that the user sees.
What I want to do, which is programmatically saved my notebook (inputs and ouputs) into HTML, each time I run it.
Is there a command to save the current version of the notebook? I tried to add %notebook before nbconvert command but a whole bunch of old inputs are saved as well.
I know I can run the notebook within nbconvert, but I'd like to avoid it as I already run it manually in jupyter.
Any idea?
I'm using jupyter through enthought canopy in Chrome browser.
Thanks
You're not telling us exactly how you are running nbconvert; from the current notebook ? Thus it is hard to figured things out.
Nbconvert converts the current file as it is on disk; as a wild guess: you haven't save your file. If you do not save your file then nbconvert will likely not have access to the outputs of cells; and wild guess again when you run it a second time autosave have kicked in.
Remember:
- Nbconvert does not execute the file
- Nbconvert is a separate process it can't magically access what is in your browser, which is potentially a different machine.
Usually think of it this way:
Run the notebook;
Save the notebook
Close the notebook
Run nbconvert.
reopen the notebook.
If you are using this command at the end of a notebook to save it in another format, then what you are looking for are save hooks that will trigger some code – server-side – every time you save a notebook.
Side note, learn about how to run shell command in IPython; ! can be used to execute shell command in CWD and does variable interpolation.
I was using iPython command line interface and after some operations I want to save my operation history to a notebook file. But I was not using iPython notebook from the beginning. Can I still make it?
From #Thomas K (I don't know why he didn't post an answer):
%notebook -e myhistory.ipynb
The short answer is in a couple of ways, the slightly longer answer is Yes - but you might not get what you expect!
Really long answer: The explanation is that when you are working in a notebook, now called a jupyter notebook of course, your work is stored in a series of cells each of which has one or more lines of code or markdown while when you are working in a console all of your work is a series of lines of python code.
From within a console session you can save, using %save some or all of your work to one or more python files that you can then paste, import, etc, into notebook cells. You can also save using %save -r to .ipy files your work including the magics as magics rather than the results of magics that again you can use from within your notebook later.
You can also use the %notebook magic to save all of your current history in one of an ipynb json file or a python .py text file with the -e export flag. However, it is not clear from the documentation if the history will end up in a single cell, one cell per command or some other division. A little testing suggests one cell per numbered line of your console, so a single command or definition, per cell.
Personally I will stick with outputting anything useful into python files using the %save command - or better yet start a notebook when I think I might be doing something that I would need later.
If you are using jupyter console (ie, the command line version of jupyter notebook) you can use the following command within a cell to save the notebook as a python file (.py)
save yourfilename
this will save the contents of the notebook as "yourfilename.py" in your current working directory (from which you started the jupyter console)
Alternatively, you can save the whole program by this command :
%save -r program-name 1-999999
The following commands were written to file program-name.ipy
I'm experienced with IPython Notebook but just upgraded to Jupyter 4 with nbpresent, and I want to use reveal.js for slideshows.
How can I incorporate reveal.js into my notebook slideshow?
The most common way to do this is using nbconvert:
jupyter nbconvert presentation.ipynb --to slides --post serve
For more information, see nbconvert documentation.
You would only use RISE if you need to execute commands on the kernel during the presentation.