I have a notebook script that I run on different datasets. I want to save the script, INCLUDING the output cells, in the data folder each time I run it.
I have the following command placed at the end of my script that I run in jupyter. I intend to save pretty much what I can see on the screen to a HTML file.
"here is my notebook script with inputs and ouput including graphs"
cmd='jupyter nbconvert --to html odnp_postprocessing.ipynb --output-dir '+dataFolder
os.system(cmd)
However, nbconvert does not export the actual cells. It will print out only the input cell without ouput the first time I run, but if I re-run, it will finally export both input and output. However then if I change something in the script, it will always export the first version. Then the only way around I found is to restart the kernel and re-run (twice) the new script with the modification.
Basically, it looks like nbconvert exports some kind of buffer that is not necesseraly the actual input and ouput cells that the user sees.
What I want to do, which is programmatically saved my notebook (inputs and ouputs) into HTML, each time I run it.
Is there a command to save the current version of the notebook? I tried to add %notebook before nbconvert command but a whole bunch of old inputs are saved as well.
I know I can run the notebook within nbconvert, but I'd like to avoid it as I already run it manually in jupyter.
Any idea?
I'm using jupyter through enthought canopy in Chrome browser.
Thanks
You're not telling us exactly how you are running nbconvert; from the current notebook ? Thus it is hard to figured things out.
Nbconvert converts the current file as it is on disk; as a wild guess: you haven't save your file. If you do not save your file then nbconvert will likely not have access to the outputs of cells; and wild guess again when you run it a second time autosave have kicked in.
Remember:
- Nbconvert does not execute the file
- Nbconvert is a separate process it can't magically access what is in your browser, which is potentially a different machine.
Usually think of it this way:
Run the notebook;
Save the notebook
Close the notebook
Run nbconvert.
reopen the notebook.
If you are using this command at the end of a notebook to save it in another format, then what you are looking for are save hooks that will trigger some code – server-side – every time you save a notebook.
Side note, learn about how to run shell command in IPython; ! can be used to execute shell command in CWD and does variable interpolation.
When I open a Jupyter notebook (formerly IPython) it defaults to C:\Users\USERNAME.
How can I change this so to another location?
jupyter notebook --help-all could be of help:
--notebook-dir=<Unicode> (NotebookManager.notebook_dir)
Default: u'/Users/me/ipynbs'
The directory to use for notebooks.
For example:
jupyter notebook --notebook-dir=/Users/yourname/folder1/folder2/
You can of course set it in your profiles if needed, you might need to escape backslash in Windows.
Note that this will override whatever path you might have set in a jupyter_notebook_config.py file. (Where you can set a variable c.NotebookApp.notebook_dir that will be your default startup location.)
%pwd #look at the current work dir
%cd #change to the dir you want
For Windows 10
Look for the jupyter_notebook_config.py in
C:\Users\your_user_name\.jupyter or look it up with cortana.
If you don't have it, then go to the cmd line and type:
jupyter notebook --generate-config
Open the jupyter_notebook_config.py and do a ctrl-f search for:
c.NotebookApp.notebook_dir
Uncomment it by removing the #.
Change it to:
c.NotebookApp.notebook_dir = 'C:/your/new/path'
Note: You can put a u in front of the first ', change \\\\ to /, or change the ' to ". I don't think it matters.
Go to your Jupyter Notebook link and right click it. Select properties. Go to the Shortcut menu and click Target. Look for %USERPROFILE%. Delete it. Save. Restart Jupyter.
As MrFancypants mentioned in the comments, if you are using Jupyter (which you should, since it currently supersedes the older IPython Notebook project), things are a little different. For one, there are no profiles any more.
After installing Jupyter, first check your ~/.jupyter folder to see its content. If no config files were migrated from the default IPython profile (as they weren't in my case), create a new one for Jupyter Notebook:
jupyter notebook --generate-config
This generates ~/.jupyter/jupyter_notebook_config.py file with some helpfully commented possible options. To set the default directory add:
c.NotebookApp.notebook_dir = u'/absolute/path/to/notebook/directory'
As I switch between Linux and OS X, I wanted to use a path relative to my home folder (as they differ – /Users/username and /home/username), so I set something like:
import os
c.NotebookApp.notebook_dir = os.path.expanduser('~/Dropbox/dev/notebook')
Now, whenever I run jupyter notebook, it opens my desired notebook folder. I also version the whole ~/.jupyter folder in my dotfiles repository that I deploy to every new work machine.
As an aside, you can still use the --notebook-dir command line option, so maybe a simple alias would suit your needs better.
jupyter notebook --notebook-dir=/absolute/path/to/notebook/directory
A neat trick for those using IPython in windows is that you can make an ipython icon in each of your project directories designed to open with the notebook pointing at that chosen project. This helps keep things separate.
For example if you have a new project in C:\fake\example\directory
Copy an ipython notebook icon to the directory or create a new link to the windows "cmd" shell. Then right click on the icon and "Edit Properties"
Set the shortcut properties to:
Target:
C:\Windows\System32\cmd.exe /k "cd C:\fake\example\directory & C: & ipython notebook --pylab inline"
Start in:
C:\fake\example\directory\
(Note the added slash at the end of "start in")
This runs windows command line, changes to your working directory, and runs the ipython notebook pointed at that directory.
Drop one of these in each project folder and you'll have ipython notebook groups kept nice and separate while still just a doubleclick away.
UPDATE: IPython has removed support for the command line inlining of pylab so the fix for that with this trick is to just eliminate "--pylab inline" if you have a newer IPython version (or just don't want pylab obviously).
UPDATE FOR JUPYTER NOTEBOOK ~ version 4.1.1
On my test machines and as reported in comments below, the newest jupyter build appears to check the start directory and launch with that as the working directory. This means that the working directory override is not needed.
Thus your shortcut can be as simple as:
Target (if jupyter notebook in path):
jupyter notebook
Target (if jupyter notebook NOT in path):
C:\Users\<Your Username Here>\Anaconda\Scripts\jupyter.exe notebook
If jupyter notebook is not in your PATH you just need to add the full directory reference in front of the command. If that doesn't work please try working from the earlier version. Very conveniently, now "Start in:" can be empty in my tests with 4.1.1 and later. Perhaps they read this entry on SO and liked it, so long upvotes, nobody needs this anymore :)
In iPython Notebook on Windows, this worked for me:
cd d:\folder\
Besides #Matt's approach, one way to change the default directory to use for notebooks permanently is to change the config files. Firstly in the cmdline, type:
$> ipython profile create
to initialize a profile with the default configuration file. Secondly, in file ipython_notebook_config.py, uncomment and edit this line:
# c.NotebookManager.notebook_dir = 'D:\\Documents\\Desktop'
changing D:\\Documents\\Desktop to whatever path you like.
This works for me ;)
UPDATE: There is no c.NotebookManager.notebook_dir anymore.
Now, the line to uncomment and config is this one:
c.NotebookApp.notebook_dir = 'Z:\\username_example\folder_that_you_whant'
Usually $ ipython notebook will launch the notebooks and kernels at he current working directory of the terminal.
But if you want to specify the launch directory, you can use --notebook-dir option as follows:
$ ipython notebook --notebook-dir=/path/to/specific/directory
Before runing ipython:
Change directory to your preferred directory
Run ipython
After runing ipython:
Use %cd /Enter/your/prefered/path/here/
Use %pwd to check your current directory
A simpler modification to the Windows Trick above - without the need to hard-code the directory.
A) Create a batch file with the following contents:
(Note: A batch file is a simple text file containing commands that can
be run in the cmd window. It must have a '.bat' extension, therefore ...
you'll need to disable the folder setting which hides extensions of known types)
rem -- start_ipython_notebook_here.bat ---
dir
ipython notebook
pause
B) Copy and paste the batch file to any folder you want to start a notebook server in.
(Make sure it's a folder that you have permission to edit. "C:\" is not a good choice.)
C) Double-click on the batch file in Windows Explorer.
The notebook server should start as it normally does.
For linux and Windows:
Just modify 1 line, and you can change it.
1. Open file
cwp.py
in
C:\Users\ [your computer name]\Anaconda2
.
2. find the line
os.chdir(documents_folder)
at the end of the file.
Change it to
os.chdir("your expected working folder")
for example: os.chdir("D:/Jupyter_folder")
3. save and close.
It worked.
Update:
When it comes to MacOS, I couldn't find the cwp.py.
Here is what I found:
Open terminal on your Macbook, run 'jupyter notebook --generate-config'.
It will create a config file at /Users/[your_username]/.jupyter/jupyter_notebook_config.py
Open the config file, then change this line #c.NotebookApp.notebook_dir = ''
to
c.NotebookApp.notebook_dir = 'your path'
and remember un-comment this line too.
For example, I change my path to '/Users/catbuilts/JupyterProjects/'
According to official Jupyter Notebook Documentation Change
%USERPROFILE%
to your folder path
Documentation Link
3.1.1. Change Jupyter Notebook startup folder (Windows)
Copy the Jupyter Notebook launcher from the menu to the desktop.
Right click on the new launcher and change the Target field, change %USERPROFILE% to the full path of the folder which will contain all the notebooks.
Double-click on the Jupyter Notebook desktop launcher (icon shows [IPy]) to start the Jupyter Notebook App. The notebook interface will appear in a new browser window or tab. A secondary terminal window (used only for error logging and for shut down) will be also opened.
OS Windows 10
Python Anaconda 2018 ver
CHANGE WORKING DIRECTORY OF JUPYTER NOTEBOOK BY CONFIGURATION FILE:
Open cmd prompt (or Anaconda Prompt), then type 'jupyter notebook --generate-config' and press enter
This auto create a file 'jupyter_notebook_config' in the 'C:\Users\username.jupyter\' folder
Look for the created file 'jupyter_notebook_config'and edit it.
Find for #c.NotebookApp.notebook_dir = ''
Put you're desired path inside double quote, it becomes ---> c.NotebookApp.notebook_dir = 'D:/my_folder/jupiter'
Note the forward slash (/) is used, and the comment (#) was remove.
Hence,
#c.NotebookApp.notebook_dir = ''
Edit to becomes
c.NotebookApp.notebook_dir = 'D:/your/desired/path'
Let's modify the path of the Jupyter Notebook shortcut icon
6.0 On start menu, right-click on the shortcut, open folder location.
6.1 Once inside the folder, recommended to create a copy of Jupyter shortcut,
6.2 right click on the new shortcut icon to open properties,
Finally,
6.3 on Target text box, remove %USERPROFILE% at the end of path, The very long path should end with jupyter-notebook-script.py
searcch my vid
Jupyter Notebook - Change working folder path from default to desired path
just change to the preferred directory in CMD, so if you are in
C:\Users\USERNAME>
just change the path like this
C:\Users\USERNAME>cd D:\MyProjectFolder
the CMD cursor then will move to this folder
D:\MyProjectFolder>
next you can call jupyter
D:\MyProjectFolder>jupyter notebook
To do the same trick described below for Windows in OS X, create this shell script
#!/bin/bash
cd $(dirname "$0") && pwd
ipython notebook
Call it ipython-notebook.command and make it executable.
Put it in the directory you want to work in, then double-click it.
When launched from the command line, the IPython Notebook will use your current working directory. I took advantage of this and created context menu entries to open it directly from Windows Explorer. No need for shortcuts or batch scripts!
I was inspired by the registry-based 'Git GUI Here/Git Bash Here' entries created by Git for Windows. This page (archived version linked) was helpful in locating the correct keys.
This first pair is for the context menu presented with nothing selected (e.g. the directory background). The notebook will open with the current directory as it's working directory.
Key: HKCR\Directory\Background\shell\ipythonnb
Value: &IPython Notebook Here
Key: HKCR\Directory\Background\shell\ipythonnb\command
Value: "<full path to IPython notebook>" "%v"
This pair is for the context menu presented when clicking on a folder. The notebook will open with the selected folder as it's working directory.
Key: HKCR\Directory\shell\ipythonnb
Value: &IPython Notebook Here
Key: HKCR\Directory\shell\ipythonnb\command
Value: "<full path to IPython notebook>" "%1"
Pay attention to %v vs %1 arguments or it won't work. Don't forget the quotes either. On my platform the full path to IPython Notebook is C:\WinPython-32bit-2.7.6.4\IPython Notebook.exe but this value will obviously dependent on your installation.
Edit: AFAICT the full path is required even if the executable is on the system path.
Locate your ipython binary. If you have used anaconda to install ipython-notebook on a mac, chances are it will be in the /Users/[name]/anaconda/bin/ directory
in that directory, instead of launching your notebook as
./ipython notebook
add a --notebook-dir=<unicode> option.
./ipython notebook --notebook-dir=u'../rel/path/to/your/python-notebooks'
I use a bashscript in my ipython bin directory to launch my notebooks:
DIR=$(dirname $0)
$DIR/ipython notebook --notebook-dir=u'../rel/path/to/your/python-notebooks'
Note - the path to the notebook dir is relative to the ipython bin directory.
Simply follow the guide on the official site, also copied below. For the first step, instead of copying the launcher, you can just go to start menu and right click to open the location.
Copy the Jupyter Notebook launcher from the menu to the desktop.
Right click on the new launcher and change the “Start in” field by pasting the full path of the folder which will contain all the notebooks.
Double-click on the Jupyter Notebook desktop launcher (icon shows [IPy]) to start the Jupyter Notebook App, which will open in a new browser window (or tab). Note also that a secondary terminal window (used only for error logging and for shut down) will be also opened. If only the terminal starts, try opening this address with your browser: http://localhost:8888/.
On MiniConda2/Anaconda2 under Windows to change Jupyter or iPython working directory, you can modify this file:
C:\Program Files\Miniconda2\cwp.py
and add your project folder location: development_folder= 'C:\Users\USERNAME\Development'
Which is My Username \ Development in my case.
also change: os.chdir(documents_folder) to os.chdir(development_folder)
try:
documents_folder = get_folder_path(FOLDERID.Documents)
development_folder= 'C:\Users\USERNAME\Development'
except PathNotFoundException:
documents_folder = get_folder_path(FOLDERID.PublicDocuments)
os.chdir(development_folder)
subprocess.call(args, env=env)
Execute by using your regular Jupiter Notebook shortcuts.
This might help someone who doesn't want to change config file. If you are on Windows/ using Anaconda3, go to Win Start ->Search for Jupyter Notebook(env).
Click on it and the Jupyter opens up. On Jupyter webpage, on right hand side go to New -> Terminal and the terminal window opens up. In this terminal windows change the directory to the working directory, using cd command. Example: cd "c:\User\<user-name>\workingdir". Now in the same terminal window type Jupyter-notebook, this will open Jupyter with the working directory we used in cd command above.
I have both 32 and 64 bit python and ipython using WinPython, I wanted both 32 and 64 bit versions to point to the same working directory for ipython notebook.
I followed the above suggestions here I was still unable to get my setup working.
Here's what I did - in case anyone needs it:
It looks like Ipython notebook was using the configuration from C:\pythonPath\winpythonPath\settings\.ipython\profile_default
Even though ipython locate returns C:\users\Username\.ipython
As a result, modifying the ipython_notebook_config.py file did nothing to change my working directory.
Additionally ipython profile_create was not creating the needed python files in C:\pythonPath\winpythonPath\settings\.ipython\profile_default
I'm sure there's a better way, but to resolve this quickly, I copied the edited python files from C:\users\Username\.ipython\profile_default to C:\pythonPath\winpythonPath\settings\.ipython\profile_default
Now (finally) ipython notebook 64 bit runs and provides me the correct working directory
Note on Windows I'm having no issue with the following syntax:
c.NotebookApp.notebook_dir = u'C:/Users/Path_to_working_directory'
If you are using ipython in linux, then follow the steps:
!cd /directory_name/
You can try all the commands which work in you linux terminal.
!vi file_name.py
Just specify the exclamation(!) symbol before your linux commands.
import sys
sys.path.append('C:/')
For Mac OS X with blanks in target directory (follow up to #pheon). Add extra pair of double quotes around $(...) in line 2 thus. See: https://stackoverflow.com/a/1308838 (Sean Bright)
#!/bin/bash
cd "$(dirname "$0")" && pwd
ipython notebook
I have a very effective method to save the notebooks in a desired location in windows.
One-off activity: Make sure the path of jupyter-notebook.exe is
saved under environment variable.
Open your desired directory either from windows explorer or by cd
from command prompt
From the windows explorer on your desired folder, select the address
bar(in a way that the path label is fully selected) and type jupyter-notebook.exe
voila!! the notebook opens from the
desired folder and any new notebook will be saved in this location.
In command line before typing "jupyter notebook" navigate to the desired folder.
In my case my all python files are in "D:\Python".
Then type the command "jupyter notebook" and there you have it. You have changed your working directory.
This question keeps coming up when I search for ipython change pwd even though I am not interested in a notebook, but a terminal or qtconsole. Not finding a relevant config entry I tried:
# lines of code to run at IPython startup.
c.InteractiveShellApp.exec_lines = ['%cd /home/paul/mypy']
This is the base level shell class; there are Terminal and Console (and probably notebook) entries that could further customize the action.
From the docs it looks like import statements are most common in the entry, but it appears that many magic commands work as well.
If you are using ipython in windows, then follow the steps:
navigate to ipython notebook in programs and right click on it and
go to properties.
In shortcut Tab , change the 'Start in' directory to your desired
directory.
Restart the kernal.
You can also use AutoHotKey with a simple script to open a Jupyter Notebook server in a default directory (CTRL+I) or a path highlighted in explorer (or elsewhere with CTRL+SHIFT+I).
#SingleInstance Force
#NoTrayIcon
SetTitleMatchMode RegEx
; Press CTRL+ALT+I in a Windows Explorer window to launch a IPython notebook server in the current folder.
^+!i::
; Get the current path.
Send ^l
; Backup the current clipboard.
ClipSaved := ClipboardAll
; Copy and save the current path.
Send ^c
ClipWait
x = %Clipboard%
; Restore the clipboard.
Clipboard := ClipSaved
ClipSaved = ; Free the memory in case the clipboard was very large.
; Now, run the IPython notebook server.
RunWait, ipython notebook --notebook-dir "%x%", , min
return
^i::
; Now, run the IPython notebook server.
RunWait, jupyter notebook --notebook-dir "C:\Path\To\Workspace", , min
return
; Press CTRL+ALT+P to kill all Python processes.
^!p::
Run, taskkill /f /im python.exe, , min
return
Upper Solution may not work for you if you have installed latest version of Python in Windows. I have installed Python 3.6.0 :: Anaconda 4.3.0 (64-bit) and I wanted to change the working directory of iPython Notebook called Jupyter and this is how it worked for me.
Step-1 : Open your CMD and type following command.
Step1 : CMD
Step-2 : It has now generated a file in your .jupyter folder. For me, it's C:\Users\Admin.jupyter . There you will find a file called jupyter_notebook_config.py .Right click and edit it. Add the following line and set path of your working directory.
Set your own working directory in place of "I:\STUDY\Y2-Trimester-1\Modern Data Science"
We are done. Now you can try restarting your Jupyter Notebook. Hope this is useful to you. Thanks