I'm using jupyter inside a conda environment in vscode on Ubuntu and all works great. However, recently I've experienced issues with AWS access tokens expiring inside my jupyter notebook.
It seems that these access tokens get cached at some level, either in the code environment or somewhere in the jupyter layer, but I couldn't figure out where and how to clean it up.
Example:
I'm using aws-vault to generate SSO credentails, that can be accessed in the environment.
After starting a session, I can access the AWS_SESSION_EXPIRATION from my terminal:
echo $AWS_SESSION_EXPIRATION
2022-04-20T17:40:36Z
when I start vscode, open a terminal, I get the same.
When I activate my environment, and run
python -c "import os;print(os.environ['AWS_SESSION_EXPIRATION'])"
I get the same.
However, when I open a new jupyter notebook and select the same environment, and run
import os
print(os.environ['AWS_SESSION_EXPIRATION'])
I get
2022-04-06T11:09:03Z
I've tried locating any files that could cache the env variables either in the conda env or jupyter extension. I've also tried restarting vscode and the machine itself - all without success.
Can anyone help me clean this up?
Thanks!!
Edit:
Another example:
If I set in my terminal
export FOO=BAR
then I can access $FOO as expected through the terminal and jupy notebook within vscode.
Then I close vscode, run
unset FOO
reboot the machine, start vscode new and open a new notebook with the same conda environment.
Within, running
import os
print(os.environ['FOO'])
STILL returns BAR
There seems to be an issue in the vscode-jupyter extension, that somehow stores the environment variables in the kernel.json file.
At the moment of writing this, the issue is still open. Working in ubuntu, this workaround from the issue's thread works for me:
Close code completely
find ~/.vscode/extensions/ms-toolsai.jupyter*
code .
Run a notebook and it will pick up the changes.
With closed vscode, issue the following command to delete cache:
find ~/.vscode/extensions/ms-toolsai.jupyter* -name kernel.json -delete
Source: https://github.com/microsoft/vscode-jupyter/issues/9774#issuecomment-1110328329
I have recently installed ipython using Enthought's EPD python install - and when starting the iPython HTML notebook from the command prompt by typing:
ipython notebook --pylab=inline
I manage to get the localhost browser notebook screen pop up correctly.
However when I try to create a new notebook by clicking "New Notebook" I get the following error message:
"Creating Notebook Failed The error was: Unexpected error while autosaving notebook: C:\Windows\System32\Untitled0.ipynb [Errno 17] No usable temporary file name found"
I am assuming this i sbecause I may not have write privilege for that particular drive. So I have tried to go into the "ipython_notebook_config.py" file and change the following:
# The directory to use for notebooks and kernels.
c.NotebookApp.notebook_dir = u'C:\Users\Stuart\Documents\iPython'
and
c.FileNotebookManager.notebook_dir = u'C:\Users\Stuart\Documents\iPython'
I have then closed down all the cmd windows and started the ipython notebook agaion. But when I click on "New Notebook" I get the same error message as before:
"Creating Notebook Failed The error was: Unexpected error while autosaving notebook: C:\Windows\System32\Untitled0.ipynb [Errno 17] No usable temporary file name found"
Could someone please help me as to how I can get this working? Any help very much appreciated.
The answer kindly provided by #Jakob in the comments above did the trick:
"Can you try switching to C:\Users\Stuart\Documents\iPython in the terminal before starting the notebook?"
Just change the directory where are run your iPython notebook. For make it, you right-click on the shortcut and edit properties. In this properties, a field named "run directory" or something like that. Put your link in this field.
I just experienced the same problem. I even erased all the untitled.ipynb files in the directory. Then I realized that I had other copies of Anaconda terminal open. When I closed them and tried again, things went back to normal.
If you run the IPython as administrator you won't run into error for starting a new notebook. To do that right click on the Ipython shortcut and click on run as administrator.
I also had the same problem, I was not able to create the new notebook or access existing notebook present in that directory.
Error Message - Unexpected error while saving file:/path/ database is locked
Turns out my old anaconda jupyter notebook terminals were open and running in the background. Every time I started jupyter notebook I used the new instance that led me to this problem. When I closed all terminals and restarted new Jupyter notebook terminal it started working again.
Many of the problems with Anaconda/Jupyter/Notebooks can be solved by examining and cleaning up what you have in your environmental variables such as Path or, if you trying to set up files to store Notebooks that you develop.
There is a very good discussion of environmental variables here:
http://johnatten.com/2014/12/07/adding-and-editing-path-environment-variables-in-windows/
It is obvious that if Anaconda/Jupyter/Notebook can't find the files they can't run them.
At a minimum your path in environmental variables should contain:
c:\users\*******\Anaconda3 where ******** is your user name
c:\users\*******\Anaconda3\Scripts
then you could create environmental variables that point to your personal Notebook code directories: (note: there can't be any spaces in the addresses) in Windows Environmental Variables (System Properties --> Environmental Variables --> add to User and System variables
variable value
NOTEBOOK address of your personal Notebook location
TESTING address of your Notebook Testing location
With this setup you can on the Anaconda Command
jupyter notebook %TESTING%
or
jupyter notebook %NOTEBOOK%
Another way you can go to your own Notebook directory is to change
jupyter_notebook_config.py
Go to:
## The directory to use for notebooks and kernels.
c.NotebookApp.notebook_dir = 'your Notebook directory address goes here'
remove the ## and enter your directory using \'s instead of \ in the address
Then anytime you enter 'jupyter notebook' you will start at your Notebook Directory.
When I open a Jupyter notebook (formerly IPython) it defaults to C:\Users\USERNAME.
How can I change this so to another location?
jupyter notebook --help-all could be of help:
--notebook-dir=<Unicode> (NotebookManager.notebook_dir)
Default: u'/Users/me/ipynbs'
The directory to use for notebooks.
For example:
jupyter notebook --notebook-dir=/Users/yourname/folder1/folder2/
You can of course set it in your profiles if needed, you might need to escape backslash in Windows.
Note that this will override whatever path you might have set in a jupyter_notebook_config.py file. (Where you can set a variable c.NotebookApp.notebook_dir that will be your default startup location.)
%pwd #look at the current work dir
%cd #change to the dir you want
For Windows 10
Look for the jupyter_notebook_config.py in
C:\Users\your_user_name\.jupyter or look it up with cortana.
If you don't have it, then go to the cmd line and type:
jupyter notebook --generate-config
Open the jupyter_notebook_config.py and do a ctrl-f search for:
c.NotebookApp.notebook_dir
Uncomment it by removing the #.
Change it to:
c.NotebookApp.notebook_dir = 'C:/your/new/path'
Note: You can put a u in front of the first ', change \\\\ to /, or change the ' to ". I don't think it matters.
Go to your Jupyter Notebook link and right click it. Select properties. Go to the Shortcut menu and click Target. Look for %USERPROFILE%. Delete it. Save. Restart Jupyter.
As MrFancypants mentioned in the comments, if you are using Jupyter (which you should, since it currently supersedes the older IPython Notebook project), things are a little different. For one, there are no profiles any more.
After installing Jupyter, first check your ~/.jupyter folder to see its content. If no config files were migrated from the default IPython profile (as they weren't in my case), create a new one for Jupyter Notebook:
jupyter notebook --generate-config
This generates ~/.jupyter/jupyter_notebook_config.py file with some helpfully commented possible options. To set the default directory add:
c.NotebookApp.notebook_dir = u'/absolute/path/to/notebook/directory'
As I switch between Linux and OS X, I wanted to use a path relative to my home folder (as they differ – /Users/username and /home/username), so I set something like:
import os
c.NotebookApp.notebook_dir = os.path.expanduser('~/Dropbox/dev/notebook')
Now, whenever I run jupyter notebook, it opens my desired notebook folder. I also version the whole ~/.jupyter folder in my dotfiles repository that I deploy to every new work machine.
As an aside, you can still use the --notebook-dir command line option, so maybe a simple alias would suit your needs better.
jupyter notebook --notebook-dir=/absolute/path/to/notebook/directory
A neat trick for those using IPython in windows is that you can make an ipython icon in each of your project directories designed to open with the notebook pointing at that chosen project. This helps keep things separate.
For example if you have a new project in C:\fake\example\directory
Copy an ipython notebook icon to the directory or create a new link to the windows "cmd" shell. Then right click on the icon and "Edit Properties"
Set the shortcut properties to:
Target:
C:\Windows\System32\cmd.exe /k "cd C:\fake\example\directory & C: & ipython notebook --pylab inline"
Start in:
C:\fake\example\directory\
(Note the added slash at the end of "start in")
This runs windows command line, changes to your working directory, and runs the ipython notebook pointed at that directory.
Drop one of these in each project folder and you'll have ipython notebook groups kept nice and separate while still just a doubleclick away.
UPDATE: IPython has removed support for the command line inlining of pylab so the fix for that with this trick is to just eliminate "--pylab inline" if you have a newer IPython version (or just don't want pylab obviously).
UPDATE FOR JUPYTER NOTEBOOK ~ version 4.1.1
On my test machines and as reported in comments below, the newest jupyter build appears to check the start directory and launch with that as the working directory. This means that the working directory override is not needed.
Thus your shortcut can be as simple as:
Target (if jupyter notebook in path):
jupyter notebook
Target (if jupyter notebook NOT in path):
C:\Users\<Your Username Here>\Anaconda\Scripts\jupyter.exe notebook
If jupyter notebook is not in your PATH you just need to add the full directory reference in front of the command. If that doesn't work please try working from the earlier version. Very conveniently, now "Start in:" can be empty in my tests with 4.1.1 and later. Perhaps they read this entry on SO and liked it, so long upvotes, nobody needs this anymore :)
In iPython Notebook on Windows, this worked for me:
cd d:\folder\
Besides #Matt's approach, one way to change the default directory to use for notebooks permanently is to change the config files. Firstly in the cmdline, type:
$> ipython profile create
to initialize a profile with the default configuration file. Secondly, in file ipython_notebook_config.py, uncomment and edit this line:
# c.NotebookManager.notebook_dir = 'D:\\Documents\\Desktop'
changing D:\\Documents\\Desktop to whatever path you like.
This works for me ;)
UPDATE: There is no c.NotebookManager.notebook_dir anymore.
Now, the line to uncomment and config is this one:
c.NotebookApp.notebook_dir = 'Z:\\username_example\folder_that_you_whant'
Usually $ ipython notebook will launch the notebooks and kernels at he current working directory of the terminal.
But if you want to specify the launch directory, you can use --notebook-dir option as follows:
$ ipython notebook --notebook-dir=/path/to/specific/directory
Before runing ipython:
Change directory to your preferred directory
Run ipython
After runing ipython:
Use %cd /Enter/your/prefered/path/here/
Use %pwd to check your current directory
A simpler modification to the Windows Trick above - without the need to hard-code the directory.
A) Create a batch file with the following contents:
(Note: A batch file is a simple text file containing commands that can
be run in the cmd window. It must have a '.bat' extension, therefore ...
you'll need to disable the folder setting which hides extensions of known types)
rem -- start_ipython_notebook_here.bat ---
dir
ipython notebook
pause
B) Copy and paste the batch file to any folder you want to start a notebook server in.
(Make sure it's a folder that you have permission to edit. "C:\" is not a good choice.)
C) Double-click on the batch file in Windows Explorer.
The notebook server should start as it normally does.
For linux and Windows:
Just modify 1 line, and you can change it.
1. Open file
cwp.py
in
C:\Users\ [your computer name]\Anaconda2
.
2. find the line
os.chdir(documents_folder)
at the end of the file.
Change it to
os.chdir("your expected working folder")
for example: os.chdir("D:/Jupyter_folder")
3. save and close.
It worked.
Update:
When it comes to MacOS, I couldn't find the cwp.py.
Here is what I found:
Open terminal on your Macbook, run 'jupyter notebook --generate-config'.
It will create a config file at /Users/[your_username]/.jupyter/jupyter_notebook_config.py
Open the config file, then change this line #c.NotebookApp.notebook_dir = ''
to
c.NotebookApp.notebook_dir = 'your path'
and remember un-comment this line too.
For example, I change my path to '/Users/catbuilts/JupyterProjects/'
According to official Jupyter Notebook Documentation Change
%USERPROFILE%
to your folder path
Documentation Link
3.1.1. Change Jupyter Notebook startup folder (Windows)
Copy the Jupyter Notebook launcher from the menu to the desktop.
Right click on the new launcher and change the Target field, change %USERPROFILE% to the full path of the folder which will contain all the notebooks.
Double-click on the Jupyter Notebook desktop launcher (icon shows [IPy]) to start the Jupyter Notebook App. The notebook interface will appear in a new browser window or tab. A secondary terminal window (used only for error logging and for shut down) will be also opened.
OS Windows 10
Python Anaconda 2018 ver
CHANGE WORKING DIRECTORY OF JUPYTER NOTEBOOK BY CONFIGURATION FILE:
Open cmd prompt (or Anaconda Prompt), then type 'jupyter notebook --generate-config' and press enter
This auto create a file 'jupyter_notebook_config' in the 'C:\Users\username.jupyter\' folder
Look for the created file 'jupyter_notebook_config'and edit it.
Find for #c.NotebookApp.notebook_dir = ''
Put you're desired path inside double quote, it becomes ---> c.NotebookApp.notebook_dir = 'D:/my_folder/jupiter'
Note the forward slash (/) is used, and the comment (#) was remove.
Hence,
#c.NotebookApp.notebook_dir = ''
Edit to becomes
c.NotebookApp.notebook_dir = 'D:/your/desired/path'
Let's modify the path of the Jupyter Notebook shortcut icon
6.0 On start menu, right-click on the shortcut, open folder location.
6.1 Once inside the folder, recommended to create a copy of Jupyter shortcut,
6.2 right click on the new shortcut icon to open properties,
Finally,
6.3 on Target text box, remove %USERPROFILE% at the end of path, The very long path should end with jupyter-notebook-script.py
searcch my vid
Jupyter Notebook - Change working folder path from default to desired path
just change to the preferred directory in CMD, so if you are in
C:\Users\USERNAME>
just change the path like this
C:\Users\USERNAME>cd D:\MyProjectFolder
the CMD cursor then will move to this folder
D:\MyProjectFolder>
next you can call jupyter
D:\MyProjectFolder>jupyter notebook
To do the same trick described below for Windows in OS X, create this shell script
#!/bin/bash
cd $(dirname "$0") && pwd
ipython notebook
Call it ipython-notebook.command and make it executable.
Put it in the directory you want to work in, then double-click it.
When launched from the command line, the IPython Notebook will use your current working directory. I took advantage of this and created context menu entries to open it directly from Windows Explorer. No need for shortcuts or batch scripts!
I was inspired by the registry-based 'Git GUI Here/Git Bash Here' entries created by Git for Windows. This page (archived version linked) was helpful in locating the correct keys.
This first pair is for the context menu presented with nothing selected (e.g. the directory background). The notebook will open with the current directory as it's working directory.
Key: HKCR\Directory\Background\shell\ipythonnb
Value: &IPython Notebook Here
Key: HKCR\Directory\Background\shell\ipythonnb\command
Value: "<full path to IPython notebook>" "%v"
This pair is for the context menu presented when clicking on a folder. The notebook will open with the selected folder as it's working directory.
Key: HKCR\Directory\shell\ipythonnb
Value: &IPython Notebook Here
Key: HKCR\Directory\shell\ipythonnb\command
Value: "<full path to IPython notebook>" "%1"
Pay attention to %v vs %1 arguments or it won't work. Don't forget the quotes either. On my platform the full path to IPython Notebook is C:\WinPython-32bit-2.7.6.4\IPython Notebook.exe but this value will obviously dependent on your installation.
Edit: AFAICT the full path is required even if the executable is on the system path.
Locate your ipython binary. If you have used anaconda to install ipython-notebook on a mac, chances are it will be in the /Users/[name]/anaconda/bin/ directory
in that directory, instead of launching your notebook as
./ipython notebook
add a --notebook-dir=<unicode> option.
./ipython notebook --notebook-dir=u'../rel/path/to/your/python-notebooks'
I use a bashscript in my ipython bin directory to launch my notebooks:
DIR=$(dirname $0)
$DIR/ipython notebook --notebook-dir=u'../rel/path/to/your/python-notebooks'
Note - the path to the notebook dir is relative to the ipython bin directory.
Simply follow the guide on the official site, also copied below. For the first step, instead of copying the launcher, you can just go to start menu and right click to open the location.
Copy the Jupyter Notebook launcher from the menu to the desktop.
Right click on the new launcher and change the “Start in” field by pasting the full path of the folder which will contain all the notebooks.
Double-click on the Jupyter Notebook desktop launcher (icon shows [IPy]) to start the Jupyter Notebook App, which will open in a new browser window (or tab). Note also that a secondary terminal window (used only for error logging and for shut down) will be also opened. If only the terminal starts, try opening this address with your browser: http://localhost:8888/.
On MiniConda2/Anaconda2 under Windows to change Jupyter or iPython working directory, you can modify this file:
C:\Program Files\Miniconda2\cwp.py
and add your project folder location: development_folder= 'C:\Users\USERNAME\Development'
Which is My Username \ Development in my case.
also change: os.chdir(documents_folder) to os.chdir(development_folder)
try:
documents_folder = get_folder_path(FOLDERID.Documents)
development_folder= 'C:\Users\USERNAME\Development'
except PathNotFoundException:
documents_folder = get_folder_path(FOLDERID.PublicDocuments)
os.chdir(development_folder)
subprocess.call(args, env=env)
Execute by using your regular Jupiter Notebook shortcuts.
This might help someone who doesn't want to change config file. If you are on Windows/ using Anaconda3, go to Win Start ->Search for Jupyter Notebook(env).
Click on it and the Jupyter opens up. On Jupyter webpage, on right hand side go to New -> Terminal and the terminal window opens up. In this terminal windows change the directory to the working directory, using cd command. Example: cd "c:\User\<user-name>\workingdir". Now in the same terminal window type Jupyter-notebook, this will open Jupyter with the working directory we used in cd command above.
I have both 32 and 64 bit python and ipython using WinPython, I wanted both 32 and 64 bit versions to point to the same working directory for ipython notebook.
I followed the above suggestions here I was still unable to get my setup working.
Here's what I did - in case anyone needs it:
It looks like Ipython notebook was using the configuration from C:\pythonPath\winpythonPath\settings\.ipython\profile_default
Even though ipython locate returns C:\users\Username\.ipython
As a result, modifying the ipython_notebook_config.py file did nothing to change my working directory.
Additionally ipython profile_create was not creating the needed python files in C:\pythonPath\winpythonPath\settings\.ipython\profile_default
I'm sure there's a better way, but to resolve this quickly, I copied the edited python files from C:\users\Username\.ipython\profile_default to C:\pythonPath\winpythonPath\settings\.ipython\profile_default
Now (finally) ipython notebook 64 bit runs and provides me the correct working directory
Note on Windows I'm having no issue with the following syntax:
c.NotebookApp.notebook_dir = u'C:/Users/Path_to_working_directory'
If you are using ipython in linux, then follow the steps:
!cd /directory_name/
You can try all the commands which work in you linux terminal.
!vi file_name.py
Just specify the exclamation(!) symbol before your linux commands.
import sys
sys.path.append('C:/')
For Mac OS X with blanks in target directory (follow up to #pheon). Add extra pair of double quotes around $(...) in line 2 thus. See: https://stackoverflow.com/a/1308838 (Sean Bright)
#!/bin/bash
cd "$(dirname "$0")" && pwd
ipython notebook
I have a very effective method to save the notebooks in a desired location in windows.
One-off activity: Make sure the path of jupyter-notebook.exe is
saved under environment variable.
Open your desired directory either from windows explorer or by cd
from command prompt
From the windows explorer on your desired folder, select the address
bar(in a way that the path label is fully selected) and type jupyter-notebook.exe
voila!! the notebook opens from the
desired folder and any new notebook will be saved in this location.
In command line before typing "jupyter notebook" navigate to the desired folder.
In my case my all python files are in "D:\Python".
Then type the command "jupyter notebook" and there you have it. You have changed your working directory.
This question keeps coming up when I search for ipython change pwd even though I am not interested in a notebook, but a terminal or qtconsole. Not finding a relevant config entry I tried:
# lines of code to run at IPython startup.
c.InteractiveShellApp.exec_lines = ['%cd /home/paul/mypy']
This is the base level shell class; there are Terminal and Console (and probably notebook) entries that could further customize the action.
From the docs it looks like import statements are most common in the entry, but it appears that many magic commands work as well.
If you are using ipython in windows, then follow the steps:
navigate to ipython notebook in programs and right click on it and
go to properties.
In shortcut Tab , change the 'Start in' directory to your desired
directory.
Restart the kernal.
You can also use AutoHotKey with a simple script to open a Jupyter Notebook server in a default directory (CTRL+I) or a path highlighted in explorer (or elsewhere with CTRL+SHIFT+I).
#SingleInstance Force
#NoTrayIcon
SetTitleMatchMode RegEx
; Press CTRL+ALT+I in a Windows Explorer window to launch a IPython notebook server in the current folder.
^+!i::
; Get the current path.
Send ^l
; Backup the current clipboard.
ClipSaved := ClipboardAll
; Copy and save the current path.
Send ^c
ClipWait
x = %Clipboard%
; Restore the clipboard.
Clipboard := ClipSaved
ClipSaved = ; Free the memory in case the clipboard was very large.
; Now, run the IPython notebook server.
RunWait, ipython notebook --notebook-dir "%x%", , min
return
^i::
; Now, run the IPython notebook server.
RunWait, jupyter notebook --notebook-dir "C:\Path\To\Workspace", , min
return
; Press CTRL+ALT+P to kill all Python processes.
^!p::
Run, taskkill /f /im python.exe, , min
return
Upper Solution may not work for you if you have installed latest version of Python in Windows. I have installed Python 3.6.0 :: Anaconda 4.3.0 (64-bit) and I wanted to change the working directory of iPython Notebook called Jupyter and this is how it worked for me.
Step-1 : Open your CMD and type following command.
Step1 : CMD
Step-2 : It has now generated a file in your .jupyter folder. For me, it's C:\Users\Admin.jupyter . There you will find a file called jupyter_notebook_config.py .Right click and edit it. Add the following line and set path of your working directory.
Set your own working directory in place of "I:\STUDY\Y2-Trimester-1\Modern Data Science"
We are done. Now you can try restarting your Jupyter Notebook. Hope this is useful to you. Thanks