I've installed TensorFlow with GPU from source on an AWS EC2 Ubuntu 14.04 system.
Using ipython results in successful tensorflow imports and training/testing with multiple GPUs.
However when I use screen ipython, I get the error:
ImportError: libcudart.so.7.0: cannot open shared object file: No such file or directory.
I suspected this was a problem with environment variables, and found that indeed
import os
os.environ['LD_LIBRARY_PATH']
returns ':/usr/local/cuda/lib64' without screen and gets a KeyError with screen.
So I created ~/.ipython/profile_default/startup/fix_ld_lib_path.py with contents
import os
os.environ['LD_LIBRARY_PATH'] = ':/usr/local/cuda/lib64'
and now screen ipython gives nearly the same environment variables (except for some that relate to terminal display that appear unrelated) but I still get the same import error as before.
Screen unsets various environment variables (including LD_LIBRARY_PATH, see this Unix StackExchange answer for details). As you've discovered, setting os.environ['LD_LIBRARY_PATH'] doesn't fix the ImportError because this environment variable must be set when the process launches. The easiest way to fix this is to modify your screen invocation as follows:
$ screen env LD_LIBRARY_PATH=$LD_LIBRARY_PATH ipython
This will set LD_LIBRARY_PATH within the screen session and it will take effect before ipython launches, as is needed.
Related
I have installed pico-go extension in VSCode. When I do any View>Command Palette>Pico-Go > Help > Show Pico Pin Map... I always get Command 'Pico-Go > Help > Show Pico Pin Map' resulted in an error (command 'picogo.extra.pins' not found).
I get a similar error for any pico-go command. I have installed different versions of the Pico-go code, did restart after each and the problem is exhibited in all the versions I have tested. Pico-go states that it is dependent on Python 3 and Python3.9.1 that is installed and running in VSCode.
I have the same problem using CircuitPython commands on both systems. The Dependencies list includes Python and Juypter which are both installed.
I have a second system that has pico-go installed and it works properly. I do not know how to isolate what is wrong in the first system.
Even if I get this problem solved the getting started code
from machine import Pin
import time
pin = Pin(25, Pin.out)
while True:
pin.toggle()
time.sleep_ms(1000)
fails with
File "e:\code.py", line 1, in <module>
from machine import Pin
ModuleNotFoundError: No module named 'machine'
Make sure you click the RUN that's at the bottom of the screen (in the window border) and not the tempting one at the top. The one at the top will run your Python code on your computer, which doesn't know what the Machine any other Pico-specific library is. The one at the bottom sends the code to the Pico and runs it there - you can confirm because "Pico Console" will be listed and not "Python".
I had started with udacity deep learning course and was setting up environments. I think the kernel notebook uses does not use python from conda environment. Following are some of the results of things I have tried.
Started conda environment
source activate tensorflow
With python terminal inside conda environment from linux terminal:
import sys
sys.executable
>>> '/home/username/anaconda2/envs/tensorflow/bin/python'
Also tensorflow gets imported with python shell
With ipython terminal inside conda environment, it shows same executable path. and tensorflow gets imported inside ipython shell.
However with jupyter notebook when I execute a cell in notebook, tensorflow module cannot be found. Also terminal spawned from notebook shows executable path of global python installation which is in anaconda/bin directoty, not of environment I had created from which I started the notebook
'/home/username/anaconda2/bin/python'
However conda environment of shell is still tensorflow
conda info --envs
# conda environments:
#
tensorflow * /home/username/anaconda2/envs/tensorflow
root /home/username/anaconda2
Does that mean kernel is linked to python installation in this location and not in conda env? How to link the same?
There is some more nuance to this question that is good to clarify. Each notebook is bound to a particular kernel. With the latest 4.0 release of Anaconda we (Continuum) have bundled a Conda-environment-aware extension that will try to associate a Notebook with a particular Conda environment. If that cannot be found then the "default" environment (or "root" environment) will be used. In your case you have a Notebook that is, I am guessing, asking for the default (or "root") environment, and so Jupyter starts a kernel in that environment, and not in the environment from which the Jupyter server was started. You can change the associated kernel by going to the Kernel->Change kernel menu and picking your tensorflow environment's kernel, along the lines of this:
Or when you create a new Notebook you can pick at that time which Conda environment's kernel should back the Notebook (note that one Conda environment can have multiple kernels available, e.g. Python and R):
We appreciate that this can be a common cause of confusion, especially when sharing notebooks, since the person who shared it either used the "default" kernel (probably called just "Python"), or they were using a Conda environment with a different name. We are working on ways to make this smoother and less confusing, but if you have suggestions for expected/desired behavior, please let us know (GitHub issue to https://github.com/ContinuumIO/anaconda-issues/issues/new is the best way to do this)
I'm using ipython 4.1.2 (installed via Anaconda). I locate the profile location (which is the default)
ipython locate profile
/home/myname/.ipython/profile_default
ipython kernel runs fine, saying
To connect another client to this kernel, use:
--existing kernel-8664.json
However, when I look inside /home/myname/.ipython/profile_default/security, it is entirely empty.
What's the issue here?
Connection files now live in a different location, because they are a component of the Jupyter architecture, rather than an IPython specific thing. The exact location depends on your system, but you can find it by running:
jupyter --runtime-dir
Or in Python code, call:
jupyter_core.paths.jupyter_runtime_dir()
The missing libcublas.so problem has been around for some time. The most common problem is that the $PATH and $LD_LIBRARY_PATH environment variable is not set properly. And solutions for command line scenarios have been posted in the NVIDIA forum and here.
But no specific solution has been out for similar symptoms in ipython or notebook. Here is my own work around.
The problem is still due to environment variables: ipython and notebook cannot propagate the settled $PATH and $LD_LIBRARY_PATH. So when this happens, the first thing to check is
import os; print(os.environ['PATH']); print(os.environ['LD_LIBRARY_PATH'])
Most probably the bin and lib paths are not in these environs.
To solve this for ipython, use sudo PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH ipython when starting ipython.
And for notebook, add these lines to the end of jupyter_notebook_config.py:
import os
os.environ['PATH'] += ':/usr/local/cuda/bin'
os.environ['LD_LIBRARY_PATH'] = '/usr/local/cuda/lib64'
I am experimenting with the music21 library, in preparation for a Machine Learning project that involves genre classification and categorization. I and following some tutorials available here. I am using MuseScore as my MusicXML program, and I am trying to run the whole thing from iPython.
Although I can run the some of the turtorials from the terminal, some elements don't seem to run well from inside iPython. For example:
In [3]: sBach.show()
Out[3]: <music21.ipython21.objects.IPythonPNGObject at 0x10da0aa10>
The line above shows that the PNG object is created, but not displayed. The expected output for 3 above is the following:
Experimenting with the following iPython command, I get a placeholder for an image, but not image.
In [6]: %load_ext music21.ipython21.ipExtension
In [7]: sBach.show()
I can't find any problem with my MusicXMLPath. This tutorial refers to the use of musc21 with Anaconda, but all my developments is done with Enthought, so I prefer not to run another virtual environment to use music21 with iPython.
Is there any way to run music21 in an Enthought/iPython notebook?
I have been grappling with this issue myself. ... Have you set your musicxmlPath in music21? If you have not, it's done via environment.set(key, value). You can query for available keys with environment.keys(). I hope this isn't too simple an answer, but it cleared up the problem for me.
This should be in the iPython music21 documentation somewhere, my apologies: iPython in music21 requires Lilypond to be installed for images to be generated within the notebook itself. There hasn't been (and won't be until MuseScore 2.0 is released) a way using MusicXML to generate PNG images of scores directly.
Edit: 2015 July; music21 2.0 w/ MuseScore 2 will generate the PNG images with MuseScore if it is installed and fallback to Lilypond if it is not installed.
If not yet tried, some steps to isolate the cause of the problem:
1) Update to the latest Canopy (Edit: currently 1.4.1) (might help this, will help generally, won't hurt).
2) Change the Pylab backend in Canopy's IPython shell to "Inline (SVG)", via Preferences / Python. (The default Qt backend in that shell conflicts with music21's use of the tkinter library.)
3) Test your script in that shell rather than in the notebook.
4) Ensure that Canopy User Python is your default Python in a Terminal window, as described here.
5) Test your scripts inside of plain ipython terminal (from Terminal, type ipython).
6) Test in ipython terminal in pylab mode (ipython qtconsole --pylab=inline).
7) Test your scripts inside of ipython notebook running in a regular browser (from Terminal, type ipython notebook, and/or ipython notebook --pylab=inline).
Had similar issues before. It's the same problem when people try to use plot function in ipython/jupyter notebook. You need to call
%matplotlib inline
For me the issue was solved by uninstalling the snap version of musescore and installing it from ppa:mscore-ubuntu/mscore3-stable via https://launchpad.net/~mscore-ubuntu/+archive/ubuntu/mscore3-stable