The activation script is not found when creating a Python virtual environment with the command mkvirtualenv of virtualenvwrapper.
After installing the 22.04 Linux Ubuntu version, I noticed that after installing virtualenv and virtualenvwrapper, I'm getting the following error when using the command mkvirtualenv test103 :
ERROR: Environment '/home/paul/.virtualenvs/test103' does not contain an activate script.
I made several research and here is my final .bashrc file update :
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
export VIRTUALENVWRAPPER_VIRTUALENV=/usr/local/bin/virtualenv
source /usr/local/bin/virtualenvwrapper.sh
export VIRTUALENVWRAPPER_ENV_BIN_DIR=usr/local/bin
Note that if I execute this command:
source /home/paul/.virtualenvs/test103/local/bin/activate
the virtual environment is activated! However, I prefer to use mkvirtualenv name and then workon name without errors.
Related
I want to run a job on Slurm and my Python script needs the evaluate package which I have on my local machine. I don't know if I could change the Python path on the server to match the one on my local machine, and if I could I'm afraid I might break the system.
So I followed this answer, and included a requirements.txt file with just evaluate==0.1.2 in it, and I get even more errors:
load GCC/10.2.0 (PATH, MANPATH, INFOPATH, LIBRARY_PATH, LD_LIBRARY_PATH, STD COMP VARS)
load ROCM/5.1.1 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, C_INCLUDE_PATH)
Set INTEL compilers as MPI wrappers backend
load mkl/2018.4 (LD_LIBRARY_PATH)
load PYTHON/3.7.4 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, PKG_CONFIG_PATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, PYTHONHOME, PYTHONPATH)
/var/spool/slurmd/job216863/slurm_script: line 12: virtualenv: command not found
/var/spool/slurmd/job216863/slurm_script: line 16: /env/bin/activate: No such file or directory
ERROR: Could not find a version that satisfies the requirement evaluate==0.1.2 (from versions: none)
ERROR: No matching distribution found for evaluate==0.1.2
Traceback (most recent call last):
File "eval_comet.py", line 1, in <module>
from evaluate import load
ModuleNotFoundError: No module named 'evaluate'
Most of the time, the Python version on HPCs are old. My Uni's HPC cluster has Python 3.7. If you wish to create a Python virtual environment (not conda) with a newer version, then there is a trick.
Activate the Anaconda Module, some system uses Module load and some uses load depending on your organisation.
[s.1915438#sl2 ~]$ module load anaconda/2021.05
[s.1915438#sl2 ~]$ conda create -n surrogate python=3.8
Here I created a Conda environment named surrogate with Python 3.8. Here, you can choose any version of your choice. Now you can activate the Conda environment and check the Python version.
[s.1915438#sl2 ~]$ source activate surrogate
(modulus) [s.1915438#sl2 ~]$ which python
~/.conda/envs/surrogate/bin/python
(surrogate) [s.1915438#sl2 ~]$ python --version
Python 3.8.13
Now navigate to the directory where you want to install your Python virtual environment and install the virtual environment using the following command.
(surrogate) [s.1915438#sl2 s.1915438]$ mkdir modulus_pysdf
(surrogate) [s.1915438#sl2 s.1915438]$ cd modulus_pysdf/
(surrogate) [s.1915438#sl2 modulus_pysdf]$ python3 -m venv modulus_pysdf
Logout (ctrl + D) from the server to exit the Conda environment and then login again. Remember, in my case the path to the Python virtual environment was /scratch/s.1915438/modulus_pysdf.
This is how I will activate the Python virtual environment.
[s.1915438#sl2 ~]$ cd /scratch/s.1915438
[s.1915438#sl2 s.1915438]$ cd modulus_pysdf/
[s.1915438#sl2 modulus_pysdf]$ source modulus_pysdf/bin/activate
Now I can check the Python version and the path.
(modulus_pysdf) [s.1915438#sl2 modulus_pysdf]$ python --version
Python 3.8.13
(modulus_pysdf) [s.1915438#sl2 modulus_pysdf]$ which python
/scratch/s.1915438/modulus_pysdf/modulus_pysdf/bin/python
As usual, I can install any package using pip. For example, to install evaluate you can download it from PyPI
pip install evaluate
Or if you have a requirements.txt file then you can do this. See this for more details.
cat requirements.txt | grep -Eo '(^[^#]+)' | xargs -n 1 pip install
I need to set up virtual environments for each language that I use. To do this, I'm running the Ubuntu 20.04 LTS Windows Subsystem for Linux (WSL) on Windows 10. Within WSL, I'm using Anaconda, installed in /usr/local/Anaconda, to create conda virtual environments for each language (i.e. one environment contains all my Python stuff, another contains my R stuff, etc.).
Since WSL doesn't come with a GUI, I'm using Visual Studio Code's (VSCode) Jupyter Notebook Extension to run Jupyter Notebooks to see plots/graphics. So far, I managed to easily create conda environments for Python (with ipython and ipykernel) and R (with IRkernel) and run their code in a notebook via the extension. Each time I set up an environment, the extension is easily able to find the kernel, connect to it and run the code.
However, I've not been able to set up an environment for Julia. I followed the documentation on the Julia website for installing the kernel, which is successfully found by the extension. But, when I try running a cell, the extension says it is trying to connect to the kernel, only for it to timeout and fail.
Here are the steps I have taken so far:
Create a clean conda environment (conda create -n Julia && conda activate Julia)
Install the latest version of Julia (conda install -c conda-forge julia)
Install the latest version of Jupyter (conda install -c conda-forge jupyter)
Install the Julia kernel with the built-in Julia package manager (using Pkg; Pkg.add("IJulia"))
Build the IJulia package (using Pkg; Pkg.build("IJulia"))
Confirm the presence of the Julia kernel (jupyter kernelspec list) which indeed shows the presence of a Julia kernel
Reload the VSCode connection to WSL (Ctrl + Shift + P; >Reload Window)
Shut down WSL via CMD (wsl --shutdown) for changes to take effect and reconnect
After I restart VSCode and WSL, the extension shows an option to use the Julia kernel installed in my conda environment: Julia 1.7.2 (~/.conda/envs/Julia/bin/julia). But when I create a cell and run code in a notebook, the extension creates a popup saying that it is connecting to the kernel and after some time an error message shows up:
Failed to start the Kernel.
Unable to start Kernel `Julia 1.7.2` due to connection timeout.
View Jupyter log for further details
I can also see the kernel spec JSON file in ~/.local/share/jupyter/kernels/julia-1.7/kernel.json
{
"display_name": "Julia 1.7.2",
"argv": [
"/home/USER/.conda/envs/Julia/bin/julia",
"-i",
"--color=yes",
"--project=#.",
"/home/USER/.conda/envs/Julia/share/julia/packages/IJulia/AQu2H/src/kernel.jl",
"{connection_file}"
],
"language": "julia",
"env": {},
"interrupt_mode": "signal"
}
The log file starts showing problems here:
info 17:50:48.378: Process Execution: cwd: ~
cwd: ~
warn 17:50:48.893: StdErr from Kernel Process [91m[1mERROR: [22m[39m
warn 17:50:49.138: StdErr from Kernel Process LoadError:
warn 17:50:49.795: StdErr from Kernel Process ArgumentError: Package IJulia not found in current path:
- Run `import Pkg; Pkg.add("IJulia")` to install the IJulia package.
The extension says it cannot find the IJulia kernel. This perplexes me because I can see the kernel spec in my home directory, the jupyter binary I installed from conda says that its there and the Jupyter Notebook extension can see the kernel. I have no explanation as to why the extension can see the kernel, match up the kernelspec but not be able to connect to it. Help would greatly be appreciated!
I'm fairly new to the Python scene. My problem is that when I launch a jupyter notebook from an Anaconda Powershell with my DataScience virtual environment activated, the notebook does not have my virtual environment in it's PATH, and therefore cannot find some packages (like plotly and progress). The same is true when I launch VS Code from Anaconda Navigator with DataScience activated. When I run import plotly in an interactive window, I get ModuleNotFoundError: No module named ‘plotly’. But when I run this line in the terminal within VS Code, it runs without error.
So I have run the following commands in various shell/terminal sessions:
import sys
print(sys.path)
In a VS Code terminal I get:
['', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience\\python37.zip', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience\\DLLs', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience\\lib', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience\\lib\\site-packages']
In an interactive window in VS Code I get:
['C:\\Users\\adiad\\AppData\\Local\\Temp\\04e2b30c-4fc3-4aa9-9567-3aba17081a73', 'C:\\Users\\adiad\\Anaconda3\\python37.zip', 'C:\\Users\\adiad\\Anaconda3\\DLLs', 'C:\\Users\\adiad\\Anaconda3\\lib', 'C:\\Users\\adiad\\Anaconda3', '', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\win32', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\win32\\lib', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\Pythonwin', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\IPython\\extensions', 'C:\\Users\\adiad\\.ipython']
In a jupyter notebook running in my browser I get:
['C:\\Users\\adiad\\Anaconda3\\envs\\test', 'C:\\Users\\adiad\\Anaconda3\\python37.zip', 'C:\\Users\\adiad\\Anaconda3\\DLLs', 'C:\\Users\\adiad\\Anaconda3\\lib', 'C:\\Users\\adiad\\Anaconda3', '', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\win32', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\win32\\lib', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\Pythonwin', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\IPython\\extensions', 'C:\\Users\\adiad\\.ipython']
The IPython session don't appear to reference my virtual environment. So my question is: what do I need to do make IPython run with same environment as my terminal?
I found the following SO question which seems to answer my question, but I find it hard to believe that everyone is following this practice.
How to start an ipython shell(not notebook) within a conda or virtualenv
Here's my configuration:
conda version : 4.7.12
conda-build version : 3.18.8
python version : 3.7.3.final.0
virtual packages :
base environment : C:\Users\adiad\Anaconda3 (writable)
channel URLs : https://conda.anaconda.org/conda-forge/win-64
https://conda.anaconda.org/conda-forge/noarch
https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : C:\Users\adiad\Anaconda3\pkgs
C:\Users\adiad\.conda\pkgs
C:\Users\adiad\AppData\Local\conda\conda\pkgs
envs directories : C:\Users\adiad\Anaconda3\envs
C:\Users\adiad\.conda\envs
C:\Users\adiad\AppData\Local\conda\conda\envs
platform : win-64
user-agent : conda/4.7.12 requests/2.22.0 CPython/3.7.3 Windows/10 Windows/10.0.18362
After doing further digging, my problem ought to be filed under, "knowing enough to be dangerous." My problem was ultimately caused by the fact that the jupyter package hadn't yet been installed in my new environment. So whenever I attempted to launch an IPython session of some kind, either in VS Code or in a browser, the application would look in my environment and see that the IPython packages weren't installed. It would then look to other conda environments and use the "nearest" equivalent, which was the base environment. Hence, most of the packages would load, but not all.
The fix to my problem was:
conda install jupyter
Another simple fix:
Launch CMD.exe prompt on Anaconda Navigator
Install: conda install jupyter
And
Conda install plotly
I tried to make a virtual environment as stated in the documentation of the visual studio code. I have already installed python 3.7.3 and anaconda on the system.
I did the following and I'm using vs code version 1.34 and windows 10:
I made an empty folder and used this line of code as stated in official documentation:
py -3 -m venv .venv
I expected a folder named .venv that has scripts folder with 'activate' file in it. As i have tested this method which worked on other systems with the same configuration, But instead i get the below error message:
Error: [Errno 2]: no such file or directory : 'C:\\ProgramData\\Anaconda3\\lib\\venv\\scripts\\nt\\python.exe'
It look like installer is broken, or so...
Copy from Python 3 installation folder:
C:\Users{user}\AppData\Local\Programs\Python\Python37
Files: python.exe, python.pdb, python_d.exe, python_d.pdb, pythonw.exe & pythonw.pdb to folder: C:\ProgramData\Anaconda3\lib\venv\scripts\nt\
That's because, in the python version you are trying create virtual environment venv [ in other cases -> vitualenv] has not been installed.
environment building process fails after python global version hand over to the specific version.
you can get idea how to install vitualenv on a specific version from the link below.
Dealing with multiple python versions and pip
I am on an Ubuntu machine, which has Python 2.7.6 as its default python. It also has Python 3.4.3, with both versions located in /usr/bin/.
I have downloaded virtualenv and virtualenvwrapper. I then downloaded the current version of Python, 3.5.1. In its directory I ran the following commands:
./configure
make
make test
sudo make altinstall
Python 3.5.1 is now installed in /usr/local/bin/.
I now run the following commands:
mkvirtualenv test1
mkvirtualenv test2 -p /usr/bin/python3
mkvirtualenv test3 -p /usr/local/bin/python3.5
Environment test1 successfully created with Python version 2.7.6, and environment test2 successfully created with Python version 3.4.3. However, test3 fails with the following error:
ImportError: No module named 'zlib'
I see mentioned that I need to have "zlib" installed when compiling python to begin with, though make test didn't seem to give any problems. Do I just need to download and compile zlib from www.zlib.net and recompile python3.5?
zlib is a built-in module for Python 3.5.
I think you just need re-compile Python 3.5...
Look that link for Python virtualenv :
https://www.reddit.com/r/linux4noobs/comments/3uwk76/help_using_python_in_linux/
Get python source and extract
wget https://www.python.org/ftp/python/3.5.0/Python-3.5.0.tgz
tar xvf Python-3.5.0.tgz
configure for local install
cd Python-3.5.0/
./configure --prefix=$HOME/python35
make
If it complains about missing dependencies, install them, make clean and repeat.
make install