I am on an Ubuntu machine, which has Python 2.7.6 as its default python. It also has Python 3.4.3, with both versions located in /usr/bin/.
I have downloaded virtualenv and virtualenvwrapper. I then downloaded the current version of Python, 3.5.1. In its directory I ran the following commands:
./configure
make
make test
sudo make altinstall
Python 3.5.1 is now installed in /usr/local/bin/.
I now run the following commands:
mkvirtualenv test1
mkvirtualenv test2 -p /usr/bin/python3
mkvirtualenv test3 -p /usr/local/bin/python3.5
Environment test1 successfully created with Python version 2.7.6, and environment test2 successfully created with Python version 3.4.3. However, test3 fails with the following error:
ImportError: No module named 'zlib'
I see mentioned that I need to have "zlib" installed when compiling python to begin with, though make test didn't seem to give any problems. Do I just need to download and compile zlib from www.zlib.net and recompile python3.5?
zlib is a built-in module for Python 3.5.
I think you just need re-compile Python 3.5...
Look that link for Python virtualenv :
https://www.reddit.com/r/linux4noobs/comments/3uwk76/help_using_python_in_linux/
Get python source and extract
wget https://www.python.org/ftp/python/3.5.0/Python-3.5.0.tgz
tar xvf Python-3.5.0.tgz
configure for local install
cd Python-3.5.0/
./configure --prefix=$HOME/python35
make
If it complains about missing dependencies, install them, make clean and repeat.
make install
Related
I want to run a job on Slurm and my Python script needs the evaluate package which I have on my local machine. I don't know if I could change the Python path on the server to match the one on my local machine, and if I could I'm afraid I might break the system.
So I followed this answer, and included a requirements.txt file with just evaluate==0.1.2 in it, and I get even more errors:
load GCC/10.2.0 (PATH, MANPATH, INFOPATH, LIBRARY_PATH, LD_LIBRARY_PATH, STD COMP VARS)
load ROCM/5.1.1 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, C_INCLUDE_PATH)
Set INTEL compilers as MPI wrappers backend
load mkl/2018.4 (LD_LIBRARY_PATH)
load PYTHON/3.7.4 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, PKG_CONFIG_PATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, PYTHONHOME, PYTHONPATH)
/var/spool/slurmd/job216863/slurm_script: line 12: virtualenv: command not found
/var/spool/slurmd/job216863/slurm_script: line 16: /env/bin/activate: No such file or directory
ERROR: Could not find a version that satisfies the requirement evaluate==0.1.2 (from versions: none)
ERROR: No matching distribution found for evaluate==0.1.2
Traceback (most recent call last):
File "eval_comet.py", line 1, in <module>
from evaluate import load
ModuleNotFoundError: No module named 'evaluate'
Most of the time, the Python version on HPCs are old. My Uni's HPC cluster has Python 3.7. If you wish to create a Python virtual environment (not conda) with a newer version, then there is a trick.
Activate the Anaconda Module, some system uses Module load and some uses load depending on your organisation.
[s.1915438#sl2 ~]$ module load anaconda/2021.05
[s.1915438#sl2 ~]$ conda create -n surrogate python=3.8
Here I created a Conda environment named surrogate with Python 3.8. Here, you can choose any version of your choice. Now you can activate the Conda environment and check the Python version.
[s.1915438#sl2 ~]$ source activate surrogate
(modulus) [s.1915438#sl2 ~]$ which python
~/.conda/envs/surrogate/bin/python
(surrogate) [s.1915438#sl2 ~]$ python --version
Python 3.8.13
Now navigate to the directory where you want to install your Python virtual environment and install the virtual environment using the following command.
(surrogate) [s.1915438#sl2 s.1915438]$ mkdir modulus_pysdf
(surrogate) [s.1915438#sl2 s.1915438]$ cd modulus_pysdf/
(surrogate) [s.1915438#sl2 modulus_pysdf]$ python3 -m venv modulus_pysdf
Logout (ctrl + D) from the server to exit the Conda environment and then login again. Remember, in my case the path to the Python virtual environment was /scratch/s.1915438/modulus_pysdf.
This is how I will activate the Python virtual environment.
[s.1915438#sl2 ~]$ cd /scratch/s.1915438
[s.1915438#sl2 s.1915438]$ cd modulus_pysdf/
[s.1915438#sl2 modulus_pysdf]$ source modulus_pysdf/bin/activate
Now I can check the Python version and the path.
(modulus_pysdf) [s.1915438#sl2 modulus_pysdf]$ python --version
Python 3.8.13
(modulus_pysdf) [s.1915438#sl2 modulus_pysdf]$ which python
/scratch/s.1915438/modulus_pysdf/modulus_pysdf/bin/python
As usual, I can install any package using pip. For example, to install evaluate you can download it from PyPI
pip install evaluate
Or if you have a requirements.txt file then you can do this. See this for more details.
cat requirements.txt | grep -Eo '(^[^#]+)' | xargs -n 1 pip install
I have installed Raspbian Lite OS in Raspberry Pi zero.
I found that Raspbian Lite comes with Python3 as default.
But I am gonna run some scripts that uses libraries that are Python2 Compatible.
So I tried to change the default Python version from Python3 to Python2 (Specifically Python2.7.18)
After so much searching and trying, instructions from [this page][1] made my job
Now if I try to check in command writing
python --version
It shows me that it is Python2.7.18
But the problem is I am not being able to install any packages using
sudo apt-get install <python-packagename>
It shows me Errors like
1.Package "python-pip" has no installation candidate (When I tried to install pip)
2. Package python-numpy has no installation candidate (when I tried to install numpy)
3. unable to locate package python-pyaudio (when I tried to install pyaudio)
I am searching but no solution.
Can anyone please help? I am frozen in a critical stage of my project .
Generally, for new raspbian/raspios os with python 3 by default, it is not suggested to replace the system python interpreter to python 2, it may break some system component's dependency.
Instead, you can create a python 2 virtual env, if you need to run python2 scripts.
sudo apt-get install python3-pip
pip install virtualenv
virtualenv -p /usr/bin/python2.7 venv
source venv/bin/activate
You can test the python version as following,
(venv) $ python --version
ruamel.yaml seems to require PIP to install, which is not the default buildroot solution to build and install a Python package.
Is is possible to -at least- install a pure Python version of ruamel.yaml into a buildroot image - and how to circumvent the pip limitation?
Is is possible to cross-build ruamel.yaml?
Forcing RUAMEL_NO_PIP_INSTALL_CHECK env. var. does not help:
test compiling test_ruamel_yaml
running install
Checking .pth file support in ...
Failed to import the site module
ModuleNotFoundError: No module named '_sysconfigdata_m_linux_arm-linux-gnueabihf'
error: command '.../output/host/bin/python' failed with exit status 1
package/pkg-generic.mk:310: recipe for target '.../output/build/python-ruamel-yaml-0.15.45/.stamp_target_installed' failed
ruamel.yaml indeed requires pip to install from PyPI (using the .tar.gz or a .whl appropriate for your platform), this is documented.
The reason for this is that the fixes necessary to enable nested package installs where only implemented for pip (and not for easy_install or python setup.py installs).
That however does not preclude you from using ruamel.yaml, especially if you don't need the C extension (which is checked for at load time).
You can either check out a tagged version from bitbucket or untar a .tar.gz from PyPI and move the result to your site-packages directory:
$ virtualenv /tmp/ruamel_yaml_no_pip
Using base prefix '/opt/python/3.7'
New python executable in /tmp/ruamel_yaml_no_pip/bin/python
Installing setuptools, pip, wheel...done.
$ cd /tmp/ruamel_yaml_no_pip/
$ source bin/activate
(ruamel_yaml_no_pip) $ mkdir lib/python3.7/site-packages/ruamel/
(ruamel_yaml_no_pip) $ touch lib/python3.7/site-packages/ruamel/__init__.py
(ruamel_yaml_no_pip) $ wget -q https://files.pythonhosted.org/packages/63/a5/dba37230d6cf51f4cc19a486faf0f06871d9e87d25df0171b3225d20fc68/ruamel.yaml-0.15.45.tar.gz
(ruamel_yaml_no_pip) $ python -m ruamel.yaml
/tmp/ruamel_yaml_no_pip/bin/python: Error while finding module specification for 'ruamel.yaml' (ModuleNotFoundError: No module named 'ruamel')
(ruamel_yaml_no_pip) $ tar xf ruamel.yaml-0.15.45.tar.gz
(ruamel_yaml_no_pip) $ mv ruamel.yaml-0.15.45 lib/python3.7/site-packages/ruamel/yaml
(ruamel_yaml_no_pip) $ python -c 'from ruamel.yaml import YAML; print(YAML().load("{hello: world}")["hello"])'
world
(ruamel_yaml_no_pip)
(ruamel_yaml_no_pip) $ python -c 'from ruamel.yaml import __with_libyaml__ as X; print(X)'
False
(The URL is copied from the 0.15.45 project download page)
For development I normally just make a soft link from a virtualenv's site-packages to my ruamel directory.
I don't know how and if that translates to a buildroot environment (if so please publish your result).
I overlooked the buildroot documentation.
There is a critical parameter to define: SETUP_TYPE = setuptools rather than SETUP_TYPE = distutils.
With the following snippet:
PYTHON_RUAMEL_YAML_VERSION = 0.15.45
PYTHON_RUAMEL_YAML_SOURCE = ruamel.yaml-$(PYTHON_RUAMEL_YAML_VERSION).tar.gz
PYTHON_RUAMEL_YAML_SITE = https://pypi.python.org/packages/63/a5/dba37230d6cf51f4cc19a486faf0f06871d9e87d25df0171b3225d20fc68
PYTHON_RUAMEL_YAML_SETUP_TYPE = setuptools
PYTHON_RUAMEL_YAML_LICENSE = MIT
PYTHON_RUAMEL_YAML_LICENSE_FILES = LICENSE
PYTHON_RUAMEL_YAML_ENV += RUAMEL_NO_PIP_INSTALL_CHECK=1
$(eval $(python-package))
ruamel.yaml installs perfectly on the target image.
I have problem with activating virtualenv.
I'm working on the server and using SSH secure shell.
My final goal is to activate virtualenv and run the latest version of tensorflow
The following is the command lines:
jeonguyoang#vision6:~$ python3 -m venv tfenv
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
apt-get install python3-venv
You may need to use sudo with that command. After installing the python3-venv
package, recreate your virtual environment.
jeonguyoang#vision6:~$ source tfenv/bin/activate
-bash: tfenv/bin/activate: No such file or directory
jeonguyoang#vision6:~$ cd tfenv
jeonguyoang#vision6:~/tfenv$ ls
bin include lib lib64 pyvenv.cfg
jeonguyoang#vision6:~/tfenv$ cd bin
jeonguyoang#vision6:~/tfenv/bin$ ls
python python3
captured image of the commands
I think that there is no activate file.
Maybe re-installing virtualenv is the answer, but I cannot interrupt server settings..
Check if you have python 2 versions of pip and python (python-all & python-pip packages). Venv installs both v2 and v3 versions of python & pip (regardless of python version of venv).
You can cd to YOUR_ENV/lib/pythonxx/site-packages/ and have a look, but is there any convenient ways?
pip freeze list all the packages installed including the system environment's.
You can list only packages in the virtualenv by
pip freeze --local
or
pip list --local.
This option works irrespective of whether you have global site packages visible in the virtualenv.
Note that restricting the virtualenv to not use global site packages isn't the answer to the problem, because the question is on how to separate the two lists, not how to constrain our workflow to fit limitations of tools.
Credits to #gvalkov's comment here. Cf. also pip issue 85.
Calling pip command inside a virtualenv should list the packages visible/available in the isolated environment. Make sure to use a recent version of virtualenv that uses option --no-site-packages by default. This way the purpose of using virtualenv is to create a python environment without access to packages installed in system python.
Next, make sure you use pip command provided inside the virtualenv (YOUR_ENV/bin/pip). Or just activate the virtualenv (source YOUR_ENV/bin/activate) as a convenient way to call the proper commands for python interpreter or pip
~/Projects$ virtualenv --version
1.9.1
~/Projects$ virtualenv -p /usr/bin/python2.7 demoenv2.7
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in demoenv2.7/bin/python2.7
Also creating executable in demoenv2.7/bin/python
Installing setuptools............................done.
Installing pip...............done.
~/Projects$ cd demoenv2.7/
~/Projects/demoenv2.7$ bin/pip freeze
wsgiref==0.1.2
~/Projects/demoenv2.7$ bin/pip install commandlineapp
Downloading/unpacking commandlineapp
Downloading CommandLineApp-3.0.7.tar.gz (142kB): 142kB downloaded
Running setup.py egg_info for package commandlineapp
Installing collected packages: commandlineapp
Running setup.py install for commandlineapp
Successfully installed commandlineapp
Cleaning up...
~/Projects/demoenv2.7$ bin/pip freeze
CommandLineApp==3.0.7
wsgiref==0.1.2
What's strange in my answer is that package 'wsgiref' is visible inside the virtualenv. Its from my system python. Currently I do not know why, but maybe it is different on your system.
In Python3
pip list
Empty venv is
Package Version
---------- -------
pip 19.2.3
setuptools 41.2.0
To create a new environment
python3 -m venv your_foldername_here
Activate
cd your_foldername_here
source bin/activate
Deactivate
deactivate
You can also stand in the folder and give the virtual environment a name/folder (python3 -m venv name_of_venv).
Venv is a subset of virtualenv that is shipped with Python after 3.3.
list out the installed packages in the virtualenv
step 1:
workon envname
step 2:
pip freeze
it will display the all installed packages and installed packages and versions
If you're still a bit confused about virtualenv you might not pick up how to combine the great tips from the answers by Ioannis and Sascha. I.e. this is the basic command you need:
/YOUR_ENV/bin/pip freeze --local
That can be easily used elsewhere. E.g. here is a convenient and complete answer, suited for getting all the local packages installed in all the environments you set up via virtualenvwrapper:
cd ${WORKON_HOME:-~/.virtualenvs}
for dir in *; do [ -d $dir ] && $dir/bin/pip freeze --local > /tmp/$dir.fl; done
more /tmp/*.fl
why don't you try pip list
Remember I'm using pip version 19.1 on python version 3.7.3
If you are using pip 19.0.3 and python 3.7.4. Then go for pip list command in your virtualenv. It will show all the installed packages with respective versions.
.venv/bin/pip freeze worked for me in bash.
In my case the flask version was only visible under so I had to go to
C:\Users\\AppData\Local\flask\venv\Scripts>pip freeze --local
Using python3 executable only, from:
Gitbash:
winpty my_venv_dir/bin/python -m pip freeze
Linux:
my_venv_dir/bin/python -m pip freeze