Tried to work with Julia in a virtual environment using below command:
virtualenv -p julia-1.0.2/bin/julia julia
I got this error:
Running virtualenv with interpreter julia-1.0.2/bin/julia
ERROR: LoadError: ArgumentError: Package os not found in current path:
- Run `import Pkg; Pkg.add("os")` to install the os package.
Stacktrace:
[1] require(::Module, ::Symbol) at ./loading.jl:823
[2] include at ./boot.jl:317 [inlined]
[3] include_relative(::Module, ::String) at ./loading.jl:1044
[4] include(::Module, ::String) at ./sysimg.jl:29
[5] exec_options(::Base.JLOptions) at ./client.jl:231
[6] _start() at ./client.jl:425
in expression starting at /home/ebasfad/.local/lib/python3.5/site-packages/virtualenv.py:5
I tires to install the "os" package but it seems no such packages
virtualenv is used to create isolated Python environments and not Julia environments. The command you give tries to find Python package os using Julia executable thinking that it is a Python interpreter. That is the reason for the error.
With Julia 1.0 and its package manager Pkg (Pkg3), you can create your own projects (similar to virtualenv environments) within Julia and manage your packages for your projects, as you please.
Note that Pkg comes built-in. See here to learn how to create a project with Pkg: https://docs.julialang.org/en/v1/stdlib/Pkg/index.html#Creating-your-own-projects-1
This is the documentation for Pkg: https://docs.julialang.org/en/v1/stdlib/Pkg/index.html
Related
I want to run a job on Slurm and my Python script needs the evaluate package which I have on my local machine. I don't know if I could change the Python path on the server to match the one on my local machine, and if I could I'm afraid I might break the system.
So I followed this answer, and included a requirements.txt file with just evaluate==0.1.2 in it, and I get even more errors:
load GCC/10.2.0 (PATH, MANPATH, INFOPATH, LIBRARY_PATH, LD_LIBRARY_PATH, STD COMP VARS)
load ROCM/5.1.1 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, C_INCLUDE_PATH)
Set INTEL compilers as MPI wrappers backend
load mkl/2018.4 (LD_LIBRARY_PATH)
load PYTHON/3.7.4 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, PKG_CONFIG_PATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, PYTHONHOME, PYTHONPATH)
/var/spool/slurmd/job216863/slurm_script: line 12: virtualenv: command not found
/var/spool/slurmd/job216863/slurm_script: line 16: /env/bin/activate: No such file or directory
ERROR: Could not find a version that satisfies the requirement evaluate==0.1.2 (from versions: none)
ERROR: No matching distribution found for evaluate==0.1.2
Traceback (most recent call last):
File "eval_comet.py", line 1, in <module>
from evaluate import load
ModuleNotFoundError: No module named 'evaluate'
Most of the time, the Python version on HPCs are old. My Uni's HPC cluster has Python 3.7. If you wish to create a Python virtual environment (not conda) with a newer version, then there is a trick.
Activate the Anaconda Module, some system uses Module load and some uses load depending on your organisation.
[s.1915438#sl2 ~]$ module load anaconda/2021.05
[s.1915438#sl2 ~]$ conda create -n surrogate python=3.8
Here I created a Conda environment named surrogate with Python 3.8. Here, you can choose any version of your choice. Now you can activate the Conda environment and check the Python version.
[s.1915438#sl2 ~]$ source activate surrogate
(modulus) [s.1915438#sl2 ~]$ which python
~/.conda/envs/surrogate/bin/python
(surrogate) [s.1915438#sl2 ~]$ python --version
Python 3.8.13
Now navigate to the directory where you want to install your Python virtual environment and install the virtual environment using the following command.
(surrogate) [s.1915438#sl2 s.1915438]$ mkdir modulus_pysdf
(surrogate) [s.1915438#sl2 s.1915438]$ cd modulus_pysdf/
(surrogate) [s.1915438#sl2 modulus_pysdf]$ python3 -m venv modulus_pysdf
Logout (ctrl + D) from the server to exit the Conda environment and then login again. Remember, in my case the path to the Python virtual environment was /scratch/s.1915438/modulus_pysdf.
This is how I will activate the Python virtual environment.
[s.1915438#sl2 ~]$ cd /scratch/s.1915438
[s.1915438#sl2 s.1915438]$ cd modulus_pysdf/
[s.1915438#sl2 modulus_pysdf]$ source modulus_pysdf/bin/activate
Now I can check the Python version and the path.
(modulus_pysdf) [s.1915438#sl2 modulus_pysdf]$ python --version
Python 3.8.13
(modulus_pysdf) [s.1915438#sl2 modulus_pysdf]$ which python
/scratch/s.1915438/modulus_pysdf/modulus_pysdf/bin/python
As usual, I can install any package using pip. For example, to install evaluate you can download it from PyPI
pip install evaluate
Or if you have a requirements.txt file then you can do this. See this for more details.
cat requirements.txt | grep -Eo '(^[^#]+)' | xargs -n 1 pip install
I need to set up virtual environments for each language that I use. To do this, I'm running the Ubuntu 20.04 LTS Windows Subsystem for Linux (WSL) on Windows 10. Within WSL, I'm using Anaconda, installed in /usr/local/Anaconda, to create conda virtual environments for each language (i.e. one environment contains all my Python stuff, another contains my R stuff, etc.).
Since WSL doesn't come with a GUI, I'm using Visual Studio Code's (VSCode) Jupyter Notebook Extension to run Jupyter Notebooks to see plots/graphics. So far, I managed to easily create conda environments for Python (with ipython and ipykernel) and R (with IRkernel) and run their code in a notebook via the extension. Each time I set up an environment, the extension is easily able to find the kernel, connect to it and run the code.
However, I've not been able to set up an environment for Julia. I followed the documentation on the Julia website for installing the kernel, which is successfully found by the extension. But, when I try running a cell, the extension says it is trying to connect to the kernel, only for it to timeout and fail.
Here are the steps I have taken so far:
Create a clean conda environment (conda create -n Julia && conda activate Julia)
Install the latest version of Julia (conda install -c conda-forge julia)
Install the latest version of Jupyter (conda install -c conda-forge jupyter)
Install the Julia kernel with the built-in Julia package manager (using Pkg; Pkg.add("IJulia"))
Build the IJulia package (using Pkg; Pkg.build("IJulia"))
Confirm the presence of the Julia kernel (jupyter kernelspec list) which indeed shows the presence of a Julia kernel
Reload the VSCode connection to WSL (Ctrl + Shift + P; >Reload Window)
Shut down WSL via CMD (wsl --shutdown) for changes to take effect and reconnect
After I restart VSCode and WSL, the extension shows an option to use the Julia kernel installed in my conda environment: Julia 1.7.2 (~/.conda/envs/Julia/bin/julia). But when I create a cell and run code in a notebook, the extension creates a popup saying that it is connecting to the kernel and after some time an error message shows up:
Failed to start the Kernel.
Unable to start Kernel `Julia 1.7.2` due to connection timeout.
View Jupyter log for further details
I can also see the kernel spec JSON file in ~/.local/share/jupyter/kernels/julia-1.7/kernel.json
{
"display_name": "Julia 1.7.2",
"argv": [
"/home/USER/.conda/envs/Julia/bin/julia",
"-i",
"--color=yes",
"--project=#.",
"/home/USER/.conda/envs/Julia/share/julia/packages/IJulia/AQu2H/src/kernel.jl",
"{connection_file}"
],
"language": "julia",
"env": {},
"interrupt_mode": "signal"
}
The log file starts showing problems here:
info 17:50:48.378: Process Execution: cwd: ~
cwd: ~
warn 17:50:48.893: StdErr from Kernel Process [91m[1mERROR: [22m[39m
warn 17:50:49.138: StdErr from Kernel Process LoadError:
warn 17:50:49.795: StdErr from Kernel Process ArgumentError: Package IJulia not found in current path:
- Run `import Pkg; Pkg.add("IJulia")` to install the IJulia package.
The extension says it cannot find the IJulia kernel. This perplexes me because I can see the kernel spec in my home directory, the jupyter binary I installed from conda says that its there and the Jupyter Notebook extension can see the kernel. I have no explanation as to why the extension can see the kernel, match up the kernelspec but not be able to connect to it. Help would greatly be appreciated!
I tried to make a virtual environment as stated in the documentation of the visual studio code. I have already installed python 3.7.3 and anaconda on the system.
I did the following and I'm using vs code version 1.34 and windows 10:
I made an empty folder and used this line of code as stated in official documentation:
py -3 -m venv .venv
I expected a folder named .venv that has scripts folder with 'activate' file in it. As i have tested this method which worked on other systems with the same configuration, But instead i get the below error message:
Error: [Errno 2]: no such file or directory : 'C:\\ProgramData\\Anaconda3\\lib\\venv\\scripts\\nt\\python.exe'
It look like installer is broken, or so...
Copy from Python 3 installation folder:
C:\Users{user}\AppData\Local\Programs\Python\Python37
Files: python.exe, python.pdb, python_d.exe, python_d.pdb, pythonw.exe & pythonw.pdb to folder: C:\ProgramData\Anaconda3\lib\venv\scripts\nt\
That's because, in the python version you are trying create virtual environment venv [ in other cases -> vitualenv] has not been installed.
environment building process fails after python global version hand over to the specific version.
you can get idea how to install vitualenv on a specific version from the link below.
Dealing with multiple python versions and pip
I have installed QuickChick using opam using the instructions given on QuickChick's GitHub page:
https://github.com/QuickChick/QuickChick
After a successful installation, I tried to run the QuickChick tutorial
https://github.com/QuickChick/QuickChick/blob/master/examples/Tutorial.v
and Coq returned the following error message
Error:
Cannot find a physical path bound to logical path matching suffix
<> and prefix QuickChick.
when I tried to process the line
From QuickChick Require Import QuickChick.
I'm running Coq 8.8, Opam 1.2.2, Ocaml 4.04.0 and Coq-QuickChick 1.0.1 on Ubuntu 16.04. Is there some configuration that am I missing?
I have tried pip install psycopg2 and following errors i got:
Error:
(ScrapeProj) C:\Users\e2sn7cy\Documents\GitHub\scraper>pip install psycopg2==2.5.4
Collecting psycopg2==2.5.4
Using cached psycopg2-2.5.4.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info\psycopg2.egg-info
writing pip-egg-info\psycopg2.egg-info\PKG-INFO
writing top-level names to pip-egg-info\psycopg2.egg-info\top_level.txt
writing dependency_links to pip-egg-info\psycopg2.egg-info\dependency_links.txt
writing manifest file 'pip-egg-info\psycopg2.egg-info\SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in c:\users\e2sn7cy\appdata\local\temp\pi
p-build-joij3x\psycopg2
Then I tried using easy_install by downloading the .exe file from Stickpole and I got following errors :
(ScrapeProj) C:\Users\e2sn7cy\Envs\ScrapeProj\Scripts>easy_install psycopg2-2.6.1.win-amd64-py2.7-pg9
.4.4-release.exe
Processing psycopg2-2.6.1.win-amd64-py2.7-pg9.4.4-release.exe
psycopg2.tests.test_module: module references __file__
psycopg2.tests.test_types_basic: module references __file__
creating 'c:\users\e2sn7cy\appdata\local\temp\easy_install-sa7asq\psycopg2-2.6.1-py2.7-win32.egg' and
adding 'c:\users\e2sn7cy\appdata\local\temp\easy_install-sa7asq\psycopg2-2.6.1-py2.7-win32.egg.tmp'
to it
creating c:\users\e2sn7cy\envs\scrapeproj\lib\site-packages\psycopg2-2.6.1-py2.7-win32.egg
Extracting psycopg2-2.6.1-py2.7-win32.egg to c:\users\e2sn7cy\envs\scrapeproj\lib\site-packages
Adding psycopg2 2.6.1 to easy-install.pth file
Installed c:\users\e2sn7cy\envs\scrapeproj\lib\site-packages\psycopg2-2.6.1-py2.7-win32.egg
Processing dependencies for psycopg2==2.6.1
Finished processing dependencies for psycopg2==2.6.1
But when I again tried to install through pip I am getting a new message that its already there :
(ScrapeProj) C:\Users\e2sn7cy\Envs\ScrapeProj>pip install psycopg2
Requirement already satisfied (use --upgrade to upgrade): psycopg2 in c:\users\e2sn7cy\envs\scrapepro
j\lib\site-packages\psycopg2-2.6.1-py2.7-win32.egg
Then I thought ok I should try to import and then I got this error:
(ScrapeProj) C:\Users\e2sn7cy\Envs\ScrapeProj>python
Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\e2sn7cy\Envs\ScrapeProj\lib\site-packages\psycopg2-2.6.1-py2.7-win32.egg\psycopg2\__
init__.py", line 50, in <module>
from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: DLL load failed: %1 is not a valid Win32 application.
Now I am not able to understand what to do. Thanks
Edit 1:
I have also added my PostgreSQL path to system path variables
path:C:\Program Files\PostgreSQL\9.4
path:C:\Program Files\PostgreSQL\9.4\bin
The correct answer was given by Craig Ringer, I am just adding this because I came to know that we should first check which version of 64 bit or 32bit Python installed on our system. If you have 32 bit version of Python and if you will try to install 64bit, then you will get the above error on importing.
For checking the version:
import platform
platform.architecture()
and you will get an output like this:
('32bit','WindowsPE')
Download the pre-packaged binary release of psycopg2 for Windows rather than compiling it yourself. It's a pain to compile on Windows. This is explained in the install documentation for psycopg2.
You might need to remove your half-installed attempts.
Choose the appropriate version from this page:
http://www.stickpeople.com/projects/python/win-psycopg/
Right click and select copy link address
Back at home, use easy_install <<Paste URL Here>>
Freeze your new requirements: pip freeze > requirements.txt
Now that psycopg is in PyPI (as of March 2017 it appears), just fire up a command prompt with administrator privileges and install like so:
python -m pip install -U pip
python -m pip install psycopg2
If you don't have pip installed, follow the instructions here to do that first:
https://stackoverflow.com/a/12476379/2540707
pip install psycopg2-binary
This worked for me
Step 1: Compilers Installation and configuration
https://wiki.python.org/moin/WindowsCompilers
Before do anything, install or upgrade the Setuptools Python package. It contain compatibility improvements and add automatic use of compilers:
pip install --upgrade setuptools
Step 2: Download and install PostgreSQL
Before you continue to install python packages inside you virtualenvs download postgres itself. It contains files that are needed when compiling the psycopg2 python package. Just use the PostgreSQL installer(version 10 for example).
Important: add the postgres C:\Program Files\PostgreSQL\10\bin folder to your path. It contains the .dll needed for psycopg2.
Step 3: Install psycopg2
pip install psycopg2
Tested on Windows 7 + python 3.7
The best/easiest way of installing packages (including this problematic psycopg2 ) for python on windows platform is to:
1) install anaconda and ...
2) use Anaconda Navigator GUI to install/update packages.
By this one avoids many inconsistencies and conflicts that occurs among different packages versions and installations.