Is it possible to run the celery command line if your django project is named "app"?
Django app structure:
|- app
| |- __init__.py
| |- settings.py
| |- celery.py
|- manage.py
__init__.py and celery.py are set up as per the docs: http://docs.celeryproject.org/en/master/django/first-steps-with-django.html
I get the following error ImportError: No module named celery when I run celery --app=app.celery:app worker. This error does not exist when I rename "app" to something else, but I prefer not to have to do this.
you need install celery package your virtual environments like
pip install celery
after installing package check your virtual environments celery version
celery --version
it display the current celery version and celery --help it shows like this
Related
I want to run a job on Slurm and my Python script needs the evaluate package which I have on my local machine. I don't know if I could change the Python path on the server to match the one on my local machine, and if I could I'm afraid I might break the system.
So I followed this answer, and included a requirements.txt file with just evaluate==0.1.2 in it, and I get even more errors:
load GCC/10.2.0 (PATH, MANPATH, INFOPATH, LIBRARY_PATH, LD_LIBRARY_PATH, STD COMP VARS)
load ROCM/5.1.1 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, C_INCLUDE_PATH)
Set INTEL compilers as MPI wrappers backend
load mkl/2018.4 (LD_LIBRARY_PATH)
load PYTHON/3.7.4 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, PKG_CONFIG_PATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, PYTHONHOME, PYTHONPATH)
/var/spool/slurmd/job216863/slurm_script: line 12: virtualenv: command not found
/var/spool/slurmd/job216863/slurm_script: line 16: /env/bin/activate: No such file or directory
ERROR: Could not find a version that satisfies the requirement evaluate==0.1.2 (from versions: none)
ERROR: No matching distribution found for evaluate==0.1.2
Traceback (most recent call last):
File "eval_comet.py", line 1, in <module>
from evaluate import load
ModuleNotFoundError: No module named 'evaluate'
Most of the time, the Python version on HPCs are old. My Uni's HPC cluster has Python 3.7. If you wish to create a Python virtual environment (not conda) with a newer version, then there is a trick.
Activate the Anaconda Module, some system uses Module load and some uses load depending on your organisation.
[s.1915438#sl2 ~]$ module load anaconda/2021.05
[s.1915438#sl2 ~]$ conda create -n surrogate python=3.8
Here I created a Conda environment named surrogate with Python 3.8. Here, you can choose any version of your choice. Now you can activate the Conda environment and check the Python version.
[s.1915438#sl2 ~]$ source activate surrogate
(modulus) [s.1915438#sl2 ~]$ which python
~/.conda/envs/surrogate/bin/python
(surrogate) [s.1915438#sl2 ~]$ python --version
Python 3.8.13
Now navigate to the directory where you want to install your Python virtual environment and install the virtual environment using the following command.
(surrogate) [s.1915438#sl2 s.1915438]$ mkdir modulus_pysdf
(surrogate) [s.1915438#sl2 s.1915438]$ cd modulus_pysdf/
(surrogate) [s.1915438#sl2 modulus_pysdf]$ python3 -m venv modulus_pysdf
Logout (ctrl + D) from the server to exit the Conda environment and then login again. Remember, in my case the path to the Python virtual environment was /scratch/s.1915438/modulus_pysdf.
This is how I will activate the Python virtual environment.
[s.1915438#sl2 ~]$ cd /scratch/s.1915438
[s.1915438#sl2 s.1915438]$ cd modulus_pysdf/
[s.1915438#sl2 modulus_pysdf]$ source modulus_pysdf/bin/activate
Now I can check the Python version and the path.
(modulus_pysdf) [s.1915438#sl2 modulus_pysdf]$ python --version
Python 3.8.13
(modulus_pysdf) [s.1915438#sl2 modulus_pysdf]$ which python
/scratch/s.1915438/modulus_pysdf/modulus_pysdf/bin/python
As usual, I can install any package using pip. For example, to install evaluate you can download it from PyPI
pip install evaluate
Or if you have a requirements.txt file then you can do this. See this for more details.
cat requirements.txt | grep -Eo '(^[^#]+)' | xargs -n 1 pip install
I'm trying to run theses commands :
docker-compose pull
docker-compose up -d
docker-compose -f other_file.yaml pull
docker-compose -f other_file.yaml up -d
Here's my Ansible code for this specific task :
- name: Run docker-compose
docker_compose:
project_src: {{ my_project_path }}
files:
- docker-compose.yaml
- other_file.yaml
I'm getting the error bellow
Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7) or docker-py (Python 2.6)) on managed's Python /usr/bin/python3.
Please read module documentation and install in the appropriate location.
If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter, for example via `pip install docker` or `pip install docker-py` (Python 2.6).
The error was: No module named 'docker'
The fact is that the python interpreter is set up in the ansible.cfg as /usr/bin/python3 which is the good one.
The version of python3 installed is 3.6.9 and the python module "docker" is installed.
Any idea on where this error comes from ? Been reading documentation, and others post all day.
Thanks !
Finally understood why the problem occured.
I was installing the python library with pip3 install <lib> the fact is that it will not work if you're using sudo to run some modules in Ansible because sudo pip3 is different from pip3 on its own.
So quick solution ? sudo pip3 install docker docker-compose
Last time I've followed recommended src layout (https://hynek.me/articles/testing-packaging/) with using tox with great success.
However VSCODE tests discovery fails because src package cannot be imported. That is expected as we want to test installed package.
But how to debug my tests in vscode?
(Q author here: I've done research on that before posting the question, so sharing what I found)
Not solution
You could modify your PYTHONPATH to point to your src directory, but it breaks the main benefit from having separate src directory (read the link from OP).
Solution
Use pip install -e path/to/your/package (usually pip install -e .) to enable development mode and test versus your codebase as it would be installed.
After that your tests should be discovered properly. Otherwise it is different issue - read vs code OUTPUT console.
Note: this requires setup.py as a build backend
workaround for poetry
pyproject.toml
[build-system]
requires = [
"poetry-core>=1.0.0",
"setuptools" # to support local installations
]
then
poetry build --format sdist && tar --wildcards -xvf dist/*.tar.gz -O '*/setup.py' > setup.py
pip install -e .
Source: https://github.com/python-poetry/poetry/issues/34
TLDR: proper solution is outside of poetry scope, links to python-list discussions: https://github.com/python-poetry/poetry/issues/34#issuecomment-732478605
I am trying to follow the example in the "First Steps with Celery" document. I have installed Celery using pip.
I created a file called tasks.py in ~/python/celery, and it contains the following:
from celery import Celery
celery = Celery('tasks', broker='amqp://guest#localhost//')
#celery.task
def add(x, y):
return x + y
I started a worker using celery -A tasks worker --loglevel=info while in the ~/python/celery directory, and it seems to be running.
In a separate Terminal window, I launched Python and ran the following:
from tasks import add
add.delay(4, 4)
I get the error:
File "/Library/Python/2.7/site-packages/celery/utils/timeutils.py", line 17, in
from dateutil import tz
ImportError: No module named dateutil
How do I install dateutils? It is listed as an installed module when I type "pip freeze"
Thanks!
That's strange, since you say you can see it as installed with pip.
I've just run pip freeze | grep date and here's what I get:
python-dateutil==1.5
Is your response something similar?
Having run the following:
$ python
>>> import dateutil
>>> help(dateutil)
I am told that my dateutil module is installed in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py (OS X). I would check your Python install to make sure nothing went wrong. There shouldn't be a need to install it separately, but you could perhaps use pip to uninstall then reinstall
Had the same issue with Python3, even though when I tried installing it, a message said it already was. So the fix was:
sudo pip3 uninstall python-dateutil
sudo pip3 install python-dateutil
As suggested here.
I am running a Django server from my run configuration which translates to:
python manage.py runserver --noreload
The stop button in the error panel doesn't kill the process, so I usually run
pskill python
How do I add this to my Eclipse run configurations?
My solution was to create a python file named "pskill.py" containing:
from subprocess import call
from sys import argv
if __name__=='__main__':
call(['pskill', argv[1]])
And add it under "Python Run" in Run Configurations.