I am migrating the configurations from setup.py to setup.cfg but I have a problem with cmdclass keyword. I looked into setuptools docs and it seems like this keyword is not documented or supported. So I tried options.entry_points instead. But I keep getting invalid command error.
Here is what I have:
setup.cfg
[options.entry_points]
console_scripts =
install = CustomInstall:run
and
setup.py
from setuptools.command.install import install
from setuptools import setup
class CustomInstall(install):
def run(self):
print('overriden install command')
setup()
The result was just a normal install command. However, I would like to replicate the behaviour that I get when I run:
# setup.py
from setuptools.command.install import install
from setuptools import setup
class CustomInstall(install):
def run(self):
print('overriden install command')
setup(cmdclass= {"install": CustomInstall})
which gives an overriden install command.
I don't believe it's possible. Leave it in the setup.py.
It is not supported.
See this PR for more information.
The cmdclass keyword in setup.cfg has been fixed in v54. It is still not documented though.
Now you can do:
setup.cfg (only part)
[options]
cmdclass =
install = build.MyInstall
build.py
from setuptools.command.install import install
class MyInstall(install):
def run(self):
print("Custom install start")
super().run()
print("Custom install done")
Both files are located in the root of project.
When you run python setup.py install, the custom install script gets triggered. This setup is equivalent to the setup(cmdclass=...) you used.
What [options.entry_points] and console_scripts does is create shell commands which run specific functions in your code base. These commands will work once your package has been installed in an environment.
So in your example, a new shell command install would be created, and it would run the run() function from a package called CustomInstall.
Based on looking at source code, I would guess the correct syntax is instead:
[global]
commands =
install = mypackage.CustomInstall
but I haven't been able to make it work either yet.
Related
my setup.py is like below:
from setuptools import find_packages, setup
tests_require = ['pytest-env', 'pytest-mock', 'pytest-cov', 'pytest-xdist', 'pytest', 'mock', 'moto<=1.3.10']
setup(
name='repo_name',
version='0.1.0',
description='repo_name',
keywords=['?'],
packages=find_packages('src', exclude=['tests', 'venv']),
package_dir={'': 'src'},
package_data={'': ['schema/*.yaml']},
install_requires=[
'boto3<=1.10.19',
'requests<=2.22.0',
'jsonschema<=3.0.1',
'objectpath<=0.6.1',
'pyyaml<=5.1.1',
'sqlalchemy<=1.3.5',
'psycopg2-binary<=2.8.3',
'auth-client<=1.0.23', # internal package
'policy_client<=1.0.9', # internal package
'audit-client<=1.1.20', # internal package
'flask<=1.1.1',
'click<=7.0',
'Werkzeug<=0.15.5',
'itsdangerous<=1.1.0',
'Jinja2<=2.10.1',
'MarkupSafe<=1.1.1',
'structlog<=19.2.0',
'python-rapidjson<=0.9.1'
],
setup_requires=['pytest-runner'],
tests_require=tests_require,
extras_require={'test': tests_require},
include_package_data=True,
zip_safe=False
)
When I run python setup.py test, it will always reinstall the most of the packages that I already installed into .eggs folder, which I understand. While it's installing the package with different versions that I provided in the install_requires section, which results in below error:
pkg_resources.ContextualVersionConflict: (urllib3 1.25.10 (/Users/***/Desktop/repo/.eggs/urllib3-1.25.10-py3.7.egg), Requirement.parse('urllib3<1.25,>=1.21.1'), {'requests'})
I have to pin the requests version to 2.22.0 which requires urllib3 version no larger than 1.25, which caused my issue. So, is there anyway I can pin the urllib3 version in somewhere so when it runs, it will only install the provided version? I have tried to add the urllib3 version in install_requires, tests_require and in requirements.txt, but no luck so far.
Probably not what you want to hear but... setup_requires and tests_require are deprecated. Although they (probably) still work (mostly) fine, you might want to look for alternative solutions. As far as I know the most common alternative solutions are to use Tox instead of tests_require and PEP 517 to replace setup_requires.
In your case, it seems that your usage of setup_requires is only a consequence of your usage of tests_require. So by switching to something like Tox you get rid of both tests_require and setup_requires.
I got it resolve to simply replace python setup.py test to be pip instll -e . [test], and then pytest directly.
It will still install all the tests_require packages and then run pytest directly. Instead of going through all the list packages and find the best match versions for all packages.
UPDATES:
The real problem is I did not remove the old .eggs/ and venv/ folder when I made the packages version change. So the solution is updating the requests version to be 2.21.0 in the setup.py file, then remove the .eggs/ and venv/ folder and rerun everything.
The version of my Python is 2.7.13.
I run the following in Jupyter Notebook.
Firstly, I installed the packages
%%bash
pip uninstall -y google-cloud-dataflow
pip install --upgrade --force tensorflow_transform==0.15.0 apache-beam[gcp]
Then,
%%bash
pip freeze | grep -e 'flow\|beam'
I can see that the package tensorflow-transform is installed.
apache-beam==2.19.0
tensorflow==2.1.0
tensorflow-datasets==1.2.0
tensorflow-estimator==2.1.0
tensorflow-hub==0.6.0
tensorflow-io==0.8.1
tensorflow-metadata==0.15.2
tensorflow-probability==0.8.0
tensorflow-serving-api==2.1.0
tensorflow-transform==0.15.0
However when I tried to import it, there are warning and error.
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow_estimator/python/estimator/api/_v1/estimator/__init__.py:12: The name tf.estimator.inputs is deprecated. Please use tf.compat.v1.estimator.inputs instead.
ImportErrorTraceback (most recent call last)
<ipython-input-3-26a4792d0a76> in <module>()
1 import tensorflow as tf
----> 2 import tensorflow_transform as tft
3 import shutil
4 print(tf.__version__)
ImportError: No module named tensorflow_transform
After some investigation, I think I have some ideas of the problem.
I run this:
%%bash
pip show tensorflow_transform| grep Location
This is the output
Location: /home/jupyter/.local/lib/python3.5/site-packages
I tried to modify the $PATH by adding /home/jupyter/.local/lib/python3.5/site-packages to the beginning of $PATH. However, I still failed to import tensorflow_transform.
Based on the above and the following information, I think, when I ran the import command, it executes Python 2.7, not Python 3.5
import sys
print('\n'.join(sys.path))
/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
/usr/local/lib/python2.7/dist-packages/IPython/extensions
/home/jupyter/.ipython
Also,
import sys
sys.executable
'/usr/bin/python2'
I think the problem is tensorflow_transform package was installed in /home/jupyter/.local/lib/python3.5/site-packages. But when I run "Import", it goes to /usr/local/lib/python2.7/dist-packages to search for the package, rather than /home/jupyter/.local/lib/python3.5/site-packages, so even updating $PATH does not help. Am I right?
I tried to upgrade my python, but
%%bash
pip install upgrade python
Defaulting to user installation because normal site-packages is not writeable
Then, I added --user. It seems that the python is not really upgraded.
%%bash
pip install --user upgrade python
%%bash
python -V
Python 2.7.13
Any solution?
It seems to me that your jupyter notebook is not using the right python environment.
Perhaps, you installed the package under version 3.5,
but the Notebook uses the other one, thus it cannot find the library
You can pick the other interpreter by clicking on: Python(your version) - bottom left.
VS-Code - Select Python Environment 1
However you can do this also via:
CNTRL+SHIFT+P > Select Python Interpreter to start Jupyter Server
If that does not work make sure that the package you are trying to import is installed under the correct python environment.
If not open up a terminal, activate the environment and install it using:
pip install packagename
For example i did the same thing here: (Note: I'm using Anaconda)
installing tensorflow_transform
After a installation, you can import it in your code directly like this:
importing tensorflow_transform
Environment info: Anaconda, windows7x64, py3.5
I setup a virtual env named as menpo to run menpoproject. For a long time I was using load_dlib_frontal_face_detector smoothly. After tinkering with a pip-install dlib command accidentally on virtualenv (I mean via Anaconda Prompt menpo) , I couldn't get my code working due to an "ImportError" ImportError: cannot import name' load_dlib_frontal_face_detector'. Code is like below:
import cv2
import menpo.io as mio
import menpodetect
Throws error at line 3.
There is this guy seems to had a similar issue.
https://github.com/menpo/menpodetect/issues/15
I did all the
conda remove dlib -y
pip uninstall dlib
conda install -c conda-forge dlib
stuff but still got the same error. Besides that;
Uninstalled Anaconda completely
Removed all Python folders wherever I found.
Installed Anaconda and setup a new menpo virtual env and still no luck. It seems like this dlib installation I made causing some issues. BTW, conda list produces dlib 18.18 py35_2 menpo and there is no pip line as mentioned in the link given above.
Wrong alarm. Seems that I've named my py file as menpodetect which I shouldn't do. Problem solved.
Does anybody know how to install h5py on datalab? pip install h5py doesn't work. apt-get install python-h5py is working in the shell but it doesn't recognize the package in datalab notebook!
Thnaks
It is true that !pip install h5py wil allow you to install the library but unfortunately, even after a successful installation, the import will fail:
The issue is rooted on an ongoing python-future issue ("surrogateescape handler broken when encountering UnicodeEncodeError") that is experienced in datalab because the underlying OS uses an 'ANSI_X3.4-1968' file system encoding.
As a hacky workaround, you may remove line 60 from h5py's __init__.py by running the following command from within a notebook cell:
!sed -i.bak '/run_tests/d' /usr/local/lib/python2.7/dist-packages/h5py/__init__.py
Just make sure you run it using bash syntax: !pip install h5py in any notebook cell.
I am trying to follow the example in the "First Steps with Celery" document. I have installed Celery using pip.
I created a file called tasks.py in ~/python/celery, and it contains the following:
from celery import Celery
celery = Celery('tasks', broker='amqp://guest#localhost//')
#celery.task
def add(x, y):
return x + y
I started a worker using celery -A tasks worker --loglevel=info while in the ~/python/celery directory, and it seems to be running.
In a separate Terminal window, I launched Python and ran the following:
from tasks import add
add.delay(4, 4)
I get the error:
File "/Library/Python/2.7/site-packages/celery/utils/timeutils.py", line 17, in
from dateutil import tz
ImportError: No module named dateutil
How do I install dateutils? It is listed as an installed module when I type "pip freeze"
Thanks!
That's strange, since you say you can see it as installed with pip.
I've just run pip freeze | grep date and here's what I get:
python-dateutil==1.5
Is your response something similar?
Having run the following:
$ python
>>> import dateutil
>>> help(dateutil)
I am told that my dateutil module is installed in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py (OS X). I would check your Python install to make sure nothing went wrong. There shouldn't be a need to install it separately, but you could perhaps use pip to uninstall then reinstall
Had the same issue with Python3, even though when I tried installing it, a message said it already was. So the fix was:
sudo pip3 uninstall python-dateutil
sudo pip3 install python-dateutil
As suggested here.