PyPI install_requires direct links - github

I have a Python library (https://github.com/jcrozum/PyStableMotifs) that I want to publish on PyPI. It depends on another library (https://github.com/hklarner/PyBoolNet) that I do not control and that is only available on GitHub, and in particular, it is not available on PyPI. My setup.py looks like this:
from setuptools import
setup(
... <other metadata> ...,
install_requires=[
'PyBoolNet # git+https://github.com/hklarner/PyBoolNet#2.3.0',
... <other packages> ...
]
)
Running pip install git+https://github.com/jcrozum/PyStableMotifs works perfectly, but I can't upload this to PyPI because of the following error from twine:
Invalid value for requires_dist. Error: Can't have direct dependency: 'PyBoolNet # git+https://github.com/hklarner/PyBoolNet#2.3.0'
My understanding is that direct links are forbidden by PyPI for security reasons. Nonetheless, PyBoolNet is a hard requirement for PyStableMotifs. What do I do? Give up on PyPI?
I just want pip install PyStableMotifs to work for my users. Ideally, this command should install the dependencies and I should not have to maintain two versions of setup.py.
Failing that, I have considered creating a "dummy" package on PyPI directing users to install using the command pip install git+https://github.com/jcrozum/PyStableMotifs. Is this a bad idea (or even possible)?
Are there already established best practices for this situation or other common workarounds?
EDIT:
For now, I have a clunky and totally unsatisfying workaround. I'm keeping two versions; a GitHub version that works perfectly, and a PyPI version that has the PyBoolNet requirement removed. If the user tries to import PyStableMotifs without PyBoolNet installed, an error message is shown that has install instructions for PyBoolNet. This is far from ideal in my mind, but it will have to do until I can find a better solution or until PyPI fixes this bug (or removes this feature, depending on who you ask).

My recommendation would be to get rid of the direct URL in install_requires, and tell your users where they can find that dependency PyBoolNet since it is not on PyPI. Don't force them on a specific installation method, but show them an example.
Maybe simply tell your users something like:
This project depends on PyBoolNet, which is not available on PyPI. One place where you can find it is at: https://github.com/hklarner/PyBoolNet.
One way to install PyStableMotifs as well as its dependency PyBoolNet is to run the following command:
python -m pip install 'git+https://github.com/hklarner/PyBoolNet#2.3.0#egg=PyBoolNet' PyStableMotifs
You could additionnally prepare a requirements.txt file and tell your users:
Install with the following command:
python -m pip install --requirement https://raw.githubusercontent.com/jcrozum/PyStableMotifs/master/requirements.txt
The content of requirements.txt could be something like:
git+https://github.com/hklarner/PyBoolNet#2.3.0#egg=PyBoolNet
PyStableMotifs
But in the end, you should really let your users choose how to install that dependency. Your project only need to declare that it depends on that library but not how to install it.

Related

fix setuptools version when installing libraries using setup.py

This question is somewhat similar to the one here, but I cannot make it work.
So suppose that I have a set of packages (say 2) to install and I want to use pipenv. If I do pipenv install on the directory with a suitable Pipfile the installation fails because there is some metadata issue when installing one of the libraries (say libX) contained in install_requirements of one of the packages. It seems that the problem can be fixed by downgrading the version of setuptools to <=58.0.0.
OK. Now, if I first install that version of setuptools<=58.0.0 in the venv and then install my packages, everything works fine. The issue is that the Pipenvfile does not respect the order when installing, so something like
[packages]
setuptools = "<=58.0.0"
pckg1 = {<github path 1>}
pckg2 = {<github path 2>}
is not ensured to work. Also, by default the seed packages added to the venv include setuptools==65.6.3.
So the idea is to be able to restrict the version of setuptools that is used to check the metadata of the libraries in libX, to mimic the above scenario in which setuptools was installed first. Is there a way to do that?
I have tried placing setuptools<=58.0.0 at the top of the requirements.txt that defines the install_requirements of the problematic package, but it does not work.
If have also tried to fix or restrict the version of libX contained in that requirements.txt file but, surprisingly, pipenvdoes not seem to care: a verbose install shows that it keeps downgrading libX well below the restriction - "using cached libX-vX.X.X"- until it uses a version for which the metadata generation fails (why on earth does it do that, even if I call it with pipenv --clear install?).
I am a bit lost about what could be the best solution here. Any help would be very appreciated.

How do I install the hg-git plugin on Debian Stretch?

Debian Jessie, as well as sid, have a mercurial-git package which contains the hg-git plugin. However, this package was (auto-)removed from Debian Stretch to to a release-critical bug.
But - I need it installed and running. Surely this should be possible, right?
Well, I followed the installation instructions on the plugin page:
I ran apt-get install python-setuptools python-setuptools-git python4-setuptools python3-setuptools-git
I ran easy_install hg-git and it seemed to work
But still, when I run various mercurial operations I get, as the first line, the error message:
*** failed to import extension hgext.git: No module named git
(regardless of whether I'm doing anything git-related or not.)
My questions:
Why is this happening?
What do I need to do in order to make the error message go away while having hggit working?
Now,
How do I correctly install dulwich to get hg-git working on Windows?
Apparently, that critical bug doesn't manifest always (and perhaps only under very specific circumstances), so you can try installing the Debian sid version of the mercurial-git package (that is, version 0.8.11-1 at the time of writing). There's a SuperUser question about how to do this:
https://linuxaria.com/howto/how-to-install-a-single-package-from-debian-sid-or-debian-testing
my personal opinion in this case is to simply install the .deb file, which you can get from here (it's not platform-specific; at the link you'll need to choose a mirror.) That makes the error message go away, at least assuming you have:
[extensions]
hgext.bookmarks =
hggit =
in your ~/.hgrc file.

installing ipython on rhel7

I'm a RHEL newbie. I'm used to a non-Linux Unix, which has a fundamentally different way of dealing with packages.
I want to install ipython for a user on a vanilla RHEL7 system with yum as the package manager.
"yum install python" was fairly straightforward, but given that I'm new to the OS and I don't completely understand what ipython is, I am stumped as to how to proceed.
"yum install ipython" obviously doesn't work and every possible solution seems to require the installation of something else that I don't know how to install in a reasonable manner.
I am trying to keep things as generic as possible so it will be obvious how to update/remove software in the future, so anything that can be done with yum, would be probably preferable.
Installation instructions refer to pip, which I don't have. I possibly need setuptools to run pip, but I can't figure out the appropriate way to get that either. Maybe I can get one or either by installation the EPEL bundle of packages, but I can't find those for RHEL7, at least not in a way that doesn't seem like a "download and install this random file, trust us" method, which seems irresponsible.
Another option is anaconda. Again, there doesn't seem to be a yum-related way to install this, and anaconda itself is only a means to an end to download ipython, so that'd be two levels of abstraction away from the goal.
Additionally, do I even want "ipython" these days, or do I want "jupyter"?
All I care about is that the user should be able to type in "ipython" at the prompt and get the thing he is expecting.
Also, the python installed by yum is 2.7.5-48.el7, which does not seem to be current. I don't care about using the current version unless that prevents me from successfully installing ipython in some other manner, but I thought it might be relevant.
Any suggestions for how to install this thing is the most easily maintainable way? Do I not want the yum version of python?
Thanks for your patience.
Install python-pip from EPEL repository first ( https://fedoraproject.org/wiki/EPEL - it's compatible with all Red Hat entrprise Linux distros - be it CentOS, RHEL, Oracl, ScientificLinux or whatever), (or if you don't trust EPEL repo providers you can use get-pip.py ( https://bootstrap.pypa.io/get-pip.py ) script, but then you have to trust its providers instead) then install via
pip install ipython

pip install from github doesn't install everything

I want to install a python package from github. It seems that pip install https://github.com/codelucas/newspaper/archive/python-2-head.zip is the way to go. However, this only installs the python files you can find here without the other folders. The package breaks because of this.
If I run pip install newspaper (which refers to the same code) the other repos are correctly installed.
I could not get if the problem is coming from pip or the package I'm trying to save (I'm kind of new to python packaging :)
The reason I don't want to use pip install newspaper is that I'm working on a fork of that code that I want to pull from github to my server directly. I have the same problem with my fork.
You can install the latest snapshot from the github repository with the following command
pip install git+https://github.com/codelucas/newspaper.git
You can find further information in the corresponding section in pip's documentation.

Why can't I cabal install --only-dependencies with mongodb?

I have gone through the following steps:
$ mkdir mongoEg
$ cd mongoEg
$ cabal init
...
Configured to run as an executable. I add mongodb to the build-depends list. I make a dummy Main.hs file and put a basic hello world in there. I then do
$ cabal sandbox init
$ cabal install --only-dependencies
Which responds with:
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: monogEg-0.1.0.0 (user goal)
next goal: mongodb (dependency of monogEg-0.1.0.0)
Dependency tree exhaustively searched.
Note: when using a sandbox, all packages are required to have consistent
dependencies. Try reinstalling/unregistering the offending packages or
recreating the sandbox.
I read up on other problems people are having, and remove ~/.ghc, remove my mongoEg directory, and repeat to get the same results. I try to run through the analogous steps at http://howistart.org/posts/haskell/1 and find that everything works just fine.
I then guess that something is wrong with the mongodb package itself. I seem to be able to cabal install mongodb in a global environment and use it outside of a sandbox without any issue. So, why wont cabal sandboxes play with the mongodb package?
See this gist for details: https://gist.github.com/anonymous/e5a548cf7d9ec59bea31
After looking here
Cabal configure in a sandbox complains "At least the following dependencies are missing" on installed packages
I saw that the answer states that package names are case sensitive. So I tried changing mongodb to the way MongoDB spells it, namely MongoDB. This did not work, so I tried changing it to mongoDB, and finally there was joy.
So even though I can do cabal install mongodb I can't use that same spelling to install it from within a .cabal file, which is, obviously, completely stupid. I'm sure I'll find the right place to channel my rage about this kind of flagrant violation of the principle of least surprise, but for now I can say that to newcomers it is most needlessly confusing.