I want to install a python package from github. It seems that pip install https://github.com/codelucas/newspaper/archive/python-2-head.zip is the way to go. However, this only installs the python files you can find here without the other folders. The package breaks because of this.
If I run pip install newspaper (which refers to the same code) the other repos are correctly installed.
I could not get if the problem is coming from pip or the package I'm trying to save (I'm kind of new to python packaging :)
The reason I don't want to use pip install newspaper is that I'm working on a fork of that code that I want to pull from github to my server directly. I have the same problem with my fork.
You can install the latest snapshot from the github repository with the following command
pip install git+https://github.com/codelucas/newspaper.git
You can find further information in the corresponding section in pip's documentation.
Related
I have a Python library (https://github.com/jcrozum/PyStableMotifs) that I want to publish on PyPI. It depends on another library (https://github.com/hklarner/PyBoolNet) that I do not control and that is only available on GitHub, and in particular, it is not available on PyPI. My setup.py looks like this:
from setuptools import
setup(
... <other metadata> ...,
install_requires=[
'PyBoolNet # git+https://github.com/hklarner/PyBoolNet#2.3.0',
... <other packages> ...
]
)
Running pip install git+https://github.com/jcrozum/PyStableMotifs works perfectly, but I can't upload this to PyPI because of the following error from twine:
Invalid value for requires_dist. Error: Can't have direct dependency: 'PyBoolNet # git+https://github.com/hklarner/PyBoolNet#2.3.0'
My understanding is that direct links are forbidden by PyPI for security reasons. Nonetheless, PyBoolNet is a hard requirement for PyStableMotifs. What do I do? Give up on PyPI?
I just want pip install PyStableMotifs to work for my users. Ideally, this command should install the dependencies and I should not have to maintain two versions of setup.py.
Failing that, I have considered creating a "dummy" package on PyPI directing users to install using the command pip install git+https://github.com/jcrozum/PyStableMotifs. Is this a bad idea (or even possible)?
Are there already established best practices for this situation or other common workarounds?
EDIT:
For now, I have a clunky and totally unsatisfying workaround. I'm keeping two versions; a GitHub version that works perfectly, and a PyPI version that has the PyBoolNet requirement removed. If the user tries to import PyStableMotifs without PyBoolNet installed, an error message is shown that has install instructions for PyBoolNet. This is far from ideal in my mind, but it will have to do until I can find a better solution or until PyPI fixes this bug (or removes this feature, depending on who you ask).
My recommendation would be to get rid of the direct URL in install_requires, and tell your users where they can find that dependency PyBoolNet since it is not on PyPI. Don't force them on a specific installation method, but show them an example.
Maybe simply tell your users something like:
This project depends on PyBoolNet, which is not available on PyPI. One place where you can find it is at: https://github.com/hklarner/PyBoolNet.
One way to install PyStableMotifs as well as its dependency PyBoolNet is to run the following command:
python -m pip install 'git+https://github.com/hklarner/PyBoolNet#2.3.0#egg=PyBoolNet' PyStableMotifs
You could additionnally prepare a requirements.txt file and tell your users:
Install with the following command:
python -m pip install --requirement https://raw.githubusercontent.com/jcrozum/PyStableMotifs/master/requirements.txt
The content of requirements.txt could be something like:
git+https://github.com/hklarner/PyBoolNet#2.3.0#egg=PyBoolNet
PyStableMotifs
But in the end, you should really let your users choose how to install that dependency. Your project only need to declare that it depends on that library but not how to install it.
pandoc-crossref must match the pandoc version, and also only the 3.10.0 release works on OSX Big Sur. Thus, it is not possible to get pandoc and pandoc-crossref running in a conda environment from the official channel or from conda-forge.
I could easily download the matching binaries from https://github.com/lierdakil/pandoc-crossref/releases/tag/v0.3.10.0 and copy them e.g. to the binpath:
$ which pandoc-crossref
/usr/local/bin/pandoc-crossref
$ curl -OL https://github.com/lierdakil/pandoc-crossref/releases/download/v0.3.10.0/pandoc-crossref-macOS.tar.xz
$ tar -xzvf pandoc-crossref-macOS.tar.xz
$ mv pandoc-crossref /usr/local/bin/pandoc-crossref
But I think that is not a clean approach, because conda will not know that I updated the version for pandoc-crossref.
What is a clean approach for updating a package managed by conda from a binary available on Github?
Update Feedstock
I updated it on the Conda Forge feedstock, which is what I regard as the "cleanest" solution.
How does one do that? First, OP had posted a comment on the feedstock in the PR that they wanted merged. This was the appropriate first step and hopefully in future cases that should be sufficient to prompt maintainers to act. In this case, it was not sufficient. So, as a follow up, I chatted on the Conda Forge Gitter to point out that the feedstock had gone stale and had non-responding maintainer(s). One of the core Conda Forge members suggested I make a PR bumping the version and adding myself as maintainer, and they merged it for me. In all, this took about 10 mins of work and ~2 hours from start to having an updated package on Anaconda Cloud.
Custom Conda Build
Otherwise, there isn't really a clean solution for non-Python packages outside of building a Conda package. That is, clone the feedstock or write a new recipe, modify it to build from the GitHub reference, then install that build into your environment. It may also be worth uploading to an Anaconda Cloud user account, so there is some non-local reference for it.
Pip Install (Python Packages Only)
In the special case that it is a Python package, one could dump the environment to YAML, edit to install the package through pip, then recreate the environment.
I have a hard time understanding where is the right place to place a code that will install the needed packages for the given docker container managed by dokku.
We have a scala application and, unfortunately, we need to have one shell call that is dependent on an environment. I would like to install the given package for the given container using "apt-get install". Right now I am using a custom plugin with a file named "post-release-build". However, I don't have the permission to install anything in that phase.
Basically, my script that should be invoked looks like this (based on a dockerfile that is available online):
apt-get update
apt-get install -y build-essential xorg libssl-dev libxrender-dev wget gdebi
wget http://download.gna.org/wkhtmltopdf/0.12/0.12.2.1/wkhtmltox-0.12.2.1_linux-trusty-amd64.deb
gdebi --n wkhtmltox-0.12.2.1_linux-trusty-amd64.deb
echo "-----> wkhtmltox installed!"
Is there a way how to make it work? I would also prefer to have such a file somewhere in the application so I don't need to setup environment before pushing the app (in the future).
EDIT:
I have found a plugin that should be capable of installing packages using apt-get (https://github.com/F4-Group/dokku-apt) however, I am a little bit unlucky because it downloads a package that is not working properly.
Since just downloading with apt-get will download a package that fails, I investigated deeper into dokku and came out with a new plugin that should install the package for you.
I have created a script, documented how to use it and licenced it over MIT license so feel free to use it. Hopefully it will save you the time I had to spend realizing what is going on.
URL: https://github.com/mbriskar/dokku-wkhtmltopdf
I'm attempting to create a local yum repo on my system containing various packages from, chiefly, the CentOS base repos. The server which is hosting the yum repo will not necessarily have the same base packages installed by default as the servers which will be using the yum repo. For this reason, I need to ensure that my repos contain the packages that I want and every single one of their dependencies.
I'm creating my repos using the yumdownloader tool provided in the yum-utils package to try to download an RPM file for a package using yum from the standard CentOS mirrors. Helpfully it provides a command line option, --resolve, which also downloads dependencies. However, because it's built on yum itself, yumdownloader will only download dependencies for the package that are not already present on the system.
For example, I wish to download package A, which depends on Packages B, C and D. If package D is already installed on the system, yumdownloader --resolve A will only download A, B and C, but not D.
Is there a way to download the RPMs for all dependencies on a package from a yum repo?
There's this bash script, which the maintainer of rpm has kindly shared with me, and I put on github. Hope you find it useful!
You can also read the original SO question, where the issue was discussed.
The script works on Fedora 23+ as it uses dnf's download plugin. It's probably very easy to make it work on Fedora 22-, as yum surely has got a similar plugin.
Additionaly, it's valuable since repotrack does not work on fedora 23 (at least it doesn't work for me).
After a lot of frustration looking around for a solution I have written a simple script that uses repotrace and wget. I've found that yumdownloader (even with the resolve flag) does not resolve all dependencies.
if you have a long list of packages you are bound to run into duplicates, downloading just the urls first with the "repotrack -u flag" and then getting unique records resolves having to download the same rpm multiple times.
#!/bin/bash
while read i; do
repotrack -u $i >> dep_rpm_urls_02.txt
done < list_of_packages_01.txt
awk '!seen[$0]++' dep_rpm_urls_02.txt > dep_rpm_urls_clean_03.txt
while read j; do
wget $j
echo dowloaded $j
done < dep_rpm_urls_clean_03.txt
happy rpming
I have to work with rdflib 4.0.1 (the latest version of it) to work with sparql, but this package is not available in the Package Manager of Canopy (Enthought).
How can I install it, in this situation?
The rdflib package is not in the Canopy / EPD repository.
Rather, it is in the "Community" (PyPi mirror) repo (marked by the "PyPI" logo in the Package Manager), which contains 11,000 untested ("as is") packages. Most of the packages are current and we are in the process of updating the rest, as well as keeping it updated from now on. Please note that we do not test for nor necessarily provide dependencies.
Meanwhile, this may help:
"Installing external packages into Canopy Python"
If you follow the link to installing external packages in Canopy that #jonathan-march provided, point three says to follow the "standard Python installation procedures." It also includes information on installing pip. I would install pip using Canopy's instructions and then simply run pip install rdflib, which will install RDFLib 4.0.1.
To install a package which is not available in the Canopy / EPD repository, follow standard Python installation procedures from the OS command line. If you want to use pip, install it first:
easy_install pip (Please check this article for more info on using enpkg)