pandoc-crossref must match the pandoc version, and also only the 3.10.0 release works on OSX Big Sur. Thus, it is not possible to get pandoc and pandoc-crossref running in a conda environment from the official channel or from conda-forge.
I could easily download the matching binaries from https://github.com/lierdakil/pandoc-crossref/releases/tag/v0.3.10.0 and copy them e.g. to the binpath:
$ which pandoc-crossref
/usr/local/bin/pandoc-crossref
$ curl -OL https://github.com/lierdakil/pandoc-crossref/releases/download/v0.3.10.0/pandoc-crossref-macOS.tar.xz
$ tar -xzvf pandoc-crossref-macOS.tar.xz
$ mv pandoc-crossref /usr/local/bin/pandoc-crossref
But I think that is not a clean approach, because conda will not know that I updated the version for pandoc-crossref.
What is a clean approach for updating a package managed by conda from a binary available on Github?
Update Feedstock
I updated it on the Conda Forge feedstock, which is what I regard as the "cleanest" solution.
How does one do that? First, OP had posted a comment on the feedstock in the PR that they wanted merged. This was the appropriate first step and hopefully in future cases that should be sufficient to prompt maintainers to act. In this case, it was not sufficient. So, as a follow up, I chatted on the Conda Forge Gitter to point out that the feedstock had gone stale and had non-responding maintainer(s). One of the core Conda Forge members suggested I make a PR bumping the version and adding myself as maintainer, and they merged it for me. In all, this took about 10 mins of work and ~2 hours from start to having an updated package on Anaconda Cloud.
Custom Conda Build
Otherwise, there isn't really a clean solution for non-Python packages outside of building a Conda package. That is, clone the feedstock or write a new recipe, modify it to build from the GitHub reference, then install that build into your environment. It may also be worth uploading to an Anaconda Cloud user account, so there is some non-local reference for it.
Pip Install (Python Packages Only)
In the special case that it is a Python package, one could dump the environment to YAML, edit to install the package through pip, then recreate the environment.
Related
I'm working on building a docker image to be able to run all of our Perl applications. The applications require hundreds of CPAN modules to be installed. The full build of the docker image takes about an hour to complete.
After doing the initial image, I'm not sure how best to handle ongoing updates.
We could keep a single Dockerfile in git, and then modify this as required, and push new builds up to dockerhub. However if the person doing the build doesn't have all of the intermediate images, then adding a single CPAN module could be an extremely tedious process, and it might take an hour before they even know if the new module installs correctly. Also it would be downloading every CPAN module again, which seems a bit risky, as there might be a breaking change in the new module.
Alternatively, the person doing the build could pull the latest docker-hub image, and then install the cpan module interactively, commit the build and push the new image to dockerhub. However then we only have our dockerhub images, but not master Dockerfile.
Or another option would be to create a Dockerfile for each new build, which references the previous dockerhub image. This seems overly complicated though.
Option 1) seems wrong. I'm fairly sure we don't want to be rebuilding the entire image from the base OS just to install one additional module. However being dependent on images without Dockerfiles seems risky as well.
You could use the standard module installer for your underlying OS on your docker image.
For example, if its RedHat then use yum and only use CPAN when they are not available
FROM centos:centos7
RUN yum -y install cpanm gcc perl perl-App-cpanminus perl-Config-Tiny && yum clean all
RUN cpanm install Some::Module; rm -fr root/.cpanm; exit 0
taken from here and modified
I would try to have a base image which the actual applications use
I would also avoid doing things interactively (e.g. script a dockerfile) as you want to be able to repeat the build when upstream dependencies change, which docker hub does for you.
EDIT
You can convert perl modules into your own packages using dh-make-perl
You can load these into your own Ubuntu repo using reprepro or a paid solution of Artifactory
These can then be installed using apt-get when you use your repo as a source from within a dockerfile.
When I have tried a similar thing before There are a few problems
Your apps don't work with the latest version of modules
There are far more dependencies than you expected
Some modules wont package
Benefits are
You keep the build tools (gcc, etc) off the app servers
You know much more about your dependencies
I want to install a python package from github. It seems that pip install https://github.com/codelucas/newspaper/archive/python-2-head.zip is the way to go. However, this only installs the python files you can find here without the other folders. The package breaks because of this.
If I run pip install newspaper (which refers to the same code) the other repos are correctly installed.
I could not get if the problem is coming from pip or the package I'm trying to save (I'm kind of new to python packaging :)
The reason I don't want to use pip install newspaper is that I'm working on a fork of that code that I want to pull from github to my server directly. I have the same problem with my fork.
You can install the latest snapshot from the github repository with the following command
pip install git+https://github.com/codelucas/newspaper.git
You can find further information in the corresponding section in pip's documentation.
I'm attempting to create a local yum repo on my system containing various packages from, chiefly, the CentOS base repos. The server which is hosting the yum repo will not necessarily have the same base packages installed by default as the servers which will be using the yum repo. For this reason, I need to ensure that my repos contain the packages that I want and every single one of their dependencies.
I'm creating my repos using the yumdownloader tool provided in the yum-utils package to try to download an RPM file for a package using yum from the standard CentOS mirrors. Helpfully it provides a command line option, --resolve, which also downloads dependencies. However, because it's built on yum itself, yumdownloader will only download dependencies for the package that are not already present on the system.
For example, I wish to download package A, which depends on Packages B, C and D. If package D is already installed on the system, yumdownloader --resolve A will only download A, B and C, but not D.
Is there a way to download the RPMs for all dependencies on a package from a yum repo?
There's this bash script, which the maintainer of rpm has kindly shared with me, and I put on github. Hope you find it useful!
You can also read the original SO question, where the issue was discussed.
The script works on Fedora 23+ as it uses dnf's download plugin. It's probably very easy to make it work on Fedora 22-, as yum surely has got a similar plugin.
Additionaly, it's valuable since repotrack does not work on fedora 23 (at least it doesn't work for me).
After a lot of frustration looking around for a solution I have written a simple script that uses repotrace and wget. I've found that yumdownloader (even with the resolve flag) does not resolve all dependencies.
if you have a long list of packages you are bound to run into duplicates, downloading just the urls first with the "repotrack -u flag" and then getting unique records resolves having to download the same rpm multiple times.
#!/bin/bash
while read i; do
repotrack -u $i >> dep_rpm_urls_02.txt
done < list_of_packages_01.txt
awk '!seen[$0]++' dep_rpm_urls_02.txt > dep_rpm_urls_clean_03.txt
while read j; do
wget $j
echo dowloaded $j
done < dep_rpm_urls_clean_03.txt
happy rpming
I'm really happy that SourceTree has Mercurial embedded. On the odd occasion I would like to run an hg command from the CLI, like
hg status
To which I get
hg: command not found
Can I make modifications to my environment path to get the command line working for the embedded version, or do I have to install and manage a separate instance of Mercurial if I want to accasionally use the command line?
I'm using SourceTree OS X version 1.7.2
(This might be a superuser question, but I thought this community has a better chance of knowing and responding.)
The binary lies within SourceTree's bundle, in the resources-folder and is called hg_local.
/Applications/SourceTree.app/Contents/Resources/mercurial_local
You might want to alias it somewhere else (like /usr/local/bin) and rename it on the go:
ln -s /Applications/SourceTree.app/Contents/Resources/mercurial_local /usr/local/bin
It might be a better idea to use some package manager and install mercurial from their repositories, eg. Homebrew. This can be done within few minutes and you gain easy access to lots of other command line tools you might want.
# First install homebrew, than run this command
brew install mercurial
I have a relatively big project that has many dependencies, and I would like to distribute this project around, but installing these dependencies where a bit of a pain, and takes a very long time (pip install takes quite some time). So I was wondering if it was possible to migrate a whole virtualenv to another machine and have it running.
I tried copying the whole virtualenv, but whenever I try running something, this virtualenv still uses the path of my old machine. For instance when I run
source activate
pserve development.ini
I get
bash: ../bin/pserve: /home/sshum/backend/bin/python: bad interpreter: No such file or directory
This is my old directory. So is there a way to have virtualenv reconfigure this path with a new path?
I tried sed -i 's/sshum/dev1/g' * in the bin directory and it solved that issue. However, I'm getting a different issue now, my guess is that this sed changed something.
I've confirmed that I have libssl-dev installed but when I run python I get:
E: Unable to locate package libssl.so.1.0.0
E: Couldn't find any package by regex 'libssl.so.1.0.0'
But when I run aptitude search libssl and I see:
i A libssl-dev - SSL development libraries, header files and documentation
I also tried virtualenv --relocatable backend but no go.
Export virtualenvironment
from within the virtual environment:
pip freeze > requirements.txt
as example, here is for myproject virtual environment:
once in the new machine & environment, copy the requirements.txt into the new project folder in the new machine and run the terminal command:
sudo pip install -r requirements.txt
then you should have all the packages previously available in the old virtual environment.
When you create a new virtualenv it is configured for the computer it is running on. I even think that it is configured for that specific directory it is created in. So I think you should always create a fresh virtualenv when you move you code. What might work is copying the lib/Pythonx.x/site-packages in your virtualenv directory, but I don't think that is a particularly good solution.
What may be a better solution is using the pip download cache. This will at least speed up the download part of pip install. Have a look at this thread: How do I install from a local cache with pip?
The clean way seems to be with virtualenv --relocatable.
Alternatively, you can do it manually by editing the VIRTUAL_ENV path in bin/activate to reflect the changes. If you choose to do so, you must also edit the first line (#) of bin/pserve which indicates the interpreter path.