I need to install tensorflow_backend.py from a specific tree in keras, ideally without updating the other keras files. This shows how to install a specific repo branch, but how can the same be done for this single file in a tree?
No. pip installs Python packages. You need a different tool to download just one file. For a remote git repository git archive is a good tool. For a remote web interface use curl or wget.
Related
I want to get started developing my own packages. I am also adding version control via Github. I mainly develop on my Mac and a Windows laptop, but there is potential for me to develop on other machines down the line. My IDE of choice is PyCharm. I need to figure out where to place my packages both on Github and on my local machines so that my packages are always in sync regardless of where I am developing. Help??
First, let's clarify that git is the version control system, and Github is a platform for hosting git repositories (there are many other platforms aside from Github). You use git commands to manage your codes, and Github is where you store a copy of your codes.
By adding version control and putting a copy on Github, you've already taken the first step in managing your codes on different machines. All you need to do is to make sure the codes on Github is always the latest updated or maintained version.
Here's a sample workflow:
On machine 1 (Mac), clone a copy of the Github repo
Develop on machine 1
When you are satisfied with your changes, push your codes from machine 1 to Github
On machine 2 (Windows), clone a copy of the Github repo
Develop on machine 2
When you are satisfied with your changes, push your codes from machine 2 to Github
On machine 1, do a fetch to check for updates to the code
If there are updates, pull those changes to machine 1
Again, when done making changes, push them from machine 1 to Github
On machine 2 again, fetch and pull changes
Repeat this fetch-pull-push- cycle for all machines
Basically, you'll need to make sure that on wherever machine you are, when you are done, you should always push those changes to the remote (Github). So that other machines, can fetch and pull those changes and continue where you left off.
UPDATE (based on comment):
Once you've got the workflow for your package source codes, next is to package them like any other regular Python package and install them to your site-packages (either directly for your system or preferably in a virtual environment).
I recommend taking a look at the Python docs on Packaging Python Projects which uses setuptools to make your package compatible with pip.
Here's a sample workflow:
git clone <mypackage#github.com> # or git pull if you already cloned it before
cd mypackage
pip install -r requirements.txt
pip install -e . or pip install --user -e .
That last step will install your package to your site-packages folder, like any other pip-compatible package (assuming you've setup your setup.py file properly). If you are using virtual environments, you'll have to activate the virtual env first, then install your package there.
If you are not going to do any modification on the source code, and you just want to install the package on a specific machine, then you can also specify the Github URL to pip:
$ pip install -e git+https://git.repo/some_pkg.git#egg=SomeProject # from git
Lastly, if you are planning to upload this package to PyPi, check out the docs on Uploading the distribution archives. This just adds an extra step to your workflow of uploading your package to PyPi and then doing pip install from there next time.
I continue to have failed builds with ReadTheDocs using a github repository with Jupyter notebooks rendered using sphinx and the readthedocs theme. I am able to render these locally, however when I try with RTD, I get the following fail message:
I tried placing the nbsphinx.py file in the repository but things still fail. Link to RTD is here:
https://readthedocs.org/projects/calculus-notes/
Github repo is here:
https://github.com/jfkoehler/calc_docs
RTD needs to install nbsphinx. To tell RTD to install packages, you must:
From RTD FAQ, My project isn’t building with autodoc:
enable the virtualenv feature in the Admin page of your project, which will install your project into a virtualenv, and allow you to specify a requirements.txt file for your project.
Create requirements.txt in your repo and add nbsphinx to its contents. See https://github.com/spatialaudio/nbsphinx/blob/0.2.14/doc/requirements.txt#L2
After you commit and push item 2 to your repo, then RTD should install nbsphinx and be able to build your docs successfully, or at least proceed to the next failure if there is one.
I use git lfs to store the large files of my git repo. I then try to build this repo with hosted agents. My build is pretty simple. It has a single task: Execute PowerShell. In the invoked script, the first thing that I want to do is to fetch my lfs dependencies. I therefore have the following in my script:
& git lfs fetch
Unfortunately, my build fails with the following error:
2016-03-04T19:49:05.7021988Z ##[error]git: 'lfs' is not a git command. See 'git --help'.
2016-03-04T19:49:05.7031986Z ##[error]Did you mean this?
2016-03-04T19:49:05.7041987Z ##[error] flow
Since I can't install anything on hosted agents, how am I supposed to have git lfs available?
EDIT
In this issue, I am not talking about git lfs authentication problems as described here. I am strictly talking about the issue of calling git lfs.
Once you are able to call git lfs, look at this answer to solve the authentication problem.
Git LFS is now supported by default on the Hosted Build Controller. But you do need to enable it in your get sources step.
You get this error message because git-lfs isn't installed on Hosted Build Agent by default.
And since you are using Hosted Build Agent, it would be a little troublesome to install git-lfs via Chocolatey on it as you don't have administrator permission. An alternative way would be download the binary files for git-lfs directly and upload it into Source Control. Then you can invoke the git-tfs.exe with an absolute path in your script.
Here are some more details around the solution provided by Eddie. git lfs is not a built-in command. It is a git custom command.
When you call git lfs, git.exe does not know about the lfs command. So it looks in the PATH environment variable and searches for a program named git-lfs.exe. Once found, it calls that program with the provided argument.
So calling git-lfs.exe pull is equivalent to calling git.exe lfs pull.
The solution suggested is therefore to download git-lfs.exe, add it your git repo (it should obviously not be tracked by LFS), and call git-lfs.exe.
It is also possible to add the folder that contains git-lfs.exe to your path environment variable. This makes it possible to use commands like git.exe lfs pull as you usually do.
If you are allowed to install software and have internet access during build you might be able to install git-lfs using the Chocolatey package in a cmd / PowerShell task prior to your git-lfs operation.
Should I re-upload an existing library to my GitHub repo if my code uses it? Or should I only reference the library?
I have some Python programs that use the Yowsup library, which is already on GitHub. Should I upload my copy of this library with my code in order to make my code easier to understand, or should I just tell people to download Yowsup from its own GitHub page?
Thanks!
Maintain your dependencies using a dependency manager.
For Python code, this usually means using pip to maintain a requirements file:
pip install yowsup
pip freeze > requirements.txt
Commit the requirements.txt file to your repository. Don't commit the yowsup code itself.
Now other users can clone your repository and install all of your project's dependencies using
pip install -r requirements.txt
Generally you will want to do this inside a virtual environment, which in the Python world generally means using virtualenv (and optionally virtualenvwrapper).
Many other languages have similar tools, so you can apply the same general technique.
What are the best practices for using one mercurial project in another? I've got a django app that I'm working on, but I'm also using mercurial to version control a website that uses that app. I've looked at mercurial subrepositories, but apparently this is considered a "feature of last resort". Is there a good way of doing what I want to do, or do I just have to copy the code from my app into my website repo when I want to update to a new version of my app?
In your specific case I like to let pip handle my django application dependencies: http://guide.python-distribute.org/pip.html#installing-from-a-vcs
We have in our "website" repo a requirements.txt and our deploy does a pip install --upgrade -r requirements.txt That pulls the latest from the repo and installes it into the application's virtual env. This gives nice flexibility and separation while leaving the package management up to pip. With those VCS urls in pip you can point to a specific tag or branch too if you want different sites using different revisions from the same underlying repo.
pip also has a -e /path/to/file mode for pointing to an "editable" clone that's outside the website repo, which would work too, but I've not tried it.
That said, if you think subrepos fit your workflow better by all means use them. They work just fine, but people often get hung up on the workflow constraints ("What do you mean I can't commit my parent repo w/o also committing in the subrepo?!")