What are the best practices for using one mercurial project in another? I've got a django app that I'm working on, but I'm also using mercurial to version control a website that uses that app. I've looked at mercurial subrepositories, but apparently this is considered a "feature of last resort". Is there a good way of doing what I want to do, or do I just have to copy the code from my app into my website repo when I want to update to a new version of my app?
In your specific case I like to let pip handle my django application dependencies: http://guide.python-distribute.org/pip.html#installing-from-a-vcs
We have in our "website" repo a requirements.txt and our deploy does a pip install --upgrade -r requirements.txt That pulls the latest from the repo and installes it into the application's virtual env. This gives nice flexibility and separation while leaving the package management up to pip. With those VCS urls in pip you can point to a specific tag or branch too if you want different sites using different revisions from the same underlying repo.
pip also has a -e /path/to/file mode for pointing to an "editable" clone that's outside the website repo, which would work too, but I've not tried it.
That said, if you think subrepos fit your workflow better by all means use them. They work just fine, but people often get hung up on the workflow constraints ("What do you mean I can't commit my parent repo w/o also committing in the subrepo?!")
Related
I want to get started developing my own packages. I am also adding version control via Github. I mainly develop on my Mac and a Windows laptop, but there is potential for me to develop on other machines down the line. My IDE of choice is PyCharm. I need to figure out where to place my packages both on Github and on my local machines so that my packages are always in sync regardless of where I am developing. Help??
First, let's clarify that git is the version control system, and Github is a platform for hosting git repositories (there are many other platforms aside from Github). You use git commands to manage your codes, and Github is where you store a copy of your codes.
By adding version control and putting a copy on Github, you've already taken the first step in managing your codes on different machines. All you need to do is to make sure the codes on Github is always the latest updated or maintained version.
Here's a sample workflow:
On machine 1 (Mac), clone a copy of the Github repo
Develop on machine 1
When you are satisfied with your changes, push your codes from machine 1 to Github
On machine 2 (Windows), clone a copy of the Github repo
Develop on machine 2
When you are satisfied with your changes, push your codes from machine 2 to Github
On machine 1, do a fetch to check for updates to the code
If there are updates, pull those changes to machine 1
Again, when done making changes, push them from machine 1 to Github
On machine 2 again, fetch and pull changes
Repeat this fetch-pull-push- cycle for all machines
Basically, you'll need to make sure that on wherever machine you are, when you are done, you should always push those changes to the remote (Github). So that other machines, can fetch and pull those changes and continue where you left off.
UPDATE (based on comment):
Once you've got the workflow for your package source codes, next is to package them like any other regular Python package and install them to your site-packages (either directly for your system or preferably in a virtual environment).
I recommend taking a look at the Python docs on Packaging Python Projects which uses setuptools to make your package compatible with pip.
Here's a sample workflow:
git clone <mypackage#github.com> # or git pull if you already cloned it before
cd mypackage
pip install -r requirements.txt
pip install -e . or pip install --user -e .
That last step will install your package to your site-packages folder, like any other pip-compatible package (assuming you've setup your setup.py file properly). If you are using virtual environments, you'll have to activate the virtual env first, then install your package there.
If you are not going to do any modification on the source code, and you just want to install the package on a specific machine, then you can also specify the Github URL to pip:
$ pip install -e git+https://git.repo/some_pkg.git#egg=SomeProject # from git
Lastly, if you are planning to upload this package to PyPi, check out the docs on Uploading the distribution archives. This just adds an extra step to your workflow of uploading your package to PyPi and then doing pip install from there next time.
I've been looking for a solution how to manage my project with git for quite some time now. I want to have one instance as the main repo for connecting the entire project. Each app should be its own git instance.
During my search I found both git submodule and git subtree. For both tools I found an instruction how to insert an existing reppo. However, I am interested in how to proceed from the beginning. I mean here from the command $ django-admin startproject myproject Where do I enter the git init? When I create a new app
$ django-admin startapp new-app and how do I use this as subtree/submodule?
Until now I have always found instructions that refer to a remote repo. Is this always necessary? I am not sure if I want to publish every Django app on Github. But I want a version control system just for me. Is this possible?
I have to say that so far it has been enough to manage my "projects" locally. Now I want to work together with others and I don't want to install the whole Django Project locally but only provide me with single functions or modules.
It would be a great help if you could explain to me how that works.
TL;DR
How to manage (start and expand) a Django Project with git. The apps should be their own git repos.
The purpose of submodules is to allow you to graft an existing repo/library into your git. Rarely do you want to do this. Instead you want to use PIP tools to install your libraries as part of library management.
This is essentially a git question. If you don't have a remote repository, you can still use git. With that said, the reason you want a remote repository is so that you can collaborate with others, and have a stored version of the code separate from your workstation.
There are services that let you have private repos even without a paid account. Bitbucket is the most well known of these services and is comparable to Github in most ways.
I have a Github wiki that I am maintaining and I want to include a section that displays whether or not the python dependancies within requirements.txt are outdated. I know how to do this manually using:
pip list --outdated
What is the easiest way that I can dynamically have that called whenever the page is accessed? or another method that gives the same result?
"whenever the page is accessed"
I am not aware of a webhook trigger by accessing the page: that would be costly whenever a lot of people want to read said page.
But since a GitHub wiki is a repo, it should be possible, as a pre-push commit to trigger a local command pip list --outdated and modify the wiki locally cloned repo in order to push that output to the wiki, before finally pushing your commit on your regular repo.
I've got Composer installed and I've used it to download HTMLPurifier locally. Now I'd like to push that download to my OpenShift Git repo. So, in a Git Bash window, I run the following...
git add -A :/
git commit -a -m "Uploading HTML Purifier"
git push origin master
At this point Git reports that the push was successful but when I ls the directory through SSH, it shows that the HTMLPurifier directory is empty. Why is that? How do I get Git to push those files?
Additional Info: I noticed that the HTMLPurifier directory is indeed a Git repo itself and contains a .gitignore file in its root directory. I tried deleting it and re-running the above commands but to no avail...
You should try to avoid pushing downloaded dependencies into a repository. It is recommended to add the vendor directory into the .gitignore file at top level. But what you must do instead is commit and push both composer.json and composer.lock.
Here's why: The vendor directory is managed by Composer. Running Composer will probably do minor things during an update, but may also be doing heavy stuff if the Composer team decides to optimize things.
Also, if you require a branch of a package, and Composer knows the repository of that package, it will default to cloning a Git repo or do a SVN checkout instead of trying to grab a ZIP package of that branch (often there is no way to get such a package for branches, and even tagged versions in a plain Git repository do not have such download ability. Composer knows that Github offers such downloads, and detects Github by looking at the repo URL.)
So you can assume that Composer will put a lot of repository meta data into the vendor file, and if you blindly commit these, things will get ugly. First of all, you are committing way too many files, increasing your repository by an unnecessary amount, which will slow down things. Then, if cloning Git repositories, these will be treated as submodules, and that has another bunch of nastiness I am told. If you are just learning Git, it probably isn't a good idea to start with these. And if you are sufficiently known to the tools (Git and Composer), you probably won't need them either.
There really is only one reason why you'd try to commit a modified version of the vendor directory: If your release process is completely depending on all files being present in your one repository, without any way to run a composer install during the release to make these files appear on the target server.
In such cases, you'd install or update the packages with Composer, and then go through all created directories and delete any .git and .svn (and probably also .hg for Mercurial) folders you encounter. Only then you'd be able to commit the files into your own repository.
But note that this step might be a tedious step to do manually - you probably want to create an update script that does all that work for you. You also might run into issues when updating dependencies because Composer expects files to simply go away when deleted, and not be in the way when being written. I cannot tell you exactly what you'd be experiencing because it depends on how you'd do stuff, but I expect you stumbling upon random puzzling issues.
Bottom line: Avoid committing the dependencies into your own repository if possible.
Try using the -force option, you will also most likely need to delete the .git directory inside the HTMLPurifier directory too.
Can I create an apt repository from my GitHub repository?
I found How To Setup A Debian Repository but it is too complex to understand.
I believe that it would be possible, but it not very practical. Git and GitHub are designed for storing source code, while an apt repository is setup to serve built packages. As far as git would be concerned, even a Debian source package would not be source code. But, since GitHub pages allow serving of largely arbitrary content via http, it should be possible to serve an apt repository in that way.
The page to which you referred doesn't seem at all complex to me, it's just a list of tools which can be used to manage an apt repository. The part listing Archive Generation Tools should be especially helpful. In the past I've used mini-dinstall to manage my own package archive, you'd probably want use either that or reprepro; the other tools are probably either trying to do too much (providing the capability to manage an entire distribution repository) or too little.