We have several product lines built around a common core and currently maintain them in SVN using externals. Moving to mercurial, it is natural to move to use hg sub-repositories.
The thing is the core is quite large (probably >GB, judging by the SVN repo), and a typical developer sometimes wishes to work simultaneously on several products, say 3-4.
Did I get it correctly that it usually means a developer would have the core replicated 3-4 times for each developer, with its entire history?
Also, if a developer wishes to perform some simple operation in another product, it would mean the core have to be pulled first, even though it is already available at the client (several time...)?
In order to truly share the subrepository (and not its working copy), you can use the share extension. However, that makes the cloning process a bit counter-intuitive:
hg clone -U remote_core core
hg clone -U remote_projectA projectA
cd projectA
hg share ../core core
hg update
cd ..
hg clone -U remote_projectB projectB
cd projectB
hg share ../core core
hg update
And so on. But I warn you that you are going to have more than one headache with this setup. At work, we have a similar setup, but instead the shared subrepository has a branch (not a named branch, but a clone branch, a dedicated master repository) for each project that uses it. That way projects can modify the shared code independently while still having the easy merging between them.
Related
I use svn2git but it fails always on the same line:
My command:
svn2git https://my-svn/proj/ --revision 1000:1001 --username xxx
After one houre this command stoped running and it fails with:
Broken pipe at /usr/lib64/perl5/vendor_perl/SVN/Ra.pm line
623.
What would cause this issue?
For a one-time migration git-svn is not the right tool for conversions of repositories or repository parts. It is a great tool if you want to use Git as frontend for an existing SVN server, but for one-time conversions you should not use git-svn, but svn2git which is much more suited for this use-case.
There are plenty tools called svn2git, the probably best one is the KDE one from https://github.com/svn-all-fast-export/svn2git. I strongly recommend using that svn2git tool. It is the best I know available out there and it is very flexible in what you can do with its rules files.
The svn2git tool you use is based on git-svn and thus suffers from most of the same drawbacks.
You will be easily able to configure svn2gits rule file to produce the result you want from your current SVN layout and you can also tell it to keep empty directories by giving it a commandline option that causes it to put empty .gitignore files in the directories to keep them.
If you are not 100% about the history of your repository, svneverever from http://blog.hartwork.org/?p=763 is a great tool to investigate the history of an SVN repository when migrating it to Git.
Even though git-svn (or the svn2git you used) is easier to start with, here are some further reasons why using the KDE svn2git instead of git-svn is superior, besides its flexibility:
the history is rebuilt much better and cleaner by svn2git (if the correct one is used), this is especially the case for more complex histories with branches and merges and so on
the tags are real tags and not branches in Git
with git-svn the tags contain an extra empty commit which also makes them not part of the branches, so a normal fetch will not get them until you give --tags to the command as by default only tags pointing to fetched branches are fetched also. With the proper svn2git tags are where they belong
if you changed layout in SVN you can easily configure this with svn2git, with git-svn you will loose history eventually
with svn2git you can also split one SVN repository into multiple Git repositories easily
or combine multiple SVN repositories in the same SVN root into one Git repository easily
the conversion is a gazillion times faster with the correct svn2git than with git-svn
You see, there are many reasons why git-svn is worse and the KDE svn2git is superior. :-)
From what I can tell, the "broken pipe" error is due to a problem receiving a file. It might be really big or there may be something else odd about it.
If you run into the error, fortunately in most cases you can simply re-run the git svn clone command or other related command and usually it can seamlessly resume from where it left off.
If you are using git-svn to do a one time migration of an SVN repo to Git, just be sure to follow Atlassian's guide. They have a tool for properly converting the SVN tags to Git tags, but it is a destructive process and you won't be able to use the git repo to sync with SVN anymore, so only do this when you're finally ready to abandon the SVN repo.
I'm using Bazaar quite some time now, but at the moment I'm searching a solution to the following problem:
Assuming you've got several developers with everyone developing in its own branch, like this:
Project
|
|----Branch 1
|
|----Branch 2
|
...
Now, we've got a project manager who wants to have an overview over all branches.
Is there any possibility (using only bzr functions) that he can manage those branches at once?
With "manage", I mean update, commit and perhaps even checkout (last one could perhaps be done with multi-pull but I think this would overwrite existing local data)
Greetings Florian
P.S. I know that this use-case could easily be achieved with SVN (by simply using subdirectories - but without the features of a dvcs) or more or less easily with shell-scripts (something like bzr list-branches|xargs bzr update), but I'd prefer a built-in bzr function
You can see all the branches in a directory tree with:
bzr branches -R /path/to/base/dir
However this works only on the local filesystem. If you need to find branches in a remote system, you need to run the command through ssh or something.
Once you have the list of branches, the manager should branch from them into his local shared repository, preferably configured with the --no-trees option for space efficiency. Existing branches should be pulled instead (using multi-pull for example), removed branches should be removed.
Once he has the branches, the easiest way to overview is using Bazaar Explorer. Open the shared repository location. I especially like the Log button, which will show the tree of logs.
When you say commit... The manager should not commit to the developer branches. If fixes are needed it's better to ask the developer to fix it, otherwise the manager will always have to clean up their mess for them. The manager should only merge other branches to the trunk/main/master. In other words use the gatekeeper workflow.
You can try the bzr-externals plugin or the bzr-scmproj plugin.
I want to store a project in Mercurial that contains external code (which can be modified by me) coming from Git and SVN repositories. In SVN I would solve this with vendor branches and copy the code around, but I understood that in Mercurial it's better to have different repositories for different projects, and pull between them when needed.
The project layout will be like this:
- externalLibraryA [comes from a SVN repo]
- ...with some extra files from me
- externalLibraryB [comes from a SVN repo]
- ...with some extra files from me
- externalPluginForExternalLibraryB [comes from a Git repo]
In Subversion I would create vendor dir and a trunk dir, copy all external libraries first in vendor, and then in the right place in trunk. (I think) I can do this in Mercurial too, with subrepositories, but is this the best way to do this?
I tried setting up different repositories for the external libraries, but then it seems I can't pull the externalLibraryARepo into the externalLibraryA directory of my main repository? It goes in the main directory, which is not what I want. I can also create a Mercurial mirror repository and include it as a subrepo in my main repository, but then the changes in this subdirectory go to the mirror repository, while I want them to stay in the main repository.
I'd probably just store this in one repository - note that in the link you give they are using their build system in the end to bring together the binary output from the different repos. I'm not clear on their rationale there.
If the underlying problem you're trying to solve is how to update the externals in a clean way, I'd probably use anonymous branching for that.
I.e. add the external lib to your project, and your modifications. Make sure it works. Tag with ExternalA-v1.0. Hack away on your actual project. Now ExternalA, Inc. has a new version of their stuff. Update your repo to ExternalA-v1.0 tag. Import their new version and apply your modifications on top. Commit. Now you have two heads: one with the latest version of your code (that works with ExternalA-v1.0) and one with the latest version of ExternalA (that does not work with your code, maybe). So then you merge and reconcile the two. Tag again, now with ExternalA-v2.0. Repeat as needed.
You can still keep your externals in separate repositories, but I assume that the project that is using those does not need to be up to date with changes there all the time - looks like the whole point of vendor branches is to have some point of isolation between dependee and dependants. Of course, moving the changes from the externalA project to the project that is using that will then be a manual affair (well, a copy, much like in SVN really).
It depends on whether your vendor code is going to be customized by your team or not. Our teams have had a great deal of success maintaining a named "vendor" branch on repositories with our own customizations on branches named by project name. This vendor code is then easily included in a project as a subrepository.
A caveat to this approach: if active development is going on in the subrepository, best keep it to directly editing the subrepository as a separate clone, otherwise it becomes necessary to pay close attention to the top-level repository so you don't inadvertantly bump your .hgsubstate forward to the wrong revision and break your build.
Watch out for merges of the top-level repository (your project) between versions which point to different named branches of your subrepository, as this can result in a merge between the "vendor" and "project" branches in the subrepository as it recurses, which may not be desirable.
Note that this functionality may change in the future as well, as some "warm" discussions have been taking place in recent months on the mercurial-devel mailing lists about the future of subrepository recursion.
edit:
I just saw this discussion in the related links as well, which seems relevant: https://stackoverflow.com/a/3998791/1186771
The question
An open source program uses CVS for version control. I would like to make a number of bug-fixes and submit patch bombs to the developers with commit access. I would also like to maintain my own semi-private fork that mainly tracks the main code-base but that includes my own features (these features, right now, should not be incorporated into the main code-base.)
I prefer to use mercurial for my own version control needs, but I am open to other version control systems if necessary.
I'd like to:
Be able to easily create patch-bombs against the current CVS source with my own bug-fixes
Keep track of history on my own features
Have fixes and improvements from the main tree easily incorporated in my new-feature fork
Easily apply my own bug-fixes to my new-feature fork
Be able to work and track change history without an Internet connection.
What suggestions do you have for doing this?
My current idea
My own best guess is below, to give you a better idea of what I am thinking about.
I will have 3 mercurial repositories.
The first two repos are managed as specified at (https://wiki.mozilla.org/Using_Mercurial_locally_with_CVS). One just mirrors the latest changes from the CVS upstream. I do "cvs update" then "hg commit" in this repo. The second repo holds my bug-fixes as patches using the mq extension and I pull from the the first repo and re-base my patches every so often. When my patches are incorporated into the main tree, I remove the patches from the patch queue/make them permanent commits.
The third repo is my local fork. It will start out as a clone of the first repo. Then each time I do an update of the first repo, I'll pull from it into repo 3. My own features will be directly present as commits in this repo. When I fix a bug, I'll export a patch from repo 2 and apply it to the appropriate pull from repo 1.
I have used Git to manage changes on top of a CVS repository in a similar way. My solution in Git uses local branches instead of multiple repositories, but it sounds essentially similar to your proposed idea.
I found that this arrangement works best if you commit all the CVS metadata (in the CVS/) subdirectories) to your mirrored repository. This means that the CVS metadata gets replicated in the other repositories, but it doesn't cause any harm (and lets you run commands like cvs diff if you need to).
I want to use Bazaar on Windows XP for web-development and related tasks. Most of the files are edited locally and then transferred via FTP to the server. Just now the repository sits on my local workstation. Later on it should be shared locally with some co-workers. Perhaps we will use a local Linux server as a centralized repository, but this structure is not decided for now. But first I need to understand the impacts of the different repository setups, which I do not at all.
Using Bazaar-Explorer on Windows XP I’ve created a ‘shared tree repository’ from the option list of the init-dialogue in some location dev-filter/. Bazaar Explorer tells me:
Created repository with treeless branches at F:/bzr.local/dev-filter
Created branch at F:/bzr.local/dev-filter/trunk
Created working tree at F:/bzr.local/dev-filter/work
OK so far. Now I move a bunch of files into the work directory and add and commit them as Rev 1 ‘Start Revision’. Then I work on some of these files and commit them again as Rev 2. Here my confusion starts. Shouldn’t both revisions go into the trunk? The trunk is still empty, beside the .bzr directory which only holds some management information. If I delete my working directory, which I have tried during these first experiments, everything is gone. There’s obviously no hidden storage of those files.
OK. Perhaps I need to push it into the trunk? This does not work either. Entering the work/ directory and initializing the ‘push’ to the trunk, Bazaar-Explorer tells me
No new revisions to push.
So what? This looks like a severe conceptual misunderstanding about what should happen on my side.
Edit, 2010-02-03: Some conclusions
What I learned meanwhile is this:
I think I should switch to the command line until I really understand what’s going on, at least for creating the repositories and branches. Bazaar Explorer introduces a new level of abstraction which I only can handle if I understand the level beneath
One of the secrets of working with Bazaar at least for me is to understand those .bzr directories, their particular properties and states when created with ‘bzr init’, ‘bzr init-repository’, ‘bzr branch’ etc. in all their variants and how they are plumped together.
While there’s a whole chapter of ‘Organizing your workspace’ in the Bazaar User Guide, it’s more or less workflow oriented. The manual contains a lot of directory structures for the given examples. What I would prefer beside this and have not (or only rudimentary) found so far is some graphical representation of those ‘Lego like’ .bzr building blocks which create the linking of all the parts. So I started to invent some simple notation while working through the examples and looking into the .bzr directories to document what information is stored there, where does it come from, how and to what is it linked, is it complete or shared, etc.
Erich Schreiber
Created repository with treeless
branches at F:/bzr.local/dev-filter
This part of the output looks suspicious to me. Are you sure you chose 'Shared repository' and not 'Shared repository with treeless branches' from the init dialog?
Treeless Branches are branches without the working tree, if you indeed created a treeless branch for trunk then it makes sense that there are no files there.
Your changes are still saved in the F:/bzr.local/dev-filter/trunk/.bzr, and have indeed been committed there. You don't see those changes reflected in the file system because Bazaar has created trunk as a treeless branch, with `` as a lightweight checkout. See checkouts in the Bazaar User Reference.
If you open F:/bzr.local/dev-filter/trunk in Bazaar Explorer, you should see your revisions. If you create a new branch with a working tree or checkout based on trunk, Bazaar will create the files with your changes for you.
typically it goes like this.
bzr init-repo --no-trees F:/bzr.local/dev-filter
cd F:/bzr.local/dev-filter
bzr init trunk
bzr branch trunk work
---all above will not create any tree
Now in new directory say F:\temp
cd F:\temp
bzr checkout F:/bzr.local/dev-filter/work
bzr add
bzr commit
---back to F:/bzr.local/dev-filter/work
cd F:/bzr.local/dev-filter/work
bzr push F:/bzr.local/dev-filter/trunk