We are trying to migrate from Subversion to Mercurial but we are encountering some problems. First a bit of background:
Desired workflow:
We would like to have just two named branches, stable and default, within one repository.
Development takes place on default branch.
Bug fixes are committed to stable branch and merged to default.
After every Sprint we tag our default branch.
Eventually we can release a new version, for which we bring some code (possibly the latest Sprint tag) from default over to stable (update stable, merge Sprint_xyz), tag the branch (tag Release_xyz) and release.
We also want the following jobs on our Jenkins build server for CI:
End-of-Sprint job: This job should tag default with something like Sprint_xyz
Release job: This job should bring the latest "Sprint" tag changes over to the stable branch, then tag stable with something like Release_6.0.0 and build a release.
Some more background:
Mercurial is new to us, but for what we have seen, this seems like a sane approach. We chose tags to mark releases over named-branches and cloned-branches trying to make the development workflow as straightforward as possible (single merge step, single checkout, only a couple of branches to keep track of...).
We use scrum and potentially (but not necessarily) release a version after each sprint which may (or not) become part of the stable branch and turn into a "shipable" release.
The problem we are encountering (and which is making us wonder if we are approaching this the right way...) is the following:
We work on the default branch ('d' on the poor-man's-graph that follow):
d -o-o-o-o-
We finish a sprint and trigger an End-of-Sprint job (using Jenkins) which tags default with "Sprint 1":
d -o-o-o-o-o-
|
Sprint 1
To release Sprint 1 we update to stable branch ('s') and merge changes from the Sprint 1 tag revision and commit:
Sprint 1
|
d -o-o-o-o-o-
\
s -o-o-o-o-o-o-
Tag stable and commit:
Sprint 1
|
d -o-o-o-o-o-
\
s -o-o-o-o-o-o-o-
|
Release 1
Update to default and merge stable since default should stay a superset of stable, commit and push:
Sprint 1
|
d -o-o-o-o-o-o-o-o-o-
\ /
s -o-o-o-o-o-o-o-
|
Release 1
The problem is that when merging .hgtags from 's' to 'd' mercurial encounters a conflict which holds the release job from completing. The resulting .hgtags should contain information from both involved tags.
We have searched for a solution to this, and could probably automate these type of merge conflicts with some hooks and scripts, but it looks like an unnecessary and error-prone hack to support a workflow that otherwise seems nothing out of the ordinary.
Is there something inherently wrong with our approach that causes us to encounter these problems?
If not, what is the best way to solve these issues without having to rely on a scripts/hooks approach?
Is there a better approach that would support our workflow?
I would go for the special case hooks. The problem you're seeing is related to the Mercurial philosophy of versioning metadata in the same way as normal repository data. This is simple and effective, and leads to a system that's overall easier to understand. But in this case it also leads to your merge conflict.
The reason it leads to a merge conflict is relatively simple. The .hgtags file is just a text file with a bunch of lines in it. Each line contains a hash and the associated tag. In one branch you've added the Sprint 1 tag. In another branch you've added the Release 1 tag. These show up as one line being added to the end of the file in one branch, and a different line being added to the end of the file in another branch.
Then you merge the two branches. Suddenly Mercurial is faced with a decision. Which line should it take? Should it take both of them? If it were source code, there would really be no way to tell without human intervention.
But it isn't source code. It's a bunch of tags. The rule should be 'if the two lines being added refer to different tags, just take both of them'. But it isn't because Mercurial is treating it like a bog-standard text file that could be important source code.
Really, the .hgtags file should be handled in a fairly special way for merges. And it might actually be good to add code that handles it that way into mainline Mercurial to support your use-case.
IMHO Mercurial should be modified so that the .hgtags file would only give you a conflict warning if you have two different hashes for the same tag. The other weird case would be if you have a tag with a hash that isn't an ancestor of the change in which the tag appears. That case should be called out somehow when doing a merge, but it isn't really a conflict.
I suspect you're merging the tagged changeset from default to stable. If you merge the tagging changeset instead, you shouldn't get the merge conflict when you merge the second (probably also tagging!) changeset back to default.
Related
First off: I know that hg branches are immutable and they cannot be renamed. I am also aware of the existence of the mutable branches extension for hg. But I'd prefer a different approach, as I can never be sure that all of our developers have it installed and active, it's still "only" an extension.
My question: We have a repo with about 20 branches in it. Due to various reasons (inexperienced use, bad choices, experiments that became production environments) some of those branches were named badly and now our repo is a little confusing. What we'd like to do is rename a few of those branches, because obviously, the more we work with them, the more it's becoming a problem.
Do you have any suggestions? I already thought of a "tool" or some kind of script that recreates the whole repo from scratch, getting changesets of the old repo and committing them - with new branch names - to a new repo, "rebuilding" it. But before I go and waste time in writing something like that, I'd like to hear if there are other possibilities.
FYI: there are about 600 commits with frequent merges across the various branches.
You can rebuild the repository by doing a Mercurial to Mercurial conversion using hg convert. Enable the convert extension first and create a branchmap to do the mapping of branch names from old to new:
a-bad-name new-name
another-bad-name better-name
You can use that to map multiple bad names into a single good name, for example.
After the conversion, you will have a new repository with the same history, but with different branch names. The changeset hashes will thus be different and people will have to reclone (but I think you're aware of this already).
Perhaps I'm biased from years of svn but why would I ever want to merge with a different branch then make some changes before committing. The hg output itself suggests otherwise:
(branch merge, don't forget to commit)
Dear hg, then why didn't you just do it for me?
Because a merge operation is not guaranteed to produce correct output (automatic merges can use bad heuristics for how to deal with conflicts, manual merges can suffer from human error). Usually, what you will do is the following:
Merge.
Build.
Test.
If the test failed, fix the issue, go back to 2.
Commit.
Mercurial does not suggests otherwise, it 's just reminder: "All merge-related changes stored only on Working Dir, commit results into repository for permanent storing" and doesn't prohibit you to make some additional changes in code before commit.
But separation of tasks (merge and not-related to merge changes - i.e changes not initiated by merge-conflicts) into separate changesets is good idea anyway
Suppose we have a big TFS 2010 project with three branches: MAIN, TST and PRD.
The strategy is: whenever a Sprint finishes MAIN is copied/merged into TST. Whenever TST is deemed stable it is copied/merged into PRD. Whenever TST or PRD have fixes, they're merged back to MAIN, or MAIN and TST. (Don't ask me why, I can't control this and I don't particularly like it.)
At each promotion step, as I understand, one can either:
delete the target branch and branch again - this entails losing immediate access to that branch's history (it can always be recovered, right?);
merge and resolve with acceptTheirs - this entails loosing changes that may not have been merged back from target to origin.
For the merge-backs, it is important to have ancestry information. With 1. I would expect ancestry information to be kept. With 2. I am unsure.
So, two questions:
Are those two the possible/desirable ways to go about promoting software between branches?
I which cases is ancestry information not kept?
Extra points for any additional tradeoffs that might be relevant for big-size repositories.
1.Are those two the possible/desirable ways to go about promoting software between branches?
If MAIN has a child branch TST, which has a child branch PRD then without resorting to baseless merges, these are the only merges possible to promote changes between branches.
If this is a desirable branching strategy, depends on many factors like how many parallel releases are put out and team sizes. A good reference guide on this is the branching guidance of the TFS Rangers http://vsarbranchingguide.codeplex.com/ The version you are seem to be using is a variation of the basic dual branch plan (what you call main, they call dev and your production branches aren't unique labeled). This branching strategy works best if only one version is in production and releases should always contain everything made.
2.In which cases is ancestry information not kept?
If files are copied or branches are destroyed. However if you need to delete and/or recreate branches all the time and/or need to use acceptTheirs continuously, than often its an indication of; inadequate branching strategy, inadequate TFS training, or issues with the testing and patching strategy (bugs found in production and development are found and fixed at the same time, resulting in merge conflicts).
In CVS I have a branch (b) off another branch (a) which is off the trunk/head.
Some bug fixes were made in branch (a) that I'd like to go ahead and use in branch (b). How can I pull those fixes into my branch in Eclipse?
head
|
v
a (with bug fixes)
|
v
b (needs bug fixes)
Ideally what you need is to have two tags on a for every feature you want to merge, and then merge the difference between those two tags into b. However, you would also need to remember which ones you have already merged, because CVS doesn't remember that.
When I was working in a company that used CVS and branches, our policy was that bugfixes from branches (a in this case) that ought to be used by other branches need to get merged into the trunk first, and all the other branches merge them from there.
However, it was still very painful if you wanted to cherry-pick individual bugfixes. Essentially, you'd have to remember every fix you've merged (by two tags, marking the beginning and the end of the changes making up that fix).
Generally, in CVS it's much better to remember (in a tag) up to which revision you have merged, and merge everything from there to the head (and then move the tag to the head). In CVS, cherry-picking is painful and requires you to store the merge history somewhere.
When SVN with merge tracking works, it's really nice, I love it. But it keeps getting twisted up. We are using TortoiseSVN. We continuously get the following message:
Error: Reintegrate can only be used if revisions 1234 through 2345 were previously merged from /Trunk to the reintegrate source, but this is not the case
For reference, this is the method we are using:
Create a Branch
Develop in the branch
Occasionally Merge a range of revisions from the Trunk to the Branch
When branch is stable, Reintegrate a branch from the branch to the trunk
Delete the branch
I Merge a range of revisions from the trunk to the branch (leaving the range blank, so it should be all revisions) just prior to the reintegrate operation, so the branch should be properly synced with the trunk.
Right now, the Trunk has multiple SVN merge tracking properties associated with it. Should it? Or should a Reintegrate not add any merge tracking info?
Is there something wrong with our process? This is making SVN unusable - 1 out of every 3 reintegrates forces me to dive in and hack at the merge tracking info.
This problem sometimes happens when a parial merge has been done from trunk to branch in the past. A partial merge is when you perform a merge on the whole tree but only commit part of it. This will give you files in your tree that have mergeinfo data that is out of sync with the rest of the tree.
The --reintegrate error message above should list the files that svn is having a problem with (at least it does in svn 1.6).
You can either:
Merge the problem files manually from trunk to branch, using the range from the error message. Note: you must subtract 1 from the start of the range, so the command you'd run would be:
cd <directory of problem file in branch working copy>
svn merge -r1233:2345 <url of file in trunk>
svn commit
or
If you're certain that the contents of the files in your branch are correct and you just want to mark the files as merged, you could use the --record-only flag to svn merge:
cd <directory of problem file in branch working copy>
svn merge --record-only -r1233:2345 <url of file in trunk>
svn commit
(I think you can use --record-only on the entire tree, but I haven't tried it and you'd have to be absolutely sure that there are no real merges that need to come from trunk)
Bunny hopping might be the solution.
Basically, instead of continuously merging trunk changes into a single branch (branches/foo, let's call it), when you want to pull those changes from trunk:
Copy trunk to a new branch (branches/foo2).
Merge in the changes from the old branch (merge branches/foo into branches/foo2).
Delete the old branch (delete branches/foo).
Your problem is that you're trying to use Reintegrate merge on a branch that has been 'corrupted' by having a 'half merge' already done on it. My advice is to ignore reintegrate and stick to plain on revision merging if this is your workflow.
However, the big reason you get errors is because SVN is performing some checks for you. In this case, if the merge has extra mergeinfo from individual files in there, then svn will throw a wobbly and prevent you from merging - mainly because this case can product errors that you might not notice. This is called a subtree merge in svn reintegrate terminology (read the Reintegrate to the Rescue section, particularly the controversial reintegrate check at the end).
You can stop recording mergeinfo when you perform your intermediary merges, or just leave the branch alone until its ready - then the merge will pick up changes made to trunk. I think you can also byass this check by only ever merging the entire trunk to branch, not individual files thus keeping mergeinfo safe for the final reintegrate at the end.
EDIT:
#randomusername: I think (never looked too closely) at moving is that it falls into the 'partial merge' trap. One cool feature of SVN is that you can do a sparse checkout - only get a partial copy of a tree. When you merge a partial tree in, SVN cannot say that the entire thing was merged as it obviously wasn't, so it records the mergeinfo slightly differently. This doesn't help with reintegrate as the reintegrate has to merge everything back to the trunk, and now it finds that some bits were modified without being merged, so it complains. A move appears kind of the same thing - a piece of the branched tree now appears differently in the mergeinfo than it expects. I would not bother with reintegrate, and stick with the normal revision range merge. Its a nice idea, but it trying to be too many things to too many users in too many different circumstances.
The full story for mergeinfo is here.
I suspect you're not following the merge instructions correctly:
"Now, use svn merge with the --reintegrate option to replicate your branch changes back into the trunk. You'll need a working copy of /trunk. You can get one by doing an svn checkout, dredging up an old trunk working copy from somewhere on your disk, or using svn switch (see the section called “Traversing Branches”). Your trunk working copy cannot have any local edits or contain a mixture of revisions (see the section called “Mixed-revision working copies”). While these are typically best practices for merging anyway, they are required when using the --reintegrate option.
Once you have a clean working copy of the trunk, you're ready to merge your branch back into it:"
I have few problems with merging.