In the project that I'm working on, we are using SVN with 'Stable Trunk' strategy. What that means is that for each bug that is found, QA opens a bug ticket and assigns it to a developer. Then, a developer fixes that bug and checks it in a branch (off trunk, let's call this the bug branch) and that branch will only contain fixes for that particular bug ticket
When we decided to do a release, for each bug fixes that we want to release to the customer, a developer will merge all the fixes from several bug branch to trunk and proceed with the normal QA cycle.
The problem is that we use trunk as the codebase for our CI job (Hudson, specifically), and therefore, for all commits to the bug branch, it will miss the daily build until it gets merged to trunk when we decided to release the new version of the software. Obviously, that defeats the purpose of having CI.
What is the proper way to fix this issue?
As you note, one purpose of using a branch is to segregate specific code fluctuations from ticket-fixing and feature development away from the trunk. But once the feature or ticket is complete, you should merge it back. In Subversion, branches are better used to track sets of related features (like those for a release), not individual features. Otherwise you will quickly wind up with unmanageable numbers of branches.
Furthermore, why delay the integration at all? The longer you wait between releases, the higher the likelihood that your isolated change will conflict with another change made since then and/or produce further instability in your system once merged back.
My preferred strategy is to do something like this:
[begin work on 0.4 branch]
|
|
v
(*)---(*)-------(a)--(b)---(c)-- <-- Trunk is "unstable".
\ | | Contains all commits.
ver \ [merge from trunk] Developers commit to trunk.
<-- 0.3 \ v v
+---(a)--------(c)-- <-- Branch is "stable".
Contains selected commits from trunk.
Know beforehand what's going onto branch.
Now, once you're ready for release:
[trunk]
(*)---(*)---(*)----------------------------[development continues]--->
[0.4 branch] No further development on branch unless
(*)---(*)---(*)---[0.4-release] spot fixes are needed. Then re-tag (0.4.1)
^ and re-release.
|
|
[make tag on branch; release from stable branch,
not unstable trunk]
I know you asked about the best way to coerce your continuous integration system to do this. But I would respectfully suggest that given that Hudson is recognized as a relatively capable CI system, the fact that you're having a lot of trouble shoehorning your development model into it is a possible sign that it's not a process that lends itself well to CI in the first place.
Our typical practice is to have two base builds per project: one against trunk and one against the current release branch. This way you know that:
Whatever is being updated is being integrated correctly (trunk)
Whatever your release target is, if you stopped working now you would still have a correct and working (just not fully featured) build.
Here is what we do (inspired from Version Control for Multiple Agile Teams by Henrik Kniberg):
development is done in a development branch and features are pushed to trunk when "done done"
trunk is the "done" branch
at release time, we tag the trunk
when a defect comes up, we create a release branch from the tag
defects is patched in the release branch
patch is merged from release branch to trunk immediately after releasing (to include it in future releases)
(source: infoq.com)
CI runs on all branches (development branches, trunk, release branches).
That sounds painful and overly complicated (from a branch/merge standpoint).
I would branch at release and have developers check in to the trunk. Anything that needs to go out as a hot fix could be merged with the release branch.
Do a nightly merge to an "unstable-evil-twin-trunk" that merges all bug branches to the evil-twin-trunk.
Or set up nightly builds on each bug-branch (which sounds like a lot of nightly builds).
In my opinion, this sounds like an awful lot of branching for a centralized style source control solution. Maybe you need a distributed version control system and a build server on each workstation, which it seems would accomplish the same thing (isolated check ins for each developer and daily builds on what the developers check in)
Instead of creating branches for your bug fixes, why don't you try creating branches for the version before the bug fix, and then apply the fix to the trunk.
This way, if you want to give your customers the bug fix, you give them the trunk version.
If you do not want to give them the bug fix, you can give them the branch version before your fix was applied.
This way, you can have Hudson build the trunk line, and your nightly builds will include all your bug fixes.
I answer this a lot, and here's how IBM recommends it with ClearCase (UCM), and I do it in the real world:
- Project
|- development mainline
|- TAG: version-1.0
|- version-1.0-bugfix#123
|- version-1.0-bugfixes
|- TAG: version-1.0-sp1
|- version-1.0-sp1-bugfix#234
|- version-1.0.sp1-bugfixes
|- TAG: version-1.0-sp2
Anything not prefixed by TAG is a branch.
Related
Here is what i want to know.
We will develop a software and releases a major version v1.0 of it.
After moving that version into production if we find any bugs we will fix those and deliver a patch version p1.0.1 to be applied over the existing major versionv1.0 which consists only modified classes for the bug fix. And it continues like whenever any bug found or an enhancement has to be applied, we will release a new patch version to be applied over the existing major version and existing patches.
So, How can we maintain only modified source(java) files for particular patch release? Like we need to have complete source code and separately we have to maintain patch source on top of that. We have to sync this code in SVN as multiple people involve in bug fixing, and patching.
Trunk, Branches, Tags will it full-fill our needs?
Please suggest.
There are several standard ways of doing development. I like what's called the unstable trunk method.
In unstable trunk, you do all of your development on trunk. Everyone works together. We have over 100 developers and don't have issues with it. It forces developers to work together, make small changes, and constantly commit these changes.
It also works well with a continuous integration builder like Jenkins.
When you get to some magic point, you create a release branch. What is this magic point? It depends. The idea is that as you get close to a release, you will have some developers working on the up coming release and some developers working on features that will go into the very next release. Two groups working in parallel means branching.
In some groups, this is when you get to the point of a release candidate or when you are feature complete on a release, and now you're just fixing bugs. At that point, you create a branch for your release. Our standard is to call that branch after your release.
Example: Creating a release
You have a team of 10 developers. You are working on Release 1.2. All work is currently on trunk. Now that the release is almost complete, you assign two of those developers to work with QA and UAT to fix up the code and work on the release. All other developers are continue their work.
You create a branch called 1.2 using svn cp to copy trunk to branches/1.2. Now, those two developers working with QA and UAT work on Branch 1.2 and the rest continue working on trunk.
Once you are ready to release the code, you can use svn cp to copy branches/1.2 to tags/1.2.
You now have some bugs that have been fixed on your 1.2 release that may apply to that 1.3 release you're working on in trunk. In that case, you can use svn merge -c $REV where $REV is the revision in Subversion where you fixed the pertinent bug. Notice there's no reintegration. You simply apply the patches in branches/1.2 to trunk.
Example: Patch Release
You've released Release 1.2, and a critical bug was discovered. The customers can't wait for Release 1.3, so you'll fix the bug now and create a Release 1.2.1.
In this case, since you already have branches/1.2, you simply patch that branch, and then when you are ready to do your release, you copy the 1.2 branch to tags/1.2.1.
Once again, you merge individual changes from branches/1.2 to trunk via svn merge -c $REV.
NOTE: There's no need to create a 1.2.1 branch. However, there's not anything preventing you to do that if you so prefer. You could have copies tags/1.2 to branches/1.2.1 and do your work there. Then, you could copy branches/1.2.1 to tags/1.2.1 when you do a release.
The only possible issue is that your branching is from trunk->branches/1.2->tags/1.2->branches/1.2.1. Merging /branches/1.2.1 back into trunk use to cause problems in older versions of Subversion. This shouldn't realy be an issue in Subversion 1.6 or greater.
One more piece of advice: Do not reuse release names or tags. You can talk about release 1.2, but each patch should be tagged 1.2.1, 1.2.2, 1.2.3, etc. Or, 1.2p1, 1.2p2. The point is that you should be able to point to each and every release you've delivered to production. If you want to know what changed in patch #3, you can do a diff between 1.2.2 and 1.2.3.
there's an "issue" (actually a nuisance, but a major one) with our development process.
We've setup a good continuous integration-deployment-delivery pipeline; basically, our process is like that:
New development happens on the default branch (we use Mercurial, but git would have the same issue); usually we develop just one small improvement there;
Once the improvement is ready and tested (something that usually happens twice or three times a week, our cycle is pretty short), it's merged into the release branch (which actually stands for release candidate), it's tested again in a production-like environment and if it passes all our testing it's pushed to the production environment.
If we need to make a bugfix, we do it on the release branch, we test it and then push it to the production env.
An artifact (usually an RPM package) is built from each branch; when there's a release in production, the very artifact from the release branch is promoted to our public repository. There's no branch for the production environment, because we usually don't need it (there's a very short timespan when something it's on the release branch but it's not in production; code doesn't linger there unattended).
In this context, there's a small issue for us.
In order to distinguish packages, we usually set version to 1 for packages on the release branch, and 2 to packages on the default branch. Our CI system then adds its build number, so we've got packages like oursoftware-1.0-build37 for release and oursoftware-1.0-build902 for default. Such version is written either in our .spec file or in our pom.xml files (or other files for different softwares, like .gemspec or simila)
Whenever we want to put something in release candidate, we merge the default branch to the release branch.
What happens there? the version gets merged as well. So after each merge we need to get back at the file containing the version and put the old version in them. We should need to do it just once (Mercurial does not try to re-merge things that were already merged) but whenever we need to do a merge from release from default (e.g. in order to merge a fix done on release to the dev branch) the problem rises again.
I'd like some advice on how to prevent this problem completely; how do I set versions outside of my branches? I sometimes actually need to change them (e.g. I want to change from 1 to 2 on release and from 2 to 3 on default) and I wouldn't like to hardcode them in our CI system, I'd like some system which is "outside" the scope of branches and it's kind of "global" for a repository. Or I would need something like "ignore modification for some files".
In many situations I actually need versions in my files as well; for example I cannot omit version info from a Maven pom.xml file or from a .gemspec, otherwise it won't work when developing.
Thanks,
Well, I see 3 4 a lot of possible solution, more or less "native" (random order without priorities and own personal preferences)
Instead of "Monolitic $DEVEL branch" you can use "Branch Per task": in this case file with (hardcoded) version-info will not appear as mergeable file (TBT!)
Branch-specific version change can be moved from (permanent) changeset of branch into patch of MQ (as results - versions /and files/ are identical across branches, permanent default changes have to appear as result of applying patch)
Files with version-info doesn't contain hardcoded versions, they are more "templates", than "data", and real data appear as result of some preprocessing inside build process (you know transformation-rules and can code it)
You can change nothing in developent|deploy process and if version-info files are unique inside repository, you can use trick, similar to merging hgtags (special mergetool for only some files in repo - in your case in must be internal:local special mergetool)
Maybe using just merge-patterns will be OK for you, check Mercurial - Exceptions to internal:local merge-patterns? and Default to trunk version for Mercurial merges? for details and some additional notes+discoveries
I try to understand the right approach for managing hotfixes with TFS /source-control.
Say I have a small project to contains only two file (app.js and util.js). Both files are under source control (Team Foundation Server).
Assuming I applied two weeks ago a label that indicates the "milestone-1" that was deployed to the server. Since then I applied several other modifications (extended app.js and added another files).
Today I found a bug so I get the version with the label "milestone-1" from the server, modify the util.js and deploy this version to the server. The goal is that on the server only the version "milestone-1" including the hotfix-1 is deployed but not the modification I applied the last two weeks.
The question is: How /where do I check-in the hotfix to the TFS? Because I might find another bug tomorrow so I have to get the version "milestone-1" and the hotfix-1 base base for appling the second hotfix.
Is there a way to check-in a version of my code into the TFS that "sits" between label "milestone-1" and my latest version?
With branching and merging!
We have the following development cycle:
The initial project is created, permissions assigned, and imported in to TFS under branch name of "Main".
Because this is the initial iteration of development, we branch "Dev" from main, and that's where the developers work. Note at this point there is no official release.
When development is complete, it is merged back in to Main. No Dev work happens on the Main branch.
We branch "Release" from main, and that is the point at which the code is actually deployed, and version numbers are updated.
Ok, so, further down the line, we have to make some amendments. Once again, "Main" is branched out again and the development work continues down the Dev Branch. All is well. Then a user reports a bug that needs fixing, and it needs fixing fast. But we're half-way through our other development!! What to do?
At this point, we branch from main again. We'll call it "HotFix". The bug is fixed, and then HotFix is merged back in to main. We can then merge it in to our release branch, and the update is sent out. Our current dev work is not interrupted, and none of the work we have been doing up to this point exists on Main).
We finally finish our dev work, and we merge that back in to the main branch, and then in to release. Because our Hotfix branch got merged in to main, our new development work also contains the hotfix we did on the fly earlier.
You should use branching. Check out the "Rangers'" guidance.
I have successfully used the Advanced Branch Plan in the past, but you can customize it to your needs. You need to be disciplined with branching, merging, checking in, etc. Also consider shelvesets for code changes that aren't ready to be committed even to the DEV branch.
http://vsarbranchingguide.codeplex.com/
Let me know if you want more information.
I am wondering what the best way might be to use Mercurial to create a release revision that includes artifacts that provide evidence of process execution. For example, we'd like to attach system test results, check lists, release notes, etc. so that if we are audited by a customer we can easily show that we performed our process. This is important to us due to the safety aspects of our product.
Our release management process will go something like the drawing below. All developers are developing in local repos and periodically pushing to main. Main is the latest and greatest but not necessarily safe for even internal engineering purposes outside of the dev team.
When we want to create a release for the customer or for other internal engineering departments, we start with a release candidate branch (e.g. RC1). If any fixes are required, we commit to the RC branch. Testing occurs on this branch. When the RC is determined to be good, the changes are merged back to main.
We think that what we would like to do is merge into a Releases branch. However, we have a chicken and an egg problem related to the artifacts: The artifacts that we would like include in the release revision contain, among other things, the hash for the revision. This provides clear traceability that the testing and other process steps were performed on this precise revision. But, to add these items, I need to create a new revision and I obviously can't know what the hash for that revision will be before I create it. I am wondering if there is some way to "amend" a revision without changing the hash?
The only way to do this, that I can think of, would be, for example, in the drawing below, to create a revision RC2.3 that contains the necessary process artifacts but actually merge RC2.2 to Releases.
Then, of course, I have another problem and that is that the merge of RC2.2 into Releases will generate a new hash. So, my artifacts are out of date again. So the next question is whether there is some way to have the Release branch "point to" RC2.2.
BTW, we're open to changing this process if necessary. Our reasons for using this methodology are:
CI system is monitoring main and kicks off a series of builds and performs automated unit testing on each push. There are frequent changes to main and we don't want people using it.
Development can continue on main with no impact to the release.
Any revision on the Releases branch kicks off a different set of tasks on our CI platform, including the creation of a distribution of a field flash utility and the required images (we're developing firmware). This is how we provide releases to outside entities.
Main A--B--C--D--E--F--G--H--I--J--K--L---------M-------------N
\ / \ /
RC1 RC1.0--RC1.1 \ /
\ \ /
RC2 \ RC2.0--RC2.1--RC2.2
\ \
\ \
Releases ER1.0-----------------------------------PR1.0
To answer your question:
I am wondering if there is some way to "amend" a revision without changing the hash?
No, that's impossible by design. The changeset hash is a cryptographic checksum derived from the content of the commit — this is why the changeset hash uniquely identifies the content of all the files in the changeset.
The Mercurial wiki has a page with a standard branch setup. I'll describe how it works with pictures below.
What people normally do in your situation is to create a branch for release candidates like you suggest. If you're about to release version 2.0 of your firmware, then I would call the branch 2.x. The commits on the branch are release candidates for your version 2.0:
Main A--B--C--D--E--F--G--H--I--J--K--L---------M-------------N
\ /
2.x 2.0-RC1--2.0-RC2
Bugfixes are committed on the branch as needed and merged back into Main regularly. You should always be able to merge the bugfixes back since they are by their nature bugfixes and stability improvements that you want to have on Main too. In other words, commit I above has the bugfixes in 2.0-RC2 plus the new features in D to H.
You can work on the release candidate branch as much as you want and keep fixing bugs:
Main A--B--C--D--E--F--G--H--I--J--K--L-------M-------------N
\ / /
2.x 2.0-RC1--2.0RC2 -- ... --2.0-RC8
All bugfixes are merged back since the rule is that you can only do fixes on the release candidate branch that are safe. Here you had 8 release candidates on the release branch.
When you like the status of the release candidate, you can tag it and run the more comprehensive tests on the revision you tagged. The output of a succesfull test is committed on the branch and you then tag that commit as the final released version:
Main A--B--C--D--E--F--G--H--I--J--K--L-------M-----------N
\ / / /
2.x 2.0-RC1--2.0RC2 -- ... --2.0-RC8 -- T -- 2.0
Here T has the test output for 2.0-RC8 included. Commit T is the one you tag as version 2.0 of your software and the one you package and ship to your customers.
Should you need to create a version 2.1 with a bugfix to 2.0 (but no new features), then you continue with the same scheme:
Main A--B--C--D--E--F--G--H--I--J--K--L-------M-----------N
\ / / /
2.x 2.0-RC1--2.0RC2 -- ... --2.0-RC8 -- T -- 2.0 -- 2.1-RC1
That is why I called the branch 2.x.
Your usage of mercurial seems a bit complicated with all those merges all over the place and will be painful to handle.
You are using the branches feature of mercurial, but seems to overlook 2 other important features:
Tags
Cherry Pick
I'll describe below our general usage of mercurial to highlight those features:
Note that the following only applies to projects for which different versions need to be maintained over time.
The main branch (called default in mercurial) is dedicated as you to the current development of the application.
Every time a developer complete the development/modification/update of a functionality, he/she pushes their change into default, which is then available to everybody and to our CI (jenkins) which will ensure nothing is broken.
When we reach a stage where a new version is about to be released a branch is created. In reference to your schema, the branch could be named MY_PRODUCT_1_0
Developers will continue to work on the default branch, and any time a commit need to be in the coming release, the related changeset will be copied to the MY_PRODUCT_1_0 branch via the command hg graft REV_CHANGESET (cherry pick) (note you can also copy from MY_PRODUCT_1_0 branch to default).
So you basically select which changesets from the default branch will make it to the current release, without having to merge the 2 branches.
This requires developers to push clean and atomic changesets, which is how things should be done in mercurial in the first place.
As your commits evolve in the MY_PRODUCT_1_0 you can tag it multiple times along the way as your schema from MY_PRODUCT_1_0_RC_1, MY_PRODUCT_1_0_RC_2,...
When the final changeset is made on this branch you just need to tag it MY_PRODUCT_1_0_PR_1_0
You then get only 2 branches at first, default (the dev branch) and MY_PRODUCT_1_0 (your first release), and as time goes on when you need to release a new version of your product, you create a new branch MY_PRODUCT_2_0 and restart the cycle as described above.
With this approach you are sure that you only have the required changes in your releases, and not extra ones that you would get when merging branches.
I hope I was clear enought, if not let me know.
For a team of 4-5 developers using Visual Source Safe(VSS) 2005 what is best practice to maintain source versions?
Our requirement is to maintain and identify a version that is currently in production. This way at all times we will know what piece of code was pushed during deployment.
So far this is what I had in mind.
-trunk - Main solution and project
-Dev branches - branch created for development and enhancements
-Dev (some enhancement)
-Release - release branches
-Release 2.1.0
-Release 2.1.1
-Release 3.0
Everything that starts with a dash(-) above is a folder in the repository.
I am thinking that each time we need to make an enhancement to our project, we make a dev branch from the main trunk of the entire project. This will be where the developers will work on. Once development is complete and UAT is done we proceed to the deployment.
After the project(in this case a web application) is deployed and confirmed that it is stable we make a Release(version #) branch of the Dev branch above. Lets say this is Release 3.0, we now know that in July 2010 deployment we pushed the code in the Release 3.0 folder.
We then merge(is this easy in VSS2005) the Release 3.0 code with the trunk. This way the trunk is always the latest deployed code and the next enhancement will be branched from trunk.
HotFix
Some things that I have not yet figured out is what happens when there exists a Dev branch that is being worked on and there is a hot-fix that is needed to be deployed immediately.
Perhaps, we make a separate branch from the latest Release folder, call it Hot-fix 3.0. Have the developers make the fix, merge it with Release 3.0 code after deploying the hot-fix and then merge with trunk again.
Clean up
After release there is really no need to have the dev branches. Should these be deleted?
Since we are branching, I read that in VSS deleting a branch does not free up the space in the database unless the original is deleted.
How should we delete, or should we delete the dev branches?
These are my thoughts, how do you manage your versions and what are your recommendations for my requirements.
We MIGHT be moving to TFS in the future, so anything I implement now in VSS should consider this move in the future.
trunk -> should only contain stable code
tag -> add tags to the trunk code to identify versions (e.g. releases 2.1.0, 2.1.1, 3.0, etc.)
branch -> create a branch for every single issue
update -> update branches often to keep the code as close as possible to the trunk (because meanwhile other fixes where committed)
merge -> merge branches to trunk if the solution is stable (include the issue number so the code changes are related to the issue number)
Most important of all:
use a source control system which supports you in doing all this -> VSS will not ;)
Also take a lok at this interesting article: http://betterexplained.com/articles/a-visual-guide-to-version-control/