I have a somewhat complicated branching structure at work (at least for me). It is something like this:
Main
|
1
|
2
| \
3 \
Ver2
|
1
| \
2 \
| ProjectA
3 |
1
There are 2 branches off of main. "Ver2" which has everyone's changes for the next version, and "ProjectA" which is my work.
My question is: Is there a way to create a config spec that knows what has been merged so I get:
Anything from ProjectA that has not been merged
If the LATEST from ProjectA has been merged to Ver2, then get the LATEST from Ver2 branch
If there is not a ProjectA branch, get from Ver2
If there is no Ver2, get from MAIN
For example, in the above case, if I merged version 1 from ProjectA to version 2 in Ver2 branch, then I would want to see version 3 on Ver2. However, if I have not yet merged those files, I would want version 1 from ProjectA in my view.
You have to remember why you define a branch:
To isolate a development effort.
So to better manage your complex config spec, you should know precisely what role play the 'main' branch, v2 branch and project A branch.
V2 and project A, for instance, should be there for two different reasons.
If Project A is there to develop the current version of the project, merges to V2 branch should occur to allow for retrofitting some of the current developments to the V2 branch.
By that reasoning, you should not want to see "both" in a same view: they represent two different set of files, V2 could include large refactorings with very different API.
However, should you insist on such a configuration, you could use the ability of moving a "MERGE_FROM_PA" label: each time you merge some files from Project A to V2 branch, you set again the "MERGE_FROM_PA" label for each merged file/directory, moving that label from previous V2 versions to their latest.
The config spec could be:
element * MERGE_FROM_PA
element * .../ProjectA/LATEST
element * .../V2/LATEST
element * /main/LATEST
But then again, that would not make much sense.
You need to define the different development efforts you want to modelize, and then define a coherent workflow, allowing your config spec to focus only on one of those environments.
I don't think you can do it quite like that. What you can get, though, is latest on ProjectA; anything on Ver2 that has not been changed in ProjectA; and anything on MAIN that has not been changed in Ver2 or ProjectA. The rest of the trick is ensuring that everything necessary has been merged from Ver2 or MAIN. For that, you can use a reference view with a config spec for Ver2 (you don't need one for MAIN unless Ver2 is not kept up to date), and then, in the ProjectA view, do:
cleartool findmerge . -fta view-tag-for-ver2 -merge
The -fta means "from tag". There are a myriad extra options, of course.
This ensures that ProjectA is fully up to date with respect to Ver2.
Why not? Not using a branch if it's been merged somewhere makes sense to me.
Here's the Config Spec:
element * {version(.../ProjectA/LATEST)&&!hltype(Merge,->)}
element * {version(.../Ver2/LATEST)&&!hltype(Merge,->)}
element * /main/LATEST
what makes this workflow incoherent, as long as you label it all before a build?
Related
I am looking for a good process to manage the version number of my project when my master branch has GitHub branch protection turned on.
In an ideal world, when you merge to your 'release' branch your continuous integration server would run its tests then automatically increment the version number of your project and commit back to your SCM system.
However, with branch protection, you cannot commit to your branch without a Pull Request, so you have a catch-22 where your CI server cannot push to your protected branch when it tries to update the version number.
I can think of a few work arounds, all of which are sub optimal:
Rig up a system so your CI server would make a PR to update the version. I'm not sure if you can even do that with GitHub, but even if it is that ends up creating two Pull Requests for every one 'real' PR, which is awkward.
Remove Branch Protection - now anyone can push anything to the branches, and you have to manage your developers through a manual process
Update the version manually before the pull request. this is slightly better than #2, but it opens the door for developers to make mistakes when choosing the new version number, or merging the right version number to the wrong branch.
I'm hoping there are other options I haven't thought of.
We are using Javascript and npm, but I believe the problem is language agnostic, certainly the same problem would exist with Java and Maven for example.
I used git tags to automate versioning using Jenkins. So every time the job ran it took the last tag and incremented it. Since tags are different than commits, adding a tag doesn't conflict with Branch Protection.
# get last tag
last=$(git describe --abbrev=0 --tags)
# increment and tag again
git tag $(($last + 1))
git push origin $(($last + 1))
This script is for a simple integer version number but you could follow semver if you like. This was for a Android project so I added a gradle function that would take out the last tag and use if for version number while building.
/*
* Gets the version name from the latest Git tag
*/
def getVersionName = { ->
def stdout = new ByteArrayOutputStream()
exec {
commandLine 'git', 'describe', '--tags'
standardOutput = stdout
}
return stdout.toString().trim()
}
I am sure you could use a similar setup with Javascript.
This is what I am trying to do in TFS (a trivial operation in CVS)
Statements:
There are 2 independent code trees in TFS, corresponding to:
HEAD development
STABLE version.
Something like this:
$/MyProduct/HEAD/<files>
$/MyProduct/STABLE/<files>
I have a few changes in HEAD that I need to backport into STABLE.
The changes are represented in a ChangeSet
Does you know if it is possible to do the following operation in TFS:
Create a patch file from a TFS changeset number, which is based on the HEAD tree
Apply the patch in a different TFS tree, in case STABLE
Have the merge tool to merge the difference.
What I have now is a bunch of scripts that work outside TFS - but it would be much better to have this feature inside TFS.
This list seems to discuss this, Apply specific changeset from one TFS instance to another - But what it basically says is that there is NOT an internal way of doing my operation.
I wonder if someone know some other secret.
The way to do this in TFS is to use branches. HEAD and STABLE should both be branches with a branch relationship in place. Then you can simply merge from HEAD to STABLE whenever you desire, and the VS merge tool will help you with the merging.
If you don't have this setup as branches you should do this ASAP. What I would do is backup the files in HEAD somewhere on your local disk. Then delete the HEAD folder in TFS (and locally). Make STABLE a branch, then branch off of STABLE to create HEAD. Now HEAD is a branch with a relationship to the STABLE branch.
Then in your local workspace overwrite the files in the HEAD folder with the ones you backed up previously. So long as you are using VS 2012+ you can use the Local Workspace functionality, and VS will detect all the changes and allow you to check them into HEAD.
From this point on you can merge HEAD into STABLE whenever you wish.
there's an "issue" (actually a nuisance, but a major one) with our development process.
We've setup a good continuous integration-deployment-delivery pipeline; basically, our process is like that:
New development happens on the default branch (we use Mercurial, but git would have the same issue); usually we develop just one small improvement there;
Once the improvement is ready and tested (something that usually happens twice or three times a week, our cycle is pretty short), it's merged into the release branch (which actually stands for release candidate), it's tested again in a production-like environment and if it passes all our testing it's pushed to the production environment.
If we need to make a bugfix, we do it on the release branch, we test it and then push it to the production env.
An artifact (usually an RPM package) is built from each branch; when there's a release in production, the very artifact from the release branch is promoted to our public repository. There's no branch for the production environment, because we usually don't need it (there's a very short timespan when something it's on the release branch but it's not in production; code doesn't linger there unattended).
In this context, there's a small issue for us.
In order to distinguish packages, we usually set version to 1 for packages on the release branch, and 2 to packages on the default branch. Our CI system then adds its build number, so we've got packages like oursoftware-1.0-build37 for release and oursoftware-1.0-build902 for default. Such version is written either in our .spec file or in our pom.xml files (or other files for different softwares, like .gemspec or simila)
Whenever we want to put something in release candidate, we merge the default branch to the release branch.
What happens there? the version gets merged as well. So after each merge we need to get back at the file containing the version and put the old version in them. We should need to do it just once (Mercurial does not try to re-merge things that were already merged) but whenever we need to do a merge from release from default (e.g. in order to merge a fix done on release to the dev branch) the problem rises again.
I'd like some advice on how to prevent this problem completely; how do I set versions outside of my branches? I sometimes actually need to change them (e.g. I want to change from 1 to 2 on release and from 2 to 3 on default) and I wouldn't like to hardcode them in our CI system, I'd like some system which is "outside" the scope of branches and it's kind of "global" for a repository. Or I would need something like "ignore modification for some files".
In many situations I actually need versions in my files as well; for example I cannot omit version info from a Maven pom.xml file or from a .gemspec, otherwise it won't work when developing.
Thanks,
Well, I see 3 4 a lot of possible solution, more or less "native" (random order without priorities and own personal preferences)
Instead of "Monolitic $DEVEL branch" you can use "Branch Per task": in this case file with (hardcoded) version-info will not appear as mergeable file (TBT!)
Branch-specific version change can be moved from (permanent) changeset of branch into patch of MQ (as results - versions /and files/ are identical across branches, permanent default changes have to appear as result of applying patch)
Files with version-info doesn't contain hardcoded versions, they are more "templates", than "data", and real data appear as result of some preprocessing inside build process (you know transformation-rules and can code it)
You can change nothing in developent|deploy process and if version-info files are unique inside repository, you can use trick, similar to merging hgtags (special mergetool for only some files in repo - in your case in must be internal:local special mergetool)
Maybe using just merge-patterns will be OK for you, check Mercurial - Exceptions to internal:local merge-patterns? and Default to trunk version for Mercurial merges? for details and some additional notes+discoveries
I am wondering what the best way might be to use Mercurial to create a release revision that includes artifacts that provide evidence of process execution. For example, we'd like to attach system test results, check lists, release notes, etc. so that if we are audited by a customer we can easily show that we performed our process. This is important to us due to the safety aspects of our product.
Our release management process will go something like the drawing below. All developers are developing in local repos and periodically pushing to main. Main is the latest and greatest but not necessarily safe for even internal engineering purposes outside of the dev team.
When we want to create a release for the customer or for other internal engineering departments, we start with a release candidate branch (e.g. RC1). If any fixes are required, we commit to the RC branch. Testing occurs on this branch. When the RC is determined to be good, the changes are merged back to main.
We think that what we would like to do is merge into a Releases branch. However, we have a chicken and an egg problem related to the artifacts: The artifacts that we would like include in the release revision contain, among other things, the hash for the revision. This provides clear traceability that the testing and other process steps were performed on this precise revision. But, to add these items, I need to create a new revision and I obviously can't know what the hash for that revision will be before I create it. I am wondering if there is some way to "amend" a revision without changing the hash?
The only way to do this, that I can think of, would be, for example, in the drawing below, to create a revision RC2.3 that contains the necessary process artifacts but actually merge RC2.2 to Releases.
Then, of course, I have another problem and that is that the merge of RC2.2 into Releases will generate a new hash. So, my artifacts are out of date again. So the next question is whether there is some way to have the Release branch "point to" RC2.2.
BTW, we're open to changing this process if necessary. Our reasons for using this methodology are:
CI system is monitoring main and kicks off a series of builds and performs automated unit testing on each push. There are frequent changes to main and we don't want people using it.
Development can continue on main with no impact to the release.
Any revision on the Releases branch kicks off a different set of tasks on our CI platform, including the creation of a distribution of a field flash utility and the required images (we're developing firmware). This is how we provide releases to outside entities.
Main A--B--C--D--E--F--G--H--I--J--K--L---------M-------------N
\ / \ /
RC1 RC1.0--RC1.1 \ /
\ \ /
RC2 \ RC2.0--RC2.1--RC2.2
\ \
\ \
Releases ER1.0-----------------------------------PR1.0
To answer your question:
I am wondering if there is some way to "amend" a revision without changing the hash?
No, that's impossible by design. The changeset hash is a cryptographic checksum derived from the content of the commit — this is why the changeset hash uniquely identifies the content of all the files in the changeset.
The Mercurial wiki has a page with a standard branch setup. I'll describe how it works with pictures below.
What people normally do in your situation is to create a branch for release candidates like you suggest. If you're about to release version 2.0 of your firmware, then I would call the branch 2.x. The commits on the branch are release candidates for your version 2.0:
Main A--B--C--D--E--F--G--H--I--J--K--L---------M-------------N
\ /
2.x 2.0-RC1--2.0-RC2
Bugfixes are committed on the branch as needed and merged back into Main regularly. You should always be able to merge the bugfixes back since they are by their nature bugfixes and stability improvements that you want to have on Main too. In other words, commit I above has the bugfixes in 2.0-RC2 plus the new features in D to H.
You can work on the release candidate branch as much as you want and keep fixing bugs:
Main A--B--C--D--E--F--G--H--I--J--K--L-------M-------------N
\ / /
2.x 2.0-RC1--2.0RC2 -- ... --2.0-RC8
All bugfixes are merged back since the rule is that you can only do fixes on the release candidate branch that are safe. Here you had 8 release candidates on the release branch.
When you like the status of the release candidate, you can tag it and run the more comprehensive tests on the revision you tagged. The output of a succesfull test is committed on the branch and you then tag that commit as the final released version:
Main A--B--C--D--E--F--G--H--I--J--K--L-------M-----------N
\ / / /
2.x 2.0-RC1--2.0RC2 -- ... --2.0-RC8 -- T -- 2.0
Here T has the test output for 2.0-RC8 included. Commit T is the one you tag as version 2.0 of your software and the one you package and ship to your customers.
Should you need to create a version 2.1 with a bugfix to 2.0 (but no new features), then you continue with the same scheme:
Main A--B--C--D--E--F--G--H--I--J--K--L-------M-----------N
\ / / /
2.x 2.0-RC1--2.0RC2 -- ... --2.0-RC8 -- T -- 2.0 -- 2.1-RC1
That is why I called the branch 2.x.
Your usage of mercurial seems a bit complicated with all those merges all over the place and will be painful to handle.
You are using the branches feature of mercurial, but seems to overlook 2 other important features:
Tags
Cherry Pick
I'll describe below our general usage of mercurial to highlight those features:
Note that the following only applies to projects for which different versions need to be maintained over time.
The main branch (called default in mercurial) is dedicated as you to the current development of the application.
Every time a developer complete the development/modification/update of a functionality, he/she pushes their change into default, which is then available to everybody and to our CI (jenkins) which will ensure nothing is broken.
When we reach a stage where a new version is about to be released a branch is created. In reference to your schema, the branch could be named MY_PRODUCT_1_0
Developers will continue to work on the default branch, and any time a commit need to be in the coming release, the related changeset will be copied to the MY_PRODUCT_1_0 branch via the command hg graft REV_CHANGESET (cherry pick) (note you can also copy from MY_PRODUCT_1_0 branch to default).
So you basically select which changesets from the default branch will make it to the current release, without having to merge the 2 branches.
This requires developers to push clean and atomic changesets, which is how things should be done in mercurial in the first place.
As your commits evolve in the MY_PRODUCT_1_0 you can tag it multiple times along the way as your schema from MY_PRODUCT_1_0_RC_1, MY_PRODUCT_1_0_RC_2,...
When the final changeset is made on this branch you just need to tag it MY_PRODUCT_1_0_PR_1_0
You then get only 2 branches at first, default (the dev branch) and MY_PRODUCT_1_0 (your first release), and as time goes on when you need to release a new version of your product, you create a new branch MY_PRODUCT_2_0 and restart the cycle as described above.
With this approach you are sure that you only have the required changes in your releases, and not extra ones that you would get when merging branches.
I hope I was clear enought, if not let me know.
I want to store a project in Mercurial that contains external code (which can be modified by me) coming from Git and SVN repositories. In SVN I would solve this with vendor branches and copy the code around, but I understood that in Mercurial it's better to have different repositories for different projects, and pull between them when needed.
The project layout will be like this:
- externalLibraryA [comes from a SVN repo]
- ...with some extra files from me
- externalLibraryB [comes from a SVN repo]
- ...with some extra files from me
- externalPluginForExternalLibraryB [comes from a Git repo]
In Subversion I would create vendor dir and a trunk dir, copy all external libraries first in vendor, and then in the right place in trunk. (I think) I can do this in Mercurial too, with subrepositories, but is this the best way to do this?
I tried setting up different repositories for the external libraries, but then it seems I can't pull the externalLibraryARepo into the externalLibraryA directory of my main repository? It goes in the main directory, which is not what I want. I can also create a Mercurial mirror repository and include it as a subrepo in my main repository, but then the changes in this subdirectory go to the mirror repository, while I want them to stay in the main repository.
I'd probably just store this in one repository - note that in the link you give they are using their build system in the end to bring together the binary output from the different repos. I'm not clear on their rationale there.
If the underlying problem you're trying to solve is how to update the externals in a clean way, I'd probably use anonymous branching for that.
I.e. add the external lib to your project, and your modifications. Make sure it works. Tag with ExternalA-v1.0. Hack away on your actual project. Now ExternalA, Inc. has a new version of their stuff. Update your repo to ExternalA-v1.0 tag. Import their new version and apply your modifications on top. Commit. Now you have two heads: one with the latest version of your code (that works with ExternalA-v1.0) and one with the latest version of ExternalA (that does not work with your code, maybe). So then you merge and reconcile the two. Tag again, now with ExternalA-v2.0. Repeat as needed.
You can still keep your externals in separate repositories, but I assume that the project that is using those does not need to be up to date with changes there all the time - looks like the whole point of vendor branches is to have some point of isolation between dependee and dependants. Of course, moving the changes from the externalA project to the project that is using that will then be a manual affair (well, a copy, much like in SVN really).
It depends on whether your vendor code is going to be customized by your team or not. Our teams have had a great deal of success maintaining a named "vendor" branch on repositories with our own customizations on branches named by project name. This vendor code is then easily included in a project as a subrepository.
A caveat to this approach: if active development is going on in the subrepository, best keep it to directly editing the subrepository as a separate clone, otherwise it becomes necessary to pay close attention to the top-level repository so you don't inadvertantly bump your .hgsubstate forward to the wrong revision and break your build.
Watch out for merges of the top-level repository (your project) between versions which point to different named branches of your subrepository, as this can result in a merge between the "vendor" and "project" branches in the subrepository as it recurses, which may not be desirable.
Note that this functionality may change in the future as well, as some "warm" discussions have been taking place in recent months on the mercurial-devel mailing lists about the future of subrepository recursion.
edit:
I just saw this discussion in the related links as well, which seems relevant: https://stackoverflow.com/a/3998791/1186771