We use a combination of GitHub, TeamCity, CodeReviews, Octopus deploy to manage our overall Release Management process. We develop websites, and several internally used API's.
We're looking to implement better versioning of our API's in-particular and will use Semver.
My question is, at which stage do you assign a version number to it?
Example:
Current version in Master is 1.2.1
User creates branch (Branch2) to implement some new functionality
User deploys Branch2 into QA for review and sign-off
User merges Branch2 into Master and Releases to Production.
At what stage should the version have been incremented to 1.3.0? If it is updated at the stage when it goes into QA, it is possible that another developer in the meantime creates another feature branch which is tested and ready for Release into Production much quicker than Branch2 was - so really the later should have been 1.3.0 whereas Branch2 which will be pushed out to Production weeks later should have [possibly been 1.4.0.
So, am i right in thinking that the version number should only be incremented once QA has been signed off, and prior to the final merge back into the Master branch?
Thanks for your time in advance
Regards,
dotdev
As you describe I see these versions:
1.2.1 Current version in Master is 1.2.1
1.3.0-alpha+branch2 User creates branch (Branch2) to implement some new functionality
1.3.0-beta+branch2 User deploys Branch2 into QA for review and sign-off
1.3.0 User merges Branch2 into Master and Releases to Production.
Increment to the next version must be done immediately after any official/production release. Here is a sketch that could help.
feature *-(1.3.0-beta+feature1)---* *-(1.4.0-beta+feature2)--
/ \ /
master -[v1.2.1]-(1.3.0-beta)-*-----------------------------*-[v1.3.0]-(1.4.0-beta)--*-----
\ \
release *---- *---
You manually apply tags to official releases represented by square brackets, while the version (normal parenthesis) takes in account tags in history.
I suggest to take a look at the wonderful GitVersion tool to help you in managing the calculation.
Related
At work, we're now using GitHub, and with that GitHub flow. My understanding of GitHub flow is that there is a master branch and feature branches. Unlike git flow, there is no develop branch.
This works quite well on projects that we've done, and simplifies things.
However, for our products, we have a development and production environment. For the production environment, we use the master branch, whereas for the development environment we're not sure how to do it?
The only idea I can think of is:
When a branch is merged with master, redeploy master using GitHub actions.
When another branch is pushed, set up a GitHub action so that any other branch (other than master) is deployed to this environment.
Currently, for projects that require a development environment, we're essentially using git flow (features -> develop -> master).
Do you think my idea is sensible, and if not what would you recommend?
Edit:
Just to clarify, I'm asking the best way to implement development with GitHub Flow and not git flow.
In my experience, GitHub Flow with multiple environments works like this. Merging to master does not automatically deploy to production. Instead, merging to master creates a build artifact that is able to be promoted through environments using ChatOps tooling.
For example, pushing to master creates a build artifact named something like my-service-47cbd6c, which is a combination of the service name and the short commit hash. This is pushed to an artifact repository of some kind. The artifact can then be deployed to various environments using tooling such as ChatOps style slash commands to trigger the deloy. This tooling could also have checks to make sure test environments are not skipped, for example. Finally, the artifact is promoted to production.
So for your use case with GitHub Actions, what I would suggest is this:
Pushing to master creates the build artifact and automatically deploys it to the development environment.
Test in development
Promote the artifact by deploying to production using a slash command. The action slash-command-dispatch would help you with this.
You might also consider the notion of environments (as illustrated here)
Recently (Feb. 2021), you can:
##Limit which branches can deploy to an environment
You can now limit which branches can deploy to an environment using Environment protection rules.
When a job tries to deploy to an environment with Deployment branches configured Actions will check the value of github.ref against the configuration and if it does not match the job will fail and the run will stop.
The Deployment branches rule can be configured to allow:
All branches – Any branch in the repository can deploy
Protected branches – Only branches with protection rules
Selected branches – Branches matching a set of name patterns
That means you can define a job to deploy in dev environment, and that job, as a condition, will only run if triggered from a commit pushed from a given branch (master in your case)
For anyone facing the same question or wanting to simplify their process away from gitflow, I'd recommend taking a look at this article. Whilst it doesn't talk about Github flow explicitly it does effectively provide one solution to the OP.
Purests may consider this to be not strictly Gitflow but to my mind it's a simple tweak that makes the deployment & CI/CD strategy more explicit in git. I prefer to have this approach rather than add some magic to the tooling which can make a process harder for devs to follow and understand.
I think the Gitflow intro is written fairly pragmatically as well:
Different teams may have different deployment strategies. For some, it may be best to deploy to a specially provisioned testing environment. For others, deploying directly to production may be the better choice...
The diagram in the article sums it up well:
So here we have Master == Gitflow main and the useful addition is the temporary release branch from which you can deploy to other environments such as development. What is worth considering is what you choose to call this temporary branch, in the above it's considered a release, in your process it may be a test branch, etc.
You can take or leave the squashing and tagging and the tooling will change between teams. Equally you may or may not care about actual version numbers.
This isn't a million miles away from VonC's answer, the difference is the process is more tightly defined and it's more towards having multiple developers merge into a single branch & apply fixes in order to get a new version ready for production. It may well be that you configure the deployment of this temporary branch via a naming convention as in his answer.
The way I've implemented this flow is using PRs. I did it with Azure DevOps, but I'd say that the same can be achieved with GitHub Actions.
When you have a branch that you intent to test and eventually merge to master and release to production, you create a PR from that branch to master. The PR will trigger a pipeline, which will run your build, static analysis and tests. If that passes, the PR is deployed to a test environment where further automated and manual testing can happen. That PR can be reviewed and approved by other developers and, if you need to, by QA after manual testing. You can configure GitHub PR rules to enforce the approvals. Once approved, you can merge the PR to master.
What happens once in master is independent of the workflow above, but most likely a new pipeline will be triggered, which will build a release candidate and run the whole path to production (with or without manual intervention).
One of the tricks is how the PR pipeline decides which environment to deploy the PR too. I can think of three options:
Create an environment on the fly which will be killed once the PR is merged or closed. This is the most advanced and flexible option. This would require the system to publish the environment location to the PR.
Have a pool of environments and have the automation figure out which are free and automatically choose one. The environments could be stopped, so you find an environment which is stopped, start it up and deploy there. Once the PR is closed/merged, stop the environment again.You can publish the environment location to the PR.
Add a label to the PR indicating the environment (ie. env-1, env-2, etc.). This is the simplest option, but it requires that developers look at the open PRs to see which environments are already in use in other PRs to avoid overwriting other people's code.
With all these options, once the PR is created, you can just push new commits to the branch and the environment will be updated.
You also need to decide what you want to do when a new commit is pushed to master. You most likely want to trigger a new PR build to update the environments with the latest master, but you can do this automatically or manually, depending on how busy your master is.
Nathan, adding a development branch is good idea, you can work on development changes in new branch and test them in dev environment and after getting signoff to move to production environment you can merge your changes in master branch.
Don't forget to perform regression testing on merged master branch to test both old features and new features are working fine before releasing your code for installation in production
We are getting cases where our Work Items are getting included in many builds and without sufficient documentation I can't seem to work out why. Wondered if anyone here could shine any light on this.
Example scenario is
We run "Release Flow" branch strategy (like MS describe they do for Azure DevOps)
Bug 30368 was linked to a feature branch and got back into master via PR 2991 and commit cb9120d0.
All the appropriate links to PR and Branch were in place for this so both the PR and the master commit show the link to the work item correctly
We create our release branch off of master and 30368 was included in v2.97.0 build pipeline and was linked (correctly) with an "integrated in build" to that build
Build 1 is created
We then had a bug in v2.97.0 during release testing (not to do with Bug 20368) and so we Cherry Picked the master commit with the fix into the release branch (as per Release Flow guidance).
I'm wondering that this Cherry Pick is the cause of the problem I describe later
Of course the work item wasn't included in this second build of v2.97.0, only the work item with the related fix was
Build 2 is created
So all is good so far. v2.97.0 release is deployed from Build 2 and all the links are correct
Now we branch from master for v2.98 release and the release pipeline does another build.
Build 3
For some reason it determines that 30368 should be linked to this build also and adds a "integrated in build" to the v2.98 build?
The only other clue that I've found is to do with the commits that are linked with the build. Here I can see that I get the following
Build 1
from e0faff8f (master 2019-12-02 commit release 2.97 was branched from )
to db225f14 (from feature branch just before PR completed 2019-11-21)
Build 2
from aeaf3dc5 (release branch 2019-12-03 cherry picked)
to e0faff8f (pull request branch 2019-12-03)
Build 3
from f2ce6d3d (release branch 2019-12-06)
to f3db8c7b (2019-12-02)
It isn't even clear to me how the build pipeline knows what the range of git commits are included in it and perhaps this is the key to understanding the link. Of course it gets the From part from the commit that the build ran from but how does it know which commit it should go to?
Also in the above example, neither the range given for Build 1 or Build 3 seems to have the commit that is related to the item (cb9120d0) in it - even though this work item has been linked to these builds?
It is all pretty confusing and nebulous.
Has anyone got this feature working correctly/reliably?
I also wondered if we need to create specific pipelines for each branch so that each branch is independent but I'm not sure as even if you create a brand new pipeline and point it to the master branch it doesn't include every single commit ever (and workitem) in that build. Again, how does it know where to start?
I'm trying to setup a library project using DevOps. The branching strategy choosed is git-flow in order to have use git version and follow semver with releases and hotfixes.
So i tried was to protect both develop and release branches by activating pull requests.
Everything works fine unless a release is going to be closed. Here I see two problems:
In git-flow the system try to realign develop from master, but devops doesn't know this is git-flow and just merges release to master.
Develop branch is protected, so if i want to realign manually i should create a new pull request, from master to a feature branch in order to solve conflicts
I'm not sure about the correctnes of the implemented workflow. Is there a better way to accomplish this realignment?
Any suggestions are welcome
Thanks
At present, azure devops is designed like this. Devops can only be one-to-one branch merge at the same time.The release branch cannot be merged into the master branch and into the develop branch at the same time.
When the release is finished, the release branch is merged into master
and into develop too, to make sure that any changes made in the
release branch aren’t accidentally lost by new development.
This feature described in git flow cannot be implemented in azure Devops.
So, your understanding is correct, you can do it manually by your method, although this may be a bit inconvenient.
I am working on the project where currently we have following three fixed branches
Develop - The code is deployed to development environment. It's a base branch for anyone who want to add new feature.
Release - Deployed in QA environment, And QA can start testing on.
Master - Deployed to Production environment and available to clients.
And we have two more dynamic branches, feature and hotfix.
Anyone wants to start new development or bugfix, forks new feature from develop branch, then then creates a pull request.
once development is done it's testing in Development environment from develop branch it then creates a merge request for release branch
QA deploys the release branch on Test environment and start testing, once testing is done. it's merged to master and then deployed to production.
This all works well for most of the part. However, it has following problem
Not every feature in QA (release branch) is tested and ready for deployment (merge to master) at the end of the release. And so we are not sure how to create a pull request as it will select all the commits.
I am thinking Github releases might be a solution for this. I can create a new release with which ever feature is ready for deployment and then merge these releases with master branch.
However, what I am not sure is when to deploy to production, from releases or from master ?
You should be following the standard Git Flow:
Instead of merging any feature to the release branch (from develop) after testing, what you'll want to do is make use of a 'release cut'. This is a dedicated point in time at which you decide you want to make a release. Features that are planned to be in this release are evaluated and tested, and the release is confirmed.
You'll want to tag your releases, and optionally include any assets that are associated with that release for future reference. If it's a mobile app, this is a great place to include your .apk or .ipa. After the release cut has been made and tagged, you'll then want to release the release branch to production. Once that has been deployed out to production, you'll then want to merge release into both master and develop.
Under this approach, every feature branch will end up in develop, then release (with a corresponding tag), then master. If a feature is still in progress when the release cut is made, it simply doesn't make the cut! It will get added to the next release. This will give it adequate time for testing before it comes anywhere near release, let alone master. In this regard, master should be a mere representation of your last-known 'good state' -- a point at which you were happy to release to the public.
Also, note that hotfixes and bugfixes are two different things, and neither should branch from develop. Bugfixes should be made to the release branch itself, and hotfixes should be created from master, and merged back into both master and develop.
We are reaching a point in our project where we need to make a production deployment but also need to have ongoing development for future features. Our source control currently has a single development branch. In my previous company a 3 branch system was set up with Development, Integration, Production. Feature development was done in Development, tesing in Integration and Production always was the code running in the current production environment (except for a breif time when a merge was done from Integaration into Production and the deployment was done).
Each time there was a Production change or a merge into Production that was deployed live, a new archive branch was taken off Production and given a version number. Any change to a higher branch was merged back so changes in Production would make it back to Development for example. It worked but I always didn't see the need for an ongoing Integration and Production branch.
Two aspects of this system I really didn't like: 1) the merge from Integration branch into Production branch: I would prefer to have a "clean" Production branch each time even though they should be in sync after a merge and 2) this system doesn't allow for multiple deployments of the system running different versions of the code at the same time, though this has not been a requirement in either team I have worked for (yet).
I have heard this model is common but in the system I am setting up now I am proposing the following:
Have a Development branch, create a new Release branch each time Development is ready for the next deployment to production. The Release branches are given a version number and then branched again to an archive branch. Testing is done on the Release branch. Once deployed any production fixes are done in the Release branch then a new archive branch is created with a minor version number increment once deployment is approved. When a new deployment from Development is ready the a new Release branch is created...
To me this is simpler and is actually better: There is NO Integaration branch (less merging) and there is a fresh Production (Release) branch for each deployment and caters for current Production versions. What am I missing? Why go for the Development, Integration, Production model?
Thanks
The advantage of the 3-branch system is that your developers, testers, and customers each have their own isolated version/branch of the code. This provides a "gated" development where features can only be promoted to the next branch when they pass a certain completeness/quality bar, giving high confidence that the release branch is always in a fit state to send to a customer.
Pros:
You (almost) always have a build that could be sent to a customer at a moments notice.
Your testers (almost) always have a working build to test
The release branch is stable so you don't often have to fix bugs in it and reverse-merge them into the test/dev branches.
Cons:
You have to merge frequently to promote completed features up to the release branch. This isn't too bad though, as this merge is simply a quick copy operation (as long as you don't make edits in the test/release branches, you never have a merge conflict to address)
If you take the approach of a single development branch from which you split off a release branch only when a release is required, then you are effectively working in an unbranched system most of the time. That is, you have an unstable work-in-progress build in your dev branch, which you then copy into a release branch, where you work on it as a work-in-progress until it is stabilised to release quality. If you continue developing features in the dev branch while you bugfix the release branch, you will then get a lot of merge conflicts to resolve. You spend a lot of time without a good build to test, and without a releasable build that you can ship if needed.
There also isn't really much need to branch the code into an archive branch when you release - all you need to do is label the code when you do a release to get a reliable "snapshot" of it that you can recover in future.
Your suggestion isn't too far off the recommendations.
The primary difference in the model is that you're suggesting that you develop directly on what was previously the 'integration' branch. A lot of people suggest taking a branch off this branch to do their work on, then merging back in. But it depends on the size of your team.
An invaluable resource for working with TFS branching is here:
http://tfsbranchingguideiii.codeplex.com/
Your proposal sounds exactly the way I am working:
I develop in the main branch until my developments are ready
Then a release branch is created, let's say release 1.5
After the release branch has been tested, it is branched again, calling it 1.5.1
Bug fixing is done in the main branch and all active release branches (e.g. 1.3, 1.4, 1.5)
Regularly new versions (branches) of the release branches are created and sent to customers (e.g. 1.5.2, 1.5.3, ...)
In my experience this works quite good, but probably it also depends on how your company is organized.
One place I used to work had a branch for each release. We spent more time branching than we did merging. It was a bit excessive.
If you think of a mental model with a graph of connected nodes, the Production branch is just another node connected to the main Integration branch.