I work for a small company that currently has only two developers. I'm not an expert with GitHub and I know that our current workflows aren't necessarily standard. I'm not looking to re-design our whole workflow, just a reasonable solution to this specific challenge:
We have two main branches: Development and Master. We use the Master branch for client installs, so it is always behind Development, until we merge the two before a major release.
Due to the nature of our software and target market, it is important for us to be able to periodically apply custom code for specific clients to the Master branch between releases so that it is ready for their use when we install. We also need to apply these changes to the Development branch so that they are included in the next release. This custom code is included in all future client installs/updates, but only accessible to the specific client based on configuration settings.
My current solution is to create the "Custom Feature" branch based on the Master branch. When the custom work is done, we will create a pull request for both the Master branch and the Development branch. Since the Master branch always has the same code as development, just in an earlier state - this seems to me like it should work. But like I said, I'm not an expert with GitHub and I'm sure this could be dangerous for any number of reasons.
I know it's risky to apply these types of periodic changes to a live release branch. However due to the nature of our software, most of our clients expect at least a small level of customization when we install for them.
Edit
I'm aware this is very similar to this question:
Avoid merging master into development branch
But I'm proposing branching from the release branch rather than the development branch, so I think it's a different case (I'll admit some of the concepts from that question go over my head though). I apologize if this is deemed to be a duplicate.
I recommend getting familiar with GitFlow as it might be a solution to your problem. https://datasift.github.io/gitflow/IntroducingGitFlow.html
Basically, you are right, the correct way to do this is to branch off from master, create the change in the branch and then merge to master and develop. Follow the "hotfix" section in the attached link.
Related
I have three environments; dev, test and staging/prod. In our previous model of using Team Foundation Server, we would have three branches of code that matched up to each of these three environments.
Developers would work locally and when they had their code complete, they'd check it into the dev branch. When checking in, TFS automatically creates something called a changeset. This check-in would kick off a build of the files into code which then gets deployed to the dev environment.
When a developer was happy with their code in dev, they'd merge just their changeset into the test branch. They'd pull up a complete list of all of the available changesets that dhad not been merged into test, they'd select theirs and check those into the test branch. Again, this would kick off a build and the output files would get deployed to test.
Once QA was happy with the changes, the dev would merge this changeset into the prod branch. Kicking off a build and the files would be deployed to the staging area. The developer and QA would them promote these files to prod.
All of would allow multiple developers to work on the same files using this changeset mentality. When a specific changeset (or set of changesets) was merged into another environment, only those changes would get merged.
In my relatively new exposure to git, I cannot seem to find a way to select specific "pull requests" (which I assume is similar to a TFS changeset) from one branch to another branch. When I try to make a pull request from one branch to another branch, it wants to pull in not only my pull request, but every other pull request made in the lower branch by other developers too. What is the magic way to make this happen?
Note: Unfortunately we don't have the notion of a "release". We have five scrum teams working on one website with over 200 pages. Each scrum team has their own sprints and can release multiple scrum stories during their sprint. We have internally only one DEV environment, and one TEST environment and one PROD environment. Not only are our environments used by these five scrum teams, but these DEV/TEST/PROD sites are is also used by various other teams for integration efforts with applications we sell and also for customer account management and purchasing. We cannot change that infrastructure.
Note: this is not for a discussion as to if this "changeset" methodology is correct or proper. This is a question of how to achieve this behavior in github/git.
Note: we are a set of scrum-based agile teams. We work from stories. As many as 60 stories can be actively in development at any one time with our large team of 25+ developers. When one story is ready for prod, we promote it to the prod environment as an atomic unit. So think of a changeset as a story.
I have two thoughts:
Don't do it this way. Instead, you should look to git-flow. http://danielkummer.github.io/git-flow-cheatsheet/ and http://nvie.com/posts/a-successful-git-branching-model/ are good explanations. At it's core, git-flow is a naming convention for branches, so it's really not tied to git at all. In essence, you have feature branches that each developer or dev team works on. Once they complete a feature, they merge into develop. develop is "done features" -- not "done done" but rather "feature complete." When we deem it time to release, we fork to a new release/someversion branch (name to match the release name), and then work with QA to harden the release. Commits on the release/someversion branch are only bug fixes. Once it's good enough to deploy, we fast-forward the master branch up to the release branch as we push it into production. master then represents what's in production. As we deploy, we also merge release/someversion into develop so the bug fixes get into the mainline of development. The project manager / product owner can then think of the develop branch as "the latest," and developers can continue on their feature branches until they're feature complete. (Hint, make features small -- like an hour or a day. Features are not releases.)
So why is this better than the way you were doing it? If the feature is done, ready enough for QA to start banging on it, it's done enough to be part of the next release. Picking and choosing features around each other will lead you into very subtle and unpredictable bugs. Since you're re-merging at each step, you have the possibility that you'll merge incorrectly, creating a bug. You're also now creating unique product with each step, so you could get to production with a completely different set of features than you vetted in dev and test. (Will this do bad things? Ask your pharmacist if these drugs interact when taken together.)
Git-flow works great for cadences where you have well coordinated, infrequent, larger releases. As you get closer to continuous delivery, this ceremony will get in your way. At that point, you may choose to flip to GitHub flow or a similar lighter-weight naming convention.
If you're really, really, really (see the above "you shouldn't do it this way" comment) convinced you should do it this way, first, go convince a rubber duck and hopefully you will have talked yourself out of it. If you're still really, really convinced you need to do this, you'll frequently need to squash your commits together creating one large commit for the entire feature, then cherry pick the changeset between the branches.
There's a few disadvantages to this "squash and cherry-pick" approach. 1. You lose history. Since you're squashing the history together, you have to now keep features in very contained bundles, and frequently edit the bundle as a whole. One of the primary premises of source control is you get an audit history -- both to roll back to if something goes wrong, and to reference when you need to learn why something works this way or who to talk to about it. (See "git blame".) When you squash, you intentionally remove that learning tool. 2. You're playing features into place in different orders. So you're frequently doing merges. What makes git so awesome is merging is easy. What makes git merging easy is you do it in small pieces. This methodology of squashing everything associated with this feature into one huge commit and cherry-picking it between branches means you're doing very large merges ... which means it will be hard.
Yeah, I know you're quite enamored with the way it's always been, and you really don't want someone telling you your baby is ugly. Sorry. Your baby was ugly. On the bright side, it doesn't need to be. Git flow is awesome, and can definitely facilitate the velocity your team needs.
You previous behavior was dysfunctional. Although not unusual: http://nakedalm.com/avoid-pick-n-mix-branching-anti-pattern/
In Git you most likely want to do two things. The first is to follow Git Flow: http://nvie.com/posts/a-successful-git-branching-model/
Once you have this you can look at creating a deployment pipeline for binaries, not for source. You should do a build from MASTER and that build goes through your environments. Happy to discuss here and offline.
I am trying to come up with a good branching merging strategy for a scenario when multiple features are being tested simultanously and at least one of them is not signed-off by stakeholders. I want to get the signed off changes pushed through to production with the least effort in terms of SCM operations and retesting.
I am using CVS (and I can't change this), but the question is SCM technology agnostic to certain extent.
Imagine that at any given time there are multiple features being developed on a common baseline. They're all worked on in isolation in their own branches. At some stage a build is pushed to test envrionment from the trunk that contains all the finished/tested and merged back to trunk changes. Let's say there are 5 branches merged in total and one is not good enough to pass testing or for some reason is not signed-off by the business.
If there was a silver bullet like 'unmerge' operation it would be perfect, but as far as I know there isn't.
My other idea is not to merge all the changes to trunk but to a separate branch forked off trunk and push a build off that branch to test server. If some change is withdrawn for any reason it would require forking a new branch off trunk and merging all but the withdrawn changes. Once all the changes are accepted this temporary branch can be merged to trunk.
I am wondering if this is not an overkill though.
Any other ideas?
While this doesn't really answer your question as asked, you might want to look into the concepts of feature toggles and "branch by abstraction" as discussed in Continuous Delivery. These are a couple of the core concepts that allow iterative mainline/trunk development.
Should multiple developers work within the same branch, and update - modify - commit ? Or should each developer have his/her own each branch exclusively? And how would sharing branches impact an environment where you are doing routine maintenance as opposed to unmaintained code streams? Also, how would this work if you deploy each developers work as soon as it is done and passes testing (rapidly, as opposed to putting all of their work into a single release).
In general, I have found that having developers (who are working on the same project) use the same branch is better for finding integration problems sooner. If developers are each using an individual branch, then you're just delaying possible integration problems until later, when you merge the branches.
Of course, having developers work on the same branch means you need to have actual communication between those developers, but that's a social problem and not a technical one.
Developers would work on separate branches when there is a good reason for that branch to exist in the first place (such as a patch release of a previous version of the software, or a special build for a specific customer).
Note that tools such as Git and Mercurial allow developers to easily create their own private branches to organise their own work. This is a different situation from more than one developer sharing a branch, and (usually short-lived) private branches should be encouraged.
Branches are meant as a way to version control any feature or experimental piece of code that may break the mainline/trunk.
While it is common for developers to have their own personal branches for deep experimentation, often branches center around a new feature being added. These new features often require more than one person to be committing.
For example, on a web project, two developers and a designer may be doing a facelift to their company website. They still need to keep their mainline/trunk code clean in case they need to make a quick change to it before the facelift is complete. So they create a "facelift" branch and work on that instead. While the developers are committing javascript, the designer can be committing CSS and images. Once the facelift feature is complete, they can merge it into the mainline and send it live.
The only reason any of them would need personal branches would be for experimenting. Perhaps the designer is trying to implement "sliding door" tabs and can't get the padding right in IE6, for example. If he solves the problem, he can merge it into the facelift branch, if he can't, he simply ignores it and continues with the rest of the design back in the facelift branch.
To some extent, the version control software you are using will nudge you into a particular approach. GIT is geared toward open-source contributors and resembles the "one developer" model (branching isn't even a concept in GIT. GIT is more about managing changes). Clearcase is more corporate, so you do have multiple developers on a branch, but each developer gets to play in his or her own view.
I agree with Greg's answer, this is more a social planning issue. Lots of devs on one branch will step on each other's toes. I've been on a project where there were more developers than individual source files :)
I think that merging of branches can be problematic (dropped or inconsistent functionality), regardless of how good the source control tools are. I would more readily opt for multiple developers working on a single main branch. There could be other branches for things like production bug fixes or proof-of-concepts (POCs), where merging could/should happen very soon after change (bug fixes) or good chance that merging may not need to happen (POCs).
Is there a specific rule I should be using for when to branch in source control? Branches seem to be expensive because they require that the team have extra knowledge about where the features they want to work on should go.
Our development team sometimes finds itself working on a long term feature and a shorter term feature at the same time. That means we end up with:
Trunk
-Branch A (Short Term)
-Branch B (Long Term)
After they complete we have to merge A in to the trunk, then merge the changes to the trunk back in to B to make sure those edits still function. It's messy.
I am wondering if we can cut down on branches by using Labels (Or tags, or pins or whatever your Source Control Software of choice calls it). Maybe it makes sense to branch for the longer term project, but we could just do the edits for the short term project right in the trunk after applying a label to the stable release. That way we can always retrieve the source code that was stable if we have to do an emergency bug fix, but we don't have to deal with the branch.
What rules do you use to decide when to branch?
One way to reduce branching is to implement new features (especially smaller ones) directly on trunk. This is how we do it:
small features, which will are guaranteed to be completed before the next release, are implemented on trunk
for larger features, we create a feature branch ("Branch B" in your example)
once we are ready to create a release, we create a release branch (from trunk), e.g. named "branches/2.x". This branch is then used for testing and finalizing the release.
once the release is built, we tag the corresponding revision from the release branch (e.g. tags/2.0.0).
normal development then continues on the trunk. the release branch is used for maintenance of the 2.x line of the product (e.g. bug fixes are merged from trunk, or implemented directly on that branch)
In a small team, the time to branch is when you can't commit directly into the trunk. With svn (as I guess with other version controls as well), it is possible to postpone the decision to branch till the time one realizes that one cannot commit into the trunk.
To minimize the need to branch, a new feature can be worked on in the trunk itself by restricting the new-feature code within compile-time or run-time flags. This approach also allows to later turn off feature if not needed, do A/B split testing experiments with the feature, etc.
Of course with this approach it always helps to have a continuous testing that gives an early alert whenever the build/test-suite breaks on the trunk.
For one thing, this depends on the tool you use. Branches are more 'expensive' in Subversion than in Mercurial or git, because merges are harder to do. For another, it depends on your project/organization: you should probably have at least one branch per maintained version.
It depends on the VCS you are using. If you are using a tool that has good support for merging, then you should branch whenever you feel like it. When in doubt, create a new branch. If the UNIX epoch time is even, then you should branch. If it's, odd, you should wait a second, and then branch. If you are using a tool that doesn't support merging well, then you should consider changing tools. In other words, stop using a tool that makes it necessary to ask this question.
It’s normally poor practice to develop against the mainline or trunk. The trunk should be used as the master code set and should reflect the code that currently represents production. If you are not in production yet, it should represent the gold copy and should always build and be subjected to automated regression tests. It should not be used to show development status or activity. Protect your trunk from change and resist the temptation to allow developers to check out and lock code on a trunk. The only updates in my view should be via the merge process, when you are ready to repatriate your code to the mainline.
When branching you should consider the purpose, complexity and duration of the development.
• Is it to support a team of developers working on a new feature or a substantial piece of development?
• Are you using traditional processes or the various agile flavors that are out there?
• It is to accommodate the development of a patch or fix for production?
• What development and in particular, test activity will you accommodate on the branch and will you retain the branch until the derived artifacts are built, tested and deemed releasable?
There are many models out there but few give sufficient consideration to the "build" process and the implications of regenerating your releasable artifacts.
Let’s assume you have the following lifecycle: DEV->SYSTEM-INTEGRATIONTEST->UAT->PRE-PROD->PRODUCTION. Assume you create a branch from mainline to accommodate the development and build processes. Your development\build\test cycle continues right through to UAT. The artifacts produced from this branch have been exposed to sufficient testing to deem them potentially suitable for release. You are able to state that the artifacts signed off by the users were also exposed to system and integration testing.
Some folks advocate merging the source code to the trunk at this point and recommend that you create a RELEASE branch upon a successful trunk rebuild. For me this is fine if the solution is stable and requires no further change prior to production, otherwise you risk propagating bugs elsewhere. In variably it will need to change.
If you do unearth issues in PRE_PROD, where are these “Fix” changes going to be made? Many suggest that you can make the code changes directly in the release branch. If you proceed, this modification will produce a new build and a new set of artifacts. You may elect to push these artifacts back through PRE_PROD and on to production, as the underlying code has been validated through previous testing and the modifications made to stabilize the release are deemed risk free? But you have a problem.
You cannot state that the executables\artefacts released to pre-prod and subsequently production, have been tested in your lower environments. Despite confidence being high, the output from the release branch build is different from that produced from the development builds. This may fail audit.
Branching for me is about managing your code and not the build output or solely the release. If you advocate branching for release and release stabilization (pre-prod fixing), you must take the above risk combined with the need for significant regression testing into consideration.
On the basis that the trunk should represent production code, you cannot push code to it unless it has been pushed to production first. I advocate creating a branch that supports the development, build and release as a single cycle. To avoid branch longevity and unnecessary divergence from the trunk (and potential big bang conflict issues) limit the development as much as you can and release and repatriate often with the trunk to keep other development efforts current.
We use a JIRA as our ticket system. New bugs/tickets are submitted to that system. Once a bug is fixed, we create a new build and test it on our dev server. If everything is good we push it to the live server. Now I usually work on the trunk without any branching to fix the bugs. This is of course a problem. Because there can be many bugs in our system, but only certain ones get fixed at a time. However if I fix all of them in the trunk instead of a branch, then we are forced to test them all even if we did not had enough time to test them all. How do you usually fix bugs and branch etc..? (I am not sure if I explained it very well).
Here is the strategy I use. I develop in the main trunk. When the software is released, I branch it (say v1.0). When bugs come in, fix in the branch trunk and then merge back to main trunk. Here is a good synopsis of strategies that are available: http://www.cmcrossroads.com/bradapp/acme/branching/branch-structs.html
I'm not sure if it's the normal strategy but we do all work on the trunk and then backport bugfixes into release branches. Our main trunk is always 'unstable' and when we feel we have a trunk in a releasable state we branch it into a release branch. From then on buyfixes are ported back into the release branch and new functionality only gets added to the trunk. It means you can run your release branch through testing and focus on testing the bugfixes.
One common mode of operations is that the deployed software lives in a separate branch which receives only bugfixes. Where you actually develop those fixes is mostly irrelevant; to avoid interference with the current development, it makes sense to develop the fix on top of the "live" branch, then test/commit/deploy to the live system and aftewards merge the fix back into the trunk.
We have the same problem (or almost), and I think every developer team has it. I can unfortunately not yet give you an answer by experience, but only a theoretical one.
In my opinion, as long as it's a bug fix, it should be deployed as soon as possible.
What I am about to implement is a feature branch strategy, and a release branch.
This means we have to differentiate features from bugs. and what is deployed is branched separately (or labeled, in our case)
Doing this, you can still work on the trunk for the bugs, and deploy them to your testing server, and once it's tested and approved branch it to the release branch and deploy it.
you can also merge-in the bug fixes into your feature branch, or try to merge the feature later when you plan to deploy it to the testing server.
Anyway, the most important I think is to branch the long work that prevent you from deploying smaller bug fixes.
If you branch too much, you will have a merging problem. If you don't branch enough, you will have a deployment flexibility issue.
It depends on your version control system. If you're using git, where branches are cheap and merges are easy, then I would definitely create a new branch for each fix. This allows you to keep bug fixes independent of each other, allowing greater flexibility with respect to what gets merged into the master/trunk, and when.
On the other hand, if you're using Subversion, branches are more expensive (in terms of creating and switching/updating) and merging is more difficult. Often the cost/benefit ratio of a new branch isn't high enough (especially for small bugs) to make it worthwhile.
I wouldn't recommend branching on every reported bug. As you said, you may not decide to fix every bug that's reported, which would mean that you'd have a lot of dead branches to prune at some point.
If your tools & language support it, branching on every bug you decide to fix (and feature you decide to implement) isn't a bad idea. It allows you to develop and test each bugfix/feature when you have the budget and schedule to do so, and merge them back into trunk when you are ready.
We split our branches into product versions / release, so that each release has its own branch. The release from the branch is tested, and so we only need to test the fixes applied to that branch.
Additionally each product version has a dev and a main branch. Developers are allowed to freely commit to the dev branch without fear of interfering with the Release (only other developers!)
Unless you're using a distributed SCM (Mercurial, Git, ...) where branching is basically free, branching on every bug sounds like an unreasonnable amount of work.
The usual strategy with central repository SCM is to note the revision that is supposed to fix the bug, and test against a build made with a later revision. You can then merge the concerned revision back into the release branch.
We are using mercurial, and branching to fix bugs and then merge changes back is quite doable in a distributed SCM