We've been using Mercurial for a couple of months now and it's improved our deployment process A LOT already. This is the good part.
The system we have in place is working but it's still error prone if you're not careful or rushing. This leave me wondering if there's ways we could improve it or... maybe we're completely off the track :)
Our environment consist of:
Local development workstation (each developer)
Development server (hosting the DB & the central repository)
An acceptance server (Where QA is done)
A staging server (Where we stage the release branch, we then robocopy to the live systems)
A little background on the reason why we switched:
We are in a work environment that often have us switching from task to task, leaving many pending tasks. Many would become stale and clutter up the main branch when we were back on CVS. Deployment was a nightmare as you had to work around lines that needed to go live and others that didn't using Beyond Compare.
Mercurial with named branches and easy merging solves this for us. So not knowing what to expect we set it up.
First we cleaned cleaned up our production source, pruning dead files, etc.
We FTP'd that on staging and made this our new repository as "default", this was to be our stable branch ready to be deployed at all time.
Afterward, we did an hg clone to the development server and had each developer hg clone from the development default branch.
On acceptance where we do QA we did an hg clone of the development server's default branch.
At this point we have a stable copy of the code everywhere, everyone is eager to start!
local machine are pushing to dev, acceptance pulls from dev and staging is completely isolated and can pull from wherever if the path is provided.
Now the idea behind this was that the default branch on our system would always be a copy of the code on the live server, provided that we remembered to pull before starting a new branch. When starting a new feature we would:
hg pull -b default #synch up
hg update default
hg branch {newFeature} #newFeature completely isolated from other changes.
*work on {newFeature}
Oh no! A bug! this is unrelated to what I am currently working on lets call it {bugFix111}. This appear to call for a new branch independant of my feature; go back to updated default. This will isolate newFeature and bugFix111 from each other and each can go live independently as they are based on default.
hg update default
hg pull -u
hg branch {bugFix111}
Once work is completed on say {bugFix111}
hg push -F {bugFix111} #send our fix to the main central repo on dev.
Go to acceptance:
hg pull -b {bugFix111} #pull the fix from the central repo (dev).
hg merge {bugFix111} #merge the code in the default QA branch.
hg commit -m "Merging {bugFix111} into default"
*QA sign off on the fix
We have to branch off acceptance - default were QA take place and release where we merge the stuff as it get signed off.
hg update release
hg merge {bugFix111} #fix is now ready to go live
hg commit -m "Merging {bugFix111} into release"
On staging:
hg pull -b release {PATH TO ACCEPTANCE REPO}
hg merge release
hg commit -m "Merging {bugFix111} into staging default"
hg tag release{date}
*robocopy to live
*run task that pull from staging to dev so that they synch up again.
This has been working for us and save up some deployment time as it's a breeze to just robocopy the stable release branch.
Issues
What we have noticed is:
It's easy to goof up a merge when merging the second time on release, this seem against the flow. We can break it after the QA sign off.
Could get QA to test our release branch as well but it seems like duplicating resource, the goal is just to have features isolated and being able to send them one at a time.
We can completely blow it up by merging release over something wrong, e.g. hg merge release when you are on the default on acceptance completely overwrites it.
If we forget to pull before starting a new branch we are working off the wrong base.
Few other oddities but those are the biggest hurdles.
I realize this is a long post, but hopefully the answers will help other Mercurial newbies like me trying to set up a decent workflow at their company.
Why not pull on staging from the QA branch? Then the merge job has already been done and validated, i.e if the commit has some manual merge you will import it on staging also. Otherwise you have to replicate the merge on staging as you are doing it now.
Related
I have two branches in Github. Master and CoolFeature.
When i check CoolFeature in Github i see that it is even with Master.
When i deploy through Team City the Master branch somehow i get a commit from the Master branch that is five commits behind. I can see it in Team City by checking the SHA1. Is there some feature i have missed?
When i deploy the CoolFeature branch then the latest changes works cause CoolFeature deploys the latest commit.
Edit:
When i checkout the master branch i get this message:
> git checkout origin/master
Note: checking out 'origin/master'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at e86824af5...
The SHA1 hash refered e86824af5... IS the latest commit. And this is the one i want to be deployed. But TeamCity deploys a commit that is five commits behind this one. I can see it in the log comparing the SH1 hash ids.
I am planning to add my iOS project to GIT hub and I am new to the GIT branching and taging.
Please suggest a simple and good branching structure for development and production.
If i create one branch for development and one for production(master), is it possible to create sub branches under development?
Help is highly appreciable.
You can branch from any commit in git. All branches are equal. Why don't you try it out?
SCM organisation tends to be project-specific, so, whatever works for you is good. Your clients should never see your entire repository anyway.
First of all, branching in Git is unlike branching in Subversion or any other Central VCS.
Guess you need one major branch, and from there you can make all possible branches you need. Just remember to merge what you want to keep.
E.g. you branch for production (guess solving problems (branch even per problem)) and you branch for development (sub branching again if needed). And later on you can combine (merge) the changes from the production branch back to the master and all the stuff from your development branches that you want to keep.
But there is no best way - that depends on your use cases.
I usually branch per problem (production / issues) and merge it back if solved.
The development branch(es) I only add when needed (f.i. before acceptance testing).
YMMV.
Assume you have a tarball named ios_project.tar.gz with your initial work. You can place it under git revision control as
follows.
$ tar xzf ios_project.tar.gz
$ cd project
$ git init
Git will reply
Initialized empty Git repository in .git/
You've now initialized the working directory--you may notice a new directory created, named ".git".
Let the master be as is. I would recommend you to create two branches one for development and another for production. Do all your code modifications which you are unsure of in your development branch, and when you are sure about your work, push it to the production branch and apply a tag on it so that it is each to track the production line at a later stage.
In addition to this if you would like to share it with others, then have the repository as a shared directory. And clone it on any other development machine, create a local branch tracking the development branch on this machine, do changes, push to origin.
Following set of commands would be useful at a very basic level:
git clone git#repository_path_on_network/folder_name
git branch usrname_activity_dev --track origin/branch_in_repository
git checkout usrname_activity_dev
git add filenames
git commit -m "comments"
git push origin usrname_activity_dev:branch_in_repository
git checkout local_production_branch
git rebase usrname_activity_dev
git push origin local_production_branch:production_branch_on_repo
git tag tag_name
git push origin local_production_branch:production_branch_on_repo --tags
These are very basic commands. I would suggest you to further go through the internet. You will find a lots of commands according to every situation you are in.
Take a look at Git flow. Python scripts to help apply this are available here.
Where I work, we work (mostly) in pairs. We have seen the need for version control, and we will be using bazaar as our distributed version control system, due to its apparent flexibility.
After some experimentation, we have agreed that in order to set up a project, we should use the following steps:
On Server
bzr init (initializes the project)
bzr add (tells bzr to track all files in current directory, so please make sure you do not have unnecessary files in your project skeleton before you run this command)
bzr commit -m "initial commit" (commits the added files to bzr for version control)
On Development Machine
On your local machine, do a bzr branch project_dir
Daily routine
We are currently trying to establish a workflow that will work for us. This is what we have agreed to do daily:
Pull down latest changes from pull_path
Code and commit. NB. Your commits will be saved on your local machine.
See step 1.
Push your changes to push_path (NB. push_path = pull_path)
If there is any conflict:
Try bzr resolve first.
If that fails, get your partner and do a manual resolve (open file.OTHER, file.BASE and file.THIS and make relevant changes).
Commit your changes (bzr commit)
Push again (bzr push)
Repeat the above sub-points (#5) until all conflicts are resolved.
In terms of the workflow, is this the right way to do version control with bazaar? We have encountered problems where our commit comments 'change ownership' everytime the other team member pushes changes to the server. I'm pretty sure this is not how it's supposed to work, but it may also be due to certain options selected during the project setup phase.
As the VCS evangelist here, I am working on a guide to be used by the team, and particularly by new people as the team grows, and it would be great to have a 'proper' set of steps to follow in getting work done. Your contributions in establishing a nice and simple step-by-step flow to get the best out of bzr would be greatly appreciated. Please add your contributions here.
Thank you all in advance :)
What operating system(s) do you run on the server and development machines? And file systems? Windows file systems' permissions and sometimes the owner / group sometimes differ from the same files on a unix file system. That might be the first stumbling block.
Bazaar workflow:
Run a main tree on the repo server, and do a checkout locally:
bzr checkout sftp://path/to/repo/project /var/source/project
Branch the checkout locally / to your dev environment:
bzr branch sftp://path/to/repo /var/www/project
Don't work on the checkout, only work on the dev branch. Work and commit there, using the various bzr commands.
Once a work module / bug fix / task is finished, merge (not push) into the main repo:
//In /var/source/project
bzr merge /var/www/project
//Resolve any conflicts
bzr resolve
//Commit the merge
bzr commit -m "Work module | task | bug fixed"
Because /var/source/project is a checkout, the repo on the repo server will be updated automatically. This enables two or more developers to work on the same project concurrently, without needing to push and pull the whole time.
I'm not sure how your commit message changes ownership, if you do a merge and commit then the new commit is under the name of the guy who did the merge, but the original commits are still tracked. See bzr log -n0
We are in the process of converting from CVS to Mercurial hg.
Our infrastructure is Windows 2003/IIS6
Each developer develop on their machine
1xDevelopement server
1xStaging server
Production environment
Here's what I have done so far:
Installed Mercurial on my machine, on the development server and on the staging server.
Created a repository on the dev. server.
Imported our CVS repository in there.
Cloned that repository to my local machine.
Cloned that repository to our staging server.
For development in the past we always shared 1 branch, not ideal but merging was such a pain that we never bothered and dealt with it.
Now if I understand correctly, we should be doing this:
Local:
hg branch myfeature1 //Start work on feature1
Urgent bugfix required, using the affecting the SAME files as our feature
hg update default
hg branch bugfix1 //fix bug
hg commit --rev bugfix1 //done fixing bug we commit
hg push --rev bugfix1 -f //-f seems odd here, forcing to create a new branch
hg update feature1 //we return to work on feature1
hg commit --rev feature1 //done work commit
hg push --rev feature1
DEV
hg branches //we see the bugfix and feature1
hg merge --rev bugfix1
hg commit //commiting bugfix1
hg merge --rev feature1
hg commit //commiting feature1 //Dev master is now our main trunk ready for a new developer containing the feature1 and bugfix1.
Staging (The QA needs to signoff on that important bugfix before testing feature1
hg incoming //we see the new stuff
hg pull --rev bugfix1
hg merge --rev bugfix1
hg commit
//QA test and signoff bugfix1 we clone to a production repo and sync.
//QA can now test feature1 which we finished some days after bugfix1
hg pull --rev feature1
hg merge --rev feature1
hg commit //commiting merge feature1
//QA test feature1 and signoff
We clone to a production repo and sync.
Does this way make sense? Am I complicating things and will it bite me in the ass later?
It looks like you've got the concepts down great, but you've got some oddities and some questions in there, I'll hit them as a list below:
commit doesn't take a --rev option. It commits the current working directory as a new changeset whose parent (or parents if it's a merge) is whatever hg parents returns, which is always whatever you last hg updateed to.
your hg push --rev bugfix1 -f won't require a -f in very new (1.5+) versions of mercurial. Historically the warning "you're creating a new head" usually meant you forgot to merge, but now if the new head is a new named branch the warning is suppressed.
If I were your person doing the emergency bug fix on my local machine I'd just create a new clone and do the branch and fix in there. If you set up your webserver/webapp config to support that you can allow new instances at new path locations easily/automatically
In your staging environment, I recognize the desire to have them test the bugfix independent of the feature and that's a good idea, your QA team shouldn't be merging. Make/let the developers merge and have the QA team pull the merge revisions into their working area. For example, the developer changeset created in DEV step 3 already provides the proper integration of the bugfix and what-used-to-be, so have the QA team pull that revision, which will be on the 'default' branch. Similarly, after QA has approved the bugfix and is ready to move on to the feature, have them pull from dev the changeset created in dev step 5.
Remember, merging is coding too -- the person doing the merge is making choices about what should and shouldn't be. The QA people might be capable of it, but it's the developer's job. Also, why do it twice? The usual handoff for this is something like "QA, pull revision 897a9d9f9a7 and test please -- the developers". If you want to get fancy you can have a tag like 'readyforQA' that the developers move along the 'default' branch as they go (in this example they'd hg tag after their steps 3 and 5 and let QA know there's new stuff to pull.
The one piece of advice I'd give you is don't try to over-engineer the process. DVCSs lead to a sort-of haphazard way of working, that's a little scary at first, but tends to work out. YOu'll find sub-teams and pairs of folks have clones you never knew about and in the end so long as you have a few firm rules like "nothing goes to production without first passing through QA" the rest sort of works itself out.
I'm just starting out with bazaar, and I've found that the checkout feature is the most useful for the way I work - namely I can c/o from a "master copy", do some development and then commit my changes in the new directory. This then updates the "master copy".
But what if I'm working on (eg) two projects, changing different portions of code? Say:
~/master - master copy
bzr co master ./gui
bzr co master ./engine
So I'm doing gui-related stuff in the ./gui directory and under-the-hood stuff in ./engine. How should I commit my changes? If I commit gui first, then engine, I guess any conflicts will be flagged in engine?
Is there a way to merge gui and engine, and then do just one commit to the master copy?
To make things a little more complicated, how about if I do this:
bzr branch gui ./mouse
Now I perhaps I've been working on mouse, but also on gui. If I want to merge the code from gui AND mouse, and then commit to master, what is the best way to manage this? Or indeed, if I also:
bzr branch gui ./keyboard
If I've changed altered gui, keyboard and mouse, should I hierarchically merge - ie mouse+keyboard, then merge this with gui, then commit gui to master?
I hope it is clear what I'm trying to achieve! Thanks in advance for your time.
If you have two checkouts, any time you commit changes to one, you will first have to pull down any changes from the other one, potentially having to resolve conflicts at each step. This is generally a good idea, since it's easier to resolve conflicts over time and make sure your code doesn't diverge too much.
However, it sounds like you want to have separate developers working on "gui" and "engine", or you just want to save your conflict resolution till development on both branches has completed. In this case, you should probably create them as independent branches with "bzr branch". Each branch can use local commits and not worry about conflicts with each other. Then when it comes time to merge you can do it one of 3 ways, all of which get the same end result:
1. Merge one branch into the other, then push it up to master:
cd gui
bzr merge ../engine
# manually fix any conflicts
bzr commit
bzr push #back up to main
The downside to the above method is that your "gui" branch now has the "engine" changes in it. Which is fine if you're going to throw away both branches once they're pushed back into the mainline. But if you want to keep the branches longer, you can:
2. Merge into the mainline:
cd master
bzr merge ../gui
bzr commit
bzr merge ../engine
# manually fix conflicts
bzr commit
This has the upside that you still have "gui" and "engine" as separate branches, but you've had to commit one to master before you were sure that they would both work together. So you really probably want to:
3. Create a merge branch:
bzr branch ~/master gui-engine-merge
cd gui-engine-merge
bzr merge ../gui
bzr commit
bzr merge ../engine
# manually fix conflicts
bzr commit
bzr push ~/master
# since this branch was only for merging, you don't need it anymore:
cd ..
rm -r gui-engine-merge
Yes, bzr should prevent you from checking in changes from the engine repo if it detects conflicts. Normally, you first do "bzr up" just prior to check-in and then make sure your stuff plays nice with others.
As for the second part of your question, dealing with mouse/keyboard branches, this is how I would normally do it. Simply cd into the gui dir, and then do:
bzr merge ../mouse
After merging the changes, you can then commit from the gui directory and it will send the changeset to the "master" directory.
Note that I'm hardly a bzr expert, but this is the way I've been dealing with SVN repos.