Our team is looking to migrate our perforce server over to Git. Is there a way to sync check-ins from a branch in our Github Server back to Perforce to keep them in sync? I have been looking at git-p4 and there seems to be lots of documentation of how to sync Perforce -> Git but not the other way around. I would ideally like to have it syncing both ways perforce <-> git, is that possible with p4-git?
Perforce GitFusion can do this, but the developer would have to push changes to the GitFusion server instead of the github server.
For most of the time, merge conflicts were sorted out automatically. Perforce was the "master". People submitting from Perforce always had their change go straight through. People submitting from git would have their change submitted via this process:
lock master git repo
fetch upstream p4 changes
rebase against upstream p4
submit to p4 if that all went ok
That still left a very small window between (3) and (4) where someone in Perforce land could submit a conflicting change to the same files. But in practice that only happened a couple of times, over the course of several years of probably hundreds of commits per week.
In those two cases I went in and manually patched things up which was a bit clunky. I think you could just automatically discard the problematic change quite easily, but in the end, switching to git entirely made this problem go away.
Git-p4 is designed such that the git repository is initialized with data imported from Perforce. After this initial import bidirectional communication between git and Perforce repositories is fully supported, with the exception of branches/merges that have limited support.
To import updates from Perforce to git:
git p4 sync
To submit changes from git to Perforce:
git p4 submit
For more details regarding git-p4 configuration please consult its documentation.
Update: I would always advise to test any flows in temporary repositories before deploying.
In the past I setup something like this. I had a gitolite repo and a p4 server. Changes pushed to the gitolite repo would be picked up by a cron job and turned into P4 commits (via git-p4 submit). Similarly, changes submitted to p4 would be picked up by the same cron job and synced back to git (via git p4 rebase).
There were two git repos involved: the gitolite repo that normal git developers committed to, and a separate git-p4 based repo where the git-p4 operations took place.
I had a fairly small shell script to co-ordinate everything. The main tricky areas were:
locking: you have to forcibly rebase the gitolite repo so you need some way to ensure developers don't lose changes when this happens (I used a lock file).
Merge conflicts. Very occasionally two people would edit the same section of a file in P4 and git at the same time.
I have faced this problem myself and as far as I know there's nothing of the kind.
In my case I have to basically:
Create a patch for the changes in Git: git diff sha1..sha2 > mypatch.diff
Make a list of the affected files: git diff --name-only sha1..sha2 > files.list
Apply the patch in the P4 repo: git apply mypatch.diff
Reconcile changes in P4: for each file in files.list; p4 reconcile $file; done
Edit your P4 change list and then submit it (I'm sure this can be automated too but I haven't had the need for it).
I have a couple of scripts that help me with this, you can find them here: https://github.com/pgpbpadilla/git-p4-helpers#sharing-changes-git-p4
I have been using this workflow for ~6mo and it works for most cases.
Caveats
This creates a single change lists for all commits in the sha range (a..b).
Related
Summary:
We used to be on SVN and working with offshore team building our automation.
We periodically merged to keep us in sync(local and offshore team). Like every other day or so.
Local team switched to GIT while offshore continued their work on SVN. Security policies, still flushing out new git processes and such...
We have 2 different projects now that are off sync.
Question:
Now that we want them to move to GIT so that we can sync up I can think of couple of options once their code is in GIT:
Manual merge (its been a month, but I think I can do it). Manually copy files over, build fix etc.
Have offshore team rename their files a little so that GIT thinks its a new file and then manually resolve some duplicates perhaps.
There shouldn't be many
Any advise would be much appreciated.
The manual merge should be a straightforward solution:
Compare and merge two different folders, one with your Git repo, one with the latest SVN HEAD revision.
Once the merge is done, you can send them back a Git bundle (one file), which they can clone and start working from, within their own Git cloned repo.
I have project A. The project A is in perforce as well as in github( Business decision). They are both out of sync. I see that I can use git-p4 tool http://git-scm.com/docs/git-p4 to submit git changes to perforce. Problem is first instruction of it says is:
git p4 clone //depot/path/project
This command will create a new repo in git hub. I do not want that. I want my existing perforce depot to connect to exiting github repo and then sync files from github to perforce. Is it possible?
No. git p4 is basically just a wrapper around Perforce. It reads changes to a Git repo and issues the proper Perforce commands to commit them (and vice-versa -- it reads changes to a Perforce checkout and issues the proper Git commands to update a Git repo accordingly). So you need to have a Perforce checkout and a local Git repo that can talk to that Perforce checkout.
Fundamentally you have two different histories for the same project, so you need to find some way to reconcile them.
git-p4 can sync from Perforce (create a new git repository), then you can add a new remote pointing to github, and sync the branches from there.
However that doesn't help with the problem of merging the histories.
You can either:
Go with Perforce, and rewrite github history
Go with Github history, and get your perforce admin/Perforce (company) to rewrite your Perforce history.
2 is probably (politically) impossible. May be impossible technically. Perforce is designed to prevent history from being altered.
1 will break anyone cloned from your github repository.
You may want to take a look at Git Fusion, which is a bridge between Git and Perforce:
http://www.perforce.com/git-fusion
This KB article gives an example of how you may wish to work with a project that is both in Perforce and a public Git repo:
http://answers.perforce.com/articles/KB/7481/
Hope this helps!
I often see the below errors on doing git pull on totally untouched files which I am NOT working on.
Pull is not possible because you have unmerged files
It is clear the conflicted files have changed in the git repo. But I don't understand why git pull cannot over-write on these untouched files - which I've not touched?
What can I or my team do to avoid getting these errors?
Edited -
Along with the solution I want to understand why the errors are coming.
Just to make clear, the other team members are working on other files (say xyz). And I am working on a separate set of files having no dependency on xyz files. So if I pull the repo changes after a long time with no changes from my side in xyz, why the conflicts in those files?
use git diff to see problem files
look at this git cheat shets for usefull commands
http://www.cheat-sheets.org/saved-copy/git-cheat-sheet.pdf
http://jan-krueger.net/wordpress/wp-content/uploads/2007/09/git-cheat-sheet-v2-back.svg
There are some tips from my own experience. i'm not sure whether they're 100% corerect)
Split work with your team on paralel threads. Try not to work in the same files.
Try to avoid situations when 2 or more persons are adding new files simalteniously. When one added new files others should make pull as soon as possible
the last but not least - make git push very often. this will keep your project git up to date
Since git pull does not pull individual files, it's git merge phase will stop for manual correction of any merge conflicts. git merge and/or git pull (and by nature of the fact that git pull is essentially git fetch followed by git merge) is an all-or-nothing operation - you either successfully merge the changes introduced on both (or all) of the branches you are merging together, or you don't merge any of them. The catch in that is when conflicts arise that must be manually resolved - but in that situation, you are in an in-between state, having neither completed and committed the merge nor rolled it back to your previous state.
The message you are getting implies that you have previously done a git pull or a git merge, which stopped in the middle, requesting that you manually resolve some conflicts, which you have not yet done, but have rather continued on doing other stuff, and are now trying to do another git pull / git merge without ever having completed the first one.
Take a look at git status, and follow the suggested directions for either resolving your in-progress merge issues or resetting back to a not-in-the-middle-of-a-merge state.
There is a project on GitHub that I have forked. My workflow involves cloning my fork to my local machine.
Occassionally I commit my changes to my local branch. And eventually I push those changes to my fork on GitHub. I do this so I can work on my project at work, push my changes, then go home, pull the latest from my fork and continue working.
At this point, on my branch, I have several commits. I want to link my changes to some people for code reviews, but when I view an individual commit, it's diff is only compared to the previous commit from me. I don't want my peers to have to sift through all my commits trying to understand what changed from the beginning.
Is it possible to specify exactly which commit you are diffing against? In my case, I want to diff my latest commit against the latest commit in the origin branch.
If not, is there a way to rebase a fork on GitHub so that it combines my selected commits into one commit?
In most git based code reviews, it's normal practice to go through and review every commit and comment separately for each one. This is normally to help enforce good coding practices in terms of commit size and scope and testability.
In the command line, almost all commands support a revision interval as an argument
git diff <some commit>..<some other commit>
git log <some commit>..<some other commit>
and that will give you the results for changes solely within that range
also there is a difference between
git diff <some commit>..<some other commit>
and
git diff <some commit>...<some other commit>
as the third dot means inclusive, also showing results for which defaults to HEAD
In the github web client I don't think you can do this, I just checked their native client and it's a no go as well.
I did just look at Atlassian/bitbucket's SourceTree and that allows you to shift-select a range of commits as you're expecting
Where I work, we work (mostly) in pairs. We have seen the need for version control, and we will be using bazaar as our distributed version control system, due to its apparent flexibility.
After some experimentation, we have agreed that in order to set up a project, we should use the following steps:
On Server
bzr init (initializes the project)
bzr add (tells bzr to track all files in current directory, so please make sure you do not have unnecessary files in your project skeleton before you run this command)
bzr commit -m "initial commit" (commits the added files to bzr for version control)
On Development Machine
On your local machine, do a bzr branch project_dir
Daily routine
We are currently trying to establish a workflow that will work for us. This is what we have agreed to do daily:
Pull down latest changes from pull_path
Code and commit. NB. Your commits will be saved on your local machine.
See step 1.
Push your changes to push_path (NB. push_path = pull_path)
If there is any conflict:
Try bzr resolve first.
If that fails, get your partner and do a manual resolve (open file.OTHER, file.BASE and file.THIS and make relevant changes).
Commit your changes (bzr commit)
Push again (bzr push)
Repeat the above sub-points (#5) until all conflicts are resolved.
In terms of the workflow, is this the right way to do version control with bazaar? We have encountered problems where our commit comments 'change ownership' everytime the other team member pushes changes to the server. I'm pretty sure this is not how it's supposed to work, but it may also be due to certain options selected during the project setup phase.
As the VCS evangelist here, I am working on a guide to be used by the team, and particularly by new people as the team grows, and it would be great to have a 'proper' set of steps to follow in getting work done. Your contributions in establishing a nice and simple step-by-step flow to get the best out of bzr would be greatly appreciated. Please add your contributions here.
Thank you all in advance :)
What operating system(s) do you run on the server and development machines? And file systems? Windows file systems' permissions and sometimes the owner / group sometimes differ from the same files on a unix file system. That might be the first stumbling block.
Bazaar workflow:
Run a main tree on the repo server, and do a checkout locally:
bzr checkout sftp://path/to/repo/project /var/source/project
Branch the checkout locally / to your dev environment:
bzr branch sftp://path/to/repo /var/www/project
Don't work on the checkout, only work on the dev branch. Work and commit there, using the various bzr commands.
Once a work module / bug fix / task is finished, merge (not push) into the main repo:
//In /var/source/project
bzr merge /var/www/project
//Resolve any conflicts
bzr resolve
//Commit the merge
bzr commit -m "Work module | task | bug fixed"
Because /var/source/project is a checkout, the repo on the repo server will be updated automatically. This enables two or more developers to work on the same project concurrently, without needing to push and pull the whole time.
I'm not sure how your commit message changes ownership, if you do a merge and commit then the new commit is under the name of the guy who did the merge, but the original commits are still tracked. See bzr log -n0