Team Foundation Server - Moving Source with History - version-control

I was wondering what the best approach might be in moving source code, with history, from one Team Project to another Team Project. I am not concerned with work items, reporting, or SharePoint sites, as the system we are going to be restoring from did not use these functionalities. The reason for wanting to move to a different Team Project also is driven by the fact that the original implementation (being restored from a backup that was maintained by a third party) were using a third-party process template that we do not wish to use going forward. We want to start utilizing work item tracking and reporting after the migration is complete.
The TFS Integration Platform seems to be one likely scenario. It can be used to change the process template, according to the documentation. However, I was curious if the tf.exe move syntax might work? Something like:
tf.exe move $/ProjectA $/ProjectB
It is my understanding that this command operates much like a rename operation, whereas moving with the "Move" context menu item in Source Control Explorer is more like a delete and add operation. Also, would the tf.exe move path actually associate the code under the folders with the appropriate Team Project, assuming that $/ProjectA is the root source control folder for one project and $/ProjectB is the root source control folder for the other? The key is to be able to preserve the history, if possible.
Any advice or tips would be greatly appreciated!
Edit - Could branching to another project handle this scenario - much like Microsoft discusses in the Branching Guidance documentation? I think that this could be the answer, since the history would likely be preserved with the branch. However, I do not have access to a Team Foundation Server 2008 instance at the moment to test it.

Move and Rename are aliases. There is absolutely no difference, in any version of TFS, from the command line or the UI.
Both of them preserve history. At least in 2005/2008, you keep the same physical item in the VersionedItem table no matter how often or how drastically the name and/or parent path changes. There is actually no way to get a "fake" rename (delete + add) without a lot of manual work on your part.
However, while this versioning model is very pure in a theoretical sense, it has some practical gotchas. Because different items can occupy the same name at different points in time, TFS needs the full name + version to uniquely identify any inputs you send it. Normally you don't notice this restriction, but once have you renamed items in the system, if you say tf [doSomething] $/newname -version:oldversion then it will get confused and either throw an error or operate on an item you may not have intended. You have to be careful to pass valid combinations (newname+newversion or oldname+oldversion) to ensure commands behave the way you want.
TFS 2010 changes the story somewhat: it's a branch+delete under the covers, causing the itemID to change. Even so, everyday commands like Get and History are "faked" very well; old clients are about 95% compatible. The advantage is that when you have multiple renames in the system and path-based item lookups start to become ambiguous as alluded to above, the server will simply accept the name you specify and run with it. This improves overall system performance and eliminates several traps that unfamiliar users often fell into, at the cost of not being quite as flexible and not preserving history with 100% precision (eg when there are name collisions during a Merge of two branches).
Returning to the problem at hand...
It's not as simple as saying tf rename $/projectA $/projectB. Top level folders in the source control tree are reserved for the Team Project Creation Wizard; you can't run standard tf commands against them. What you need is a script like:
Get-TfsChildItem $/ProjectA |
select -Skip 1 | # skip the root dir
foreach {
tf rename $_.serveritem $_.serveritem.replace("$/ProjectA", "$/ProjectB")
}
[of course, you can do it by hand if there aren't too many children under $/ProjectA]
As far as the gotchas I mentioned, I'll elaborate on one right now since looking up old history seems very important to you. Once you checkin the rename, tf history $/ProjectA/somefile.cs will NOT work. By default, tf commands assume version = "latest." Any of these alternatives will the full history you want:
tf history $/ProjectA/somefile.cs;1234 where changeset 1234 was before the move
tf history $/ProjectB/somefile.cs;5678 where changeset 5678 was after the move. Or you could just omit the version.
A final alternative for completeness & debugging purposes:
tf history $/ProjectA/somefile.cs -slotmode. You will only see the changes that happened prior to the move; however you'll also see the history of any other items that may have lived in the $/ProjectA/somefile.cs "slot" prior to or subsequent to the item you moved underneath B.
(In TFS 2010, "slot mode" is the default behavior; there's an -ItemMode option to request that your lookup be traced across history like it was 2008 rather than path-based.)
EDIT - no, branching is not a great alternative. While branching does leave enough metadata in the system to trace the full history to & from ProjectB, it's not terribly user friendly in 2008. Plan to spend a lot of time learning the tf merges command (no UI equivalent). 2010 dramatically improves your ability to visualize changes across multiple branches, but it's still not the clean unified experience you'd get from a Rename.

Richard's answer above is well written and explains the situation well. I did have a couple more practical gotcha to add, however.
In TFS2010, the default behavior makes it seem like moving a file causes you to lose all history from before the move. The command my users are likely to use (and the one used, it seems, by the VS2010 GUI) is:
tf history $/ProjectB/somefile.cs
My users intend to get all the history of somefile.cs, both before and after the move. They want "the history of the code that is currently stored in $/ProjectB/somefile.cs", regardless of the filename at any point in time. Maybe other people see it differently.
The first gotcha is that the GUI that appears for me in VS2010 using TFS2010 initally shows only the history since the move. The least-recent item in the list is the rename operation. It can be expanded with a subtle little drop-down arrow. Underneath is the history from the previous location. If you don't know to look for this, it can look like your history is gone.
The second gotcha is that if you later delete ProjectA (because you've finished the migration to ProjectB, say), the history really is gone. Expanding the drop-down in the history for $/ProjectB/somefile.cs doesn't produce the older history.

Another option (and I think easier) is to import to Git then export back to TFS using the Git-TF command line tools.
Install Git-TF from binary, Choclatey or source code.
Clone a TFS folder:
git tf clone https://myAcc.visualstudio.com/mycollection $/TeamProjectA/Main --deep
Disassociate the Git Repo from the TFS server by deleting the .Git/tf folder and the .Git/git-tf file.
Configure the new Git Repo to connect to an empty TFS folder.
git tf configure https://myAcc.visualstudio.com/mycollection $/TeamProjectB/Main --deep
Don't forget the --deep
git tf pull
You should get a message at this point "git-tf: this is a newly configured repository. There is nothing to fetch from tfs."
git commit -a -m "merge commit"
git tf checkin --deep

Regarding the original command above:-
Get-TfsChildItem $/ProjectA |
select -Skip 1 | # skip the root dir
foreach {
tf rename $_.serveritem $_.serveritem.replace("$/ProjectA", "$/ProjectB")
}
Both the source and target names need to be surrounded by quotes in case there are any spaces in the full pathfilename. I found it difficult to do this on $_.ServerItem as surrounding it with escaped " returns that whole child object and not just the .serverItem string. Or if I did manage to get the string, I got unwanted carriage returns such as
"
$proj/folder/file
"
Eventually I got the command to work with the following, but I found that the history still doesn't get transferred, which was the whole point! So I believe this command is the direct equivalent of just using a right mouse click in source explorer and selecting rename (or move).
$tfsServerString = "http://machine:8080/tfs/DefaultCollection"
$tfs = Get-TfsServer $tfsServerString
Get-TfsChildItem -server $tfs "$/Dest Project/MyTestProject" | select -Skip 1 | foreach { $sourceName = $_.serveritem; $targetName = $_.serveritem.replace("$/Dest Project/MyTestProject/", "$/Dest Project/Source/StoreControllers/dSprint/dSprint2/") ; ./tf rename `"$sourceName`" `"$targetName`" /login:myUser }
Also note, it requires use of the backtick ` to escape "

Related

How to recover Perforce history on a moved directory

I have a branch on Perforce, where I changes the directory structure of the project using Rename/Move command.
During merging back to the mainstream, Something went wrong that caused Perforce to think of the new structure as a whole-new directories.
Subsequently, the history of the files in the new directory structure is totally unrelated to the history of the same files before changing the structure.
Is there anyway to recover this situation ? Or ask Perforce to append the old history with the new history ?
Something went wrong that caused Perforce to think of the new structure as a whole-new directories.
Usually if this happens it means someone didn't use the "rename/move" command and used some other method to rename instead (i.e. they did something that adds the new directory as a new set of files independent of the originals rather than an atomic rename of an existing set of files). It's impossible for me to say how to "recover" without seeing what the history of the files looks like now so I can reverse-engineer what the "something went wrong" was.
I'd recommend either posting on the Perforce forums or contacting Perforce technical support so that somebody with expertise can wheedle the necessary data out of you (I can intuit that this will require an amount of back and forth that stackoverflow frowns on -- "what were the branches you were merging from and to", "okay, now run THIS command to see the history of that branch and send me the output," "okay, which of these five merge operations I can see in the history is the one you're talking about,") and propose a solution.
From another answer:
So, for a file a/b/c, you can look at the by using the -i option where appropriate. For example, p4 filelog -li a/b/c.
This is not necessary if files are renamed via the "move/rename" command, so if you need to use "filelog -i" to see file history, the files were definitely renamed by some other method. (The "p4 move" command was added in 2009 so long-time Perforce users will sometimes use other workflows.)

Multiple repositories in one directory (same level) - is it possible?

My original problem is that I have a directory where I write various scripts. Each of them is independent of others, and usually one-file-long. I want to have some versioning applied to them, but I have the following problems/requirements:
I don't want to have to store each small script in a separate directory!
I don't want to store them all in one repository OTOH, as they are completely unrelated, and:
some of them may later grow to more files (and then they will need a separate dir),
I sometimes want to copy one of them to a different machine (and I want to clone the whole repo).
I want to benefit from (distributed) version control mechanisms -- at least:
"infinite" number of revisions,
ability to clone repositories on different computers,
ability to do "atomic" multi-file commits.
Is it possible?
I'd prefer to do it in some mainstream distributed VCS (a solution using Mercurial would be preferable, but I'm not fixed).
EDIT: the solution has to be free (at least "as in beer") and cross-platform (at least Win32 & Linux).
Related, but didn't help:
"two-git-repositories-in-one-directory" -- didn't find it helpful: the accepted answer looks like point 2. (above) to me; the current "community voted" answer sounds like 1.
"Version control of single files using Subversion" -- also too much of 2. or 1.
These requirements seem pretty "special" to me, so here is a solution on par with them ^^
You may use two completely different VCS, in the same directory. Even two "instances" of SVN might work: SVN stores its metadata in a directory called .SVN and has (for historical reasons regarding ASP) the option to use _SVN. The Directory listing should look like this
.SVN // Metadata for rep1
_SVN // Metadata for rep2
script1 // in rep1
script2 // in rep2
...
Of course, you will need to hide or ignore the foreign scripts or folders from each VCS...
Added:
This only accounts for two scripts in one folder and needs one additional VCS per script beyond that, so if you even consider this route and need more repositories, rename each Metadir and use a script to rename it back before updating:
MOVE .SVN-script1 .SVN
svn update
MOVE .SVN .SVN-script1
Why don't you simply create a separate branch (in the git sense) for each (group of) script(s)?
You can develop them individually as you please. Switching to a branch will show you only the scripts from that branch. It's sort of like directories but managed by the version control system. If you later want to pluck a branch out into another repository, you can do that and if you want to combine two scripts into a single project, you can do that as well. The copying them to the different machine point might be a problem but you can clone the branch you're interested in and you it should work for you.
Another proposition for my own consideration is "Using Convert to Decompose Your Repository" article on hgtip.com. It fails as a "standalone" solution, but could be helpful as an addition to the "mv .hgN .hg / MOVE .SVN-script1 .SVN" idea.
You can create multiple hidden repository directories and symlink .hg to whichever one you want to be active. So if you have two repositories, create directories for them:
.hg_production
.hg_staging
Then to activate either of them just do:
ln -sf .hg_production .hg
You could easily create a bash command to do this. So instead you could write something like activate-repo production, which would run ln -sf .hg_production .hg.
Note: Mac doesn't seem to support ln -sf so instead you'll need to do:
rm .hg; ln -s .hg_production .hg
I can only think of these two lightweight versioning systems:
1) Using Dropbox with the Pack-Rat upgrade, to keep a full history of versions for each file automatically backed up and with the possibility to be shared with multiple Dropbox users: https://www.dropbox.com/help/113
If you have multiple machines managed by the same user (you), the synching would be automatic. Also if the machines are in the same LAN, Dropbox is smart enough to sync the files over the local network, so big files shouldn't be a worry.
2) Using a 'Versions' aware text editor for Mac OS X Lion. I'd expect TextMate, Coda and other popular Mac code editors to be updated to support this feature when Lion is released.
How about a compromise between 1 and 2? Instead of a folder+repo for each script, can you bundle them into loosely related groups, such as "database", "backup", etc. and then make one folder+repo for each group? Then if you clone a repo on another machine, you're only pulling down a smaller number of unrelated files. (Is the bandwidth/drivespace really a concern?) To me, this sounds WAAAY simpler than all of the other suggestions so far.
(Technically this approach meets your requirements because (1) each script isn't in its own directory, (2) not all scripts are in the same repository, and (3) you can easily do this with any popular DVCS. :D)
UPDATE (2016): Apparently, a guy named Cosmin Apreutesei created a tool named multigit, which seems to implement what I wished for in this question! If you ever read it, thanks a lot Cosmin! I've started using your tool this year and find it awesome.
I'm starting to think of some kind of an overlay over Mercurial/git/... which would keep a couple "disabled" repository meta-directories, let's say:
.hg1/
.hg2/
.hg3/
etc., and then on hg commit FILENAME would find the particular .hgN that is linked to FILENAME, and would then temporarily:
mv .hgN .hg
hg commit FILENAME
mv .hg .hgN
The main disadvantage is that it would require me to spend some time writing the tool. Or does anybody know of some ready-made one like this? If you do, please post as a full-featured answer (not a comment), I'm more than willing to accept it.

How to join two files in a version control system

I am doing a refactoring of my C++ project containing many source files.
The current refactoring step includes joining two files (say, x.cpp and y.cpp) into a bigger one (say, xy.cpp) with some code being thrown out, and some more code added to it.
I would like to tell my version control system (Perforce, in my case) that the resulting file is based on two previous files, so in future, when i look at the revision history of xy.cpp, i also see all the changes ever done to x.cpp and y.cpp.
Perforce supports renaming files, so if y.cpp didn't exist i would know exactly what to do. Perforce also supports merging, so if i had 2 different versions of xy.cpp it could create one version from it. From this, i figure out that joining two different files is possible (not sure about it); however, i searched through some documentation on Perforce and other source control systems and didn't find anything useful.
Is what i am trying to do possible at all?
Does it have a conventional name (searching the documentation on "merging" or "joining" was unsuccessful)?
You could try integrating with baseless merges (-i on the command line). If I understand the documentation correctly (and I've never used it myself), this will force the integration of two files. You would then need to resolve the integration however you choose, resulting in something close to the file you are envisioning.
After doing this, I assume the Perforce history would show the integration from the unrelated file in it's integration history, allowing you to track back to that file when desired.
I don't think it can be done in a classic VCS.
Those versioning systems come in two flavors (slide 50+ of Getting git by Scott Chacon):
delta-based history: you take one file, and record its delta. In this case, the unit being the file, you cannot associate its history with another file.
DAG-based history: you take one content and record its patches. In this case, the file itself can vary (it can be renamed/moved at will), and it can be the result of two other contents (so it is close of what you want)... but still within the history of one file (the contents coming from different branches of its DAG).
The easy part would be this:
p4 edit x.cpp y.cpp
p4 move x.cpp xy.cpp
p4 move y.cpp xy.cpp
Then the tricky part becomes resolving the move of y.cpp and doing your refactoring. But this will tell Perforce that the files are combined.

Which Version control?

If a project has multiple people, say, A,B,C working together and they all edit a same source file.
Couple months later, they realize that what A has been doing is wrong and they want to roll back the file in such a way that only parts/functions/lines/... that A "touched" are removed and the work B and C did is still in the roll back version. In other words, the roll back version has only the work of B and C up to the time they decide to remove A's work.
Is there any version/source control software out there (free/commercial) can do that?
Thanks.
Git and a bit of scripting will do that. Probably a bit of hand work too, but you can resort commits using interactive rebase.
Most VCSs should be able to do this -- it's a reverse merge. In Subversion you would identify the revisions made by A and merge them in again, but the other way round. To oversimplify, this means turning line additions into line removals, and vice versa.
# Don't want revision 37 because A made it.
$ svn merge -r 37:36 path
http://svnbook.red-bean.com/en/1.5/svn.branchmerge.basicmerging.html#svn.branchmerge.basicmerging.undo
I use TFS and Git. But, there are a lot of free and open source version control softwares. You can find all the source control softwares here.
In Git, you would probably do something like
git revert `git rev-list --author=A`
[Note: completely untested.]
I bet it can (easily) be done with Monotone by using `mtn local kill_certs selector certname [certval]' command (see reference) which:
This command deletes certs with the given name on revisions that match the given selector. If a value is given, it restricts itself to only delete certs that also have that same value. Like kill_revision, it is a very dangerous command; it permanently and irrevocably deletes historical information from your database.
So, by using A's certificate, the above command will eliminate 'wrong work' done by him.

TFS - files ending up in wrong folder

I'm investigating this for someone else but I hope this explanation is correct:
We have a lot of files and a lot of folders in TFS source control, but two of them are these (made up names):
$/Root/Shared/...
$/Root/Solutions/...
5 files from the folder $/Root/Shared/Client/Main are now checked in, and when looking at the changeset they all says 'edit' in the change field.
But, when looking at the paths, 3 of them are checked into $/Root/Solutions/Client/Main instead of $/Root/Shared/Client/Main. The last two are at the expected location.
And it gets worse; there is no, and is not supposed to be a $/Root/Shared/Client/Main folder. When browsing source control this location does not exist. And the files are not at their original locations either, they are just gone - except for when viewing them in the changeset.
What could have happened here?? I do have the code, since I can see them in the changeset, but I don't want to lose the history by just creating them again and copying in the code.
Reading things more carefully, it sounds like you're describing files that have been renamed / moved over time.
TFS considers namespace information just as important as file contents. If you ask for an old version of $/Root, you don't just get the old version of those files, you get the old file & folder structure too, preserved exactly the way it was at that time. This design permeates the system, including the View Changeset dialog as you've seen.
The remaining question seems to be, where have my files gone? Quickest way to find out is to use a cmdlet from the Power Tools:
Get-TfsItemHistory -all .\rentest2\grand2\child2\parent\foo3.cs |
Select-TfsItem |
Format-Table -auto path, #{label="version"; expression={$_.versions[0].changesetid}}
Path version
---- -------
$/Test-ConchangoV2/rentest2/grand2/child2/parent/foo3.cs 10725
$/Test-ConchangoV2/rentest/grand2/child2/parent/foo3.cs 10148
$/Test-ConchangoV2/rentest/grand2/parent/foo3.cs 10147
$/Test-ConchangoV2/rentest/grand2/child2/foo3.cs 10146
$/Test-ConchangoV2/rentest/grand2/child/foo2.cs 10145
$/Test-ConchangoV2/rentest/grand2/parent/child/foo2.cs 10144
$/Test-ConchangoV2/rentest/grand/parent/child/foo2.cs 10143
$/Test-ConchangoV2/rentest/grand/parent/child/foo.cs 10142
I have a little GUI tool that does this and more but haven't had time to get it to a publicly usable state, unfortunately.
My guess is you have some screwy workspace mappings. Check for yourself in File -> Source Control -> Workspaces.
For best results (read: least amount of lost sleep on issues like this), create one single mapping at the highest level you need. If that's too broad, make this root mapping a "one level" mapping (using the * syntax; see docs) and then create additional recursive mappings with the same relative paths underneath it as needed.