Perforce: Find source changelist for a branch - version-control

Short version:
After branching in P4, how can I find out the "source" changelist of the branch?
Long version:
Let's say I have a main branch of my project at
//project/main/...
The latest changelist submitted here is #123, when I decide to create a branch for release 1.0 in
//project/1.0/...
From P4V, a new changelist is created (say #130), resolved and submitted.
From the CLI, it would look something like this:
p4 integrate -c 123 -o //project/main/... //project/1.0/...
p4 submit
Later, I look at the changelists under //project/1.0, and see the #130 changelist containing a lot of branched files.
How can I find out the changelist no. that this was originally branched from (that is, #123) ?

p4 changes will display a list of submitted changelists, optionally filtered to a specific path.
p4 changes //project/main/...
Change 123 ... 'Very last change.'
Change 122 ... 'Next-to-last change.'
Change 100 ... 'Only two changes to go...'
...
No surprise there, but, as you've found, p4 changes is less helpful when you integrate all those changes in a single change:
p4 changes //project/1.0/...
Change 130 ... 'Integrated everything from main.'
The trick is to use the -i option which includes any changelists integrated into the specified files.
p4 changes -i //project/1.0/...
Change 130 ... 'Integrated everything from main.'
Change 123 ... 'Very last change.'
Change 122 ... 'Next-to-last change.'
Change 100 ... 'Only two changes to go...'
...
To get exactly what you want (123) you'll need to write a script which filters the output from p4 changes -i //project/1.0/... to remove any change listed by p4 changes //project/1.0/... (and then take the most recent remaining change).
(When exploring, I frequently also find the -m max option useful. This limits changes to the 'max' most recent. This helps your output not flow offscreen when there are many changes.)

I don't know of any simple command that performs what you would like to do. If you are willing to script a little bit and the command doesn't have to execute fast you could perhaps try to script something like the following for all branched files:
Find the source file/revision for a target file.
p4 filelog //project/1.1/foo.bar#1
//project/1.1/foo.bar
... #1 change 6416 branch on 2009/07/10 by foo#bar (text) 'Release 1.1'
... ... branch from //project/main/foo.bar#1,#2
Get the change list at which the source file/revision was submitted.
p4 fstat //project/main/foo.bar#2
... depotFile //project/main/foo.bar
... headAction edit
... headType text
... headTime 1201771167
... headRev 2
... headChange 5353
... headModTime 1201770971
Repeat for all files in branch and select the highest change no (headChange above), which should be the latest change submitted to the parent before branching for that specific file. You could get a complete listing of all branched files using e.g. "p4 files //project/1.0/...#1".
(or perhaps take the easy way out and ask Perforce support)

Since none of the answers thus far provide the code for finding the source or root changelist of a branch, I thought I'd provide a one-liner to do just that. This approach is based on #Cwan's suggestion, and will print the "parent" changelist from which the branch was created. The FIRST_BRANCH_CL argument needs to be replaced with the branch creation changelist (i.e. the first changelist submitted to the new branch). As a concrete example, replacing FIRST_BRANCH_CL with 130 from the original question, this one-liner would output 123.
p4 describe -s FIRST_BRANCH_CL | perl -lne 'if(/^\.\.\. (.+#[0-9]+) .+$/) {print quotemeta $1}' | xargs p4 filelog -m1 | perl -lne 'if(/^\.\.\. \.\.\. branch from (.+#[0-9]+)/) {print quotemeta $1}' | xargs p4 fstat | perl -lne 'if(/^\.\.\. headChange (\d+)/) {$MaxCL=$1 if($1 > $MaxCL)} END {print $MaxCL}'

Short answer:
Use the Revision Graph in P4V is step back through time and investigate the integration history. Video on the Perforce website.
I have successfully used the revision graph on branches with thousands of files to track when an particular change was integrated into a branch. That is why I recommended it and linked to a training video as most people under-estimate it because they do not know how to use it.
Long answer:
... [Removed]
UPDATE: As the Revision Graph apparently is unfeasible, you can perhaps solve this using a process/policy, i.e., when you perform the integrate, add a note in the description "Branched # CL 123". We used this approach ourselves when integrating from a trunk to release lines.

If you use the history tab in p4v it will show you all changelists submitted against a branch, so look at this for
//project/1.0/...
once you have found the oldest submitted changelist, then on any one the files in that changelist view the Revision Graph for it, this will show you the branch that the file (and the rest of the files) were integrated from.
I'll see if I can come back with the p4 commands to do the same thing.

Updated Answer: I think this will work. Try this:
p4 interchanges from_branch to_branch
This will show unintegrated changes from your main branch to your release branch. I believe you can use the top changelist number minus 1 to find your source changelist. interchanges is an undocumented Perforce CLI feature. To find out more, type p4 help interchanges to find out more about this command.
Again, I think this will work. There may be some special cases where it will not, but it's my best guess to a tough and important problem.

"p4 integrated" worked for me. Look for "copy from" in the description

Related

How can I see my place in history during an 'hg rebase' conflict?

Whenever I perform an hg rebase and there are merge conflicts, it immediately pulls up an editor for me to resolve the conflict. However, it doesn't give me any information about where in the rebase process I am at. For example, if my history looks as follows:
o 12
|
o 11
|
10 o |
\ /
o 9
Performing hg rebase -s 11 -d 10 may have a conflict trying to apply either 11 or 12. It is difficult to tell at a glance just from the merge conflict where I am stopped, especially when the graph is larger than this. How can I tell where in the rebase process the conflict is?
Very recent Mercurials have two configuration options: [ui] mergemarkertemplate, and [ui] pre-merge-tool-output-template, which can be used to improve this situation a bit.
pre-merge-tool-output-template
pre-merge-tool-output-template is printed before running any external merge tool. This can be used to print something before your editor or kdiff3 pops up; note that if you use a terminal-based merge tool (such as most editors unless they're the gui version), it'll likely be hidden by the merge tool. Depending on OS and what program you're using, you may be able to hit Ctrl-Z to suspend your merge tool to see this output.
Example output:
merging path/to/file
Running merge tool for path/to/file (/usr/bin/vim):
- local (working copy): 10:2d1f533d add binary file (#2) tip default
- base (base): 6:abcd1234 some other description default
- other (merge rev): 9:1e7ad7d7 add binary file (#1) default
... vim runs here ...
See https://www.mercurial-scm.org/repo/hg/file/14589f1989e9/tests/test-merge-tools.t#l1956 for the template that produced that output, hg help config.ui.pre-merge-tool-output-template and hg help templates for more information on that.
mergemarkertemplate
mergemarkertemplate controls the conflict markers you see in your editor. Set [ui] mergemarkers=detailed and see if this is sufficient; if not, you can use [ui] mergemarkertemplate to customize it; this can also be customized on a per-merge-tool basis, so see hg help config.ui.mergemarkers, hg help config.ui.mergemarkertemplate, and hg help config.merge-tools.
Programs with customizable labels
Merge tools like kdiff3 often have the ability to customizable the labels. In the default configuration, this should be the operation-provided name for the base/local/other (in my example above, this would be base, working copy, and merge rev, respectively. I believe if you have [ui] mergemarkers=detailed or [merge-tools] kdiff3.mergemarkers=detailed, these will include additional information. See hg help config.merge-tools for more information on the per-merge-tool configuration options.
(Not an answer, exactly, but a bit long for a comment...)
When you write:
Performing hg rebase -s 11 -d 10 may have a conflict trying to apply
either 10 or 11.
do you mean to write
may have a conflict trying to apply either 11 or 12
? Because you are trying to rebase those csets to 10, so it doesn't make sense to talk about applying 10. Also, consider using the Evolve extension if you aren't already. It makes everything append-only, which is much better.
Also, test in a clone. And also, try rebasing 12 first, if possible. Anyway, Mercurial is just trying to rebase the changes from both 11 and 12, and I don't think it distinguishes between those changes. And why would you expect it to? Isn't it obvious to you which changes belong to which cset?
Also, consider configuring your merge setup for use with kdiff3, if you aren't already. It makes things much clearer to do things in a merge editor, and you can also see both sides of the merge clearly. See
https://www.mercurial-scm.org/wiki/MergeToolConfiguration and https://www.mercurial-scm.org/wiki/KDiff3
Personally, I have the following lines in my ~/.hgrc, but they've been there a long time, and I don't remember where I got them from. Also, I don't do merges much these days. But for whatever it is worth...
[merge-tools]
kdiff3.args=--auto --L1 base --L2 local --L3 other $base $local $other -o $output
kdiff3.regkey=Software\KDiff3
kdiff3.regappend=\kdiff3.exe
kdiff3.fixeol=True
kdiff3.gui=True
kdiff3.diffargs=--L1 '$plabel1' --L2 '$clabel' $parent $child
Hope that helps.

Perforce: How can I get files "as of" a date or revision from before a branch action?

Consider the following revisions of a file:
branch -1---2---
/
/
main 100--101-102--103---104---
I am currently in my branch and the file is synced to revision 2. I know that p4 has kept the history of the file, because I used merge and not copy, and indeed the history shows revisions previous to 103 (inclusive) when I look at the history of the file and check the option "Follow branch actions".
I would like to roll it back to revision 103 or before. Is this possible?
Answers for p4v preferred, but I'll take command line statements if that's the only way to do it.
The copy command is the best way of doing this:
p4 copy main#103 branch
Another option would be to use integ -f:
p4 integ -f main#103 branch
p4 resolve -at
One benefit of copy is that if you're doing this across a bunch of files (e.g. main/...#2018/01/01 to get all of main as of Jan 1) and some of them are already identical to the rev you're copying from because they haven't changed in that time, copy will leave them alone, which makes the result less noisy.

Get all changelists (excluding user) between "mainline" and branch

Background
I have a "mainline" depot and a "beta" branch in Perforce (using simple branches: no streams, etc).
I would like to merge the latest code from "mainline" to my "beta" branch. I have to do this about once every day, and there are about 100+ commits/submissions to the "mainline branch" every day.
Normally, I would do something like so:
p4 integ //prod/mainline/... //prod/beta/...
cd $(p4 where //prod/beta/... | cut -d ' ' -f3 | sed 's/\.\.\.$//g')
p4 resolve ./...
Problem
However, we have an annoying hourly build process that updates version numbers in various Makefiles, build scripts, etc; that updates version/branch numbers, and is checked in to Perforce using a "dummy" account (i.e. dummy_user) by our build servers. This is done across all branches, causing any merge operation to have arbitrary conflicts.
This version number submission now prevents my p4 integ/p4 resolve operation from completing cleanly, and I have to hand-merge all of these files affected by the "version number update operation". I would like to just have to hand-merge actual code changes, rather than this version number nonsense.
Question
Is there a way to p4 integ a set of change-lists not yet present in the branch (but present in main/another-branch), excluding a user? I could always do something like:
for i in $(p4 changes //prod/mainline/... | grep -v dummy_user | cut -d ' ' -f2)
do
p4 integ //prod/mainline/...#${i},${i} //prod/beta/...
done
However, I don't have an automated way to get a list of all change-lists that:
Exist in mainline...
But haven't yet been merged/integrated into beta...
And aren't authored by dummy_user.
How can I accomplish this?
Sounds like you've found a solution that works for you, but FWIW here's what I'd do:
0) Make a branchspec since that makes this next part easier:
p4 --field "View=//prod/mainline/... //prod/beta/..." branch -o prod-main-beta | p4 branch -i
1) Ignore the robo-commits (I assume you want to just leave these things alone):
p4 -Ztag -F #=%change% ichanges -u dummy_user -b prod-main-beta | p4 -x - integ -b prod-main-beta
p4 resolve -ay
p4 submit -d "Begone, robo-cruft!"
2) Do the "real" integrate. If you find yourself still having to pick around the "dummy" changes, try the -Rs option -- that might give you more merges to do but it will bend over backwards to make sure those merges don't include anything you've already integrated.
p4 integ -b prod-main-beta [-Rs]
p4 resolve [-am]
p4 submit
Optional 3) Improve your build tooling so that the version info is located in a dedicated file, and other build files link to it. Then you can exclude your version file from your branch specs, or just always ignore changes to it without having to cherry-pick, or whatever. Here's a real-world example: https://swarm.workshop.perforce.com/files/guest/perforce_software/p4/2015-1/Version

Listing perforce changes in one branch, but not another; when there are deletions

When I want to compute a list of changes in one branch but not an older branch, I do something like this:
diff -u \
<( p4 changes -s submitted -i #OLD_BRANCH_LABEL ) \
<( p4 changes -s submitted -i #NEW_BRANCH_LABEL ) | grep ^+'
NOTE: These are automatic labels containing a view and a revision, but I can also put in the view paths #REVISION
which gives a good answer when both labels are actually in the same view, and good answers generally.
But I find that I get some poor results in this case:
If a commit #A adds a file to MAIN_BRANCH, and commit #D removes the file (and makes some other change), followed by commit #F which forks a NEW_BRANCH, I find that the MAIN_BRANCH contains both commits #A and #D, but NEW_BRANCH contains commits #D and #F
The NEW_BRANCH does not contain commit #A
So my p4 changes recipe above insists that the mainline contains #A which is not in the branch, or in other words was made since the branch, even though it is not in the mainline any more than it is in the branch.
An obvious but unwieldy fix would be to take another fork of MAIN_BRANCH at the point I want to compare, knowing that #A would be excluded again in the same way.
If this were git, I would use git-merge-base or something to find a common point and remove that from the changes of both branches, but perforce branching is too flexible, being just another integration.
Are there any ways that I can convince perforce that NEW_BRANCH does contain #A? (for the branch fork #F occurred after #A was committed).
Or to get perforce to ignore changes whose files are entirely deleted, that WOULD be ignored if a branch were made?
The command p4 interchanges does exactly what you are after, if I read your question correctly.
'p4 interchanges' lists changes that have not been integrated from
a set of source files to a set of target files.
Have a look at p4 help interchanges for the full description. The command does take indirect integrations into account, too.

mercurial hg partial checkin

It is a very simple and stupid question,
I am working on 2 tasks and modified 2 sets of files in the code.
Now when I type 'hg ci', it checks in all the files. Can I remove certain files from the checkin i.e. do checking for only one task?
If I remove the files in the 'check in message' will they be removed from the checkin
Thanks for all the answers
This seems like a big flaw, My use case is very simple and general. Most of the time dev are working on various tasks , some are ready , some are not, bug fixes etc.
Now the only simple solution seems to be multiple local repos or clones
Use hg ci <files>... to commit only certain files in your working directory.
If you want to pick file by file you can use the record command. It ships with mercurial, but you have to turn it on if you want to use it by putting: record= in the [extensions] section of your ~/.hgrc.
It goes hunk by hunk, but you can answer for a whole file:
y - record this change
n - skip this change
s - skip remaining changes to this file
f - record remaining changes to this file
d - done, skip remaining changes and files
a - record all changes to all remaining files
q - quit, recording no changes
? - display help
I'll point out that if you're committing some files but not others it's certain that you've not run your test suite on the one change without the other, but maybe that doesn't apply in your case.
This isn't possible with mercurial out of the box. As have been suggested there are several ways of selecting what files you want to commit. To get this function via the commit message template, you would need an extension or a shell script wrapping the commit command. Here's one way to do that:
[alias]
ci = ! hg-partial-commit
hg-partial-commit:
#!/bin/sh
# partial commit
edit=$(mktemp ${TMPDIR:-/tmp}/$(basename $0).XXXXXXXXXXXX)
filelist=$(mktemp ${TMPDIR:-/tmp}/$(basename $0).XXXXXXXXXXXX)
logmessage=$(mktemp ${TMPDIR:-/tmp}/$(basename $0).XXXXXXXXXXXX)
cleanup="rm -f '$edit' '$filelist' '$logmessage'"
trap "$cleanup" 0 1 2 3 15
(
echo user: $(hg debugconfig ui.username)
echo branch: $(hg branch)
hg parents --template 'parent: {rev}:{node|short} {author} {date|isodate}\n'
echo
echo 'Enter commit message. Select files to commit by deleting lines:'
hg status 'set:not unknown()' | sed -e 's/^/#/'
) | sed -e 's/^/HG: /' >"$edit"
${VISUAL:-${EDITOR:-vi}} "$edit"
egrep -v '^HG:' "$edit" >"$logmessage"
egrep '^HG: #' "$edit" | cut -c8- >"$filelist"
hg commit -l "$logmessage" "listfile:$filelist"
$cleanup
The real problem here is the fact that you're doing changes related to different tasks jumbled together. Mercurial has a few ways you can keep things separate.
Task Branches
Suppose you've been working on a task and you've checked in a few times since you last pulled, but things aren't ready to share yet.
o----o----B----o----o----o
Here, B is the revision where you started your changes. What we do is (after making sure our current changes are checked in):
> hg update -r B
<do our work on the other task>
> hg commit
We've now created a new branch with the changes for this task separated from the changes for our original task.
o----o----B----o----o----o
\
----o
We can do this for as many different tasks as we want. The only problem is that sometimes remembering which branch is which can be awkward. This is where features like bookmarks come in useful. A bookmark is a tag which moves forward as commits are made so that it always points at the head of a branch.
Mercurial Queues
MQ adds the ability to work on changes incrementally and move between them by pushing and poping then off a stack (or "Queue" I guess). So if I had a set of uncommitted changes that I needed to split up I'd:
> hg qrecord taska
> hg qrecord taskb
> hg qrecord taskc
I'd use the record extension (or more likely the crecord extension) to select which parts of files I want to select.
If I needed to go back to taska and make some changes:
> hg qpop; hg qpop # pop two off the queue to go back to task a
<Do changes>
> hg qrefresh # update task a with the new changes
When I want to turn the queue into normal changesets:
> hg qpush or hg qpop # get the changes I want converted onto the queue
> hg qfinish -a # converts mq changes to normal changesets
There's other methods too, but that will do for now.
You will unavoidably have to either specify the files that you want to add or the files you want to leave out. If you have a lot of files, as you indicate above, the following steps will simplify the procedure (I'm assuming a UNIX-ish system here, Windows would be slightly different).
First, generate a list of changed files:
hg status -mard -n >/tmp/changedlist.txt
The -mard options will list all files that were modified, added, removed, or delated. The -n option will list them without the status prefix so that all you have is a raw list of files.
Second, edit /tmp/changedlist.txt with your favorite text editor so that it contains only the files you wish to commit.
Third, commit these files:
hg commit `cat /tmp/changedlist.txt`
You will be able to review the files to be committed before actually performing the commit.
Alternatively, you can add more files to a commit incrementally by using
`hg commit --amend file...`
Here, hg commit --amend will not create a new commit, but add the new files to the existing commit. So, you start out with committing just a couple of files, then incrementally adding more until you are done. This still requires you to type all of them in.
For yet another alternative, you can use Mercurial Queues to split a commit in more sophisticated ways, but that's a bit more of an advanced topic.