I am subscribed to multiple GitHub repository channels and I am constantly seeing stuff like this:
File Changes
M .circleci/config.yml (2)
M .gitignore (2)
M doc/_static/style.css (9)
M doc/conf.py (4)
M doc/documentation.rst (2)
R examples/visualization/plot_make_report.py (8)
M mne/report.py (48)
M mne/tests/test_report.py (32)
What do the Ms and Rs mean when describing these file changes?
This list is a summary of the changes that were done in a specific commit. These letters indicate what happened to the file next to them:
M - modified
A - added
D - deleted
R - renamed
and there are a few more.
These shortcuts are taken from the output for the git status --short command which you can read about more in its documentation.
Related
This should be simple, but something isn't quite right. Here is my scenario and then I'll give a brief overview of the commands I'm using. It helps to know that we have 3 specific dev areas, Live, Staging, and of course our own local dev areas.
I developed a new "beta" area of my site which has gone live and had appropriate testing. Now I'm ready to move it from a beta directory, to where it really should be and move out the old. When I do it locally, it seems fine, but when I try to merge my local branch into the staging branch, it doesn't seem to map the files correctly, and gives me a bunch of those use (c)hanged version, (d)elete, or leave (u)nresolved? prompts. The problem comes when my old directory has files that are named the same as the beta directory (like index.php for instance). Here's a quick example of what I mean:
currentDir/index.php
currentDir/update.php
currentDir/another_file.php
currentDir-beta/index.php
currentDir-beta/update.php
currentDir-beta/a_new_file.php
currentDir-beta/another_new_file.php
This is my process.
# creates a new branch from the live branch
hg branch new-branch-name
# move the current directory somewhere else
hg mv currentDir/* currentDir-old/
# commit...
hg com -m "moved current to -old"
# everything is fine up to this point
# move the beta directory to where the old one was
hg mv currentDir-beta/* currentDir/
# when I run hg st, it only shows that files are being removed from the -beta directory and added to the new/old directory
# commit
hg com -m "moved -beta to currentDir"
# when this commits is when the problems start happening.
# At this point when I run this next command, it shows that
# currentDir/index.php and other common files are now "modified" instead of "added"
hg st --rev "max(ancestors('new-branch-name') and branch(live)):'new-branch-name'"
# then try to merge to staging
hg up staging
hg merge new-branch-name
# errors happen with "common" file names like index.php. It treats them as though they were only modified instead of added.
Even if I ignored the above "modified" quirk, when I go to merge this new branch into the staging branch with other changes programmers have done, it complains that "local has this which remote deleted". I really wouldn't care with most of this as I could just throw this live and then any new branches would have this change. The thing I do care about is that any work done in the currentDir-beta folder on those "common" files from other programmers will no longer map to the new location. I can copy/paste the code and commit it, but it basically means that those branches are hosed as it pertains to keeping the changes other programmers did on those common files. To give you an example of what I mean, when I merge and type hg st it might look something like this.
M currentDir/index.php
M currentDir/update.php
M currentDir/a_new_file.php # why is this M? It should be A right?
M currentDir/another_new_file.php # why is this M? It should be A right?
M currentDir-old/another_file.php # why is this M? It should be A right?
R currentDir/another_file.php
R currentDir-beta/index.php
R currentDir-beta/update.php
R currentDir-beta/a_new_file.php
R currentDir-beta/another_new_file.php
Any suggestions on how to get around this? My goal is to make it so existing code changes that took place in currentDir-beta are "forwarded" to currentDir/ in the staging environment. All the other "not common" file changes are mapped, just not these common files.
UPDATE
Forgot to mention, I'm using Mercurial 3.9 on macOS Sierra.
I don't know
Your version of Mercurial
OS
but on my Win-box with Mercurial-3.9.1 my impressions (and results) differ
Clean initial state (folders shortened due to lazyness)
>hg st -A
C Current-beta\a_new_file.php
C Current-beta\another_new_file.php
C Current-beta\index.php
C Current-beta\update.php
C Current\another_file.php
C Current\index.php
C Current\update.php
First rename
>hg mv Current Current-Backup
moving Current\another_file.php to Current-Backup\another_file.php
moving Current\index.php to Current-Backup\index.php
moving Current\update.php to Current-Backup\update.php
...commit details skipped...
Second rename
>hg mv Current-beta Current
moving Current-beta\a_new_file.php to Current\a_new_file.php
moving Current-beta\another_new_file.php to Current\another_new_file.php
moving Current-beta\index.php to Current\index.php
moving Current-beta\update.php to Current\update.php
and working directory after it (as expected)
>hg st
A Current\a_new_file.php
A Current\another_new_file.php
A Current\index.php
A Current\update.php
R Current-beta\a_new_file.php
R Current-beta\another_new_file.php
R Current-beta\index.php
R Current-beta\update.php
...commit details skipped...
If you want to see how it was recorded by Mercurial: I used such slightly puzzling at first glance log for better interpreting of output
hg log -T "{rev}:{node|short}\n{if(file_adds,'\tAdded: {join(file_adds,', ')}\n')}{if(file_copies,'\tCopied: {join(file_copies,', ')}\n')}{if(file_dels,'\tDeleted: {join(file_dels,', ')}\n')}{if(file_mods,'\tModified: {join(file_mods,', ')}\n')}\n"
and here it's result
2:98955fcb7e71
Added: Current/a_new_file.php, Current/another_new_file.php, Current/index.php, Current/update.php
Copied: Current/a_new_file.php (Current-beta/a_new_file.php), Current/another_new_file.php (Current-beta/another_new_file.php), Current/index.php (Current-beta/index.php), Current/update.php (Current-beta/update.php)
Deleted: Current-beta/a_new_file.php, Current-beta/another_new_file.php, Current-beta/index.php, Current-beta/update.php
1:61068c6ba8a7
Added: Current-Backup/another_file.php, Current-Backup/index.php, Current-Backup/update.php
Copied: Current-Backup/another_file.php (Current/another_file.php), Current-Backup/index.php (Current/index.php), Current-Backup/update.php (Current/update.php)
Deleted: Current/another_file.php, Current/index.php, Current/update.php
0:454486bc43e5
Added: Current-beta/a_new_file.php, Current-beta/another_new_file.php, Current-beta/index.php, Current-beta/update.php, Current/another_file.php, Current/index.php, Current/update.php
As you can see - no edits ("Modified") at all (and here log /per changeset/ is more correct than aggregated status)
PS: I couldn't see on the fly purpose of your revset in hg st and necessity of branching+merging
PPS: OK, I saw
>hg st --rev "0:"
M Current\index.php
M Current\update.php
A Current-Backup\another_file.php
A Current-Backup\index.php
A Current-Backup\update.php
A Current\a_new_file.php
A Current\another_new_file.php
R Current-beta\a_new_file.php
R Current-beta\another_new_file.php
R Current-beta\index.php
R Current-beta\update.php
R Current\another_file.php
Aggregated results in considering only the boundary conditions is (correctly, technically speaking) modified files for files 1) in the same location 2) with the same name 3) and with changed content
I have two directories on two different computers - machine A (Windows) and machine B (OSX) - and I want to keep the two directories via Mercurial in sync. [*]
The restriction is that the two machines are not connected via LAN/WAN; the only way to move data between them is via email. So I thought emailing Mercurial bundles as deltas could do the trick.
My current workflow is roughly this (using a local tag lcb for the latest change bundle):
Say I work on machine A. At the end of the day I do:
hg commit -A -m "changes 123"
hg bundle --base lcb bundle_123.hg
hg tag --local -f lcb --rev tip
finally then I email that bundle to machine B.
Then sitting at machine B I do
hg unbundle bundle_123.hg
hg merge
hg commit -A -m "remote changes 123"
hg tag --local -f lcb --rev tip
Now I'm working on machine B and at the end of the day I do what's listed under 1., but on machine B. And the cycle continues...
However, I'm worry this system is not robust enough:
In-between changes: What happens when after creating a bundle (Step 1) and before applying it remotely (Step 2) a changes occurrs on the remote machine B? I had a case where it just overwrote the changes with the new bundle without conflict warning or merge suggestion.
Double-applying of bundle: What happens when by accident a bundle is applied twice? Would be needed to record the applied bundles somehow with local tags?
Or is there another better workflow to transfer Mercurial deltas via email?
[*] From the answer to a superuser question I figured that Mercurial might be the most feasible way to do this.
In-between changes: What happens when after creating a bundle (Step 1) and before applying it remotely (Step 2) a changes occurs on the remote machine B? I had a case where it just overwrote the changes with the new bundle without conflict warning or merge suggestion.
If a change is made on machine B, then this change will have been made in parallel with the changes you bundled from machine A. It doesn't really matter if the changes are made before or after you create the bundle (time-wise), it only matters that the changes on machine B don't have the head from machine A as their ancestor.
In other words, the world looks like this when the two machines are in sync:
A: ... [a]
B: ... [a]
You then create some new commits on machine A:
A: ... [a] --- [b] --- [c]
B: ... [a]
You bundle using [a] as base, so you get a bundle with [b] and [c]. Let us now say that someone (perhaps yourself) makes a commit on machine B:
A: ... [a] --- [b] --- [c]
( bundled )
B: ... [a] --- [x]
So far nothing has been exchanged between the two repositories, so this is just a normal case of people working in parallel. This is the norm in a distributed version control system — people working in parallel is that creates the need for merge commits.
The need for a merge is not evident in either repository at this point, they both have linear histories. However, when you unbundle on machine B, you see the divergence:
A: ... [a] --- [b] --- [c]
( bundled )
B: ... [a] --- [x]
\
[b] --- [c]
( unbundled )
It is helpful to realize that hg unbundle is exactly like hg pull, except that it can be done offline. That is, the data stored in a bundle is really just the data that hg pull would have transferred if you had had an online connection between the two repositories.
You would now proceed by merging the two heads [x] and [c] to create [y] on machine B:
A: ... [a] --- [b] --- [c]
B: ... [a] --- [x] --- [y]
\ /
[b] --- [c]
on machine B your last bundle was created with [a] as a base. However, you also know that machine A has commit [c], so you can specify that as an additional base if you like:
$ hg bundle --base a --base c stuff-from-machine-b.hg
That will put [x] and [y] into the bundle:
bundle: (a) --- [x] --- [y]
/
(c)
Here I use (a) and (c) to denote the required bases of the bundle. You can only unbundle this bundle if you have both [a] and [c] in your repository. If you leave out the second base (only use [a]), you will also bundle [b] and [c]:
bundle: (a) --- [x] --- [y]
\ /
[b] --- [c]
Here you included everything except [a] in the bundle. Bundling too much is okay, as we will see next.
Double-applying of bundle: What happens when by accident a bundle is applied twice? Would be needed to record the applied bundles somehow with local tags?
Applying a bundle twice is exactly like running hg pull twice: nothing happens the second time. When unbundling, Mercurial looks in the bundle and imports the missing changesets. So if you unbundle twice, there is nothing to do the second time.
Initial state
A>hg log --template "{rev}:{node|short} \"{desc}\" - files: {files}\n"
2:415231dbafb8 "Added C" - files: C.txt
1:6d9709a42687 "Added B" - files: B.txt
0:e26d1e14507e "Initial data" - files: .hgignore A.txt
B>hg log --template "{rev}:{node|short} \"{desc}\" - files: {files}\n"
1:72ef13990d0d "Edited A" - files: A.txt
0:e26d1e14507e "Initial data" - files: .hgignore A.txt
i.e:
Identical repos diverged at revision 1 at both sides: independent changes appeared
Test for case 1 - parallel changes
72ef13990d0d in B doesn't interfere with 6d9709a42687:415231dbafb8 in A
A>hg bundle --base e26d1e14507e ..\bundle1-2.hg
2 changesets found
B>hg pull ..\bundle1-2.hg
pulling from ..\bundle1-2.hg
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
because B had own child for e26d1e14507e, pulling from bundle added additional head (and anonymous branch for changesets from A)
B>hg glog --template "{rev}:{node|short} \"{desc}\" - files: {files}\n"
o 3:415231dbafb8 "Added C" - files: C.txt
|
o 2:6d9709a42687 "Added B" - files: B.txt
|
| # 1:72ef13990d0d "Edited A" - files: A.txt
|/
o 0:e26d1e14507e "Initial data" - files: .hgignore A.txt
Test for case 2 - applying bundle twice
I know apriori, that existing in repo changesets will not be pulled again (and prefer unified style of hg pull from bundle instead of hg unbundle), but show it
B>hg pull ..\bundle1-2.hg
pulling from ..\bundle1-2.hg
searching for changes
no changes found
Additional benefit from pull's behavior - you can don't worry about moving base changeset for bundle and always use one, oldest point of divergence - it will (slightly) increase size of bundle (slightly - because by default bundle is bzip2-compressed archive), but also it will it guarantees the inclusion of all child changesets into bundle and pulling all missing (and only missing) changesets in destination repository
And, it any case, even unbundle the same bundle twice will not have any backfires.
Same bundle in same repo B, attempt to unbundle already pulled bundle
B>hg unbundle ..\bundle1-2.hg
adding changesets
adding manifests
adding file changes
added 0 changesets with 0 changes to 2 files
(run 'hg update' to get a working copy)
My mercurial installation refuses to merge via command line. Here's the output after I turn on verbose and debug.
C:\src\StackOverflow>hg resolve --all --verbose --debug
couldn't find merge tool codecompare_diff
couldn't find merge tool codecompare_merge
couldn't find merge tool bcompare
couldn't find merge tool beyondcompare3
couldn't find merge tool beyondcompare3-noauto
couldn't find merge tool rekisa
couldn't find merge tool UltraCompare
couldn't find merge tool araxis
couldn't find merge tool meld
couldn't find merge tool diffuse
picked tool 'kdiff3' for StackOverflow/StackOverflow.csproj (binary False symlink False)
merging StackOverflow/StackOverflow.csproj
my StackOverflow/StackOverflow.csproj#4f4faeac0fea+ other StackOverflow/StackOverflow.csproj#ae29e8c6bb88 ancestor StackOverflow/StackOverflow.csproj#399376fedfc5
The system cannot find the path specified.
merging StackOverflow/StackOverflow.csproj failed!
Note: the merge was actually successful (kdiff3 pops up, I merge and save), but mercurial is confused.
How do I determine which path cannot be found by Mercurial?
I've tried updating to the latest version (2.8) and looking at my mercurial.ini for any suspicious files. I've also used a different merge tool, p4merge, to the same exact effect.
Hmm... Actually.
It may be stupid "solution", but you may look into the filemerge.py file that should come with mercurial.
This file defines how mercurial is doing merge (calling external merge-tools, processing result, etc).
Here you see the following lines inside the filemerge function:
ui.debug("picked tool '%s' for %s (binary %s symlink %s)\n" %
(tool, fd, binary, symlink))
[...]
ui.status(_("merging %s\n") % fd)
ui.debug("my %s other %s ancestor %s\n" % (fcd, fco, fca))
They are giving correspondent messages in your output:
picked tool 'kdiff3' for StackOverflow/StackOverflow.csproj (binary False symlink False)
merging StackOverflow/StackOverflow.csproj
my StackOverflow/StackOverflow.csproj#4f4faeac0fea+ other StackOverflow/StackOverflow.csproj#ae29e8c6bb88 ancestor StackOverflow/StackOverflow.csproj#399376fedfc5
Then _xmerge function is called from filemerge function with the following lines:
needcheck, r = func(repo, mynode, orig, fcd, fco, fca, toolconf,
(a, b, c, back))
And then the ui.warn(onfailure % fd) is reached either at line 334 or at line 368 (I don't know), which gives your last error message:
merging StackOverflow/StackOverflow.csproj failed!
Somewhere between ui.debug("my %s other %s ancestor %s\n" % (fcd, fco, fca)) and ui.warn(onfailure % fd) there is a reason why ui.warn(onfailure % fd) is actually reached (maybe in _xmerge). You can try to discover it by inserting various debug output in various places of filemerge.py file, and retrying merge. That's all I can currently recommend (as I can't reproduce your errors and context on my machine).
P.S.: As santiagopim suggests, you may prefer to copy WC to the clean PC with clean mercurial installation and retry there. As the errors are most probably caused by some misconfiguration. However, if you prefer to try to resolve problem in-place...
As I understand one of the main advantages of distributed revision control system like Mercurial is that you should not worry about breaking something in your super-important main repo (which is used by quite a lot other developers), and do all you work and researches in your personal remote clone until you understand that everything stable and you can push your work back.
And hence I got my question: if it possible to push back not all your changes history (with several revisions which you made for yourself), but only one which is actually diff between your repo and current state of master.
One example:
hg init master-repo; cd master-repo
echo -e 'Important file\nWith Bug!' > file
hg commit -A -m "initial commit"
cd ..; hg clone master-repo hotfix-repo; cd hotfix-repo
echo "fix1" >> file
hg commit -m "first attempt to fix bug"
echo "fix2" >> file
hg commit -m 'Fixed it!'
Now (possibly after pull and merge with newest master-repo' changes) I want to push back only one changeset containing all changes that I've done without my local commits history.
One possible solution is to create one more clone then use diff/patch between two clones to extract/apply changes from first one and commit them all at once in second repo. Then do push as in normal case. But is it possible to so using only mercurial commands?
Thanks in forward!
Opinions differ on whether or not it's good to collapse trial-and-error changesets into a single changeset before pushing:
Pros: you avoid having changesets in your history where your test suite fails — those changesets are bad for hg bisect and add noise.
Con: you cannot collapse changesets that you have published to other repositories — doing so would only rewrite your local changesets and you would then have to clean up the other repositories manually.
Technically, it's perfectly safe to collapse a set of changesets into a single changeset before pushing. You start with
... a --- b --- c --- x --- y --- z
and you rewrite this into
... a --- b --- c --- w
where w has exactly the same repository state as z had (but a different parent changeset, obviously). There are no merges here (and hence no merge conflicts) so it cannot fail.
After rewriting, you can pull and merge with the upstream (d and e):
... a --- b --- c --- w --- v
\ /
d ----- e
You need an extension to do any kind of history rewriting. Here I would suggest one of:
Collapse extension: as the name implies, this extension is dedicated to collapsing changesets.
Histedit extension: full-fledged history editing, but the fold command let's you collapse changesets.
Rebase extension: this standard extension can move changesets around and collapse them at the same time. In the example above, it would move x --- y --- z after e:
... a --- b --- c --- d --- e --- x' --- y' --- z'
You can then optionally collapse x' to z':
... a --- b --- c --- d --- e --- w'
Compared to just collapsing x to z, rebasing does involve merges so it can fail.
Intro
I think, rewrite history and send/get "polished" changesets (in Git-boys style) is, in common, bad idea - history is history, it have own value.
After all, for "mainline" branch (you use branches, isn't it), all your changes will be presented as one mergeset, regardless of changesets count in branch
Short answer
No. In Mercurial you pull/push and get&accept full set of changesets, which created difference in repo history
Long answer
Somehow you can, by rewting your history of changesets before exchange to "other side". Just remember - each changeset represent not new state of object, but diff between old and current (if describe it shortly), thus - by ordinary removing changests you can get wrong final result.
Anyway, you have a lot of ways to rewrite own history (as extensions):
Collapse
History Edit
MQ (with mq-patches per se, folding changesets in mq-patch and splitting the same way)
Maybe some others, unknown for me
You can use hg grep, but it searches the contents of all files.
What if I just want to search the file names of deleted files to recover one?
I tried hg grep -I <file-name-pattern> <pattern> but this seems to return no results.
using templates is simple:
$ hg log --template "{rev}: {file_dels}\n"
Update for Mercurial 1.6
You can use revsets for this too:
hg log -r "removes('**')"
(Edit: Note the double * - a single one detects removals from the root of the repository only.)
Edit: As Mathieu Longtin suggests, this can be combined with the template from dfa's answer to show you which files each listed revision removes:
hg log -r "removes('**')" --template "{rev}: {file_dels}\n"
That has the virtue (for machine-readability) of listing one revision per line, but you can make the output prettier for humans by using % to format each item in the list of deletions:
hg log -r "removes('**')" --template "{rev}:\n{file_dels % '{file}\n'}\n"
If you are using TortoiseHg workbench, a convenient way is to use the revision filter. Just hit ctrl+s, and then type
removes("**/FileYouWantToFind.txt")
**/ indicates that you want to search recursively in your repository.
You can use * wildcard in the filename too. You can combine this query with other revision sets using and, or operators.
There is also this Advanced Query Editor:
I have taken other answers and improved it.
Added "--no-merges". On large project with dev teams, there will lots of merges. --no-merger will filter out the log noise.
Change removes("**") to sort(removes("**"), -rev). For a large project with over 100K changesets, this will get to the latest files removed a lot faster. This reverses the order from starting at rev 0 to start at tip instead.
Added {author} and {desc} to ouput. This will give context as to why the files was removed by displaying the log comment and who did it.
So for my use case, it was hg log --template "File(s) deleted in rev {rev}: {author} \n {desc}\n {file_dels % '\n {file}'}\n\n" -r 'sort(removes("**"), -rev)' --no-merges
Sample output:
File(s) deleted in rev 52363: Ansariel
STORM-2141: Fix various inventory floater related issues:
* Opening new inventory via Control-Shift-I shortcut uses legacy and potentinally dangerous code path
* Closing new inventory windows don't release memory
* During shutdown legacy and inoperable code for inventory window cleanup is called
* Remove old and unused inventory legacy code
indra/newview/llfloaterinventory.cpp
indra/newview/llfloaterinventory.h
File(s) deleted in rev 51951: Ansariel
Remove readme.md file - again...
README.md
File(s) deleted in rev 51856: Brad Payne (Vir Linden) <vir#lindenlab.com>
SL-276 WIP - removed avatar_skeleton_spine_joints.xml
indra/newview/character/avatar_skeleton_spine_joints.xml
File(s) deleted in rev 51821: Brad Payne (Vir Linden) <vir#lindenlab.com>
SL-276 WIP - removed avatar_XXX_orig.xml files.
indra/newview/character/avatar_lad_orig.xml
indra/newview/character/avatar_skeleton_orig.xml
Search for a specific file you deleted efficiently, and format the result nicely:
hg log --template "File(s) deleted in rev {rev}: {file_dels % '\n {file}'}\n\n" -r 'removes("**/FileYouWantToFind.txt")'
Sample output:
File(s) deleted in rev 33336:
class/WebEngineX/Database/RawSql.php
File(s) deleted in rev 34468:
class/PdoPlus/AccessDeniedException.php
class/PdoPlus/BulkInsert.php
class/PdoPlus/BulkInsertInfo.php
class/PdoPlus/CannotAddForeignKeyException.php
class/PdoPlus/DuplicateEntryException.php
class/PdoPlus/Escaper.php
class/PdoPlus/MsPdo.php
class/PdoPlus/MyPdo.php
class/PdoPlus/MyPdoException.php
class/PdoPlus/NoSuchTableException.php
class/PdoPlus/PdoPlus.php
class/PdoPlus/PdoPlusException.php
class/PdoPlus/PdoPlusStatement.php
class/PdoPlus/RawSql.php
from project root
hg status . | grep "\!" >> /tmp/filesmissinginrepo.txt