I know how to configure the Mercurial signing extension. The problem that I'm having is that I don't want to sign each individual change set, I only want to sign revisions that introduce new version tags.
That's easily accomplished locally, however I can't come up with a way to enforce this on the remote server. I'd like people to continue to be able to push their changes as normal, unless adding a release tag, which should be accompanied by a signature.
The end result should be that anyone cloning our repository can easily see a list of signed revisions, which point to a list of signed releases.
Hopefully, I've just missed something obvious in hooklib. Has anyone else accomplished this, if so, how?
You could do it on the server with a pretxnchangegroup hook. More efficient in-process in python, but off the top of my head in shell you'd do:
In your hgrc:
[hook]
pretxnchangegroup = all-tags-checked.sh
and in all-tags-checked.sh:
for therev in $(seq $(hg id -n -r $HG_NODE) $(hd id -n -r tip)) ; do
if hg log --template '{files}' -r $therev | grep --quiet '^.hgtags' ; then
if hg sigcheck $therev | grep --quiet '^No valid' ; then
exit 1
fi
fi
done
That goes through every new changeset and checks to make sure that if it edits .hgtags (add a tag) then it must also be signed.
Is that what you're looking for?
Related
Background
I have a "mainline" depot and a "beta" branch in Perforce (using simple branches: no streams, etc).
I would like to merge the latest code from "mainline" to my "beta" branch. I have to do this about once every day, and there are about 100+ commits/submissions to the "mainline branch" every day.
Normally, I would do something like so:
p4 integ //prod/mainline/... //prod/beta/...
cd $(p4 where //prod/beta/... | cut -d ' ' -f3 | sed 's/\.\.\.$//g')
p4 resolve ./...
Problem
However, we have an annoying hourly build process that updates version numbers in various Makefiles, build scripts, etc; that updates version/branch numbers, and is checked in to Perforce using a "dummy" account (i.e. dummy_user) by our build servers. This is done across all branches, causing any merge operation to have arbitrary conflicts.
This version number submission now prevents my p4 integ/p4 resolve operation from completing cleanly, and I have to hand-merge all of these files affected by the "version number update operation". I would like to just have to hand-merge actual code changes, rather than this version number nonsense.
Question
Is there a way to p4 integ a set of change-lists not yet present in the branch (but present in main/another-branch), excluding a user? I could always do something like:
for i in $(p4 changes //prod/mainline/... | grep -v dummy_user | cut -d ' ' -f2)
do
p4 integ //prod/mainline/...#${i},${i} //prod/beta/...
done
However, I don't have an automated way to get a list of all change-lists that:
Exist in mainline...
But haven't yet been merged/integrated into beta...
And aren't authored by dummy_user.
How can I accomplish this?
Sounds like you've found a solution that works for you, but FWIW here's what I'd do:
0) Make a branchspec since that makes this next part easier:
p4 --field "View=//prod/mainline/... //prod/beta/..." branch -o prod-main-beta | p4 branch -i
1) Ignore the robo-commits (I assume you want to just leave these things alone):
p4 -Ztag -F #=%change% ichanges -u dummy_user -b prod-main-beta | p4 -x - integ -b prod-main-beta
p4 resolve -ay
p4 submit -d "Begone, robo-cruft!"
2) Do the "real" integrate. If you find yourself still having to pick around the "dummy" changes, try the -Rs option -- that might give you more merges to do but it will bend over backwards to make sure those merges don't include anything you've already integrated.
p4 integ -b prod-main-beta [-Rs]
p4 resolve [-am]
p4 submit
Optional 3) Improve your build tooling so that the version info is located in a dedicated file, and other build files link to it. Then you can exclude your version file from your branch specs, or just always ignore changes to it without having to cherry-pick, or whatever. Here's a real-world example: https://swarm.workshop.perforce.com/files/guest/perforce_software/p4/2015-1/Version
I am using bookmarks in mercurial to emulate a git branches-like workflow.
One thing I'm finding is that whenever I push, I invariably want to push just the current bookmark. Rather than typing
hg push -B <bookmark_name>
all the time, I'd like to alias hg push to just push the current bookmark. To do that, I need a way of referring to the current bookmark without mentioning its name. Is there a way to do that?
I understand it was asked two years ago, but I found this page in Google and none of the answers helped. So here's what did the trick for me (Linux):
[alias]
currentbranch = !cd `$HG root` && cat .hg/bookmarks.current
pushb = !$HG push -B `$HG currentbranch`
cd is required for this to work from non-root directories.
Current bookmark name stored in .hg/bookmarks.current file of your repository. As an alias you can use something like this:
pushb = push -B `cat .hg/bookmarks.current`
Also note that when you update you repository state to any other revision, there won't be file .hg/bookmarks.current.
OK, platform independent solution, somehow ugly
pushb = push -B `hg log --template "{bookmarks}\n" -r "bookmark() & ."`
or, with nested command in more natural way it must be: hg parents --template="{bookmarks}\n"
Ugly because pure Mercurial-way using nested shell-aliases in hgrc
[alias]
cb = !$HG log --template "{bookmarks}\n" -r "bookmark() & ."
pushb = push -B cb
does not work for me
>hg pushb
...
bookmark cb does not exist on the local or remote repository!
Edit
Long time later, with new solution. According to hg help push
If -B/--bookmark is used, the specified bookmarked revision, its
ancestors, and the bookmark will be pushed to the remote repository.
Specifying "." is equivalent to specifying the active bookmark's name.
if order to "push the current bookmark" you can use just
hg push -B .
This works for me (and handles bookmarks with spaces):
alias pushb="hg push -B \"\$(hg bookmarks 2> /dev/null | awk '/\*/ { \$1=\"\"; \$NF=\"\"; printf }' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')\""
You can use a shell call substitution:
hg push -B `hg bookmark --active`
This can be put into an alias (see alias section of hg help config)
It is a very simple and stupid question,
I am working on 2 tasks and modified 2 sets of files in the code.
Now when I type 'hg ci', it checks in all the files. Can I remove certain files from the checkin i.e. do checking for only one task?
If I remove the files in the 'check in message' will they be removed from the checkin
Thanks for all the answers
This seems like a big flaw, My use case is very simple and general. Most of the time dev are working on various tasks , some are ready , some are not, bug fixes etc.
Now the only simple solution seems to be multiple local repos or clones
Use hg ci <files>... to commit only certain files in your working directory.
If you want to pick file by file you can use the record command. It ships with mercurial, but you have to turn it on if you want to use it by putting: record= in the [extensions] section of your ~/.hgrc.
It goes hunk by hunk, but you can answer for a whole file:
y - record this change
n - skip this change
s - skip remaining changes to this file
f - record remaining changes to this file
d - done, skip remaining changes and files
a - record all changes to all remaining files
q - quit, recording no changes
? - display help
I'll point out that if you're committing some files but not others it's certain that you've not run your test suite on the one change without the other, but maybe that doesn't apply in your case.
This isn't possible with mercurial out of the box. As have been suggested there are several ways of selecting what files you want to commit. To get this function via the commit message template, you would need an extension or a shell script wrapping the commit command. Here's one way to do that:
[alias]
ci = ! hg-partial-commit
hg-partial-commit:
#!/bin/sh
# partial commit
edit=$(mktemp ${TMPDIR:-/tmp}/$(basename $0).XXXXXXXXXXXX)
filelist=$(mktemp ${TMPDIR:-/tmp}/$(basename $0).XXXXXXXXXXXX)
logmessage=$(mktemp ${TMPDIR:-/tmp}/$(basename $0).XXXXXXXXXXXX)
cleanup="rm -f '$edit' '$filelist' '$logmessage'"
trap "$cleanup" 0 1 2 3 15
(
echo user: $(hg debugconfig ui.username)
echo branch: $(hg branch)
hg parents --template 'parent: {rev}:{node|short} {author} {date|isodate}\n'
echo
echo 'Enter commit message. Select files to commit by deleting lines:'
hg status 'set:not unknown()' | sed -e 's/^/#/'
) | sed -e 's/^/HG: /' >"$edit"
${VISUAL:-${EDITOR:-vi}} "$edit"
egrep -v '^HG:' "$edit" >"$logmessage"
egrep '^HG: #' "$edit" | cut -c8- >"$filelist"
hg commit -l "$logmessage" "listfile:$filelist"
$cleanup
The real problem here is the fact that you're doing changes related to different tasks jumbled together. Mercurial has a few ways you can keep things separate.
Task Branches
Suppose you've been working on a task and you've checked in a few times since you last pulled, but things aren't ready to share yet.
o----o----B----o----o----o
Here, B is the revision where you started your changes. What we do is (after making sure our current changes are checked in):
> hg update -r B
<do our work on the other task>
> hg commit
We've now created a new branch with the changes for this task separated from the changes for our original task.
o----o----B----o----o----o
\
----o
We can do this for as many different tasks as we want. The only problem is that sometimes remembering which branch is which can be awkward. This is where features like bookmarks come in useful. A bookmark is a tag which moves forward as commits are made so that it always points at the head of a branch.
Mercurial Queues
MQ adds the ability to work on changes incrementally and move between them by pushing and poping then off a stack (or "Queue" I guess). So if I had a set of uncommitted changes that I needed to split up I'd:
> hg qrecord taska
> hg qrecord taskb
> hg qrecord taskc
I'd use the record extension (or more likely the crecord extension) to select which parts of files I want to select.
If I needed to go back to taska and make some changes:
> hg qpop; hg qpop # pop two off the queue to go back to task a
<Do changes>
> hg qrefresh # update task a with the new changes
When I want to turn the queue into normal changesets:
> hg qpush or hg qpop # get the changes I want converted onto the queue
> hg qfinish -a # converts mq changes to normal changesets
There's other methods too, but that will do for now.
You will unavoidably have to either specify the files that you want to add or the files you want to leave out. If you have a lot of files, as you indicate above, the following steps will simplify the procedure (I'm assuming a UNIX-ish system here, Windows would be slightly different).
First, generate a list of changed files:
hg status -mard -n >/tmp/changedlist.txt
The -mard options will list all files that were modified, added, removed, or delated. The -n option will list them without the status prefix so that all you have is a raw list of files.
Second, edit /tmp/changedlist.txt with your favorite text editor so that it contains only the files you wish to commit.
Third, commit these files:
hg commit `cat /tmp/changedlist.txt`
You will be able to review the files to be committed before actually performing the commit.
Alternatively, you can add more files to a commit incrementally by using
`hg commit --amend file...`
Here, hg commit --amend will not create a new commit, but add the new files to the existing commit. So, you start out with committing just a couple of files, then incrementally adding more until you are done. This still requires you to type all of them in.
For yet another alternative, you can use Mercurial Queues to split a commit in more sophisticated ways, but that's a bit more of an advanced topic.
I want to know if there's a way to turn off the default push, but keep the default pull when using Mercurial. I don't want to accidentally pollute the master repository by inadvertently pushing from an experimental repository.
I was able to solve this by putting the following in my .hg/hgrc file, but I was wondering if there's a better/official way.
[paths]
default = http://server/hg/repo
default-push = .
Your solution probably is the quickest and is certainly effective. If there's any official way it would be using a preoutgoing hook:
[hooks]
preoutgoing = bash -c 'read -p "Really push to $HG_URL? " -n 1 RESP ; [ "$RESP" == "y" ]'
which will ask you if you want to push and provide the URL to which it would go as a reminder.
I like your own answer of setting paths.default-push = . -- it is simple and it is clear that it will work.
Another option would be a pre-push hook:
[hooks]
pre-push = if [ $HG_PATS == "[]" -o $HG_PATS == "['default']" ]; then
read -p "Really push to default? " -n 1; echo
[ "$REPLY" == "y" ]
fi
(Here I'm taking advantage of how you can split a long value over several lines by indenting them in a Mercurial config file.)
A push to default looks this
% hg push
Really push to default? n
warning: pre-push hook exited with status 1
where I typed the n. The hooks checks for both no arguments ($HG_PATS == "[]") and a default as the argument ($HG_PATS == "['default']") and will only prompt you in those cases. The $HG_PATS variable was introduced in Mercurial 1.6.
PS: I saw you updated the question and asked for a solution in PowerShell, but I'm afraid I know nothing about that language. However, you should be able to lift the important concepts from this answer yourself.
The answer previously posted, in hgrc setting
default-push = .
is ALMOST but not quite correct. It can break, e.g. if you hg your home directory.
Here is the my current BKM to disable default-push:
I've embellished the idea of setting paths.default-push in ~/.hgrc, making it a little bit more self documenting and less error-prone - since, as I point out below, setting default-push = . does not always disable pushing.
in ~/.hgrc
[paths]
# my main project master repo
project-master = ...
#DISABLING IMPLICIT PUSH
# to prevent embarassment from accidentally pushing to the project master repo
# instead of, in my case, a repo that has fine grain commits
# that the rest of the team does not want to see in the project master repo
#default-push = .
# this works mostly, but NOT if you use hg on your home directory
# since '.' in ~/.hgrc seems to be interpreted as -R ~
#default-push = /NONEXISTENT_default-push_--_must_specify_push_target_explicity
# this works ok, but I can clean up the error message using blanks
# keeping this around because blanks in pathnames confuses many UNIX tools
default-push = /'NONEXISTENT default-push -- must specify push target explicitly'
# this amounts to disabling implicit push targets.
This should be a very simple thing to have run, but for some reason it won't work with my Mercurial repository. All I want is for the remote repo to automatically run hg update whenever someone pushes to it. So I have this in my .hg/hgrc file:
[hook]
changegroup = hg update
Simple, right? But for some reason, this never executes. I also tried writing a shell script that did this. .hg/hgrc looked like this:
[hooks]
changegroup = /home/marc/bin/hg-update
and hg-update looked like this:
#!/bin/sh
hg help >> /home/marc/works.txt;
hg update >> /home/marc/works.txt;
exit 0;
But again, this doesn't update. The contents of hg help are written out to works.txt, but nothing is written out for hg update. Is there something obvious I'm missing here? This has been plaguing me for days and I just can't seem to get it to work.
Update
Okay so again, using the -v switch on the command line from my workstation pushing to the remote repo doesn't print any verbose messages even when I have those echo lines in .hg/hgrc. However, when I do a push from a clone of the repo on the same filesystem (I'm logged in via SSH), this is what I get:
bash-3.00$ hg -v push ../test-repo/
pushing to ../test-repo/
searching for changes
1 changesets found
running hook prechangegroup: echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
Remote repo is at 821:1f2656753c98
Remote repo wdir is at 821:1f2656753c98
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
running hook changegroup: echo "Updating.... `hg update -v`"
echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
Updating.... resolving manifests
getting license.txt
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
Remote repo is at 822:389a6c7276c6
Remote repo wdir is at 822:389a6c7276c6
So it works, but again only when I push from the same filesystem. It doesn't work if I try pushing to the repo from another workstation over the network.
Well, after going through the same steps of frustration as Marc W did a while ago, I finally found the solution to the problem, at least when remote serving is done with the hgwebdir WSGI script.
I found out that when using this kind of remote push via HTTP or HTTPS, Mercurial simply ignores everything you write into the .hg/hgrc file or your repository. However, entering the hook in the hgwebdir config does the trick.
So if the bottom line in your hgwebdir.wsgi script is something like
application = hgwebdir('hgweb.config')
the [hooks] config section needs to go into the mentioned hgweb.config.
One drawback is that these hooks are executed for every repository listed in the [paths] section of that config. Even though HG offers another WSGI-capable function (hgweb instead of hgwebdir) to serve only a single repository, that one doesn't seem to support any hooks (neither does it have any config).
This can, however, be circumvented by using a hgwebdir as described above and having some Apache RewriteRule map everything into the desired subdirectory. This one works for me:
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/reponame
RewriteRule ^(.*)$ reponame/$2 [QSA]
Have fun using your remote hooks over HTTP :D
I spent some time researching this myself. I think the answer to problem is described concisely here:
Output has to be redirected to stderr (or /dev/null), because stdout
is used for the data stream.
Basically, you're not redirecting to stderr, and hence polluting stdout.
First of all, I want to correct a few comments above.
Hooks are invoked also when pushing over file system.
It is not necessary to keep the hook in the repo on which you want them to operate. You can also write the same hook as in your question on the user end. You have to change the event from changegroup to outgoing and also to specify the URL of remote repo with the -R switch. Then if the pushing user has sufficient privileges on the remote repo, the hook will execute successfully.
.hg/hgrc
[hooks]
outgoing = hg update -R $HG_URL
Now towards your problem.... I suggest creating both prechangegroup and changegroup hooks and printing some debugging output.
.hg/hgrc
[hooks]
prechangegroup = echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
changegroup = echo "Updating.... `hg update -v`"
echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
And also push with the -v switch, so that you may know which hook is running. If you still can't figure out, post the output. I might be able to help.
My problem was that my hgwebdir application ran as the "hg" user, but the repository was owned by me, so I had to add in this bit of config to hgweb.config to get it to run the hooks:
[trusted]
users = me
You need to have it in the remote repositiory's hgrc. It sounds as if it's in your local repo.
Edit: It also depends on how you're pushing. Some methods don't invoke hooks on the right side. (ssh does, I think HTTP does, file system does not)
Edit2: What if you push "locally" at the remote repo's computer. You might have different users/permissions between the webserver and the hgrc-file. (See [server] and trusted directives for hgrc.)
I had the same problem pushing from Windows Eclipse via http, but after capturing stderr, I found that the full path was needed to the hg.bat file. My hooks section now looks like:
[hooks]
incoming = c:\Python27\Scripts\hg.bat update > hg_log.txt 2>>hg_err.txt
Hope this helps someone else.
SteveT
Try turning on hook debugging to see why it's not running.
Likely a permissions issue or something like that.
took a while but I got it working.
I started with
[hooks]
tag=set >&2
commit=set >&2
the >&2 pipes it to standard error so remote consoles will show it.
when remote this should output in console if it is running
hg push https://host/hg -v
It wasn't.
I was using hgweb.cgi so I switched to hgweb.wsgi with no difference.
what I discovered is that some hooks don't get called on remote.
when I switched it to
[hooks]
incoming= set >&2
the hooks tag and commit don't seem to get called but incoming and changeset do get called. I haven't confirmed the others.
now that I got it working I switched back to hgweb.cgi and everything works the same.
Tthe reason I've found for this has nothing to do with redirecting stdout to stderr. As you may see in the wiki page it is not specified on the wiki's current version
https://www.mercurial-scm.org/wiki/FAQ#FAQ.2FCommonProblems.Any_way_to_.27hg_push.27_and_have_an_automatic_.27hg_update.27_on_the_remote_server.3F
The problem I've found is around permissions.
In my original setup, I had a user, lets say hguser with a repo on its home, and a script /etc/init.d/hg.init to launch hg serve. The problem being hg serve was being run by root, while most files under the repo pertained to hguser (some of them switched to root at some point, but it won't mind, since I'll correct them with chown)
Solution:
chown -R hguser:hguser /home/hguser/repo (to correct ALL files, back to hguser)
launch su hguser -c "hg serve ..." (in my case from /etc/init.d/hg.init)
changegroup = hg update -C under [hooks] in repo/.hg/hgrc as usual
Now it should work on push
PS: in my case, I rather update to the head of a specific branch, so I use hg update -C -r staging, to make the staging server update only to the head of the intended branch, even if the tip is from another branch (like development for instance)
BTW my hg.init script ended up like this: (notice the su hguser part)
#!/bin/sh
#
# Startup script for mercurial server.
#
# #see http://jf.blogs.teximus.com/2011/01/running-mercurial-hg-serve-on-linux.html
HG=/usr/bin/hg
CONF=/etc/mercurial/hgweb.config
# Path to PID file of running mercurial process.
PID_FILE=/etc/mercurial/hg.pid
state=$1
case "$state" in
'start')
echo "Mecurial Server service starting."
(su hguser -c "${HG} serve -d --webdir-conf ${CONF} -p 8000 --pid-file ${PID_FILE}")
;;
'stop')
if [ -f "${PID_FILE}" ]; then
PID=`cat "${PID_FILE}"`
if [ "${PID}" -gt 1 ]; then
kill -TERM ${PID}
echo "Stopping the Mercurial service PID=${PID}."
else
echo Bad PID for Mercurial -- \"${PID}\"
fi
else
echo No PID file recorded for mercurial
fi
;;
*)
echo "$0 {start|stop}"
exit 1
;;
esac
PS: due credit to http://jf.blogs.teximus.com/2011/01/running-mercurial-hg-serve-on-linux.html