I created azure devops pipeline where I need to download code from another git repository. As per recommended solution I've read somewhere, I added a command line script task with git clone command. Unfortunatelly this doesn't work.
The error that I get is:
remote: TF200016: The following project does not exist: My0Test0Project. Verify that the name of the project is correct and that the project exists on the specified Azure DevOps Server.
fatal: repository 'https://dev.azure.com/myCompany/My0Test0Project/_git/Service.Azure.Core/' not found
My project in Azure has spaces, maybe there is a bug in azure related to that? Does anybody knows any workarounds?
This is some code that I have tried already:
git -c http.extraheader="AUTHORIZATION: Basic bXl1c2VyOmxtNjRpYTYzb283bW1iYXp1bnpzMml2eWxzbXZoZXE2azR1b3V2bXdzbnl5b3R5YWlnY2E=" clone https://dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core
git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" clone https://dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core
git clone https://oauth:lm64ia63oo7mmbazunzs2ivylsmvheq6k4uouvmwsnyyotyaigca#dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core
git clone https://test:$(System.AccessToken)#dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core
The reason is because you need to add the system access token so that Azure can have access to the external repo. (your credentials need access to that repo)
NOTE:
I wrapped my URL in quotes to escape the % signs in the URL string because Powershell treats those as variables otherwise.
- powershell: |
git clone -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" "https://{YOUR PIPELINE URL}"
displayName: '{YOUR PIPELINE NAME}'
I had the same issue from a bash script and solved it by escaping the % character with %%.
So, you can use this :
git clone https://test:$(System.AccessToken)#dev.azure.com/myCompany/My%%20Test%%20Project/_git/Service.Azure.Core
Can you try to change your Command Line task to something like this one?
git clone --single-branch --branch $(branchName) https://$(gitUserName):$(gitPassword)#dev.azure.com/myCompany/My%20Test%20Project/_git
Where $(gitUserName) and $(gitPassword) configured as Alternate git credentials in
your Azure DevOps account.
Also wanted to ask about ending of git repository URL:
".../_git/Service.Azure.Core", you are trying to specify which folder to clone or maybe working directory? You can also try your previous code without that ending, just
https://dev.azure.com/myCompany/My%20Test%20Project/_git
I tried this from Powershell task and it worked
Its very simple just use
- powershell: |
git config user.email "$(Build.RequestedForEmail)"
git config user.name "$(Build.RequestedFor)"
git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" clone https://<org_Name>.visualstudio.com/<project_name>/_git/<repository_name>;
displayName: "<your task display name>"
This will happen when your path has space characters in it.
The workaround for this is to not use %20 in your URL and use actual space characters instead. Since your URL now has space characters, also make sure that your URL is wrapped inside double quotes.
So your
git clone https://dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core/
should be
git clone "https://dev.azure.com/myCompany/My Test Project/_git/Service.Azure.Core/"
I've made a simple script in bash to get subfolders from my repo on GitHub.
The problem i'm having is that I want to be able to run the script several times getting several subfolders. The script currently run once. Then i get the message "branch master already up to date" how can I change it so this does not happen and I can pull several folders?
getFolder()
{
repository="$1"
folder="$2"
remote="$3"
branch="$4"
if [ "$repository" = school ]
then
repository=https://github.com/mergin/School
fi
git init
git remote add "$remote" "$repository"
git config core.sparsecheckout true
echo "$folder"/ > .git/info/sparse-checkout
git pull "$remote" "$branch"
}
This is the current function that i use.
In the moment, each time I want to make a merge-request, I commit my changes and push it online with
git commit -am 'My changes'
git push origin my_branch:my_branch
and then on Gitlab, manually, I make a merge-request and assign the CTO.
How directly from bash could I create a merge-request an assigning the CTO?
I know there exists the following answer Is it possible to create merge requests in pure Git from the command line?, but that didn't answer my question.
How can I create a merge-request from my_branch to master?
Could I do git --assign=CTO request-pull my_branch master?
Use push options. Here is an example.
git push -o merge_request.create -o merge_request.assign="CTO"
I use a local repository cache so my clones are faster.
When I clone I clone from the local cache, which auto-updates before the clone actually starts. When I push, I push to the local cache which first updates and then (when the push is successfull) pushes the changesets onwards to the canonical repository.
However: I want "hg incoming" to work as follows:
I make a clone of the local cache repository
..hack hack hack in the clone of the local cache..(hey are there any incoming changes? I wonder)
$ hg in
The local cache repository gets asked "are there any new changesets?"
The local cache then first does hg pull
Then it answers the question
Is this possible? Is there a hook I can use in the local cache repository that fires when it gets a "hg incoming" command?
The .hgrc in my local cache looks like:
[hooks]
# We want to make sure this repository has the latest changesets just before
# we do a clone of it.
preoutgoing.update_local_cache1 = echo "======================"
preoutgoing.update_local_cache2 = echo "hg pull from canonical"
preoutgoing.update_local_cache3 = echo "======================"
preoutgoing.update_local_cache4 = hg pull
# Before we push commits to the local cache make sure it's up to date.
prechangegroup.update_local_cache1 = echo "======================"
prechangegroup.update_local_cache2 = echo "hg pull from canonical"
prechangegroup.update_local_cache3 = echo "======================"
prechangegroup.update_local_cache4 = hg pull
# Just after a bunch of changesets have been succesfully pushed to this local
# cache repository, push it further to the canonical repository.
changegroup.push_to_canonical1 = echo "===================="
changegroup.push_to_canonical2 = echo "hg push to canonical"
changegroup.push_to_canonical3 = echo "===================="
changegroup.push_to_canonical4 = hg push
There's no hook you can use in the local cache to catch when one of your working clones is checking for incoming available stuff. Your best best is probably to get the working clones to do the pull in the local cache. Something like in your user-wide ~/.hgrc file should do it
[hooks]
pre-incoming = $HG --repository /path/to/local/cache pull
THen whenever you do hg incoming in any repository you'll do a pull in the local cache first. That, of course, won't work well when you're running hg incoming in the actual local cache, turning it into a pull instead, but no tragedy.
This should be a very simple thing to have run, but for some reason it won't work with my Mercurial repository. All I want is for the remote repo to automatically run hg update whenever someone pushes to it. So I have this in my .hg/hgrc file:
[hook]
changegroup = hg update
Simple, right? But for some reason, this never executes. I also tried writing a shell script that did this. .hg/hgrc looked like this:
[hooks]
changegroup = /home/marc/bin/hg-update
and hg-update looked like this:
#!/bin/sh
hg help >> /home/marc/works.txt;
hg update >> /home/marc/works.txt;
exit 0;
But again, this doesn't update. The contents of hg help are written out to works.txt, but nothing is written out for hg update. Is there something obvious I'm missing here? This has been plaguing me for days and I just can't seem to get it to work.
Update
Okay so again, using the -v switch on the command line from my workstation pushing to the remote repo doesn't print any verbose messages even when I have those echo lines in .hg/hgrc. However, when I do a push from a clone of the repo on the same filesystem (I'm logged in via SSH), this is what I get:
bash-3.00$ hg -v push ../test-repo/
pushing to ../test-repo/
searching for changes
1 changesets found
running hook prechangegroup: echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
Remote repo is at 821:1f2656753c98
Remote repo wdir is at 821:1f2656753c98
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
running hook changegroup: echo "Updating.... `hg update -v`"
echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
Updating.... resolving manifests
getting license.txt
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
Remote repo is at 822:389a6c7276c6
Remote repo wdir is at 822:389a6c7276c6
So it works, but again only when I push from the same filesystem. It doesn't work if I try pushing to the repo from another workstation over the network.
Well, after going through the same steps of frustration as Marc W did a while ago, I finally found the solution to the problem, at least when remote serving is done with the hgwebdir WSGI script.
I found out that when using this kind of remote push via HTTP or HTTPS, Mercurial simply ignores everything you write into the .hg/hgrc file or your repository. However, entering the hook in the hgwebdir config does the trick.
So if the bottom line in your hgwebdir.wsgi script is something like
application = hgwebdir('hgweb.config')
the [hooks] config section needs to go into the mentioned hgweb.config.
One drawback is that these hooks are executed for every repository listed in the [paths] section of that config. Even though HG offers another WSGI-capable function (hgweb instead of hgwebdir) to serve only a single repository, that one doesn't seem to support any hooks (neither does it have any config).
This can, however, be circumvented by using a hgwebdir as described above and having some Apache RewriteRule map everything into the desired subdirectory. This one works for me:
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/reponame
RewriteRule ^(.*)$ reponame/$2 [QSA]
Have fun using your remote hooks over HTTP :D
I spent some time researching this myself. I think the answer to problem is described concisely here:
Output has to be redirected to stderr (or /dev/null), because stdout
is used for the data stream.
Basically, you're not redirecting to stderr, and hence polluting stdout.
First of all, I want to correct a few comments above.
Hooks are invoked also when pushing over file system.
It is not necessary to keep the hook in the repo on which you want them to operate. You can also write the same hook as in your question on the user end. You have to change the event from changegroup to outgoing and also to specify the URL of remote repo with the -R switch. Then if the pushing user has sufficient privileges on the remote repo, the hook will execute successfully.
.hg/hgrc
[hooks]
outgoing = hg update -R $HG_URL
Now towards your problem.... I suggest creating both prechangegroup and changegroup hooks and printing some debugging output.
.hg/hgrc
[hooks]
prechangegroup = echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
changegroup = echo "Updating.... `hg update -v`"
echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
And also push with the -v switch, so that you may know which hook is running. If you still can't figure out, post the output. I might be able to help.
My problem was that my hgwebdir application ran as the "hg" user, but the repository was owned by me, so I had to add in this bit of config to hgweb.config to get it to run the hooks:
[trusted]
users = me
You need to have it in the remote repositiory's hgrc. It sounds as if it's in your local repo.
Edit: It also depends on how you're pushing. Some methods don't invoke hooks on the right side. (ssh does, I think HTTP does, file system does not)
Edit2: What if you push "locally" at the remote repo's computer. You might have different users/permissions between the webserver and the hgrc-file. (See [server] and trusted directives for hgrc.)
I had the same problem pushing from Windows Eclipse via http, but after capturing stderr, I found that the full path was needed to the hg.bat file. My hooks section now looks like:
[hooks]
incoming = c:\Python27\Scripts\hg.bat update > hg_log.txt 2>>hg_err.txt
Hope this helps someone else.
SteveT
Try turning on hook debugging to see why it's not running.
Likely a permissions issue or something like that.
took a while but I got it working.
I started with
[hooks]
tag=set >&2
commit=set >&2
the >&2 pipes it to standard error so remote consoles will show it.
when remote this should output in console if it is running
hg push https://host/hg -v
It wasn't.
I was using hgweb.cgi so I switched to hgweb.wsgi with no difference.
what I discovered is that some hooks don't get called on remote.
when I switched it to
[hooks]
incoming= set >&2
the hooks tag and commit don't seem to get called but incoming and changeset do get called. I haven't confirmed the others.
now that I got it working I switched back to hgweb.cgi and everything works the same.
Tthe reason I've found for this has nothing to do with redirecting stdout to stderr. As you may see in the wiki page it is not specified on the wiki's current version
https://www.mercurial-scm.org/wiki/FAQ#FAQ.2FCommonProblems.Any_way_to_.27hg_push.27_and_have_an_automatic_.27hg_update.27_on_the_remote_server.3F
The problem I've found is around permissions.
In my original setup, I had a user, lets say hguser with a repo on its home, and a script /etc/init.d/hg.init to launch hg serve. The problem being hg serve was being run by root, while most files under the repo pertained to hguser (some of them switched to root at some point, but it won't mind, since I'll correct them with chown)
Solution:
chown -R hguser:hguser /home/hguser/repo (to correct ALL files, back to hguser)
launch su hguser -c "hg serve ..." (in my case from /etc/init.d/hg.init)
changegroup = hg update -C under [hooks] in repo/.hg/hgrc as usual
Now it should work on push
PS: in my case, I rather update to the head of a specific branch, so I use hg update -C -r staging, to make the staging server update only to the head of the intended branch, even if the tip is from another branch (like development for instance)
BTW my hg.init script ended up like this: (notice the su hguser part)
#!/bin/sh
#
# Startup script for mercurial server.
#
# #see http://jf.blogs.teximus.com/2011/01/running-mercurial-hg-serve-on-linux.html
HG=/usr/bin/hg
CONF=/etc/mercurial/hgweb.config
# Path to PID file of running mercurial process.
PID_FILE=/etc/mercurial/hg.pid
state=$1
case "$state" in
'start')
echo "Mecurial Server service starting."
(su hguser -c "${HG} serve -d --webdir-conf ${CONF} -p 8000 --pid-file ${PID_FILE}")
;;
'stop')
if [ -f "${PID_FILE}" ]; then
PID=`cat "${PID_FILE}"`
if [ "${PID}" -gt 1 ]; then
kill -TERM ${PID}
echo "Stopping the Mercurial service PID=${PID}."
else
echo Bad PID for Mercurial -- \"${PID}\"
fi
else
echo No PID file recorded for mercurial
fi
;;
*)
echo "$0 {start|stop}"
exit 1
;;
esac
PS: due credit to http://jf.blogs.teximus.com/2011/01/running-mercurial-hg-serve-on-linux.html