I have problem with uploading my Polymer component into gh pages.
I'm try this from tutorial:
# git clone the Polymer tools repository somewhere outside of your
# element project
git clone git://github.com/Polymer/tools.git
# Create a temporary directory for publishing your element and cd into it
mkdir temp && cd temp
# Run the gp.sh script. This will allow you to push a demo-friendly
# version of your page and its dependencies to a GitHub pages branch
# of your repository (gh-pages). Below, we pass in a GitHub username
# and the repo name for our element
../tools/bin/gp.sh <username> <test-element>
# Finally, clean-up your temporary directory as you no longer require it
cd ..
rm -rf temp
But it's not working.
In terminal I have this errors:
There is something I'm, missing?
Here is your problem:
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
For the script to run as intended, you need to add your public ssh key to your github project. Settings -> Deploy keys -> Add deploy Key.
Alternatively, you can manually execute the steps in gp.sh that involve pulling from and pushing to github.
If you don't feel like splitting up the script, try running the commands manually, that should work. The only multi-line command in the script is this one:
echo "{
\"directory\": \"components\"
}
" > .bowerrc
Good luck.
Related
I created azure devops pipeline where I need to download code from another git repository. As per recommended solution I've read somewhere, I added a command line script task with git clone command. Unfortunatelly this doesn't work.
The error that I get is:
remote: TF200016: The following project does not exist: My0Test0Project. Verify that the name of the project is correct and that the project exists on the specified Azure DevOps Server.
fatal: repository 'https://dev.azure.com/myCompany/My0Test0Project/_git/Service.Azure.Core/' not found
My project in Azure has spaces, maybe there is a bug in azure related to that? Does anybody knows any workarounds?
This is some code that I have tried already:
git -c http.extraheader="AUTHORIZATION: Basic bXl1c2VyOmxtNjRpYTYzb283bW1iYXp1bnpzMml2eWxzbXZoZXE2azR1b3V2bXdzbnl5b3R5YWlnY2E=" clone https://dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core
git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" clone https://dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core
git clone https://oauth:lm64ia63oo7mmbazunzs2ivylsmvheq6k4uouvmwsnyyotyaigca#dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core
git clone https://test:$(System.AccessToken)#dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core
The reason is because you need to add the system access token so that Azure can have access to the external repo. (your credentials need access to that repo)
NOTE:
I wrapped my URL in quotes to escape the % signs in the URL string because Powershell treats those as variables otherwise.
- powershell: |
git clone -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" "https://{YOUR PIPELINE URL}"
displayName: '{YOUR PIPELINE NAME}'
I had the same issue from a bash script and solved it by escaping the % character with %%.
So, you can use this :
git clone https://test:$(System.AccessToken)#dev.azure.com/myCompany/My%%20Test%%20Project/_git/Service.Azure.Core
Can you try to change your Command Line task to something like this one?
git clone --single-branch --branch $(branchName) https://$(gitUserName):$(gitPassword)#dev.azure.com/myCompany/My%20Test%20Project/_git
Where $(gitUserName) and $(gitPassword) configured as Alternate git credentials in
your Azure DevOps account.
Also wanted to ask about ending of git repository URL:
".../_git/Service.Azure.Core", you are trying to specify which folder to clone or maybe working directory? You can also try your previous code without that ending, just
https://dev.azure.com/myCompany/My%20Test%20Project/_git
I tried this from Powershell task and it worked
Its very simple just use
- powershell: |
git config user.email "$(Build.RequestedForEmail)"
git config user.name "$(Build.RequestedFor)"
git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" clone https://<org_Name>.visualstudio.com/<project_name>/_git/<repository_name>;
displayName: "<your task display name>"
This will happen when your path has space characters in it.
The workaround for this is to not use %20 in your URL and use actual space characters instead. Since your URL now has space characters, also make sure that your URL is wrapped inside double quotes.
So your
git clone https://dev.azure.com/myCompany/My%20Test%20Project/_git/Service.Azure.Core/
should be
git clone "https://dev.azure.com/myCompany/My Test Project/_git/Service.Azure.Core/"
Im currently trying to figure out how to deploy an gitlab project automatically using ci. I managed to run the building stage successfully, but im unsure how to retrieve and push those builds to the releases.
As far as I know it is possibile to use rsync or webhooks (for example Git-Auto-Deploy) to get the build. However I failed to apply these options successfully.
For publishing releases I did read https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/api/tags.md#create-a-new-release, but im not sure if I understand the required pathing schema correctly.
Is there any simple complete example to try out this process?
A way is indeed to use webhooks:
There are tons of different possible solutions to do that. I'd go with a sh script which is invoked by the hook.
How to intercept your webhook is up to the configuration of your server, if you have php-fpm installed you can use a PHP script.
When you create a webhook in your Gitlab project (Settings->Webhooks) you can specify for which kind of events you want the hook (in our case, a new build), and a secret token so you can verify the script has been called by Gitlab.
The PHP script can be something like that:
<?php
// Check token
$security_file = parse_ini_file("../token.ini");
$gitlab_token = $_SERVER["HTTP_X_GITLAB_TOKEN"];
if ($gitlab_token !== $security_file["token"]) {
echo "error 403";
exit(0);
}
// Get data
$json = file_get_contents('php://input');
$data = json_decode($json, true);
// We want only success build on master
if ($data["ref"] !== "master" ||
$data["build_stage"] !== "deploy" ||
$data["build_status"] !== "success") {
exit(0);
}
// Execute the deploy script:
shell_exec("/usr/share/nginx/html/deploy.sh 2>&1");
I created a token.ini file outside the webroot, which is just one line:
token = supersecrettoken
In this way the endpoint can be called only by Gitlab itself. The script then checks some parameters of the build, and if everything is ok it runs the deploy script.
Also the deploy script is very very basic, but there are a couple of interesting things:
#!/bin/bash
# See 'Authentication' section here: http://docs.gitlab.com/ce/api/
SECRET_TOKEN=$PERSONAL_TOKEN
# The path where to put the static files
DEST="/usr/share/nginx/html/"
# The path to use as temporary working directory
TMP="/tmp/"
# Where to save the downloaded file
DOWNLOAD_FILE="site.zip";
cd $TMP;
wget --header="PRIVATE-TOKEN: $SECRET_TOKEN" "https://gitlab.com/api/v3/projects/774560/builds/artifacts/master/download?job=deploy_site" -O $DOWNLOAD_FILE;
ls;
unzip $DOWNLOAD_FILE;
# Whatever, do not do this in a real environment without any other check
rm -rf $DEST;
cp -r _site/ $DEST;
rm -rf _site/;
rm $DOWNLOAD_FILE;
First of all, the script has to be executable (chown +x deploy.sh) and it has to belong to the webserver’s user (usually www-data).
The script needs to have an access token (which you can create here) to access the data. I inserted it as environment variable:
sudo vi /etc/environment
in the file you have to add something like:
PERSONAL_TOKEN="supersecrettoken"
and then remember to reload the file:
source /etc/environment
You can check everything is alright doing sudo -u www-data echo PERSONAL_TOKEN and verify the token is printed in the terminal.
Now, the other interesting part of the script is where is the artifact. The last available build of a branch is reachable only through API; they are working on implementing the API in the web interface so you can always download the last version from the web.
The url of the API is
https://gitlab.example.com/api/v3/projects/projectid/builds/artifacts/branchname/download?job=jobname
While you can imagine what branchname and jobname are, the projectid is a bit more tricky to find.
It is included in the body of the webhook as projectid, but if you do not want to intercept the hook, you can go to the settings of your project, section Triggers, and there are examples of APIs calls: you can determine the project id from there.
I'm using lftp to deploy a website via Travis CI. There is a build process before the deployment, for that reason a build directory is present and pushed to the root of the ftp server.
lftp $FTP_URL -e "glob -d mirror build . --reverse --delete-first --parallel=10 && exit"
It works quite well, but I dislike to have a downtime / temporary PHP parse errors because of missing files on my website. What is the best way to work arround that issue?
My first approach was an option to set a temporary directory, but the lftp man page says there is only a options for temporary files. I still tried the option but it didn't help.
My second approach was to use "mirror build temp" to use a temporary folder and then replace the root with it. The problem here is, that I cannot exclude the temp folder while deleting the old files and folders like rm -rf *.
For small changes not involving adding/removing php files set xfer:use-temp-file should be sufficient. Also don't use --remove-first, as it causes lftp to delete obsolete files before uploading.
For larger changes I'd create a separate directory for each version of the site and redirect the web server to the directory using .htaccess mod_rewrite or some other configuration file. This technique will allow atomic switch to the new version (and back if needed). Besides, you will be able to do final pre-production testing of the new version if you redirect to the new version conditionally based on your IP address or using some other rule.
If you don't want to re-upload whole site for each new version and the FTP server supports FXP with itself, then you can copy old version to a new directory using mirror old_directory ftp://user#example.com/new_directory, then update the new directory using mirror -eR local_dir new_directory.
This is a zero downtown pattern - each placeholder should be replaced:
lftp $FTP_URL -e "mirror {SOURCE} {TARGET}-new-{TIMESTAMP} --reverse --delete-first;
mv {TARGET} {TARGET}-old-{TIMESTAMP};
mv {TARGET}-new-{TIMESTAMP} {TARGET};
rm -rf {TARGET}-old-{TIMESTAMP};
exit"
cap is trying to create this location on my server to access my git repo (on bitbucket). Unfortunately this is a shared host, and the ssh keys are in my user directory, not in /tmp…. so this fails:
GIT_ASKPASS=/bin/echo GIT_SSH=/tmp/doman.com/git-ssh.sh /usr/bin/env git ls-remote
Can I configure this tmp dir to be in my home dir?
According to the Capistrano Github page you should set the :tmp_dir variable to a directory on your homepath like /home/user/tmp/capistrano
For example:
set :tmp_dir, '/home/user/tmp/capistrano'
This should be a very simple thing to have run, but for some reason it won't work with my Mercurial repository. All I want is for the remote repo to automatically run hg update whenever someone pushes to it. So I have this in my .hg/hgrc file:
[hook]
changegroup = hg update
Simple, right? But for some reason, this never executes. I also tried writing a shell script that did this. .hg/hgrc looked like this:
[hooks]
changegroup = /home/marc/bin/hg-update
and hg-update looked like this:
#!/bin/sh
hg help >> /home/marc/works.txt;
hg update >> /home/marc/works.txt;
exit 0;
But again, this doesn't update. The contents of hg help are written out to works.txt, but nothing is written out for hg update. Is there something obvious I'm missing here? This has been plaguing me for days and I just can't seem to get it to work.
Update
Okay so again, using the -v switch on the command line from my workstation pushing to the remote repo doesn't print any verbose messages even when I have those echo lines in .hg/hgrc. However, when I do a push from a clone of the repo on the same filesystem (I'm logged in via SSH), this is what I get:
bash-3.00$ hg -v push ../test-repo/
pushing to ../test-repo/
searching for changes
1 changesets found
running hook prechangegroup: echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
Remote repo is at 821:1f2656753c98
Remote repo wdir is at 821:1f2656753c98
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
running hook changegroup: echo "Updating.... `hg update -v`"
echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
Updating.... resolving manifests
getting license.txt
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
Remote repo is at 822:389a6c7276c6
Remote repo wdir is at 822:389a6c7276c6
So it works, but again only when I push from the same filesystem. It doesn't work if I try pushing to the repo from another workstation over the network.
Well, after going through the same steps of frustration as Marc W did a while ago, I finally found the solution to the problem, at least when remote serving is done with the hgwebdir WSGI script.
I found out that when using this kind of remote push via HTTP or HTTPS, Mercurial simply ignores everything you write into the .hg/hgrc file or your repository. However, entering the hook in the hgwebdir config does the trick.
So if the bottom line in your hgwebdir.wsgi script is something like
application = hgwebdir('hgweb.config')
the [hooks] config section needs to go into the mentioned hgweb.config.
One drawback is that these hooks are executed for every repository listed in the [paths] section of that config. Even though HG offers another WSGI-capable function (hgweb instead of hgwebdir) to serve only a single repository, that one doesn't seem to support any hooks (neither does it have any config).
This can, however, be circumvented by using a hgwebdir as described above and having some Apache RewriteRule map everything into the desired subdirectory. This one works for me:
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/reponame
RewriteRule ^(.*)$ reponame/$2 [QSA]
Have fun using your remote hooks over HTTP :D
I spent some time researching this myself. I think the answer to problem is described concisely here:
Output has to be redirected to stderr (or /dev/null), because stdout
is used for the data stream.
Basically, you're not redirecting to stderr, and hence polluting stdout.
First of all, I want to correct a few comments above.
Hooks are invoked also when pushing over file system.
It is not necessary to keep the hook in the repo on which you want them to operate. You can also write the same hook as in your question on the user end. You have to change the event from changegroup to outgoing and also to specify the URL of remote repo with the -R switch. Then if the pushing user has sufficient privileges on the remote repo, the hook will execute successfully.
.hg/hgrc
[hooks]
outgoing = hg update -R $HG_URL
Now towards your problem.... I suggest creating both prechangegroup and changegroup hooks and printing some debugging output.
.hg/hgrc
[hooks]
prechangegroup = echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
changegroup = echo "Updating.... `hg update -v`"
echo "Remote repo is at `hg tip -q`"
echo "Remote repo wdir is at `hg parents -q`"
And also push with the -v switch, so that you may know which hook is running. If you still can't figure out, post the output. I might be able to help.
My problem was that my hgwebdir application ran as the "hg" user, but the repository was owned by me, so I had to add in this bit of config to hgweb.config to get it to run the hooks:
[trusted]
users = me
You need to have it in the remote repositiory's hgrc. It sounds as if it's in your local repo.
Edit: It also depends on how you're pushing. Some methods don't invoke hooks on the right side. (ssh does, I think HTTP does, file system does not)
Edit2: What if you push "locally" at the remote repo's computer. You might have different users/permissions between the webserver and the hgrc-file. (See [server] and trusted directives for hgrc.)
I had the same problem pushing from Windows Eclipse via http, but after capturing stderr, I found that the full path was needed to the hg.bat file. My hooks section now looks like:
[hooks]
incoming = c:\Python27\Scripts\hg.bat update > hg_log.txt 2>>hg_err.txt
Hope this helps someone else.
SteveT
Try turning on hook debugging to see why it's not running.
Likely a permissions issue or something like that.
took a while but I got it working.
I started with
[hooks]
tag=set >&2
commit=set >&2
the >&2 pipes it to standard error so remote consoles will show it.
when remote this should output in console if it is running
hg push https://host/hg -v
It wasn't.
I was using hgweb.cgi so I switched to hgweb.wsgi with no difference.
what I discovered is that some hooks don't get called on remote.
when I switched it to
[hooks]
incoming= set >&2
the hooks tag and commit don't seem to get called but incoming and changeset do get called. I haven't confirmed the others.
now that I got it working I switched back to hgweb.cgi and everything works the same.
Tthe reason I've found for this has nothing to do with redirecting stdout to stderr. As you may see in the wiki page it is not specified on the wiki's current version
https://www.mercurial-scm.org/wiki/FAQ#FAQ.2FCommonProblems.Any_way_to_.27hg_push.27_and_have_an_automatic_.27hg_update.27_on_the_remote_server.3F
The problem I've found is around permissions.
In my original setup, I had a user, lets say hguser with a repo on its home, and a script /etc/init.d/hg.init to launch hg serve. The problem being hg serve was being run by root, while most files under the repo pertained to hguser (some of them switched to root at some point, but it won't mind, since I'll correct them with chown)
Solution:
chown -R hguser:hguser /home/hguser/repo (to correct ALL files, back to hguser)
launch su hguser -c "hg serve ..." (in my case from /etc/init.d/hg.init)
changegroup = hg update -C under [hooks] in repo/.hg/hgrc as usual
Now it should work on push
PS: in my case, I rather update to the head of a specific branch, so I use hg update -C -r staging, to make the staging server update only to the head of the intended branch, even if the tip is from another branch (like development for instance)
BTW my hg.init script ended up like this: (notice the su hguser part)
#!/bin/sh
#
# Startup script for mercurial server.
#
# #see http://jf.blogs.teximus.com/2011/01/running-mercurial-hg-serve-on-linux.html
HG=/usr/bin/hg
CONF=/etc/mercurial/hgweb.config
# Path to PID file of running mercurial process.
PID_FILE=/etc/mercurial/hg.pid
state=$1
case "$state" in
'start')
echo "Mecurial Server service starting."
(su hguser -c "${HG} serve -d --webdir-conf ${CONF} -p 8000 --pid-file ${PID_FILE}")
;;
'stop')
if [ -f "${PID_FILE}" ]; then
PID=`cat "${PID_FILE}"`
if [ "${PID}" -gt 1 ]; then
kill -TERM ${PID}
echo "Stopping the Mercurial service PID=${PID}."
else
echo Bad PID for Mercurial -- \"${PID}\"
fi
else
echo No PID file recorded for mercurial
fi
;;
*)
echo "$0 {start|stop}"
exit 1
;;
esac
PS: due credit to http://jf.blogs.teximus.com/2011/01/running-mercurial-hg-serve-on-linux.html