In order for Jenkins to be able to have access to multiple repositories on the same server, I set the .ssh/config as follow:
Host a.github.com
HostName github.com
User git
IdentityFile ~/.ssh/project-a-id_rsa
# same for other repos
and set the Jenkins jobs' Source Code Management (SCM), to git and git#a.github.com:user/repo_a.git. It works fine.
Problem
I want those jobs to be triggered on push events so I set a webhook service in github, .i.e, Jenkins (GitHub plugin). The request received from the webhook are "POST for https://github.com/user/repo_a" which is a different host than the one set in the SCM, .i.e, a.github.com.
Because they are different, the job does not build automatically.
Ugly Solution
I got something running by setting the SCM to github.com and override the remote url of the project's git config once cloned with a.github.com. So the SCM would match the webhook, and jenkins when running git push would use the .ssh/config info.
Question
What else can I do ? Is there a better, easily automated way to achieve this?
I stopped using the deploy key and added my own account credentials on jenkins to be able to deal with all repositories without having to change the host with .ssh/config.
Related
I would like to understand how the auto git checkout is happening azuredevops build pipelines. How the authentication to ADO repo is happening from the ADO build agent. Which user is used by the ADO pipeline to clone or checkout this repo.
My concern is ,when i trigger a pipeline, i could see that the repository is checkedout to build agent. Which user is used by the ADO ? when i try to push, its asking for AD authentication, So how the checkout and push are differ in ADO pipeline perspective?
when you set up your pipeline initially, you specified where you code resides, e.g. on GitHub. As part of that step a service connection will have been created (you have been asked for it). So whatever you specified there, will be used. You can check it out in the project settings under "Service Connections"
It depends on which source code tool you use. For example If you use Github, you will have to setup a connection with it. This can be accomplished on project settings under Github connections. You can then use this service connection and checkout your code.
This Github integration has been made from a particular User and by navigating on github -> Settings -> Integrations -> Applications, you will notice the exact permissions.
I imagine that Github/Azure Devops then will use this integration object that is created for the authentication.
On Azure Repos repositories, you do not need a service connection in place. The repository can be checked out automatically.
When you run the pipeline you can see the exact commands that have been executed for the authentication.
git remote add origin https://ORG.visualstudio.com/test-project/_git/test-project
git config gc.auto 0
git config --get-all http.https://ORG.visualstudio.com/test-project/_git/test-project.extraheader
git config --get-all http.extraheader
git config --get-regexp .*extraheader
git config --get-all http.proxy
git config http.version HTTP/1.1
git -c http.extraheader="AUTHORIZATION: bearer ***" fetch --force --tags --prune --prune-tags --progress --no-recurse-submodules origin
Microsoft-hosted agents run on secure Azure platform. However, you must be aware of the following security considerations.
Although Microsoft-hosted agents run on Azure public network, they are not assigned public IP addresses. So, external entities cannot target Microsoft-hosted agents.
Microsoft-hosted agents are run in individual VMs, which are re-imaged after each run. Each agent is dedicated to a single organization, and each VM hosts only a single agent.
There are several benefits to running your pipeline on Microsoft-hosted agents, from a security perspective. If you run untrusted code in your pipeline, such as contributions from forks, it is safer to run the pipeline on Microsoft-hosted agents than on self-hosted agents that reside in your corporate network.
When a pipeline needs to access your corporate resources behind a firewall, you have to allow the IP address range for the Azure geography. This may increase your exposure as the range of IP addresses is rather large and since machines in this range can belong to other customers as well. The best way to prevent this is to avoid the need to access internal resources.
Hosted images do not conform to CIS hardening benchmarks. To use CIS-hardened images, you must create either self-hosted agents or scale-set agents.
Taken from Microsoft-hosted agents - Security.
The most important part is probably
Microsoft-hosted agents are run in individual VMs, which are re-imaged after each run. Each agent is dedicated to a single organization, and each VM hosts only a single agent.
Next to that, check Create and manage agent pools - Security of agent pools.
I have my personal GitHub account which has been given access to Repos that exist as part of an organization. I'm trying to create a pull-request from within VSCode for these Repos, but I get the below error.
Is there a setting I'm just missing?
[Info] GitHubRepository> Creating pull requests failed: HttpError: Validation Failed: {"resource":"PullRequest","field":"head","code":"invalid"}
HttpError
Check first if you have a git config credential.helper set (to, for instance on windows, manager-core)
If you have, that means the wrong credentials (username/password) are cached in said credential helper. They are not the ones for your organization GitHub account.
See GitHub "Caching your GitHub credentials in Git" and update them.
I would like to create a webhook within Gitlab to automatically update a mirror repository on Github, whenever a push event happens. I've checked this page, but I didn't understand how it is done.
My Gitlab version is 6.5. Here is the configuration page:
What should I put in URL? Where do I need to place the script to update the repository?
You don't need a webhook for that. A regular post-receive hook will work very well.
To create and use such a hook you just have to login on the server where your gitlab is installed and create an ssh key for git user.
sudo -u git ssh-keygen -f /home/git/.ssh/reponame_key
(do not type any passphrase when prompted)
Go to your github account and add the public key (it's been created as /home/git/ssh/reponame_key.pub) to your project as a deploy key.
have a look at https://help.github.com/articles/managing-deploy-keys if you need help with that.
Once that is done, you just have to configure the connection between your git server and github's:
add an alias to git user's ssh configuration (add following lines to /home/git/.ssh/config - create it if it's not present)
Host reponame
IdentityFile /home/git/.ssh/reponame_key
HostName github.com
User git
Now add the new remote (using the alias you just created) to your repository:
cd /home/git/repositories/namespace/reponame.git
git remote add --mirror github reponame:youruser/reponame.git
Now that everything is in place you'll have to create the actual hook:
cd /home/git/repositories/namespace/reponame.git/hooks
echo "exec git push --quiet github &" >> post-receive
chmod 755 post-receive
The lastcommand is very important because git will check if a hook is executable before running it.
That's it!
(Replace reponame, namespace and youruser according to your real accounts and enjoy).
Last note: if you want your name andavatar near commits on github, make sure that the email address you are using on gitlab is one of the addresses inked to your github account as well. You'll see your gitlab username otherwise.
If you aren't hosting your own GitLab, GitLab.com has introduced this feature directly, without any workarounds.
From within a project use the gear icon to select Mirror Repository
Scroll down to Push to a remote repository
Checkmark Remote mirror repository: Automatically update the remote mirror's branches, tags, and commits from this repository every hour.
Enter the repository you want to update; for GitHub you can include your username and password in the URL, like so: https://yourgithubusername:yourgithubpassword#github.com/agaric/guts_discuss_resource.git —as noted in the comments, it is much better securitywise to use your GitHub access token here instead of login credentials; will update the answer when i've tested.
For WebHooks processing I'm using sinatra web server.
require 'sinatra'
post '/pew' do
puts JSON.parse request.body.read
# here can be placed signal code to run commit processing script
end
register webhook for push events(or other) to http://localhost:4567/pew within GitLab
and since this moment on each commit gitlab will be sending commit info to url.
I am trying to understand how does the gerrit-trigger in Jenkins works in details?
Also, how is the test for the triggered cose is being invoked ?
Thanks,
The gerrit trigger works like this:
It connects to the gerrit server using ssh and uses the gerrit stream-events command
It then watches this stream as the data comes in
It will try to match the events to triggers that have defined in your projects
Potential pit-falls:
Jenkins user has improper ssh credentials
Jenkins user does not have the stream-events rights
How to check:
Login as jenkins user
ssh -p 29418 jenkins#your.domain.com gerrit stream-events
Push a commit to the server and you should see things on your stream
Problems:
ssh connection failed? setup you ssh key pair
No streaming right? Go to the All-Projects->Access and under Global Capabilities add Stream Events to the Non-Interactive Users group
We recently moved our github from one account owner to another, and now all of the sudden when I do a git pull or any git command on the remote repo, it asks me for github username and password.
My git-config says:
[github]
user = kamilski81
token = *********
Any idea how I can stop it asking me for credentials and remember everything, does the new owner have to setup my ssh keys or something of that sort?
Following this article sorted it out for me:
https://help.github.com/articles/why-is-git-always-asking-for-my-password
git remote set-url origin git#github.com:user/repo.git
The github username and token in the gitconfig is only used for interactions with the GitHub API (for instance the hub gem and the github tool).
If you are pushing and pulling from a GitHub repository over https, then you need to specify your credentials every time, or hook into a password manager to remember the credentials for you. How to do this depends on your platform.
The easiest thing to do is to go to your github account and submit your SSH public key to your account, and then switch your github remote to push/pull over ssh instead of https.
[edit] After re-reading the question I noticed that you mentioned it was previously a GitHub repo that was just moved from one owner to another. If that's the case, and you ARE able to push and pull by specifying your credentials, it sounds to me to be one of two scenarios:
The previous remote used ssh, and GitHub has your SSH key; but when the owner changed and you updated the remote, you updated it to go over https.
The previous remote used https, but you had a password manager setup correctly to deal with your credentials, which no longer works.
After doing "remote -v"....the issue was that my remote was using an "https" remote rather than "git" remote. Changing the remote to use git rather than https solved everything.
If you execute only once git pull and Git client still asking you the credential without do the git pull for you, the problem should be because your credential is incorrect or not setup yet.
but if you get the dialog ask for credential one for each operation, it mean that your credential isn't remembered. To do that, you may use Pageant to store your SSH private key, so, everytime you do something, it will retrieve your credential from Pageant instead to asking you.