Github Jenkins webhook security problems - github

I've been using Github Jenkins webhook for a while with no problem.
I have my Jenkins running behind a Nginx proxy and the only security installed was the nginx htpasswd.
So my Github webhook looked like this:
https://user:pass#jenkins.blablabla.biz/job/script/buildWithParameters
Being user and pass the credentials from htpasswd file.
Right now I'm trying to add users to my Jenkins and I activated matrix based security to it with the option to register and login to Jenkins.
The server works perfect initially asking for nginx credentials and jenkins login second.
I created a github user in Jenkins and I generated a token for him.
Now I changed my Github webhook to use the new Jenkins github user credentials:
https://github:token#jenkins.blablabla.biz/job/script/buildWithParameters
As long as I know this should work. But it does not.
I think that the problem here is having the two security systems activated (nginx htpasswd and jenkins login). But I want both.
Do I have to use the two credentials in my webhook? How can I do that?
When I use nginx credential Github receives this response from jenkins when webhook is triggered:
Authentication required You are authenticated as: anonymous
Groups that you are in: Permission you need to have (but didn't):
hudson.model.Hudson.Read ... which is implied by:
hudson.security.Permission.GenericRead ... which is implied by:
hudson.model.Hudson.Administer
And when I use jenkins credential Github receives this response from jenkins when webhook is triggered:
401 Authorization Required 401 Authorization Required
nginx/1.11.10

You can disable basic auth in you nginx.conf for one url:
location /github-webhook/ {
auth_basic "off";
...
}
location / {
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/.htpasswd;
...
}

Finally I decided to shut down the Nginx Http based security and rely only on the jenkins security. Now my jenkins server is visible from anywhere.
I changed the format of the Github jenkins webhook in order to use the Jenkins token root plugin:
https://wiki.jenkins-ci.org/display/JENKINS/Build+Token+Root+Plugin
I didn't find a way to use both authentication methods.
The idea was to ask for an extra pass (http based) If someone tried to access from outside the office.
That's all.

Related

Azure Databricks API, cannot add repos using service principal and API calls

I need to add Azure DevOps repos to azure databricks repo by using databricks API at this link. I am using a service principal credentials for this. The service principal is already added as admin user to databricks. With my service principal I can get the list of repos and even delete them. But when I want to add a repo to a folder, it raises the following error:
{
"error_code": "PERMISSION_DENIED",
"message": "Missing Git provider credentials. Go to User Settings > Git Integration to add your personal access token."
}
I am not using my own credentials to use a PAT token, instead I am getting a bearer token by sending request to https://login.microsoftonline.com/directory-id/oauth2/token and use it to authenticate. This works for get repos, delete repos and get repos/repo-id. Just for creating a repo (adding repo by using post method to /repos) it is failing.
If I still use a PAT instead of bearer token, I get the following error:
{
"error_code": "PERMISSION_DENIED",
"message": "Azure Active Directory credentials missing. Ensure you are either logged in with your Azure
Active Directory account or have setup an Azure DevOps personal access token (PAT) in User Settings > Git Integration.
If you are not using a PAT and are using Azure DevOps with the Repos API, you must use an AAD access token. See https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/app-aad-token for steps to acquire an AAD access token."
}
I am using postman to construct the requests. To generate the error I am getting I am using the following:
method: post
url-endpoint: https://adb-databricksid.azuredatabricks.net/api/2.0/repos
body:
url: azure-devops-repo
provider: azureDevOpsServices
path: /Repos/folder-name/testrepo
header:
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbG... (Construct it by appending bearer token to key wor "Bearer")
X-Databricks-Azure-SP-Management-Token: management token (get it like bearer token by using resource https://management.core.windows.net/)
X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/azure-subscription-id/resourceGroups/resourcegroup-name/providers/Microsoft.Databricks/workspaces/workspace-name
Here the screen shot of the postman:
Please note that I have used exactly same method of authentication for even creating clusters and jobs and deleting repos. Just for adding and updating repos it is failing. I'd like to know how I can resolve the error PERMISSION_DENIED mentioned above.
To make service principal working with Databricks Repos you need following:
Create an Azure DevOps personal access token (PAT) for it - Azure DevOps Git repositories don't support service principals authentication via AAD tokens (see documentation). (The service connection for SP that you configured is used for connection to other Azure services, not to the DevOps itself).
That PAT needs to be put into Databricks workspace using Git Credentials API - it should be done when configuring first time or when token is expired. When using this API you need to use AAD token of the service principal. (btw, it could be done via Terraform as well)
After it's done, you can use Databricks Repos APIs or databricks-cli to perform operations with Repos - create/update/delete them. (see previous answer on updating the repo)
Have you setup the git credentials using this endpoint before creating the repo through the API ?
https://docs.databricks.com/dev-tools/api/latest/gitcredentials.html#section/Authentication
If you do not setup this first, you can get the error when trying to create a repo.
Listing & deleting a repo only require a valid authentication to Databricks (Bearer token or PAT) and doesn't require valid git credentials.
When trying to create a repo, you need authorizations on the target repository that is on Azure Devops in your case.
So you need to call the git-credentials endpoint (it's the same syntax on AWS and Azure) to create it.
Once your git credentials up-to-date, the creation of the repo should work as intended.

Jenkins Embeddable Build Status plugin gets redirected due to SSO

I'm trying to add Jenkins build status using the Embeddable Build Status plugin onto Github README.md file. I have SSO for Github and Jenkins and I assumed it will have the login info in the browser cookies or wherever it stored that info but looks like it doesn't work that way (The same URL works and picks up SSO info and doesn't prompt me when I open it from a browser. It just doesn't like Github pulling that info)
I see a 302 redirect on the Jenkins status badge link that is getting redirected to the SSO page and is expecting JS to be enabled for it to work which isn't possible on a Github README.md page (Even if that was possible I would not see a login prompt)
The console has a warning for this redirected URL request as Cross-Origin Read Blocking (CORB) blocked cross-origin response <the-URL> with MIME type text/html. See https://www.chromestatus.com/feature/5629709824032768 for more details.
I also set the Jenkins config to ViewStatus for anonymous uses based on JENKINS-17798 and I'm using the unprotected link but it still needs SSO
Is there a way I can fix this?
Read this first. Cors-Filter Plugin for Jenkins.
Before you go debugging. First make sure that CORS is enabled for the Jenkins instance. You can go to configure system in jenkins and then to CORS Filter:
A very short summary.
Access-Control-Allow-Origins - List domains that may access the jenkins url.
Access-Control-Allow-Headers - Headers that can be used to make actual request
Access-Control-Expose-Headers - List of headers browser are allowed to access.
Whenever one tries to go authenticating from one domain to another it usually gets redirected in jenkins if CORS Filter is not active. So try this first.

Metaflow: "Missing authentication token" when accessing the metadata/metaflow service URL in the browser

I’m currently experimenting on Metaflow. I followed the documentation and was able to deploy an aws setup with the given cloud formation template.
My question is why is that I’m always getting a:
message: "Missing Authentication Token"
when I access METAFLOW_SERVICE_URL in the browser, even if I made sure that the APIBasicAuth was set to false during the creation of cloudformation?
Shouldn’t this setting make the metadata/metaflow service accessible without the authentication/api key?
How can I resolve this? Or is this expected? That is, I cannot really view the metadata/metaflow service url via browser?
Thanks in advance
This was resolved under this github issue.
You still need to set the x-api-key header if you are trying to access the service url via the browser. To get the api-key you can go to the aws console
Api Gateway -> Api Keys -> show api key
Alternatively you can use the metaflow client in the sagemaker notebook which should be automatically setup for you via the template.
Also worth mentioning that there are two sets of endpoints: The one provided by the api gateway (which you seem to be hitting) and the one provided by the service itself. The api gateway forwards the requests the the service endpoints but needs the x-api-key to be set in the header. You can probably try hitting the service endpoints directly since you disabled auth.

How do I specify a reply URL using AKS and AAD integration?

I have a Kubernetes (AKS) setup with AAD integration, (there is a server application and client application). When I call kubectl, aauthentication pop up and after confirmed grant to the client app, it shows an error that the reply URL is missing.
How do I pick a reply URL in AAD application registration?
I found the issue. There is no need to specify a reply URL.
I use a script to request resources, but I need a manual step to grant these requests on the portal.
You may have missed any step as documented in AKS AAD Integration.

Jenkins and GitHub webhook: HTTP 403

I have a GitHub repository which I would like to have notify Jenkins of new commits via a post-receive hook. I've installed the GitHub plugin into Jenkins and have allowed for Jenkins to manage it's own hook URLs. The project has the correct git repository URL and is instructed to "Build when a change is pushed to GitHub". When I have GitHub send a test payload I find this in the nginx webserver that front's Jenkins:
207.97.227.233 - - [15/Sep/2011:07:36:51 +0000] "POST /github-webhook/ HTTP/1.1" 403 561 "-" "-"
I was running SSL so I disabled it to no effect. Do I need to provide special permissions to an anonymous user in the permissions matrix?
Please forgive the lack of configuration files: I'm happy to share those that might exist but I don't know what might be useful to share.
As I don't allow anonymous access, it turns out I needed to create a specific user for GitHub pushes and to grant it Overall read, Job create and Job read. It was also necessary to bundle the authentication into the webhook URL, like so:
https://foo:password#jenkins.example.com/github-webhook/
From Github's instructions ("Configuring global authentication", the first section),
1. Create a user in Jenkins which has, at a minimum, Job/Build permissions
2. Log in as that user (this is required even if you are a Jenkins admin user), then click on the user's name in the top right corner of the page
3. Click 'Configure,' then 'Show API Token...', and note/copy the User ID and API Token.
4. In GitLab, when you create webhooks to trigger Jenkins jobs, use this format for the URL and do not enter anything for 'Secret Token': http://USERID:APITOKEN#JENKINS_URL/project/YOUR_JOB
In my case I used http://USERID:APITOKEN#myIPaddress:808/ and no project/YOUR_JOB
As previously mentioned Jenkins does not allow anonymous access (you don't want people to trigger builds for you!). Adding a username and password in the clear is not the best solution.
1) If your Jenkins is behind a VPN, you need to make sure that your Jenkins webhooks URL are exposed
2) In the GitHub plugin configuration add the correct accessible Jenkins webhook URL(https://jenkins.example.com/github-webhook/) in Override Hook URL
3) Add a secret in the Shared secret section (preferably a long random string). This would be a token sent from Jenkins to GitHub and vice versa for authentication. This should be different from the GitHub Server Credentials!
4) Make sure the user has owner access to add webhooks to your GitHub repos
5) Don't forget to re-register the webhooks after this
https://username:password#jenkins.example.com/github-webhook/ worked for me but the password has to be encoded for the payload to work. My github was not able to contact jenkins due to nginx authorisation but passing username and encoded password worked!