I have configured a Github API token in Jenkins to be used in jobs to checkout the repos.
I also have the credentials binding plugin installed to inject other types of credentials into a pipeline using the withCredentials command.
But now each pipeline can read the GitHub API token using withCredentials. How can i restrict access to certain global credentials for all jobs?
We can create Credential object at folder level instead of Global. So that only the jobs which are in that folder can assess those credential objects. The jobs out of that folder can't access those credentials objects. This may help you to resolve the issue.
Related
I'm setting up a self-hosted github runner on a k8s cluster. The runner deployment requires that I supply it a token so that it can link to the proper repository/github account.
There are a lot of examples that I've seen that use a Personal Access Token as a secret resource. It seems like bad practice to use a single user's PAT for what should be a service account token. I am wondering if there are recommended way to use a repository or organization-level token stored as a secret.
Possibly the GITHUB_TOKEN, but this seems too transient as it appears to expire following the completion of a workflow.
Curious about best practices in this case.
You can create a registration token for an organization following this doc.
From this doc Github recommends that you only use self-hosted runners with private repositories. This is because forks of your public repository can potentially run dangerous code on your self-hosted runner machine by creating a pull request that executes the code in a workflow.
Do not store secrets in the host runner When a GitHub Action uses the self-hosted runner, it clones the code in a workdir _work.We must ensure that no secrets (application, system, ..) are accessible in this folder.
For more information follow this doc.
I need to add Azure DevOps repos to azure databricks repo by using databricks API at this link. I am using a service principal credentials for this. The service principal is already added as admin user to databricks. With my service principal I can get the list of repos and even delete them. But when I want to add a repo to a folder, it raises the following error:
{
"error_code": "PERMISSION_DENIED",
"message": "Missing Git provider credentials. Go to User Settings > Git Integration to add your personal access token."
}
I am not using my own credentials to use a PAT token, instead I am getting a bearer token by sending request to https://login.microsoftonline.com/directory-id/oauth2/token and use it to authenticate. This works for get repos, delete repos and get repos/repo-id. Just for creating a repo (adding repo by using post method to /repos) it is failing.
If I still use a PAT instead of bearer token, I get the following error:
{
"error_code": "PERMISSION_DENIED",
"message": "Azure Active Directory credentials missing. Ensure you are either logged in with your Azure
Active Directory account or have setup an Azure DevOps personal access token (PAT) in User Settings > Git Integration.
If you are not using a PAT and are using Azure DevOps with the Repos API, you must use an AAD access token. See https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/app-aad-token for steps to acquire an AAD access token."
}
I am using postman to construct the requests. To generate the error I am getting I am using the following:
method: post
url-endpoint: https://adb-databricksid.azuredatabricks.net/api/2.0/repos
body:
url: azure-devops-repo
provider: azureDevOpsServices
path: /Repos/folder-name/testrepo
header:
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbG... (Construct it by appending bearer token to key wor "Bearer")
X-Databricks-Azure-SP-Management-Token: management token (get it like bearer token by using resource https://management.core.windows.net/)
X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/azure-subscription-id/resourceGroups/resourcegroup-name/providers/Microsoft.Databricks/workspaces/workspace-name
Here the screen shot of the postman:
Please note that I have used exactly same method of authentication for even creating clusters and jobs and deleting repos. Just for adding and updating repos it is failing. I'd like to know how I can resolve the error PERMISSION_DENIED mentioned above.
To make service principal working with Databricks Repos you need following:
Create an Azure DevOps personal access token (PAT) for it - Azure DevOps Git repositories don't support service principals authentication via AAD tokens (see documentation). (The service connection for SP that you configured is used for connection to other Azure services, not to the DevOps itself).
That PAT needs to be put into Databricks workspace using Git Credentials API - it should be done when configuring first time or when token is expired. When using this API you need to use AAD token of the service principal. (btw, it could be done via Terraform as well)
After it's done, you can use Databricks Repos APIs or databricks-cli to perform operations with Repos - create/update/delete them. (see previous answer on updating the repo)
Have you setup the git credentials using this endpoint before creating the repo through the API ?
https://docs.databricks.com/dev-tools/api/latest/gitcredentials.html#section/Authentication
If you do not setup this first, you can get the error when trying to create a repo.
Listing & deleting a repo only require a valid authentication to Databricks (Bearer token or PAT) and doesn't require valid git credentials.
When trying to create a repo, you need authorizations on the target repository that is on Azure Devops in your case.
So you need to call the git-credentials endpoint (it's the same syntax on AWS and Azure) to create it.
Once your git credentials up-to-date, the creation of the repo should work as intended.
I want to run pipelines from a powershell script, and I want to do that in a secured way.
Today my powershell code runs the pipelines with System access token ( I'm talking about System.AccessToken ) by calling the REST API for Azure DevOps; I see that the URL of the API is the same as the public one.
What would happen if someone listening to the traffic discovers the System Access Token value in the request? I would like to find a way to avoid public traffic, we are avoiding PAT (Personal Access Token).
What are the risks of calling pipelines from powershell in the same azure devops server with System Access Token and a public URI of the API?
The System.AccessToken (job access token) is a security token that is dynamically generated by Azure Pipelines for each job at run time.
This means that the token is generated when the job is started up and revoked when the job is completed. And each time when you run the pipeline job, it will generated a new token for use.
The valid period of a System.AccessToken is not long-term. And our Azure DevOps also have taken the effective measures to prevent the token from being leaked when used in the pipeline.
So, you can use the System.AccessToken in your pipeline with confidence.
For more details, you can see this document.
I'm trying to use the TFVC API from a build task without using Personal Access Token.
I've read that if I'll use the Client SDK it should authenticate automatically but I'm failing to get it working.
Another alternative I'm considering is to run tf.exe but it requires authentication as well.
Basically I would like to get the changesets for a certain build using my Build Task with minimum info from the user (hence I don't want Personal Access Token or Username/Password).
Is that doable?
You should be able to authenticate with an OAuth token. It's populated in the build variable $(System.AccessToken) or the environment variable SYSTEM_ACCESSTOKEN.
You provide it to the REST API with the header Authorization set to Bearer [token], where [token] is the OAuth token value.
If you queue a build with TFVC repository and then check the logs for "Get Sources" step, you will find how to use TF.exe to get source during the build.And then you can copy the command and use it in a CommandLine/PowerShell task like following:
Make sure that "Allow Scripts to access token" option is enabled.
I have some powershell scripts in my CI server to check the state of some WebJobs.
But I have few problems.
I'm using publish settings file, but it expires and my build starts to fail.
I don't want to use a Management Certificate that will expose all management features.
And I don't want to put my user credentials on the CI server that will also expose all management features.
There is any way to create a CI user or credential with restricted permissions?
Thanks!
Azure Functions provides a good solution to this problem. You can create a Service Principle account, with certificate login and then restrict that account to whatever actions you need it to allow (via RBAC)
You can then have an Azure PowerShell script running in Functions, that is called from a webhook from your CI engine. That way the only credentials that are stored on your CI are the webhook secret, and if your CI engine has a static IP you can verify that commands only come from that address, and drop anything else.