I'm trying to use the TFVC API from a build task without using Personal Access Token.
I've read that if I'll use the Client SDK it should authenticate automatically but I'm failing to get it working.
Another alternative I'm considering is to run tf.exe but it requires authentication as well.
Basically I would like to get the changesets for a certain build using my Build Task with minimum info from the user (hence I don't want Personal Access Token or Username/Password).
Is that doable?
You should be able to authenticate with an OAuth token. It's populated in the build variable $(System.AccessToken) or the environment variable SYSTEM_ACCESSTOKEN.
You provide it to the REST API with the header Authorization set to Bearer [token], where [token] is the OAuth token value.
If you queue a build with TFVC repository and then check the logs for "Get Sources" step, you will find how to use TF.exe to get source during the build.And then you can copy the command and use it in a CommandLine/PowerShell task like following:
Make sure that "Allow Scripts to access token" option is enabled.
Related
I need to add Azure DevOps repos to azure databricks repo by using databricks API at this link. I am using a service principal credentials for this. The service principal is already added as admin user to databricks. With my service principal I can get the list of repos and even delete them. But when I want to add a repo to a folder, it raises the following error:
{
"error_code": "PERMISSION_DENIED",
"message": "Missing Git provider credentials. Go to User Settings > Git Integration to add your personal access token."
}
I am not using my own credentials to use a PAT token, instead I am getting a bearer token by sending request to https://login.microsoftonline.com/directory-id/oauth2/token and use it to authenticate. This works for get repos, delete repos and get repos/repo-id. Just for creating a repo (adding repo by using post method to /repos) it is failing.
If I still use a PAT instead of bearer token, I get the following error:
{
"error_code": "PERMISSION_DENIED",
"message": "Azure Active Directory credentials missing. Ensure you are either logged in with your Azure
Active Directory account or have setup an Azure DevOps personal access token (PAT) in User Settings > Git Integration.
If you are not using a PAT and are using Azure DevOps with the Repos API, you must use an AAD access token. See https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/app-aad-token for steps to acquire an AAD access token."
}
I am using postman to construct the requests. To generate the error I am getting I am using the following:
method: post
url-endpoint: https://adb-databricksid.azuredatabricks.net/api/2.0/repos
body:
url: azure-devops-repo
provider: azureDevOpsServices
path: /Repos/folder-name/testrepo
header:
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbG... (Construct it by appending bearer token to key wor "Bearer")
X-Databricks-Azure-SP-Management-Token: management token (get it like bearer token by using resource https://management.core.windows.net/)
X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/azure-subscription-id/resourceGroups/resourcegroup-name/providers/Microsoft.Databricks/workspaces/workspace-name
Here the screen shot of the postman:
Please note that I have used exactly same method of authentication for even creating clusters and jobs and deleting repos. Just for adding and updating repos it is failing. I'd like to know how I can resolve the error PERMISSION_DENIED mentioned above.
To make service principal working with Databricks Repos you need following:
Create an Azure DevOps personal access token (PAT) for it - Azure DevOps Git repositories don't support service principals authentication via AAD tokens (see documentation). (The service connection for SP that you configured is used for connection to other Azure services, not to the DevOps itself).
That PAT needs to be put into Databricks workspace using Git Credentials API - it should be done when configuring first time or when token is expired. When using this API you need to use AAD token of the service principal. (btw, it could be done via Terraform as well)
After it's done, you can use Databricks Repos APIs or databricks-cli to perform operations with Repos - create/update/delete them. (see previous answer on updating the repo)
Have you setup the git credentials using this endpoint before creating the repo through the API ?
https://docs.databricks.com/dev-tools/api/latest/gitcredentials.html#section/Authentication
If you do not setup this first, you can get the error when trying to create a repo.
Listing & deleting a repo only require a valid authentication to Databricks (Bearer token or PAT) and doesn't require valid git credentials.
When trying to create a repo, you need authorizations on the target repository that is on Azure Devops in your case.
So you need to call the git-credentials endpoint (it's the same syntax on AWS and Azure) to create it.
Once your git credentials up-to-date, the creation of the repo should work as intended.
I am having an issue getting my OnPrem Azure DevOps 2019 Server to allow things to authenticate to it with Personal Access Tokens (PAT). No mater what I do, I get failed to authenticate using the supplied token.
How I am creating my token:
Log into my OnPrem devops site
Go to my user profile icon in the top right, click security click personal access tokens, click new token
In Create new personal access toekn for some reasobn the organization (colleciton) I want to use is not listed, I am seeing an old XML based collection but not my new Inheritance based collection, why doesn't the newer format collection show up? My user account is an admin account, you'd think it would be here?
If I create a PAT token for the old XML based collection and give it full access plus a 90 day expiration it creates it fine
Now I have a PAT token bases off the old XML based collection, but that still doesn't work, if I run the AZ CLI I get this
AZ DEVOPS LOGIN --organization https://tfs.mydomain.com/OldXmlCollection --verbose
Token: {paste in token}
Creating connection with personal access token.
Failed to authenticate using the supplied token.
Command ran in 6.385 seconds (init: 0.167, invoke 6.12)
I also have the same problem if I try to set up a build agent using a PAT token. Fails every time, but if I change to negotiate auth it works immediately.
On the IIS end the service is running on the authentication is set up to Anonymous Authentication: Enabled, ASPS.NET Impersonation: Disabled, Basic Authentication: Enabled, Digest and Forms: Disabled and Windows Authentication: Enabled
any ideas what I am doing wrong, what to look at?
PAT Token isn't working on 2019 OnPrem Azure DevOps
You could try to disable IIS Basic Authentication.
That because when IIS Basic Authentication is enabled on your windows machine, it prevents you from using personal access tokens (PATs) as an authentication mechanism.
Please check this document Enabling IIS Basic Authentication invalidates using Personal Access Tokens for some more details.
What it turned out to be is a missing ACL in the file system. The service account that is running TFS needs to have write permission to the machine keys folder at %ProgramData%\Microsoft\Crypto\RSA\MachineKeys
Why in the world is the installer not setting this permission? PAT will not work until this is set
I am using invoke rest api task in pipeline
I have saved Azure DevOps Pat token in variable with name system.AccessToken
Its giving error unauthorized
I also tried converting PAT token to base64 and then saving in variable. Its still showing same error.
What value do I need to provide for system.AccessToken ?
You do not need to add PAT token to your variables. System.AccessToken is a special variable that carries the security token used by the running build.
In addition, you are using Invoke Rest API task in an agentless job, so you also don't need to set "allow scripts to access the OAuth token" option.
Here is my sample and it works well:
1.Create service connection:
2.Configuration of Invoke REST API task:
I'm implementing web extension for VSTS for specific release management. From that extension I need to start a new release for specific release definition. The prerequisite for creating the release is concrete set of used artifacts and their correct versions. And that's my pain.
There is a RESP API call (undocumented) which provides list of artifacts with set of versions for each of them and also with pre-selected default artifact version to be used by new release. Such API is called by VSTS itself when you open the dialog (or side panel) for new release creation. It is also defined in VSTS Release API (ReleaseManagement/Core/RestClient), the name of function is getArtifactVersions.
The underlying request look like https://{collectionName}.vsrm.visualstudio.com/{projectId}/_apis/Release/artifacts/versions?releaseDefinitionId={releaseDefinitionId}.
When I run the request directly in the browser, I'm getting the correct answer (list of artifact and their available versions). But when I run it from the web extension, I'm getting 401 error: TF400813: The user '7a6b0fe0-56fd-61f5-9a78-2e170802ac50' is not authorized to access this resource.
I'm quite sure that problem is located somewhere in the scopes set in the vss-extension.json manifest file. I have put there all possible scopes defined in the documentation, but still no success. The user identity from the error message is my identity and I have admin rights to our collection. When I run the request personally (using personal access token or just logging in the browser) the result is served correctly. But same user using generated Bearer token is then unauthorized. So from that behavior my guess is coming. Something is missing in the manifest scopes.
U put all scopes from documentation (the highest ones from each scope group) and still no success. Here is the list of them:
"vso.agentpools",
"vso.build_execute",
"vso.release_manage",
"vso.code_manage",
"vso.packaging_manage",
"vso.work_write",
"vso.agentpools_manage",
"vso.chat_manage",
"vso.dashboards_manage",
"vso.entitlements",
"vso.extension_manage",
"vso.extension.data_write",
"vso.gallery_manage",
"vso.identity",
"vso.notification_manage",
"vso.profile_write",
"vso.project_manage",
"vso.symbols_manage",
"vso.test_write"
Any idea, which scope I'm missing?
I reported an issue here: VSTS extension ReleaseManagement/Core/RestClient api 401 error, that you can follow.
You may send the http request with Personal Access Token to retrieve necessary data.
You also can get the release definition, then get each artifact version through corresponding REST API, such as Get a list of commits, Get a list of builds.
Grunt requires a github authentication token while it is executed during a docker build for an image. How do I provide grunt with the required token? Interactive passing doesn't work (as I already exacted).
The message of grunt is:
Running "volo:add:-nostamp:twbs/bootstrap/3.3.2:packages/bootstrap/" (volo) task
GitHub auth required for api.github.com/repos/twbs/bootstrap/tags: {"message":"API rate limit exceeded for XXX.XXX.XXX.XXX. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://developer.github.com/v3/#rate-limiting"}
GitHub access token required to complete action. In a web browser, go to:
https://github.com/settings/tokens
and generate a new "Personal access token" for volo. Only "repo" and "public_repo" permissions are needed. Paste the generated token below.
GitHub Personal Access Token:
You can change the URL of the repository to:
https://<token>:x-oauth-basic#github.com/MyApp/myapp.git
More informations at: https://devops.profitbricks.com/tutorials/configure-a-docker-container-to-automatically-pull-from-github-using-oauth/