How to call Azure Rest API of Azure Batch service using SharedKey authorization?
I would like to create a azure batch job's task by calling Azure Batch REST API.
I've created Azure batch account and application pool
and Mapped applicaion package with application pool
Created Azure batch job named as "TestBatchJob"
I would like to create Tasks under this "TestBatchJob" job via REST API.
https://<batchaccountname>.<location>.batch.azure.com/jobs/{{jobid}}/tasks
How to consume this API using SharedKey to create task in ADF.
This guide provides you the necessary information to create the authorization headers required for direct REST API calls using shared key.
Related
I see that Service Connection is a link between Azure Pipelines and Azure Subscription to trigger Pipelines.
But can I create a Service Connection and get client id and secret and use that to obtain access token. And with that access token I can run the Azure Pipeline via c# code with REST APIs?
Is this what a service connection used for ?
But can I create a Service Connection and get client id and secret and
use that to obtain access token.
You can generate token via this, but you can't use it in C# code to run the pipeline.
Service connection between 'Azure Pipelines and Azure Subscription' just for you to create an app in AAD, this will also create related Enterprise App(service principal) in Azure portal side.
The service principal can be assigned permissions in the Azure portal to access resources. Once the service principal has access to a resource at the Azure Portal, the devops pipeline using the service connection associated with the service principal will also have the same access.
The original purpose of the above is service connection design is to allow the pipeline to have access to the resources at the portal.
Why we can't use the app's clientid and secret to get an access token to run the pipeline?
It is clearly in this official document:
https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/authentication-guidance?view=azure-devops
The Azure DevOps API doesn't support non-interactive service access
via service principals.
I am fairly new to Azure Devops and Azure Databricks.
I have created Azure Databricks workspace using Azure Devops CI/CD Pipeline.
Now I am looking for a solution to Add Users to Azure Databricks workspace using DevOps Pipeline.
There are several methods:
Use databricks_user resource from Databricks Terraform provider - it could be as simple as example below, or you can combine it with azuread provider, and pull users from Azure Active Directory. Another advantage of Terraform provider is that you can combine it with user groups, and other things.
resource "databricks_user" "me" {
user_name = "me#example.com"
}
Use Databricks SCIM API for Users (part of general SCIM API) - creation is quite straightforward, you just need to form correct JSON as described in docs (I didn't want to copy JSON from there), and do a call with curl or something like that. Also,
There is a collection of PowerShell scripts developed by the DataThirst company, that include scripts for adding & removing users, etc. These scripts are using REST API under the hood, and could be simpler than to use REST API. Some of these tasks are also available on the DevOps marketplace.
In any case, you need to authenticate to the workspace. For automated pipelines you have two choices - service principals or managed identity associated with DevOps worker, and they should have Owner or Contributor permissions on the workspace level, or be added into workspace as admin users.
For REST API authentication of service principal is described in details in documentation, for managed identity you just get the token from internal REST API.
Databricks Terraform provider also supports both service principals and managed identity.
I have deployed an API with certain business logic in AKS. The load balancer type is internal. I am able to access this within AKS cluster. on the below address
http:servicename/myapi/
But I want to call this API from an ADF pipeline. How can I do that? What are the configurations I should do to be able to call this API from ADF?
You have to use Web Activity to make Rest API call from Azure DataFactory Pipeline.
An Azure Data Factory may be used to call a custom REST endpoint
through Web Activity. You can send datasets and connected services to
the activity to be consumed and accessed.
Note : By utilizing self-hosted integration runtime, Web Activity may also invoke URLs that are hosted on a private virtual network. The URL endpoint should be visible to the integration runtime.
Please check below documentation to know more about web activity : Web Activity in Azure Datafactory.
Is it possible to have a service principal account trigger a build pipeline? I have a CMS that whenever a document is published it fires of an event/webhook that's calls an Azure Function. The function then calls Azure DevOps using the API to trigger the correct build pipeline. Up until now we have relied on personal access tokens (PAT) from a "lucky" team member, but obviously that isn't an ideal solution. If the PAT expires or the team member leaves our pipeline breaks down. I was hoping to use the PAT Lifecycle Management API* to generate a PAT on the fly, but as the documentation states; "On-behalf-of application" solutions (such as the “client credential” flow) and any authentication flow that does not issue an Azure AD access token is not valid for use with this API".
This seems like a fairly common scenario, having an external dependency kicking of a build pipleline, so how should I go about doing this without using person-dependent tokens?
https://learn.microsoft.com/nb-no/azure/devops/organizations/accounts/manage-personal-access-tokens-via-api?view=azure-devops
Externally trigger an Azure DevOps Build using a Sevice Principal account
I am afraid it it impossible to use REST API to trigger an Azure DevOps Build using a Sevice Principal account.
That because the Azure DevOps API doesn't support non-interactive service access via service principals.
You could get the info from the document Choose the right authentication mechanism：
Hei
I have a simple scenario, where I have an on-premise system that hosts a rest api. I what based on data in that rest to fill data into a Auzure SQL database using some type of synchronization job. I just unsure of the best method to do this? Can one use Azure Data Factory for this? What other services can do the job?
Under Azure app services is a background task service called Azure Web Jobs
Here are links to help you get started:
Azure App Service Overview: https://azure.microsoft.com/en-us/documentation/articles/app-service-value-prop-what-is/
Overview from another website: http://www.informit.com/articles/article.aspx?p=2423911
Azure Web Jobs introduction: https://azure.microsoft.com/en-us/documentation/articles/web-sites-create-web-jobs/